08:01:45 GMT <nohitall> hi, i was looking into replication and clusters, but I still wonder what the best solution for geo-replication would be? meaning I got EU and US machines which ideally have to be in a master-master setup, any ideas how to solve this?
08:36:18 GMT <minus> not with redis
08:36:38 GMT <minus> at least not by itself
09:17:10 GMT <nohitall> minus: I read about some other wrapper once, just forget the name for it
09:17:14 GMT <nohitall> iirc was made by facebook
21:07:41 GMT <JordiGH> Huh, these SCAN commands are really pegging CPU.
21:08:00 GMT <minus> they're scanning the whole keyspace after all
21:08:01 GMT <JordiGH> Are they worse the bigger a cache is?
21:08:34 GMT <minus> KEYS/SCAN time is O(n) where n is the number of keys in redis
21:08:59 GMT <minus> they're not recommended for production use iirc
21:11:11 GMT <JordiGH> "Since these commands allow for incremental iteration, returning only a small number of elements per call, they can be used in production without the downside of commands like KEYS or SMEMBERS that may block the server for a long time (even several seconds) when called against big collections of keys or elements."
21:11:20 GMT <minus> it's basically like using a RDBMS table without index
21:20:10 GMT <JordiGH> Now I know why Redis isn't a database, I guess.
21:20:48 GMT <minus> yes, it's a key value store, it has nothing like indices
21:21:03 GMT <minus> (you can build them yourself though)
22:12:36 GMT <badboy_> minus: JordiGH couple of things
22:12:48 GMT <badboy_> 1. SCAN is considered ok for production use (in contrast to KEYS)
22:13:11 GMT <badboy_> 2. SCAN is O(1) for a single call (O(N) for a full iteration obviously)
22:13:37 GMT <badboy_> 3. It should not peg the CPU per single call, if it does please report it
22:13:51 GMT <badboy_> 3 1/2: I know people using Redis as their main data store ;)
22:13:52 GMT <JordiGH> No, it's the iteration.
22:14:10 GMT <JordiGH> My django code is doing scanning and then iterating over all hits.
22:14:13 GMT <JordiGH> Is O(N)?
22:14:38 GMT <badboy_> a full iteration, yes. but at least a single call is not blocking the server
22:15:02 GMT <badboy_> but if you do that regularly you might rethink your implementation
22:15:15 GMT <JordiGH> Oh, this implementation will definitely be rethought.
22:15:28 GMT <badboy_> good :)
22:16:21 GMT <JordiGH> This is doing a scan to clear out some keys, trying to invalidate a cache.
22:16:58 GMT <JordiGH> But the code to insert into Redis was actually originally disabled, so it was always scanning an empty cache and nobody noticed.
22:17:21 GMT <JordiGH> Now the cache is getting populated by something else and this is now scanning to remove keys that are never there.
22:19:03 GMT <JordiGH> My predecessor left this implementation half-done and it hadn't caused any problems yet because we always had an empty Redis instance. Now that we decided to actually use Redis, everything is terrible.