00:51:19 GMT <madsage> good morning\evening\afternoon
00:53:02 GMT <madsage> so I was able to get my redis master/slaves with sentinel slave promotion working
00:53:35 GMT <madsage> but I have noticed in a restart of the master node redis, an ID mismatch
00:53:41 GMT <madsage> Partial resynchronization not accepted: Runid mismatch (Client asked for '77db96d72609b1f460920c4e0f85570f0d8f60df', I'm '59b0c4079b841d9f46871141cc4e14682641eb87')
00:53:58 GMT <madsage> can I ignore this? or do I have a problem.
00:54:22 GMT <madsage> I googled it, and i see some things that suggest its normal.
00:59:06 GMT <madsage> so from my understanding it only rejected partial resynchronization  however I do see that Synchronization with slave's succeeded
00:59:47 GMT <madsage> i think I am talking to myself.
01:00:21 GMT <madsage> maybe answered my own question. gotta love it, just had to think it through and google a lil more
06:03:50 GMT <dbkaplun> Am I correct in saying that each of those operations are atomic with respect to each other? https://gist.github.com/dbkaplun/c21dc236f51e22e640b0c6541b73c2be
07:14:48 GMT <dbkaplun> anyone?
07:16:00 GMT <dbkaplun> Am I correct in saying that each of those operations are atomic with respect to each other? https://gist.github.com/dbkaplun/c21dc236f51e22e640b0c6541b73c2be
07:16:13 GMT <minus> no need to repost
07:20:00 GMT <minus> i wasn't familiar with WATCH/MULTI/EXEC until now, but it looks good to me
07:23:27 GMT <dbkaplun> minus: how would you solve this problem if you didn't know about them? (queueing reactions and publishing them)
07:24:13 GMT <dbkaplun> there are many ways to solve this problem it seems
07:24:18 GMT <minus> lua scripts where necessary, "lock"free access in places where consistency among multiple keys is implied
07:25:45 GMT <dbkaplun> what if you didn't have 2.6.0?
07:25:53 GMT <dbkaplun> minus: ^
07:26:27 GMT <minus> hm? you run <2.6? you should upgrade
07:27:01 GMT <minus> you could do some client-side-initiated locking i guess
07:27:09 GMT <minus> redlock i think it's called
07:27:23 GMT <minus> gotta go, bbl
08:04:46 GMT <dbkaplun> but what if two ticker processes happen at the same time? couldn't there be a deadlock?
08:08:57 GMT <dbkaplun> i.e. one ticker reads chans, c1reactions, c2reactions but hasn't EXECed yet, another ticker EXECs and DELs chans?
08:11:49 GMT <dbkaplun> Or another scenario: reactions happen constantly, ticker runs but doesn't EXEC, reaction happens during and causes EXEC to fail. there may never be a chance for a ticker to execute successfully?
13:54:25 GMT <dudeji> is there is any way to find max connection ever in happened in redis server ??
13:54:47 GMT <dudeji> not total connections max connections at one point.
14:44:44 GMT <sts> hello folks. is redis protocol resilient against network issues? Can it be that with a php->predis->redis connection, due to network issues, php gets the response of a lookup which done in parallel?
15:04:43 GMT <minus> redis uses TCP and has an internal bufffer, that's it afaik
15:05:09 GMT <dbkaplun> hi minus
15:06:23 GMT <dbkaplun> I'm still thinking about that problem from last time
15:06:33 GMT <dbkaplun> reactions happen constantly, ticker runs but doesn't EXEC, reaction happens during and causes EXEC to fail. there may never be a chance for a ticker to execute successfully?
15:06:35 GMT <dbkaplun> https://gist.github.com/dbkaplun/c21dc236f51e22e640b0c6541b73c2be
15:09:14 GMT <minus> if it's always interrupted, no
15:09:58 GMT <dbkaplun> no?
15:10:54 GMT <dbkaplun> sorry i don't understand what you're saying no to
15:21:07 GMT <dbkaplun> minus: ? (sorry if i'm bothering you i will stop)
15:24:17 GMT <dbkaplun> does anyone have experience dealing with race conditions and resource contention in redis? trying to figure out if these operations are safe https://gist.github.com/dbkaplun/c21dc236f51e22e640b0c6541b73c2be
15:37:56 GMT <minus> if the watched keys constantly get modified the transaction will always be aborted
15:39:18 GMT <minus> so if you got a lot of modifications to those you'll never manage to get your other thing executed
15:39:52 GMT <minus> i'd guess that's a thing you come across only with growing traffic to your thing
15:43:25 GMT <dbkaplun> minus: we're handling traffic on the order of tens or hundreds reactions per second conservatively
15:44:01 GMT <minus> hundreds could become problematic already, i guess
15:44:21 GMT <minus> you may want to put your logic into a lua script and EVAL that
15:44:30 GMT <minus> but why not just try?
15:47:40 GMT <dbkaplun> what do you mean just try?
16:03:59 GMT <minus> how well or badly it works, i'm sure you can generate some test traffic
16:43:00 GMT <Jason_> memcached or redis for heavy read - light write/delete ?
17:06:57 GMT <dbkaplun> is there some better way to do these operations? https://gist.github.com/dbkaplun/c21dc236f51e22e640b0c6541b73c2be
17:44:12 GMT <Asher128> Hi guys, a little question: is there any way to scale redis over different coutries?
17:44:21 GMT <Asher128> (redis cluster)
17:44:31 GMT <Asher128> I have 1 cluster in the United States
17:44:41 GMT <Asher128> and when I try to read/write from china, it is very slow
17:44:53 GMT <Asher128> is there any way to create a sort of cache in china
17:45:13 GMT <Asher128> or a sort of redis slave that is going to syncronize with the USA and give a faster result?
17:45:26 GMT <Asher128> badboy_, ?
17:46:48 GMT <badboy_> what?
17:47:13 GMT <Asher128> badboy_, any idea?
17:47:49 GMT <Asher128> about "is there any way to scale redis over different coutries?"
17:48:13 GMT <badboy_> I don't even know what's asked :D
17:48:28 GMT <Asher128> lol sry badboy_  :)
17:48:31 GMT <Asher128> badboy_, "I have a redis cluster in the United States. and when I try to read/write from china, it is very slow. is there any way to create a sort of cache in china. "or a sort of redis slave that is going to syncronize with the USA and give a faster result?
17:48:35 GMT <badboy_> There's no master-master, if that is what you want to know
17:48:49 GMT <badboy_> you can put slaves somewhere else, at least for reads that works
17:49:06 GMT <Asher128> badboy_, but if I put slaves, they could become masters
17:49:16 GMT <Asher128> badboy_, masters in china will be VERY slow in the USA
17:49:34 GMT <badboy_> you could mark them with a low priority
17:49:40 GMT <Asher128> badboy_, is there any way to prevent a redis slave to be used during the fallback?
17:50:13 GMT <badboy_> hm, at least for Sentinel, I have no idea if Cluster respects that
17:50:28 GMT <badboy_> https://github.com/antirez/redis/blob/unstable/redis.conf#L409-L422
17:50:29 GMT <Asher128> badboy_, yeah... I use cluster
17:50:57 GMT <badboy_> yeah, check if it does the same^^
17:51:30 GMT <Asher128> ok i'll check it badboy_ ^^
17:57:24 GMT <minus> Asher128: we've seen slave syncing to china go at below 100KB/s only (over OpenVPN though)
18:03:30 GMT <Asher128> minus, with redis cluster?
18:03:45 GMT <minus> no, plain master/slave setup
18:04:21 GMT <Asher128> minus, what I want is to provide a sort of caching system in china, for a redis server hosted in the USA.
18:04:48 GMT <Asher128> minus, something like a master/master replication (is know it is not possible to do with redis)
18:06:07 GMT <minus> if you're running a cluster you're shit out of luck
18:06:22 GMT <minus> slave-priority does not apply to clusters, only sentinel setups
18:06:46 GMT <minus> there currently is no way to prevent a slave from becoming a master in a cluster setup
18:08:15 GMT <minus> if you don't need the cluster but can live with a normal setup or a sentinel failover setup you could work with a slave in china
18:09:23 GMT <Asher128> minus, yeah it could be a solution...
18:09:42 GMT <minus> that's how we're currently doing it
18:09:47 GMT <Asher128> minus, redis cluster is a bit dissapointing because there is a lot of limitations
18:09:57 GMT <minus> china is a fucking pain, internet-wise
18:10:01 GMT <Asher128> minus, yeah
18:10:04 GMT <Asher128> minus, that's true
18:10:12 GMT <Asher128> minus, i'll try it, thanks minus
18:10:12 GMT <minus> do you have an ISC license?
18:10:17 GMT <Asher128> minus, yes
18:10:25 GMT <minus> is it hard to get one?
18:11:17 GMT <Asher128> minus, I'm not the person who managed the ISC license in the company ;)
18:11:41 GMT <minus> fair, i won't be either, no way i'm doing that
18:11:50 GMT <Asher128> :D
18:12:14 GMT <minus> what provider are you using in china, if i may ask?
18:19:58 GMT <Asher128> minus, txnetwork
18:20:02 GMT <Asher128> minus, what about you?
18:20:13 GMT <minus> aliyun i think it's called
18:20:49 GMT <Asher128> minus, why did you choose redis over the other key/value stores
18:21:28 GMT <Asher128> minus, performance?
18:21:45 GMT <minus> we're baking/syncing data from postgres into redis for fast read accesses. redis' data structures allowed for nice querying
18:22:12 GMT <minus> we were using mongodb at the beginning. man, what a joke that thing is
18:22:36 GMT <Asher128> minus, hahaha
18:23:13 GMT <minus> thinking about dropping redis though and going with raw sql if it's fast enough, because synching to redis is failure-prone
18:23:57 GMT <Asher128> minus, I'm curius: what dissatisfied you with mongodb
18:24:09 GMT <minus> performance wasn't as good as expected
18:24:20 GMT <Asher128> yeah... I had the same experience
18:24:23 GMT <minus> the C++ client library was very cumbersome
18:24:25 GMT <Asher128> It is SLOW...
18:24:38 GMT <minus> i think it was also buggy
18:24:44 GMT <minus> that was like 3 years ago
18:25:04 GMT <Asher128> it is less buggy
18:25:06 GMT <Asher128> but still slow
18:25:15 GMT <minus> we migrated our stack to java 2 years ago (making the huge mistake of picking vert.x as foundation)
18:27:03 GMT <Asher128> because it consumes a lot of memory?
18:28:00 GMT <minus> that's not really a problem
18:28:10 GMT <Asher128> what was the issue
18:28:15 GMT <minus> vert.x is asynchronous and doesn't play well with most java libraries
18:28:42 GMT <Asher128> ahh yeah... you should have chose a node.js library
18:28:50 GMT <minus> if i had known about using "boring" technologies back then…
18:29:46 GMT <Asher128> hehe
18:30:01 GMT <minus> if i had to redo it again i'd probably go with python (which was actually always my first choice) or back to C++ (except a different foundation instead of Qt)
18:30:38 GMT <Asher128> i love python
18:30:42 GMT <Asher128> but there is a GIL
18:31:11 GMT <minus> since it's mainly a HTTP server it would've worked pretty well with uwsgi or gunicorn
18:31:16 GMT <Asher128> yeah
18:31:29 GMT <Asher128> with uwsgi/gunicorn, you don't care about the gil
18:31:49 GMT <minus> i did start writing a PoC with asyncio and aiohttp
18:31:53 GMT <minus> but that was terribly slow
18:32:23 GMT <minus> like 300req/s-slow with protocol-parsing only
18:32:56 GMT <minus> the other stuff is at least 10 times as fast, but a lot uglier :')
18:33:41 GMT <Asher128> what slowed down your code?
18:33:59 GMT <Asher128> because it is interpreted?
18:34:10 GMT <minus> aiohttp itself isn't very fast, and JSONObject's mapping was taking its time too i think
18:34:14 GMT <Asher128> because it wasn't using the multi processors correctly?
18:34:22 GMT <Asher128> ok
18:34:30 GMT <minus> i was only comparing single-threaded performance
18:34:50 GMT <minus> also python 3.5 means no pypy support
18:38:15 GMT <Asher128> ok  minus