15:23:03 GMT <Scorchin> Hello. I'm running redis 3.2 with native clustering. There are 6 nodes in the cluster (3 masters, 3 slaves). Each node is a m4.2xlarge (32.0 GiB, 8 vCPUs). The maxmemory is set to 16GB, with a maxmemory-policy of volatile-ttl. Whenever all of my nodes have more than 90% of maxmemory used, or when evictions are high (due to maxmemory-policy) I find that my slaves disappear from the cluster and need to reload their data.
15:23:03 GMT <Scorchin> This means that my cluster is unable to take reads or writes. Has anyone seen this behaviour before? Is there a known best practice for operating at these high-ish volumes?
15:24:05 GMT <Scorchin> I can't upgrade to redis 4.0, and nor do I want to. I have a running production system that I want to keep running. When/if I want to upgrade I'll come up with a plan but it's unlikely as a short-term response to what I'm seeing.
15:25:46 GMT <Scorchin> The caveat is that the redis 4 upgrade is likely to wait until January, when I'm got more bandwidth
15:27:39 GMT <xiaomiao> Scorchin: are you using any form of persistence with redis?
15:27:45 GMT <Scorchin> AOF
15:27:57 GMT <Scorchin> with RDB
15:28:06 GMT <Scorchin> I've seen this behaviour with AOF turned off
15:28:13 GMT <xiaomiao> could it be that the extra memory pressure of writing the snapshots causes issues?
15:28:32 GMT <Scorchin> `maxmemory` is set to half of the available RAM on the node
15:29:42 GMT <xiaomiao> and for a snapshot worst case you need a copy of the data so you can persist it, which would be two times half the memory, worst case
15:29:54 GMT <xiaomiao> I hope my intuition is wrong :)
20:03:01 GMT <kaos01> hi, is running multiple instances of redis on the same server recomended ?
20:03:16 GMT <kaos01> also, shoudl one but int some memory limits than
20:03:38 GMT <Xe> kaos01: do you regularly stare into the eyes of madness?
20:03:49 GMT <minus> lol
20:04:02 GMT <Xe> also are you sure that all of the databases will have all of their data smaller than ram at every time?
20:04:09 GMT <kaos01> Xe ?
20:04:32 GMT <kaos01> what is madness ?
20:04:39 GMT <kaos01> i understand you can set maxmem
20:04:44 GMT <kaos01> which is not a hard limit
20:04:54 GMT <minus> it is
20:05:03 GMT <kaos01> well it can sort of be exceeded a bit
20:05:09 GMT <minus> well yeah
20:05:11 GMT <kaos01> im fairly sure its not hard
20:05:16 GMT <Xe> kaos01: Madness is when you create a server configuration that gaslights you
20:05:18 GMT <minus> you should leave some air for presistence anyway
20:05:19 GMT <kaos01> so where is my madness
20:05:23 GMT <Xe> please just pay for other machines
20:05:33 GMT <Xe> you will save money in the long run (your sanity)
20:05:47 GMT <minus> well, it really depends on how much data you intend to store
20:05:53 GMT <kaos01> ok fair enough, just lots of sampels out there on runnign multiple instances on same machine
20:06:10 GMT <Xe> kaos01: there's lots of samples for using sprintf for database queries too
20:06:16 GMT <kaos01> must be losts of mad people out there :P
20:06:31 GMT <kaos01> well the worst recomendations seems to be using multipl db's :P
20:06:41 GMT <kaos01> so i was looking at a midle ground :P
20:06:43 GMT <kaos01> hehe
20:06:52 GMT <kaos01> not to mad, just a bit
20:07:12 GMT <kaos01> although it is a bit extreme to setup a seperate server for a small redis instance each time
20:07:26 GMT <kaos01> after all the server uses more resources than does redis
20:07:43 GMT <kaos01> specially for small redis instances
20:07:54 GMT <kaos01> seems mad to setup a server for each as well
20:08:48 GMT <minus> if you run a few things on your personal server that each require redis, then running an extra instance for each is fine, your probably don't even need a memory limit
20:09:21 GMT <minus> if you expect to use gigabytes of memory with any one of the uses you probably want to run them on separate machines
20:09:59 GMT <minus> but there's nothing inherently wrong with running multiple redis instances
20:10:33 GMT <minus> in fact if you have a large server with many cores it's common practice to run multiple redis instances on it since each one only runs on a single core
20:11:57 GMT <kaos01> yeap so not so mad :P
20:16:02 GMT <kaos01> so please take back calling me mad :P
20:17:55 GMT <kaos01> do you need to set a custom "dir" for each isntance or just a custom name for dump dile (pidfile,logfile,bind) will do ?
20:20:28 GMT <minus> latter should do
20:21:45 GMT <kaos01> cool thats what i got :)
20:21:46 GMT <kaos01> thanks
20:22:52 GMT <kaos01> is running redis in docker mas ? :)
20:22:54 GMT <kaos01> mad
20:25:43 GMT <minus> somewhat
20:26:00 GMT <minus> it's simple enough to not need that
20:26:35 GMT <minus> if you run any kind of HA setup you'll have to turn off NAT (with --net=host)
20:28:05 GMT <kaos01> current setup is just redis in docker with no settings
20:28:22 GMT <kaos01> it sees the host RAM, no sytem tuning, etc ..
20:28:29 GMT <kaos01> hahah