02:35:34 GMT <kaos01_> hi, is 2 redis suervers sufficent, with 3 sentinels ?
02:36:00 GMT <kaos01_> the odd number got me confused, that relates to no of sentinels ?
08:32:38 GMT <cloudbud> can i have a standalone redis server
08:35:46 GMT <minus> yes
08:42:01 GMT <cloudbud> what is the latest stable redis version
08:42:45 GMT <minus> check the website, should be 3.2.something
08:43:20 GMT <minus> i wonder when 4.0 is gonna hit; it's been a RC for more than half a year already
09:10:09 GMT <King_DuckZ> 'morning
09:11:53 GMT <King_DuckZ> I'm thinking to add support for binary int to my incredis library, since it's quite easy to do and it would be a handy functionality for me to have now, but after reading this https://stackoverflow.com/questions/26799074/can-we-set-c-int-array-as-a-keys-value-in-redis-by-hiredis I'm thinking if it's really a good idea
09:13:50 GMT <King_DuckZ> I see why the person there says it's a "recipe for disaster", but my library provides a RedisInt typedef which is supposed to match the size of redis' int, and I'm planning to always convert to big-endian before sending ints
09:14:16 GMT <King_DuckZ> should I do that, or should I use strings everywhere?
09:17:45 GMT <minus> iirc i'm already storing binary data like that in redis already somewhere
09:26:24 GMT <King_DuckZ> kk ;)
14:59:13 GMT <John> yay theres a #redis!
14:59:23 GMT <John> Hi Redis, i'm not a heavy Redis user but i know of it's existance
14:59:48 GMT <John> In my field of research, people like to heavily rely on Unix pipes to do inter-process communication
15:00:20 GMT <John> So process1 sends data to process2, and that's how several tools will be chained together
15:01:08 GMT <John> However, i believe this results in lazy thinking about different ways in which problems can be solved. It's a technical debt, at the end of the day, because developers/users don't have control over a lot of things. I believe that one could perform IPC using Redis, and that would be pretty convenient
15:02:13 GMT <John> Through an inmemory database, two processes can talk to each other, rather than one-way communication, and there are an infinite number of channels, rather than just stdout, stdin, stderr.
15:02:49 GMT <John> So, um, i'd very much like to write something in my thesis that supports the idea that Redis could be used for IPC, but i wouldn't know where to look for an example of it... does anyone here know?
15:27:49 GMT <minus> redis is mostly used one-way, as a job queue, for IPC from what i've seen. though you certainly have the possibility there to signal results back
15:33:16 GMT <John> Yeah, i think that's a reflection of most problems requiring that sort of solution
15:33:42 GMT <John> it's difficult to conceive of how two processes might talk to one another to make an output
15:33:53 GMT <John> or even three, plus, processes, working together in a mesh rather than a pipeline
15:34:19 GMT <John> But still, the biggest question in my mind is, is Redis fast enough
15:34:30 GMT <John> for a job queue
15:35:12 GMT <John> I think the answer is probably yes, because i'm still thinking of Redis being at Postgres speeds - which is usually OK
15:35:40 GMT <minus> yeah, redis is faster than postgres
15:36:00 GMT <minus> zeromq is a good candidate for fast ipc
15:36:05 GMT <John> mm..
15:36:46 GMT <John> yeah i've heard of that one as well, usually in connection without protobuf and that otherone with the fun name
15:36:56 GMT <John> er, like... BAM! or something
15:37:11 GMT <John> MrFast?
15:37:32 GMT <John> gah, I totally forget
15:38:04 GMT <minus> capnproto, flatbuf, all the same category of tools
15:38:25 GMT <John> Ah, capnproto! that's the onee
15:38:34 GMT <John> MrFast was way off...
15:40:21 GMT <John> I think the best demonstration I can think of for others in my field is to take something like "process1 -in input_file | process2 -in /dev/stdin -out /dev/stdout | process3 -out output_file"
15:40:42 GMT <John> where some 4Gb text file is being passed from process1 to 2 ,and then a different 4Gb file from 2 to 3
15:41:10 GMT <minus> well, that's very useful if you're processing data and need to store intermediate results
15:41:28 GMT <John> And then show that this can all go through Redis, using 8Gb of RAM (or 10, who cares), and some logic that if process3 fails the output of process2 is still in memory
15:41:32 GMT <John> and same for process2 and process1's output
15:41:42 GMT <John> To show that you can at least unwind a pipeline with Redis
15:42:13 GMT <John> Because currently we forgo the ability to do anything like that because we only use pipes
15:42:14 GMT <minus> yeah, you don't wanna keep that in memory unless you have lots spare
15:42:38 GMT <John> Oh we do - typically the machines have 124Gb of memory minimum
15:42:43 GMT <minus> you mean you can't only partially process the dta?
15:43:03 GMT <John> well that's a whole level of stuff we cant do
15:43:24 GMT <John> Resulting in pipelines where a lot of extra processing is done because downstream programs can't tell upstream programs what they need
15:43:39 GMT <John> or, you know, work with one another
15:44:45 GMT <John> Storing everything in memory, for example, is probably not going to be a good idea 99% of the time - but it's nice to be able to. And with pipes we cant :/
15:44:56 GMT <minus> so what you need is a paradigm shift, from push (first process shoves in the data) to pull (the last process pulls/asks for the data it needs)
15:46:09 GMT <John> right, exactly - and from 70s concepts of doing IPC to 2010s concepts for doing IPC
15:46:34 GMT <John> hm, yeah, last process asks for the data. hm.
15:46:54 GMT <John> That's a really interesting idea minus :)
15:47:25 GMT <minus> i'm just gonna say one word here: microservices
15:48:05 GMT <minus> (it actually works just as well if you throw everything into one process, it's just a lot easier to manage then if it's one-shot experiments)
15:49:44 GMT <John> wow, microservices are a concept i've never even heard of - this is awesome
15:50:05 GMT <John> composability plays a big part in this section of my thesis, yet i've never come across this
15:50:14 GMT <John> thanks again :D
15:50:46 GMT <minus> modern IT has all the hype
15:51:44 GMT <John> Yeah :) But none of it filters down to academia/research :P heheh
15:51:50 GMT <John> But i'm really grateful for the leg-up
15:56:40 GMT <minus> there's also frameworks that do manage processing jobs and allow you to chain them together into pipelines, in case you haven't come across those
15:57:01 GMT <minus> though i don't think they go into finer-grained data selection/rpc
16:05:03 GMT <John> yeah, i see a new "workflow creator" being released daily :)
16:05:28 GMT <John> But if the tools fundamentally can only push output down stdout/stderr, what good are they...
16:06:05 GMT <John> hehe, i mean, i know there are benefits. but they are never quite as useful as they are made out to be
16:06:55 GMT <minus> luckily, my excursion into data evaluation was short enough not to get too deep into such a tool
21:33:39 GMT <kaos01_> hi, does one put sentinals on the app servers ?