Low Latency Architecture – AWS

Re:Low Latency Architecture – AWS by kartiksura on Monday May 08, 2017 @04:20PM (#14295230)

I work in subsecond latency domain. Can you elaborate what do you want to achieve in the sub-second: Does the ingestion have to be subsecond? or the entire cycle of ingestion and data available into redis/DDB needs to be done in sub-second?

[ Parent | on HN ]

Re:Low Latency Architecture – AWS by minormind on Monday May 08, 2017 @04:29PM (#14295306)

Entire cycle in sub-second. I agree that it depends on many factors but as per the system, we are looking subsecond from ingestion of data from client to availability of data for query.

A -> stream -> datastore (subsecond)

[ Parent | on HN ]

Re:Low Latency Architecture – AWS by kartiksura on Monday May 08, 2017 @04:31PM (#14295321)

I see. I would suggest you explore the following: 1. CitusDB (an offshoot of postgres with high ingestion rate, cloudflare used it) 2. Druid.io (popular in ad-tech) 3. Linkedin Pinot (uber uses it for similar use-cases) 4. memSQL (used in ad-tech)

[ Parent | on HN ]

Re:Low Latency Architecture – AWS by minormind on Monday May 08, 2017 @06:37PM (#14296273)

Thanks for the pointers but I am planning to revolve more around AWS ecosystem so that I have less ops burden of managing servers + regionalization becomes easy.

[ Parent | on HN ]