News for hackers. Stuff that matters.
Re:My £4 a month server can handle 4.2M requests a day by paraph1n on Tuesday September 07, 2021 @06:18PM (#28450603)
There are many binary encoding protocols. A popular one is protobufs[1], which is used by gRPC.

[1]: https://developers.google.com/protocol-buffers

Re:My £4 a month server can handle 4.2M requests a day by blacklion on Tuesday September 07, 2021 @07:04PM (#28450935)
"old skool" exchanges uses either FIX (old and really vernose), FAST (binary encoding for FIX) or custom fixed-layout protocols.

Most big USA exchanges uses custom fixed-layout protocols, where each message is described in documentation, but not in machine-readable way. European ones still use FAST.

I didn't seen FIX in the wild for data feeds, but it is used for brokers, to submit orders to exchange (our company didn't do this part, we only consume feeds).

I don't know why, but all Crypto Exchanges use JSON, not protobufs or something like this, and didn't publish any formal schemes.

Fun fact: one crypto exchange put GZIP'ed and base64'ed JSON data into JSON which pushed to websocket, to save bandwidth. IMHO, it is peak of bad design.

Re:My £4 a month server can handle 4.2M requests a day by blibble on Tuesday September 07, 2021 @07:11PM (#28450985)
there's still ascii FIX floating around on the market data side for a few esoteric venues

FAST is not particularly common in Europe

the large European venues use fixed-width binary encoding (LSE group, Euronext, CBOE Europe)

Re:My £4 a month server can handle 4.2M requests a day by blacklion on Tuesday September 07, 2021 @07:36PM (#28451152)
Eurex uses FAST for sure, but I can be wrong about "common".
Re:My £4 a month server can handle 4.2M requests a day by gpderetta on Tuesday September 07, 2021 @08:26PM (#28451484)
Eurex/XETRA EOBI no longer uses FAST. It is about 8 year old t this point, but IIRC some products are still on the older protocol.
Re:My £4 a month server can handle 4.2M requests a day by nly on Wednesday September 08, 2021 @08:58AM (#28456325)
Euronext uses SBE specifically.
Re:My £4 a month server can handle 4.2M requests a day by rewq4321 on Tuesday September 07, 2021 @06:38PM (#28450756)
And msgpack if you want an order of magnitude faster serialization/deserialisation and can put up with worse compression (I think mainly due to schema overhead since protobuf files don't store the schema?)

https://msgpack.org/index.html

Good protobuf vs msgpack comparison: https://medium.com/@hugovs/the-need-for-speed-experimenting-...