paulasmuth

Or just use the web platform: JavaScript, HTML5 Canvas API, WebGL, all of that. I think it's pretty close in terms of getting a pretty (motivating) result quickly without much prior experience.

(Unrelated, but I can also remember writing the first lines of code of my life in qbasic on a dos machine in ~1998; the blue screen definitely invokes childhood memories!)

If you can't (or don't want to!) afford that kind of money for data analytics, please consider giving the FOSS alternative EventQL [0] a try some time.

It's super simple to set up and tries to be efficient on commodity hardware, so you can run large clusters (>100TB scale) for a couple hundred dollars a month.

[0] https://eventql.io/

DISC: I'm one of the EventQL authors

Do you have any references to back that up? I'm asking because I suspect your assumption is completely wrong.

(Based on a bunch of analytics data that I have access to, which might not be representative but still contains some very large german web properties, chrome has more than twice the market share compared to FF in germany).

Also, you can still listen to their music on soundcloud [1] if you don't want to review the code. It's open source after all, which can't be said for most demos...

[0] https://en.wikipedia.org/wiki/Demoscene -- Check this video for a neat example: https://www.youtube.com/watch?v=6AUpZNq2vSQ

[1] https://soundcloud.com/nemesis-fixx/

BTW I do actually enjoy the linked music (allthough I have to admit I'm listening on soundcloud)!

Sure it takes a somewhat different approach (i.e. it requires an explicit schema), but for the use case discussed in this thread it _is_ completely relevant and a comparable open-source/on-premise alternative which parent was asking about.

>> one that you have a commercial interest in

Yes, I'm involved in the EventQL project but I thought that it was obvious from the way I phrased my posting. Usually I always include a disclaimer to prevent misunderstandings but I didn't consider it necessary in this case.

>> Please don't spam HN.

I don't think pitching a (relevant) startup is against the rules or the spirit of a startup forum (https://news.ycombinator.com/newsguidelines.html)

Please consider giving EventQL [0] a try some time! It's completely open-source and self-hostable. Still a new project though, just released this summer and still in beta.

[0] https://eventql.io/

If you want users of any standards-compliant C++ implementation to be able to use your library today, you'll still have to go with c-abi symbols or ship the sources. All other worakrounds are vendor-specific and not part of the standard.

[Of course even objects containing only C symbols are not portable across platforms either, but at least the C ABI/calling convention is more or less strictly defined for any given target platform. Assuming no other platform-specific stuff like glibc is used]

Sadly it does not give a figure/order of magnitude of the amount of data that's stored in citus after aggregation, but I guess it's just not public information. [I'm working on a system that is somewhat similar to CitusDB (eventql.io) and am always really interested in these numbers]

EDIT: I can't reply to your other comment for some reason but many thanks for digging that up, it's very interesting info to me!

See this article by Herb Sutter on the topic: https://isocpp.org/files/papers/n4028.pdf

Or, in other terms 700mbit/s per host with your kafka setup versus ~30mbit/s per host in the benchmark. Allthough your machines seem to be quite a bit beefier (I wonder if all that RAM is actually used?).

The C++ ABI is currently not portable anyway. So the concern about templates (or any C++ features) forcing you to put the implementation into the header file would not apply in the scenario dllthomas was referring to:

If you want to distribute your library as a binary object and a separate source-form interface/header today, you'll have to use a (wrapper) C API. Regardless of how "modern" the C++ code is.

Of course, it would be nice to have portable C++ objects some time in the future which would change things... :)

I think what they meant with "1000 nodes" is that the dataset they're using for the benchmark is synthetic monitoring data (where the thing being monitored are servers).

And the way they generated the synthetic data set is by having 1000 imaginative servers produce one sample per second, (i.e. have a script that writes out 1000 * duration_in_sec fake samples -- I believe this is the code that does it https://github.com/influxdata/influxdb-comparisons/tree/mast...)

On a serious note: Please check out EventQL [0] some time. It's very similar to Vertica in some ways and completely open-source. It's a new project (beta) and not nearly as mature as vertica yet though (still a long way to go).

[0] https://eventql.io/

Fair enough. I'm sure InfluxDB is very good/fast at timeseries data (allthough I have to admit to not actually having tried it out so far). Still, if that was your point, consider removing these statements from the blog.

> InfluxDB outperformed Cassandra by 4.5x when it came to data ingestion.

> InfluxDB outperformed Cassandra by delivering 10.8x better compression.

> InfluxDB outperformed Cassandra by delivering up to 168x better query performance.

I think it would help make the point and not put the reader in a defensive position (when the statements are clearly not based on a fair comparison of the two products and will not hold under most conditions). Just my two cents.

(On the other hand, columnar storage also has a bunch of tradeoffs/downsides so it's not a superior choice for every db product.)

My point about no "superior compression technology here" was specific to the linked benchmark. I.e. the lack of this potential optimization in cassandra does not appear to be the reason for the space blowup in the benchmark, but rather that they're duplicating the series ID for each sample.

I'm far from a cassandra fanboy, but this really is just dishonest marketing. Not sure if that will work if your product is open source and the target audience are developers.

Some thoughts:

- The reason why cassandra uses so much more space to store the same data is that they've set up the cassandra table schema in such a way that cassandra needs to write the series ID string for each sample (while influxdb only needs to write the values). You easily get a 10-100x blowup just from that. There is no superior "compression" technology here but just an apples-to-oranges comparison.

- Then, comparing the queries is even worse, because they are testing a kind of query (aggregation) that cassandra does not support. To still get a benchmark where they're much faster, they just wrote some code that retrieves all the data from cassandra into a process and then executes the query within their own process. If anything, they're benchmarking one query tool they've written against another one of their own tools.

- Also, if I didn't miss anythin, the article doesn't say on what kind of cluster they actually ran this on or even if they ran both tests on the same hardware. There definitely are cassandra clusters handling more than 100k writes/sec in production right now. So I guess they picked a peculiar configuration in which they outperform cassandra in terms of write ops (given a good distribution of keys, cassandra is more or less linearly scalable in this dimension)

- A better target to benchmark against would probably be http://opentsdb.net/ or http://prometheus.io/ - both seem to have somewhat similar semantics to InfluxDB (which cassandra and elasticsearch do not)

DISC: I also work on a distributed database product (https://eventql.io) but it's neither a direct competitor to Cassandra nor InfluxDB nor any of the other products I've mentioned. I hope the comment doesn't come across as too harsh. The article raised some very big (and harsh) claims so I think it's fair to respond in tone.

On top of that, LLVM is a massive dependency.

I only looked at NativeJIT a few minutes but it seems to solve both of these issues. It's pretty small in terms of API surface // total amount of code and is explicitly designed for a usecase where compilation happens as part of a user-issued query (that needs to be fast).

Will seriously consider using this to speed up expression execution in EventQL [0] (we shied away from llvm so far because it's such a massive dependency).

[0] https://eventql.io/

On the upside, massively parallel SQL databases allow you to express what probably took facebook a lot of code in a single SQL query.

On the downside, with any database that supports streaming inserts and dynamic resharding, you will have to eventually write each entry more than once. The more manual approach with spark prevents this.

[0] https://eventql.io

Still I can't help but ask one question: Your points 2 and 3 (build tooling and package management) seem like they would have the biggest, immediate impact for most current c/c++ users.

Did you ever consider separating the build tooling and dependency management effort from the core language?

I, for one, would be extremely interested in sane build tooling and dependency management for c/c++ (and what you describe sounds great). I would give it a try in no time. However, I'd be much more hesitant to consider a new language for a serious project. Even if the language is tip top, I would still be concerned if it is likely too be around and still relevant in X years. I think build tooling has less of a chicken-and-egg problem to adoption because you could always switch to another tool later. And once somebody uses your build system you're already half the way there to using the full thing.

If 30 million hours were watched with an average bandwidth of 2Mbps, the total data volume should be 25,749TB (or close to 26PB) and not 400TB.

Also, if the peak bandwidth was 600Gpbs, that peak could not have lasted very long. With that much bandwidth the claimed 400TB of total transferred data would be used up in roughly 90 minutes.

So either the total data transferred or the number of hours watched figure is off by a factor of at least 60x.

[ I figured "400Tb" was supposed to mean 400 terrabytes. If it's actually 400 terrabits, the numbers are off by one magnitude more ]