norkakn
In other domains though, R-Trees have many other advantages, and often perform better.
http://www.dpi.inpe.br/livros/bdados/artigos/oracle_r_tree.p...
"For these datasets, R-trees consistently outperformed Quadtrees by 2-3 times" ... "In short, a quadtree could be recommended for update-intensive applications using simple poly-gon geometries, high concurrency update databases, or when specialized masks such as touch are frequently used in queries. However, users have to fine-tune the tiling level to obtain best performance. R-trees, which do not require any such tuning, could be used in all other cases to obtain nearly equivalent or better performance."
I didn't realize that the auto-allocation ever shipped, but I'm glad it finally did. Hopefully there was work done to empirically show that they solved a lot of the issues around time to convert and other messy parts of the data that killed earlier efforts, but I think everyone who knew about those was gone before you joined :)
There are very subtle issues with both frequentist and bayesian stats, which makes combining them sounds insane to me.
What are you up to these days?
They are still having people make very fundamentally flawed assumptions about the data, which results in incorrect conclusions, and they are still not presenting the results in a way that people correctly interpret them. That being said, those are really hard to solve, and models that would try to correct for them would likely require a lot more data and be overly conservative for more people.
What are your reasons for disliking Optimizely?
Optimizely is in a rough spot. People don't like having to think through experimental design, and they are really, really bad at reasoning about p-values. To try to fix the people part, they came out with the sequential stopping rule stuff (their "stats engine), but they never really published much justifying it. The other alternative would be to move the experiments into a Bayesian framework, but that has a lot of it's own problems. When they acquired Synference, that was one of the likely directions to take (along with offering bandits), but that didn't work out and those guys have since left.
The lack of vector clocks in Cassandra can lead to some very non-intuitive (possible wrong) behavior - check out their counter implementation for some rage on that. It's pretty well made though, and I think C*, Hbase and Postgres all have great uses (along with Redis, and a lot of others)
Mongo tends to get things subtly wrong in ways that corrupt data, or that don't scale, and it gives up both A and C.
Mongos does weird magic as well when it gets confused, and will confirm writes to the wrong shards during "interesting" situations.
Source: tried to use it at scale
Cassandra is one of the better ones out there, but you have to deal with its data model and weird consistency promises (which however weird you think they are, are weirder)
The correct way to cluster also changes dramatically depending on your use case. Sure there are things like RAC that promise to make it just work, but those don't scale more than a few nodes.
Mongo is kind of the worst in this - it clusters in one weird way, has bad tooling, and subtly destroys your data at scale.
The general philosophy with postgres is to do it right, or not do it. There are ways to do specific kinds of clustering, but all of them (just like mongo, oracle, etc) have a lot of nuances to them.
If you have a natural shard key, use a bunch of schemas and table inheritance, and eat the downtime during re-shards. Check out citus as well. They have their issues, but they can help you hook up what you need.
https://github.com/citusdata/cstore_fdw/blob/master/LICENSE
They're really nice talented people, and a great example of a company giving back to the community.
I miss not having real scaling issues sometimes. They make my head hurt.
Find all Nodes with a property in a tree
Find all leaves L of those nodes
Find all annotations in a DAG of those leaves
Collapse similar DAG entries by backtracking up the graph based on edge weights
Writes are bulk loaded, and right now, we are just trying to push all of the graph stuff offline, but there are some limitations to that, and we could really up our accuracy by being able to perform these queries quickly.
OrientDB tries to scale writes (I'll be testing this in a few months), but still stores all of the data everywhere.
This looks like it shards the data automagically. If it works well, I might be able to bang on it a bit, but I'm guessing that it gives shit performance for complex graph questions.
Perl isn't batteries included, but its packages are usually actually tested on multiple versions. This allows for the community to move on its own. There is no dictator, by design. Try #perl for help as well, or the package channel if there is one.
Both projects suck equally. The python project is much newer (~4 years instead of 12), but the smell is about the same. The problem with new programmers isn't that they pick the wrong object creation patterns, but that they reinvent the wheel and write long stupid functions with horrible nested if/else statements. A former google employee worked on one of the projects, and wrote her own python to json function.
I like python, and I like perl, but the python culture sucks. It has this weird superiority complex where it rejects any external ideas. If I am talking to perl people, and I mention some kick ass python feature that I miss, I either get information as to how it can be done in perl, or interest into why I like it. Around python people, I generally get something like, "well, none of us know perl. Have you tried doing it the python way?"
Moose is awesome, perlbrew is awesome, perl's module culture is awesome. The module culture is the one that annoys me most often. Python programmers don't write as good of tests, or nearly as good of documentation. We still have to run a lot of python 2.4, and a lot of modules claim 2.4 support and then use 2.6+ syntax.
I like python, I think it is a much better matlab, but a lot of its syntax is unintuitive and stupid, and there are some serious cultural issues. Anyone who thinks that it is perfect, or better than another language in every way is holding it back.