News for hackers. Stuff that matters.
Re:SenseiDB: Open-source, distributed, realtime, semi-struct... by andres on Friday January 20, 2012 @11:57AM (#3490114)
SenseiDB looks really impressive. We were actually thinking about the same problem when we built ThriftDB (http://www.thriftdb.com) for Octopart so it's especially interesting to see LinkedIn's implementation.

Looking through the documentation a couple of things come to mind:

* Does SenseiDB support nested data structures?

* What happens when you modify the schema? Can you add/delete/modify attributes?

Re:SenseiDB: Open-source, distributed, realtime, semi-struct... by javasoze on Friday January 20, 2012 @01:54PM (#3490674)
re: nested data structure is in the roadmap, but no target date yet. There are 2 ways of implementing this: 1) pre-pend field name reflecting the nesting, much like elasticsearch 2) encoding payloads in the term list indicating nested nature.

1) is straightforward, but can be restrictive if you want to in-nest faceting.

We are still debating on the route to take.

As for schema changes, we should document that better: certain schema changes simply requires you bouncing the node, but if there are data integrity changes, you would need to reindex. We will work on more documentation on the specifics.

Re:SenseiDB: Open-source, distributed, realtime, semi-struct... by ericmoritz on Friday January 20, 2012 @02:29PM (#3490840)
If I change the faceting does bouncing the nodes cause a re-index automatically or is it more involved?
Re:SenseiDB: Open-source, distributed, realtime, semi-struct... by javasoze on Saturday January 21, 2012 @06:12PM (#3494982)
bouncing node will start reindexing from the last committed version on the disk. If the index is up to date, no re-indexing will be triggered.