tycho01
Alibaba Open-Sources Mars to Complement NumPy
You may want to try ParslePy, it combines CSS/XPath functionality, allowing you to declaratively specify the selector paths in a JSON file. I just made a PR to allow YAML over JSON, but not sure if Pip picked up on it yet.
New graph db helps PRC gov identify dissidents
So, the new [TigerGraph](http://www.tigergraph.com/) contains a demo (behind signup wall) called Social Graph, which uses a model of data that afaik only the Chinese government would have access to: tying everything to a person's identity including where they've slept and how they've traveled.They then demonstrate a sample query on this dataset, called 'connection mining', identifying people...
They then demonstrate a sample query on this dataset, called 'connection mining', identifying people who have enough data in common with a given person.
Now, although the Datanami article does not particularly state the Beijing government as making use of TigerGraph's services, it does mention his clients as including 2 Chinese state-owned companies, State Grid and China Mobile.
Given it's been established his clientele includes Chinese state-owned companies and no-one but the Chinese government has the data described in this 'Social Graph' data set though, let's say this may have been made for Beijing. Sounds like they might be using this query to track dissidents by association.
The First Native Parallel Graph
I tried finding what TS can do now, and figured out type-level tuple iteration, among a few others. The current roadblock seems to be getting function return types. Progress, for anyone interested: https://github.com/Microsoft/TypeScript/issues/16392
At this point I'm amazed how close we are to typing anything, despite having only 5 (!) type-level operators, with their respective warts.
I imagine in a visual environment, being able to make useful suggestions on potential ways to use/combine different nodes/types would help as well as an auto-complete for the user's intent.
I'm actually also a bit reminded here of MS Excel Power Query, which also offered a GUI for data transformation, see e.g. [this pic](https://blogs.office.com/wp-content/uploads/2015/07/6-update...).
I bring this up because I see you covered visualizing the steps, while they focused on showing the data (though after finishing a transformation the script could be generalized into a reusable function). I wonder if adding a dimension like that could be helpful for Luna as well.
If you target non-programmers, showing things as concrete as possible (e.g. their data transformed by whatever function they just pulled together) sounds like it might help make things even more accessible.
I saw it also supports both row-oriented and columnar modes. Does that mean this is meant both as a transactional and as an analytics-oriented database?
My understanding is that serialization became a thing because in-memory representations tend to use pointers to shared data structures that may thus be referenced multiple times while being stored only once. This would not translate 1:1 to serialized representations (where memory offsets would no longer hold meaning) -- much less in any language-agnostic way.
So I have this suspicion that Apache Arrow would not support reusing duplicate data while storing it only once. Would anyone mind clarifying on this point?
Mesosphere Infinity: Bundling Spark, Akka, Cassandra and Kafka on DCOS
By writing to the model, you mean programmatically adding new measures or the like?
My interest is in programmatically querying models using DAX, though to this end I'd also look to look in the direction of Microsoft's DirectQuery mode in SQL Server which supposedly did DAX-to-SQL conversion.
If one could use such a conversion plus MDX to start querying models on an Apache Spark cluster through pivot table/chart interfaces...