News for hackers. Stuff that matters.
Re:Apache Jena by openasocket on Monday March 18, 2019 @11:30AM (#19421885)
I actually stumbled across this a couple weeks ago, while thinking about the design for a project. I'm working with threat intelligence/security data and I need a nice way for users to save/share that data with the team, as well as query it in a simple fashion. People are pushing for a graph database. So I did some research and learned about RDF, OWL, Jena, etc.

I'm still not sure if this is the right road to go down. Just from reading papers it's hard to tell how much of this is hype and over-engineering and how much is solid. Anyone have some RDF/OWL/Jens/SPARQL stories they want to share?

Re:Apache Jena by th0ma5 on Monday March 18, 2019 @12:03PM (#19422249)
I had a project to map out a whole bunch of relationships between various software configuration parameters of a vendor product. This was a lot of XML but also some CSV and JSON data. There's probably a ton of ways to do it, but I just spent some time extracting "facts" from this data into RDF. I didn't know what any of it was or how it all fit together but there were a lot of UUIDs and such.

Once I had "everything" in RDF, using SPARQL queries to nose around was a really great way to find arbitrary relationships. I then used GraphViz to map out everything and it was very helpful to see how everything fit together. Could've probably used a regular database, or a key value store, but working with just flat files transformations at the command line was nice.

I've also since used it manage a knowledge base and generate documentation and presentations. I haven't even used much of OWL or higher levels of abstraction yet, but just from a hack up something standpoint I think it is a pretty nice set of ideas to get basic graph operations just about anywhere you may need it, in the shell, in the browser, or in a backend project. Large sets of data may require something else, but anything less than 1g of data is rather performant IMHO.

Re:Apache Jena by oever on Monday March 18, 2019 @11:53AM (#19422132)
EU is developing a vocabulary editor called VocBench [0]. UK is making something similar: registry-core [1]. Here in the Netherlands there an active group exchanging RDF user stories [2].

To get a good start with RDF etc, I recommend 'Semantic Web for the Working Ontologist' (second edition or later!) [3]. It explains how inferencing works by doing inferencing with SPARQL queries.

RDF is a powerful technology, but takes time to get acquainted with.

What I dislike is that there's not many libraries (that I know) for working with it. I started a Rust library (Rome) for working with Turtle files and would like to have time to make it into a SPARQL engine.

[0] http://vocbench.uniroma2.it/ [1] https://github.com/UKGovLD/registry-core [2] http://www.pilod.nl/wiki/Platform_Linked_Data_Nederland [3] http://workingontologist.org/

Re:Apache Jena by ktk on Monday March 18, 2019 @12:19PM (#19422421)
Unfortunately a lot of stuff going on in the RDF domain is behind firewalls so it's a bit hard to give a lot of details. But I can contribute some public and some private use-cases of where RDF is used:

Refinitiv (formerly Thomson Reuters Financial and Risk) knowledge graph is built completely on the RDF stack: https://www.refinitiv.com/en/products/knowledge-graph-feed

When I talked to them in late 2017, they told me they have 100 billion triples in their database, plus more in a versioning back-end. Their triplestore is open source: https://github.com/CM-Well/CM-Well

Several government-agencies all over the world start to build public RDF knowledge graphs. I'm closely involved in the one from the Swiss government, see my presentation from last week http://presentations.zazuko.com/Swiss-LOD-Platform/

There are similar projects in other countries like the Netherlands, Belgium, UK, etc. This stack makes a lot of sense for open data, as you can do some pretty crazy queries without spending 2 days on preparing your data. See for example the Swiss Open Data Advent Calendar of 2018: https://twitter.com/linkedktk/status/1076064066525949952

As I said there are many "behind the firewall" use-cases where people use the stack exactly because of its features like OWL. Yes it comes at a price (bootstrapping is not really super easy) but this is stuff we will still run in 40 years from now. I see it in:

Finance: Fraud detection, compliance, customer 360° views, ... Stardog (https://www.stardog.com/) lists Moody's, BNY Mellon and National Bank of Canada as customers, last week I've met someone from Credit Suisse which is Mr. RDF there. * Production: You have a ton of databases containing products you create but there is no way to figure out what a final product consists of as the data is scattered across at least 5 of them. The automotive supplier I talk about here is using RDF to get that view.

Life sciences: The largest RDF dataset available to the public is UniProt and related datasets. In total they provide a SPARQL endpoint (RDF database) with 50 billion (!!) triples available. This is a highly popular dataset and is used in pretty much every larger pharmaceutical enterprise as well. See https://www.uniprot.org/ as a starting point. I know at least of one large life sciences company that just recently decided that RDF will be the base of all future data unification standards within the organization.

Insurance business: One of our customers is using RDF to unify a ton of different systems and get the 360° view as well about their customers.

RDF is an absolutely amazing stack and I do not see anything else available that gets remotely close to the power of it. The day I find something more powerful, I will be the first using it. But most of the time people dismissing RDF have zero clue about what it really can do.

Re:Apache Jena by cmutel on Monday March 18, 2019 @01:00PM (#19422815)
I am part of a team building an RDF database to be used for environmental footprinting and industrial ecology (https://github.com/BONSAMURAIS), and am also slowly becoming part of the Swiss open data scene - I would appreciate a chance to chat with you about your experiences!

For us, RDF seems like the only technology that can easily adapt to the large number of data types that we envision collecting.

Re:Apache Jena by ktk on Monday March 18, 2019 @03:03PM (#19424046)
sure, more than happy to. You will find me at @linkedktk on twitter or adrian.gschwend @ zazuko . com
Re:Apache Jena by mring33621 on Monday March 18, 2019 @12:59PM (#19422810)
You sound like you may understand RDF well enough to answer.

If I have transcribed voice convo data, with date/time, names, location, sentiment and extracted subject matter keywords, would Jena/RDF and/or related tools be appropriate for exploring relationships and trends between data points?

Thank you.

Re:Apache Jena by ktk on Monday March 18, 2019 @03:05PM (#19424061)
Yes that sounds like a pretty good fit for RDF. Do you have some examples on the data? Probably pretty straight forward to do an RDF model that could be used for analytics later