News for hackers. Stuff that matters.
Re:Make the “semantic web” web 3.0 again – with the help of ... by __MatrixMan__ on Tuesday January 11, 2022 @11:48PM (#29902452)
I have a similar, "One day I will do it" project. The idea is that somebody cares enough to make the semantic web work--it's just not the people with write access to the data.

I think we can use CTPH algorithms to fingerprint the data independently of whatever names are used, and then we can use that to find representations of the "same data" submitted by other users. Probably there would be some reputation stuff involved, a web of trust, etc. The flow would go something like this:

COGNIZE:

0. Encounter messy data in the wild (has pagination, timestamps of access, etc), need other representation (human/computer/whatever)

1. Calculate CTPH fingerprints, use them to search for link: miss

3. Clean data the hard way and publish canonical representation (ipfs?)

4. Generate missing representation the hard way, publish that too

5. Calculate the fingerprints common to both the missing representation and the canonical representation, and publish it as a "link" between the two. Unlike traditional web links, this one is bidirectional.

RECOGNIZE

1. Different user encounters "same" data in the wild

2. Calculate fingerprints, use them to search for canonical representation: hit

3. Find further links to see what other representations of the "same data" are available, download them if desired.

The fingerprint stuff works, but there's a lot of work left to be done re: mapping fuzzy hashes of "in the wild" data to cryptographic ones of "canonical representations" and finding ways to incentivize users to go through the hassle of the "cognize" step so that other users can benefit from the "recognize" step.

Sorry for talking your ear off, it just feels good to know I'm not the only one working on something like this, even if our approaches are quite different (mine works on PDF's only because it works on arbitrary bytes). Good luck with yours.

Re:Make the “semantic web” web 3.0 again – with the help of ... by melony on Wednesday January 12, 2022 @01:27AM (#29903057)
Manual curation was why Berners-lee's Semantic Web failed. Unlike ARIA annotations which is required to serve certain clientele, the semantic web offers no inherent value to the developer. There is an entire field of machine learning dedicated to automatically generating knowledge graphs, I would start there rather than trying to manually curate and annotate.
Re:Make the “semantic web” web 3.0 again – with the help of ... by __MatrixMan__ on Wednesday January 12, 2022 @02:36AM (#29903472)
That's a good point. Once the tooling is there for the manual workflow, I expect that an ML-driven approach will plug in to the same hooks without fuss.

But I don't think it changes the main problem to be solved, which is that it's data consumers who want the semantic web, but so far it's been up to data providers to implement it. We need to be able to create links between data with without the participation of whoever hosts it.

Re:Make the “semantic web” web 3.0 again – with the help of ... by oever on Wednesday January 12, 2022 @04:15AM (#29904079)
WebAnnotations are meant for that.

https://www.w3.org/TR/annotation-model/

Re:Make the “semantic web” web 3.0 again – with the help of ... by __MatrixMan__ on Wednesday January 12, 2022 @09:46AM (#29906926)
Hmm, it looks like the use of "target" as a uri means that if the file gets rehosted elsewhere under a different name, the annotations will fail to adhere. I'm going for more of a virus-scanner approach, except I'm looking for the good stuff instead of the bad.

But otherwise yes. I'll have to give that a closer look and use as much of it as I can.

Re:Make the “semantic web” web 3.0 again – with the help of ... by oever on Wednesday January 12, 2022 @11:33AM (#29908928)
You can use a magnet or ipfs uri or add a triple that specifies other properties of the file that let you find it.

Semantic web uses IRIs in triples. They serve as identifiers, not locators (URL). This is because semantic web talks about more than just files.

The Web Annotation spec talks about web resources which should be dereferencable. That is a naive assumption that makes the approach brittle. But one can annotate 'sourceDate' and 'cached' which indicate the date for the annotated version and a location for a copy.

When you're annotation a resource it's best to also store a copy. The Web Annotations specification allows this.

Re:Make the “semantic web” web 3.0 again – with the help of ... by zozbot234 on Wednesday January 12, 2022 @12:25PM (#29909908)
It's preferrable to use the ni:// URI scheme since it is standardized, unlike magnet links. There is a draft standard https://datatracker.ietf.org/doc/html/draft-hallambaker-deca... defining common extensions that enable much the same featureset as magnet.
Re:Make the “semantic web” web 3.0 again – with the help of ... by __MatrixMan__ on Wednesday January 12, 2022 @01:43PM (#29911102)
This is good stuff for me to know about, thanks.
Re:Make the “semantic web” web 3.0 again – with the help of ... by __MatrixMan__ on Wednesday January 12, 2022 @01:48PM (#29911180)
> When you're annotation a resource it's best to also store a copy

Do you know if there's any precedent re: people getting upset about that copy? If I annotate an article that was originally served with ads, or behind a paywall, I'll probably not include that stuff in my copy. If people then start linking to my copy instead of the original, I could imagine that feathers would be ruffled.

Re:Make the “semantic web” web 3.0 again – with the help of ... by oever on Wednesday January 12, 2022 @03:59PM (#29912586)
If you're not permitted to publish a copy, you can keep it private and/or link to a version at an institution, such as archive.org, that is allowed to keep a public copy.
Re:Make the “semantic web” web 3.0 again – with the help of ... by webmaven on Wednesday January 12, 2022 @06:12PM (#29914102)
An implementation can dereference a resource and provide failover to a static archive. There is also probably an opportunity to use DOIs here.
Re:Make the “semantic web” web 3.0 again – with the help of ... by wnkrshm on Wednesday January 12, 2022 @07:24AM (#29905385)
What you describe has classically been the work of librarians or archivists.
Re:Make the “semantic web” web 3.0 again – with the help of ... by __MatrixMan__ on Wednesday January 12, 2022 @09:53AM (#29907063)
Ah well I guess I'm looking to make freelance librarian into something you can be.

I figure if you configure the client to keep track of whose annotations you have a history of using, you've got a real granular view of which content providers to pay. I'm imagining a game where we all put $5 in at the start of the month and we all pay each other based on whose content we use the most. Some users will pay more than they make, others will make more than they pay.