Re:1.3B Worldcat scrape and data science mini-competition
by gmcharlt on Thursday October 05, 2023 @01:34PM
(#37781456)
Depending on what you're looking for, a _lot_ more is being published by the Library of Congress as Linked Data nowadays, including LoC classification and subject headings. Check out https://id.loc.gov/.
[ Parent
| on HN ]
Re:1.3B Worldcat scrape and data science mini-competition
by dredmorbius on Thursday October 05, 2023 @02:38PM
(#37782331)
[delayed]
Re:1.3B Worldcat scrape and data science mini-competition
by gmcharlt on Thursday October 05, 2023 @06:24PM
(#37785032)
https://id.loc.gov/download/ offers bulk downloads of many of the vocabularies, including the subject headings. In addition, individual records of various types can be search for.
Re:1.3B Worldcat scrape and data science mini-competition
by dredmorbius on Thursday October 05, 2023 @07:08PM
(#37785367)
Thanks.
That seems to have Subject Headings, but not the Classification, unless I'm blind (a fairly high likelihood).
I've spent quite some time poking around the LoC website over the years without finding anything more accessible than the PDFs of the Library of Congress Classification. Are you aware of that being generally available?
Re:1.3B Worldcat scrape and data science mini-competition
by gmcharlt on Thursday October 05, 2023 @09:31PM
(#37786275)
Near as I can tell, the best that's available is starting with the class(es) you want (e.g., https://id.loc.gov/authorities/classification/H.html for social sciences) and crawling recursively to grab whatever serialization of the RDF you want to consume. There's no SPARQL endpoint for id.loc.gov, alas, but it beats resorting to the PDFs.
Re:1.3B Worldcat scrape and data science mini-competition
by dredmorbius on Thursday October 05, 2023 @09:46PM
(#37786356)
Thanks.