News for hackers. Stuff that matters.
Re:1.3B Worldcat scrape and data science mini-competition by DoctorOetker on Wednesday October 04, 2023 @10:31AM (#37765758)
> We scraped ISBNdb, and downloaded the Open Library dataset, but the results were unsatisfactory. The main problem was that there was not a ton of overlap of ISBNs.

What prevents ISBN number collisions between authors? Is there a central authority assigning them, or is there say a national prefix, with each government assigning ISBN's for local publications (perhaps delegating this to another body in that nation).

Surely such bodies would have the most complete view on all this data.

It's also bizarre that this simple metadata is not available from whatever authority assigns ISBN numbers..

Re:1.3B Worldcat scrape and data science mini-competition by yorwba on Wednesday October 04, 2023 @11:06AM (#37766250)
Companies can get a range of ISBNs before deciding what to publish under each ISBN, or whether to publish something at all. So the authorities assigning ISBNs don't necessarily know what they're being used for.
Re:1.3B Worldcat scrape and data science mini-competition by greenie_beans on Wednesday October 04, 2023 @07:52PM (#37773261)
not to mention, publishers don't always follow the rules and nobody is paying attention to them
Re:1.3B Worldcat scrape and data science mini-competition by sevenseventen on Wednesday October 04, 2023 @11:07AM (#37766274)
There are regional ISBN agencies. The US agency, Bowker, assigns ISBN prefixes by publisher, and publishers assign within their prefix as they please. They're supposed to use one ISBN per edition and format, but many publishers use ISBN as a kind of SKU so you can't 100% count on that.

If that sounds sloppy...I went to publishing conferences fairly regularly from the late 90's into the teens, and I never saw a program that didn't have at least one session or panel titled something like "Publishers must improve their metadata practices."

Re:1.3B Worldcat scrape and data science mini-competition by jszymborski on Wednesday October 04, 2023 @12:13PM (#37767301)
Sounds a bit like how DOIs are assigned.
Re:1.3B Worldcat scrape and data science mini-competition by sevenseventen on Friday October 06, 2023 @10:27AM (#37791220)
or vice versa! Although DOI RA's are organized functionally rather than regionally. CrossRef -- for journal articles -- is by far the largest RA.

DOI's are far, far more centralized in that CrossRef issues many orders of magnitude more DOI's than all of the others.

The cite-and-be-cited-by use case for scholarly articles is way more compelling than any other use case devised for DOI's.

Re:1.3B Worldcat scrape and data science mini-competition by bambax on Wednesday October 04, 2023 @12:18PM (#37767375)
It's exactly like this. Publishers get a range, and do whatever they want with it.

Also, some agencies sell the numbers range to publishers (I believe the US is in that case) and others give them away at no cost (France). As a result, some small publishers get ISBNs from more liberal agencies than their own country: one can never be 100% sure a French ISBN matches a French publisher for example.

It's also possible some publishers re-use old numbers or affix the same number to different releases/editions of a book.

It's a mess.

But a global centralized system would probably be way worse, so we have to live with that mess.

Re:1.3B Worldcat scrape and data science mini-competition by amelius on Wednesday October 04, 2023 @02:54PM (#37769900)
Why not use UUIDs?

People never enter ISBNs manually, anyways. So it might as well be longer string. Or a QR code.

Re:1.3B Worldcat scrape and data science mini-competition by dwringer on Wednesday October 04, 2023 @03:16PM (#37770188)
As someone who's worked in the field of used books, I can say from personal experience that's not quite true. It's pretty common to have to type in an ISBN for a variety of reasons. Many times the barcodes have been covered up or defaced, and many publications don't have barcodes in the first place.
Re:1.3B Worldcat scrape and data science mini-competition by amelius on Wednesday October 04, 2023 @03:22PM (#37770270)
It was true, for sure. But the question is, is it still true?
Re:1.3B Worldcat scrape and data science mini-competition by bambax on Wednesday October 04, 2023 @03:43PM (#37770553)
Schools often ask for a specific edition of a classic book and the only way to be reasonably sure you're buying the correct one is to search by ISBN.
Re:1.3B Worldcat scrape and data science mini-competition by amelius on Wednesday October 04, 2023 @04:35PM (#37771339)
Well, just copy and paste from the email you got from school.

I mean, if IT can do anything, it is to solve this problem.

Re:1.3B Worldcat scrape and data science mini-competition by greenie_beans on Wednesday October 04, 2023 @09:17PM (#37773836)
same
Re:1.3B Worldcat scrape and data science mini-competition by layer8 on Thursday October 05, 2023 @03:34AM (#37775801)
ISBNs contain a check digit, to prevent typos. So you’d have to invent a new format.
Re:1.3B Worldcat scrape and data science mini-competition by sevenseventen on Friday October 06, 2023 @10:32AM (#37791297)
The point is not just to have universally unique identifiers, but to collect common metadata that's associated with the identifiers. Like this:

https://www.bowker.com/siteassets/files/pdf-files/datasubmis...

Since this is the second time I've mentioned Bowker, let me just say that I do not and have never worked for them or with them, although I did meet with their reps several times when I worked in a different part of digital publishing. It's just that they're inescapable when you're talking about ISBNs.

Re:1.3B Worldcat scrape and data science mini-competition by gmcharlt on Wednesday October 04, 2023 @11:08AM (#37766292)
ISBNs are messy.

The International ISBN Agency coordinates assigning ISBN ranges to national agencies, who in turn will assign subranges to publishers. The publishers in turn assign specific numbers to their own works. However, the international agency does not itself maintain a universal database of assigned ISBNs - the most it operates is a global database of publishers and their assigned ranges. And since it's the publishers who are assigning numbers from their allocations, various errors can crop up, including reusing ISBNs for different works and failing to issue distinct ISBNs for different formats. (For example, if you publish hardcover, paperbook, and ebook versions of a book, you should assign three ISBNs. That rule is not always observed.)

Also, libraries hold many books that long predate ISBNs; it wasn't until 1965 that the immediate predecessor of the ISBN, the SBN, was a twinkle in a bookseller's eye.

Re:1.3B Worldcat scrape and data science mini-competition by bambax on Wednesday October 04, 2023 @12:21PM (#37767418)
Yes.

And while in most countries you can't properly publish a book without an ISBN (ie, have it sold in bookshops), you can publish a Kindle book without it (if you opt to only offer the ebook).

That leaves a huge part of publications completely out of the system. Kindle-only books are on Amazon servers and nowhere else.

[dead]
Re:1.3B Worldcat scrape and data science mini-competition by greenie_beans on Thursday October 05, 2023 @01:03PM (#37781120)
bookstores don't want to carry a book without an isbn because no isbn means it's not available to purchase from their distributor and it's easiest for a store to order through real distribution channels (like ingram in the u.s.).

but most stores carry a small amount of self-published books and sometimes those books have no isbn. those books are typically by local writers. but in my experience as a bookseller, self published books are a pain to work with. some self-published books aren't returnable, but returns are an important part of the bookstore business since a lot of books don't sell. working with a lot writers individually about ordering etc is more involved than going through a single distributor, this takes a lot of time for whoever has to do this.

> given the amount of books I've personally seen published in recent decades without ISBNs

i'd bet this on amazon? iirc, you can't always return self-published amazon books. i think the author decides this, bc they get charged a processing fee for returns.

> or even to be a 'proper' bookseller

you can totally sell your collection online without any isbns and you'd be considered a bookseller. you'll just need an sku system. there's a difference between a used/collectible seller and a bookseller who carries new books as well as used/collectibles. the new books require proper distribution channels.

Re:1.3B Worldcat scrape and data science mini-competition by crtasm on Wednesday October 04, 2023 @11:12AM (#37766341)
There's lots of agencies, they hand out blocks of numbers from their allocation. Seems there's no central database for the metadata:

https://www.isbn-international.org/content/isbn-users-manual... ISBN_FAQs_to_7ed_Manual_Absolutely_final.docx

> Will people in other countries be able to search for my books in search engines in those countries?

> This does not happen automatically ... In order for your book to be listed in other countries you should contact the respective ISBN Agency and ask them for details of how to be entered into their national catalogue for books in circulation (books in print). Sometimes you will have to obtain a distributor from that country or have an address in that country before this is possible. In some circumstances in order to be listed, the book must be in the language of that country. As well as catalogues of books in circulation, you may also want to ensure that you are listed by internet retailers... . Again, you will need to contact each of these organisations directly (including each separate international branch) with details of your book.

Re:1.3B Worldcat scrape and data science mini-competition by skuxxlife on Wednesday October 04, 2023 @02:35PM (#37769610)
For US/UK/NZ/Aus/SA, ISBNs are granted through Bowker who does maintain their "Books In Print" data set that, in theory, contains metadata for all of the ISBNs they've granted. In practice though it's a mess. It's expensive to access and relies on publishers to enter in accurate and consistent metadata, which is...variable in quality to say the least. Often publishers buy blocks of ISBNs to use later so no metadata is entered up front and has to be pushed back to Bowker at a later date. To be somewhat fair to Bowker, the history of ISBNs far predates modern data standards and I can imagine wrangling publishers to get accurate data is a difficult task. But on the other hand, you'd think they'd have a lot to gain for doing it right. As someone who runs a book website, it is endlessly frustrating.
Re:1.3B Worldcat scrape and data science mini-competition by greenie_beans on Wednesday October 04, 2023 @09:22PM (#37773871)
what is your website?