News for hackers. Stuff that matters.
Re:KanjiVG – SVGs of Kanji character strokes including order... by wahnfrieden on Tuesday February 21, 2023 @04:35PM (#34887779)
It’s too bad the pitch data out there for Japanese is either proprietary or pirated without attribution

Another resource: https://github.com/CaptainDario/DaKanji-Single-Kanji-Recogni... someone made an interesting kanji recognition lib that hasn’t gotten attention

Re:KanjiVG – SVGs of Kanji character strokes including order... by krackers on Tuesday February 21, 2023 @04:47PM (#34887937)
Given a large enough corpus of spoken lines, could you do some ML magic to get the pitch accents (maybe even just FFT and a simple classifier would do)? I'm aware that the "base" pitch accent does change in context so it's not quite trivial, but it seems like you could get pretty close?

Edit: Found https://mizoru.github.io/blog/2021/12/25/Japanese-pitch.html

Re:KanjiVG – SVGs of Kanji character strokes including order... by wahnfrieden on Tuesday February 21, 2023 @05:16PM (#34888408)
The pitch variants are also highly regional but that’s an interesting idea. I wouldn’t want to give it to learners in use cases I can think of, when correct data is available though (just with licensing headaches/costs)
Re:KanjiVG – SVGs of Kanji character strokes including order... by aikinai on Tuesday February 21, 2023 @05:36PM (#34888668)
99% (or more) of the recordings you might find for training would be in the standard accent unless you were specifically digging for regional accents.
Re:KanjiVG – SVGs of Kanji character strokes including order... by wahnfrieden on Thursday February 23, 2023 @12:52PM (#34913928)
it's also changing generationally
Re:KanjiVG – SVGs of Kanji character strokes including order... by gfaure on Tuesday February 21, 2023 @07:45PM (#34889948)
This linked article seems to have gotten special access to the data -- unfortunately only the resulting model is available.
Re:KanjiVG – SVGs of Kanji character strokes including order... by rjh29 on Tuesday February 21, 2023 @09:15PM (#34890690)
It predates modern ML, but that's what Suzuki-kun does:

https://www.gavo.t.u-tokyo.ac.jp/ojad/

Re:KanjiVG – SVGs of Kanji character strokes including order... by krackers on Tuesday February 21, 2023 @09:25PM (#34890771)
No, I don't think Suzuki-kun was trained on voice data, they trained a classifier on an annotated text corpus. And getting access to such a corpus is probably much harder than finding voice samples.

https://research.ibm.com/publications/accent-sandhi-estimati...

Re:KanjiVG – SVGs of Kanji character strokes including order... by rjh29 on Tuesday February 21, 2023 @09:16PM (#34890699)
It seems like most people use the NHK pitch accent dictionary (there are TSVs of it online). I don't feel like it's particularly "pirated" though, can you really pirate the way people actually pronounce words?
Re:KanjiVG – SVGs of Kanji character strokes including order... by wahnfrieden on Wednesday February 22, 2023 @12:30AM (#34891996)
NHK didn’t give that data an open license

Piracy is rampant. Whether lifting and reusing/redistributing copyrighted dictionaries or other stud materials, or pirating ebook/cdrom type content. That’s nice, but as a legit service provider it’s less accessible without just giving users their own ability to side load in materials

Re:KanjiVG – SVGs of Kanji character strokes including order... by kouteiheika on Wednesday February 22, 2023 @01:45AM (#34892459)
> It’s too bad the pitch data out there for Japanese is either proprietary or pirated without attribution

I have serious doubts whether this kind of data is even copyrightable, as long as you're not redistributing it verbatim.

A specific selection of words with pitch data (e.g. the NKH pitch dictionary) might be copyrightable as there was creative expression involved in picking which exact words to put into the dictionary and in what order. But the data itself? A 猫 is always going to have a HLL pitch accent (in the standard accent). That's a fact. And facts itself are not copyrightable.

You can't copyright a phone book[1]. Quoting from the case:

> "Notwithstanding a valid copyright, a subsequent compiler remains free to use the facts contained in another's publication to aid in preparing a competing work, so long as the competing work does not feature the same selection and arrangement"

Sounds exactly like the pitch accent data dump that's been floating around which a lot of people use. (Not the verbatim NHK one; the other one.) They probably used the data from NHK's pitch dictionary to compile it along with a few other dictionaries. But does it feature the same selection and arrangement? Nope.

[1] - https://en.wikipedia.org/wiki/Feist_Publications%2C_Inc.%2C_....

Re:KanjiVG – SVGs of Kanji character strokes including order... by wahnfrieden on Thursday February 23, 2023 @12:51PM (#34913914)
I see apps pay to license data from https://www.cjk.org for pitch accent data. I don't know what their data looks like though. Maybe you're right and their business case for this data is essentially a scam, or there is non-factual copyrightable data, or the law is different in Japan, I don't know.