News for hackers. Stuff that matters.

lefttoreader

Re:Show HN: Fructose, LLM calls as strongly typed functions on Wednesday March 06, 2024 @09:47PM (#39624536)
yikes.

I was responding to them sidestepping the first commenter’s question.

Re:Show HN: Fructose, LLM calls as strongly typed functions on Wednesday March 06, 2024 @06:26PM (#39623107)
Wait. So why not just contribute to an existing open source project if you’re going to implement an identical API?

If you run your own models as a part of it, surely you could hook up your models as a backend to whatever abstractions you’re copying here.

Re:Show HN: LLMs can generate valid JSON 100% of the time on Tuesday August 15, 2023 @06:52AM (#37132338)
I have some other comment on this thread where I point out why I don’t think it’s superficial. Would love to get your feedback on that if you feel like spending more time on this thread.

But it’s not obscure? FlashText was a somewhat popular paper at the time (2017) with a popular repo (https://github.com/vi3k6i5/flashtext). Their paper was pretty derivative of Aho-Corasick, which they cited. If you think they genuinely fucked up, leave an issue on their repo (I’m, maybe to your surprise lol, not the author).

Anyway, I’m not a fan of the whatabboutery here. I don’t think OG’s paper is up to snuff on its lit review - do you?

Re:Show HN: LLMs can generate valid JSON 100% of the time on Tuesday August 15, 2023 @06:33AM (#37132235)
Yep! But that’s sort of my point, and maybe this is just some misplaced academic shit of mine but if you’re going to write a paper then “easily had this idea and never discovered the paper” just doesn’t fly.

Almost all academic work is derivative tweaks of yesterday’s work, yet we still fall over ourselves to cite this stuff.

Re:Show HN: LLMs can generate valid JSON 100% of the time on Monday August 14, 2023 @10:20PM (#37129396)
Your comment here doesn’t feel like it’s in good faith, but there’s a good chance I’m misreading it.
Re:Show HN: LLMs can generate valid JSON 100% of the time on Monday August 14, 2023 @08:48PM (#37128754)
Sure! So it’s hopefully clear that the notion of constrained grammar is not novel (see every comment on here of people name-dropping their implementation from two months ago).

The novelty here is “instead of checking whether every token is allowed” to create a finite state machine that defines which tokens are allowable at each generation step. This lets them not check every token at every step.

The trick of creating an FSM to efficiently check next-token grammar is what allowed FlashText to run circles around standard regex stuff. Even FlashText guy acknowledged the shoulders he stood on, etc.

Let’s be super clear here, none of these standards apply when you’re building good ole libraries. But putting out a paper really elevates what you’re on the hook for. Most folks that write papers are dying to acknowledge the shoulders they stand on - it’s part of the toxic humility we all engage in.

Again - shill OSS all day - I’ll upvote it.

Re:Show HN: LLMs can generate valid JSON 100% of the time on Monday August 14, 2023 @05:55PM (#37127291)
The “trick” seems to blatantly rip off FlashText without citing it?

https://arxiv.org/pdf/1711.00046.pdf

I’m a fan of the approach. I normally wouldn’t care if this was just another LLM library taking inspiration, but if you’re going to go out of your way to put a paper on the ArXiv, feels like doing a literature review is a good step?