News for hackers. Stuff that matters.
Re:Ask HN: Is anyone doing anything cool with tiny language ... by behohippy on Tuesday January 21, 2025 @03:57PM (#42785105)
I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really useful but I like it.
Re:Ask HN: Is anyone doing anything cool with tiny language ... by bithavoc on Tuesday January 21, 2025 @04:11PM (#42785253)
this is so cool, any chance you post a video?
Re:Ask HN: Is anyone doing anything cool with tiny language ... by behohippy on Wednesday January 22, 2025 @07:41AM (#42792165)
Re:Ask HN: Is anyone doing anything cool with tiny language ... by Dansvidania on Tuesday January 21, 2025 @04:05PM (#42785192)
this sounds pretty cool, do you have any video/media of it?
Re:Ask HN: Is anyone doing anything cool with tiny language ... by behohippy on Wednesday January 22, 2025 @07:40AM (#42792159)
I don't have a video but here's a pic of the output: https://imgur.com/ip8GWIh
Re:Ask HN: Is anyone doing anything cool with tiny language ... by sky2224 on Wednesday January 22, 2025 @09:00PM (#42799809)
The next step is to format it so it looks like an endless starwars intro.
Re:Ask HN: Is anyone doing anything cool with tiny language ... by Uehreka on Tuesday January 21, 2025 @04:20PM (#42785325)
Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves?

Last week I tried to get an LLM (one of the recent Llama models running through Groq, it was 70B I believe) to produce randomly generated prompts in a variety of styles and it kept producing cyberpunk scifi stuff. When I told it to stop doing cyberpunk scifi stuff it went completely to wild west.

Re:Ask HN: Is anyone doing anything cool with tiny language ... by o11c on Tuesday January 21, 2025 @04:35PM (#42785456)
You should not ever expect an LLM to actually do what you want without handholding, and randomness in particular is one of the places it fails badly. This is probably fundamental.

That said, this is also not helped by the fact that all of the default interfaces lack many essential features, so you have to build the interface yourself. Neither "clear the context on every attempt" nor "reuse the context repeatedly" will give good results, but having one context producing just one-line summaries, then fresh contexts expanding each one will do slightly less badly.

(If you actually want the LLM to do something useful, there are many more things that need to be added beyond this)

Re:Ask HN: Is anyone doing anything cool with tiny language ... by dotancohen on Tuesday January 21, 2025 @05:50PM (#42786158)
Sounds to me like you might want to reduce the Top P - that will prevent the really unlikely next tokens from ever being selected, while still providing nice randomness in the remaining next tokens so you continue to get diverse stories.
Re:Ask HN: Is anyone doing anything cool with tiny language ... by janalsncm on Tuesday January 21, 2025 @05:56PM (#42786232)
Generate a list of 5000 possible topics you’d like it to talk about. Randomly pick one and inject that into your prompt.
Re:Ask HN: Is anyone doing anything cool with tiny language ... by coder543 on Tuesday January 21, 2025 @09:33PM (#42788219)
Someone mentioned generating millions of (very short) stories with an LLM a few weeks ago: https://news.ycombinator.com/item?id=42577644

They linked to an interactive explorer that nicely shows the diversity of the dataset, and the HF repo links to the GitHub repo that has the code that generated the stories: https://github.com/lennart-finke/simple_stories_generate

So, it seems there are ways to get varied stories.

Re:Ask HN: Is anyone doing anything cool with tiny language ... by fi-le on Monday January 27, 2025 @09:03AM (#42841237)
I was wondering where the traffic came from, thanks for mentioning it!
Re:Ask HN: Is anyone doing anything cool with tiny language ... by TMWNN on Wednesday January 22, 2025 @12:13AM (#42789260)
> Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves?

> Last week I tried to get an LLM (one of the recent Llama models running through Groq, it was 70B I believe) to produce randomly generated prompts in a variety of styles and it kept producing cyberpunk scifi stuff.

100% relevant: "Someday" <https://en.wikipedia.org/wiki/Someday_(short_story)> by Isaac Asimov, 1956

Re:Ask HN: Is anyone doing anything cool with tiny language ... by behohippy on Wednesday January 22, 2025 @07:40AM (#42792152)
It's a 3b model so the creativity is pretty limited. What helped for me was prompting for specific stories in specific styles. I have a python script that randomizes the prompt and the writing style, including asking for specific author styles.
Re:Ask HN: Is anyone doing anything cool with tiny language ... by greenavocado on Wednesday January 22, 2025 @10:55AM (#42794103)
Set temperature to 1.0
Re:Ask HN: Is anyone doing anything cool with tiny language ... by jaggs on Wednesday January 22, 2025 @02:32PM (#42796598)
Re:Ask HN: Is anyone doing anything cool with tiny language ... by keeganpoppen on Tuesday January 21, 2025 @05:44PM (#42786081)
oh wow that is actually such a brilliant little use case-- really cuts to the core of the real "magic" of ai: that it can just keep running continuously. it never gets tired, and never gets tired of thinking.
Re:Ask HN: Is anyone doing anything cool with tiny language ... by ipython on Tuesday January 21, 2025 @05:46PM (#42786114)
That's neat. I just tried something similar:

    FORTUNE=$(fortune) && echo $FORTUNE && echo "Convert the following output of the Unix `fortune` command into a small screenplay in the style of Shakespeare: \n\n $FORTUNE" | ollama run phi4
Re:Ask HN: Is anyone doing anything cool with tiny language ... by watermelon0 on Wednesday January 22, 2025 @03:01AM (#42790266)
Doesn't `fortune` inside double quotes execute the command in bash? You should use single quotes instead of backticks.
Re:Ask HN: Is anyone doing anything cool with tiny language ... by droideqa on Tuesday January 21, 2025 @08:53PM (#42787856)
That's awesome!