News for hackers. Stuff that matters.
Re:The Coming of Local LLMs by gamegoblin on Tuesday April 11, 2023 @12:41PM (#35527634)
I already have a custom browser plugin that calls out to GPT (gpt-3.5-turbo is cheap and good enough for this) to classify and filter out low-effort, overly negative, or intellectually dishonest HN comments. It significantly improves the experience on this site.

Bonus points: I had never written a browser plugin, but GPT4 helped me do it in under half an hour.

Re:The Coming of Local LLMs by darkgreene on Tuesday April 11, 2023 @12:47PM (#35527706)
Do you have a repo you can share? I'd be very interested in running my own local copy
Re:The Coming of Local LLMs by tough on Tuesday April 11, 2023 @05:55PM (#35532123)
+1
Re:The Coming of Local LLMs by artificial on Tuesday April 11, 2023 @02:21PM (#35529070)
This sounds great, would love to see some more about this endeavor.
Re:The Coming of Local LLMs by metalliqaz on Tuesday April 11, 2023 @02:24PM (#35529114)
How do you define the cost function for 'low effort'?
Re:The Coming of Local LLMs by gamegoblin on Tuesday April 11, 2023 @02:36PM (#35529333)
I give GPT some examples of what I consider low-effort comments and non-low-effort comments, and then I just ask it to do classification. It's quite good at few-shot classification of fuzzy stuff like this.
Re:The Coming of Local LLMs by jpe90 on Tuesday April 11, 2023 @03:37PM (#35530317)
This sounds super cool, I’m very curious how you implemented it.

I’m only vaguely familiar with the API.. if I had to guess I would say you send system instruction that its job is to filter unwanted content. And you mentioned you give few examples of unwanted content, so maybe those are additional messages along with an instruction like “filter the following html:”

For every web request you want to filter, you would re-send all of those messages followed by the page HTML as the final message. Is that close?

Re:The Coming of Local LLMs by gamegoblin on Tuesday April 11, 2023 @04:32PM (#35531129)
Close, but it's a bit more specialized to just work on HN:

- Examples of unwanted content

- Then I give a large numbered list of comments and ask which numbers should be filtered

- The plugin then just deletes those comment nodes from the DOM. If HN ever updates their HTML I will have to tweak this code.

The reason to send a large list of comments is just to save on costs. It's cheaper to do it this way than one comment at a time.

So the main difference from what you've proposed is GPT never sees the HTML. My code enumerates the comments in the HTML and splices them in to the prompt in a nice numbered list, then does the reverse translation from list number to DOM element in the other direction.

Re:The Coming of Local LLMs by jpe90 on Tuesday April 11, 2023 @06:06PM (#35532241)
Oh that’s clever! Thanks for answering!
Re:The Coming of Local LLMs by xk_id on Tuesday April 11, 2023 @03:42PM (#35530413)
I don’t know how anyone can trust to run code on their machine that they don’t understand and hasn’t been reviewed by a third party.
Re:The Coming of Local LLMs by gamegoblin on Tuesday April 11, 2023 @04:33PM (#35531163)
I understand the code now, I just didn't know how to write a browser plugin and having GPT4 walk me through it was 10x faster than reading a tutorial.
Re:The Coming of Local LLMs by sliken on Tuesday April 11, 2023 @04:23PM (#35531008)
How many line of codes are in the kernel, drivers, and apps on your laptop/desktop?

How sure are you that all that code has been reviewed by a 3rd party? How many CVEs a year impact your laptop/desktop?

Do you have any reason to think that increased productivity with LLM assistance will result in lower quality code? Personally I find LLM assistance increases productivity, decreases the penalty of using a more difficult language like rust, and makes it more palatable to spend more (LLM assisted) time writing tests.

Re:The Coming of Local LLMs by xk_id on Tuesday April 11, 2023 @08:31PM (#35533592)
I meant reviewed by a (competent) human; not, audited by a security team/whatever. Generative computer code is in a whole new category of risk, because it can hallucinate a `rm -rf *` that you might overlook if you don't understand the code yourself.
Re:The Coming of Local LLMs by sliken on Tuesday April 11, 2023 @08:47PM (#35533710)
Well at least for now, any non-trivial code is going to be written by a human+AI, likely one function at a time. It's pretty rare for the AI to give me exactly what I want, often I have to tweak my prompts multiple times, and even just give up and write a function based on the useful bits from previous queries.

I just don't see it chatbot assisted programming any worse than what we have today.

Re:The Coming of Local LLMs by rcme on Tuesday April 11, 2023 @03:47PM (#35530479)
Isn't that pretty much all code?
Re:The Coming of Local LLMs by ChatGTP on Tuesday April 11, 2023 @05:07PM (#35531576)
Some negative, even overly negative comments contain good ideas though so maybe it’s not a great idea?
Re:The Coming of Local LLMs by heyoni on Wednesday April 12, 2023 @12:32AM (#35535032)
Your comment has been detected to contain oppositional tones and has thus been filtered out.

You are hereby removed from the discourse.

/s

Re:The Coming of Local LLMs by ChatGTP on Wednesday April 12, 2023 @08:06AM (#35538328)
Love it!
Re:The Coming of Local LLMs by 4dayworkweek4u on Thursday April 13, 2023 @08:27PM (#35564045)
It's very inclusive to filter out all "negative" people.
Re:The Coming of Local LLMs by duckmysick on Wednesday April 12, 2023 @03:03AM (#35535959)
That's ok. If it's a good idea, somebody else will talk about in a positive way.

Also, unless you are reading every comment in every thread, you are going to miss a few interesting ideas anyway. That's ok too.

Re:The Coming of Local LLMs by ChatGTP on Wednesday April 12, 2023 @08:37PM (#35549242)
Stay calm and keep summarising!
Re:The Coming of Local LLMs by 4dayworkweek4u on Thursday April 13, 2023 @08:25PM (#35564034)
You are handsome, clever and very virtuous... You are in a positivity filter bubble -- that is super positive. (What about DEI though? Are you being inclusive? Filtering is exclusive).

You are so good.. because you don't acknowledge negativity. Best way to solve problems is to ignore them. Let's all be super loving and positive! Yay.