News for hackers. Stuff that matters.
Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by sanderjd on Sunday November 05, 2023 @08:12AM (#38150843)
How does one efficiently learn how to do such things, and what kinds of problems such approaches are fruitful for?

I find there to be a giant gap in learning about this stuff between material that boils down to "use magic words and system prompts to improve results from one of the big models" and "how do LLMs work from first principles".

I still haven't found a great resource that covers this middle ground, which seems to me to be where a lot of the power of these approaches is going to reside.

Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by Swizec on Sunday November 05, 2023 @09:13AM (#38151274)
> I still haven't found a great resource that covers this middle ground, which seems to me to be where a lot of the power of these approaches is going to reside.

Read papers, build institution, experiment.

That last part may be the most important.

Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by sanderjd on Monday November 06, 2023 @11:52AM (#38165048)
I think this is the disconnect: It doesn't strike me that what I'm talking about has anything to do with "papers". So from your comment, I'm once again left wondering what you mean.

My sense is that I have a much better grasp of the foundational material here, having read in depth books and papers about that, but still can't quite wrap my head around the question of how people are actually "operationalizing" this into useful software.

But to your point about experimentation, it might just be the kind of thing where there is no path to enlightenment besides working on a project and running into and overcoming all the hurdles along the way.

Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by somenameforme on Sunday November 05, 2023 @09:23AM (#38151356)
Basic neural net principles apply directly on over to LLMs. The middle ground would be in something like taking an number-image-to-text parser and seeing how it works. If you just search for neural net on Github you'll get a million hits, and most of all major projects will have some sort of an interactive image parser demo because it's surprisingly simple, sexy, shows a lot of different concepts, and is just awesome for learning from.
Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by kcorbitt on Sunday November 05, 2023 @10:06AM (#38151733)
If you're looking for a practical guide to getting started with fine tuning, I wrote one a couple of months ago that got pretty popular here on HN. Might be helpful if you're interested in playing around with it! https://news.ycombinator.com/item?id=37484135
Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by Tostino on Sunday November 05, 2023 @10:20AM (#38151837)
So I described my approach to how I fine tune a specific task below to another user, but I'll copy it here:

> Design your tasks to be repeatable and small steps, call the OpenAI API and log all requests/responses. > Filter out any bad responses and take a representative sample of the data you have collected from OpenAI,and train a Mistral or Llama2 model with the request/response pairs.

> Measure the quality of your model vs OpenAI for the same inputs, and then swap out the model in your workflow once happy with the results.

Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by capableweb on Sunday November 05, 2023 @11:02AM (#38152286)
If you do this, be careful how/if you publish your weights trained on OpenAI output as if they look into how it was generated and it becomes clear you broke the ToS, they'll most likely ban you from the platform.
Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by leereeves on Sunday November 05, 2023 @11:33AM (#38152635)
How would they "look into how it was generated"?
Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by capableweb on Sunday November 05, 2023 @11:37AM (#38152680)
You train your model, publish it on huggingface and then write in the README:

> This is how I made this model: Design your tasks to be repeatable and small steps, call the OpenAI API and log all requests/responses. > Filter out any bad responses and take a representative sample of the data you have collected from OpenAI,and train a Mistral or Llama2 model with the request/response pairs.

Just one example.

Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by Tostino on Sunday November 05, 2023 @05:32PM (#38156354)
I'm not competing with OpenAI in any sense of the word.
Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by jiveturkey on Sunday November 05, 2023 @06:10PM (#38156708)
I haven't read the ToS, but it may not specifically require competing to be a violation.
Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by jachee on Sunday November 05, 2023 @01:57PM (#38154353)
The industry term for that middle ground is a “moat”, and the people who are most familiar with it are getting paid for what they know, so they’re not giving it away.
Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by sanderjd on Monday November 06, 2023 @11:48AM (#38164986)
I think that may be right, but if so, that seems pretty unusual to me.

I've gone through a few of these "new kinds of software becoming useful" transition periods - most notably applications moving to the web, and then native smart phone applications - and in none of those transitions was there a dearth of resources on how to spin up on doing useful things due to this "moat" concern.

Nobody was protecting their iphone app dev moat by not publishing books and training courses on Objective-C and XCode...

Re:Email Obfuscation Rendered Almost Ineffective Against Cha... by danielmarkbruce on Sunday November 05, 2023 @09:11PM (#38157987)
huggingface is your friend.