News for hackers. Stuff that matters.
Re:The architecture of today's LLM applications by lmeyerov on Sunday November 19, 2023 @01:03PM (#38335569)
Yes, we work with security analysts and other folks doing data-intensive investigations & analysis in Louie.ai (think being able to use natural language to talk to you Splunk, OpenSearch, Databricks, etc. logs/news/.., and many Python / Pandas viz + wrangling + AI tools, to investigate more easily & further, build autonomous hunts, smarter monitors & detections, etc)... and LLMs work amazingly here.

Blue team is a lot easier than red team, and as part of that, code tasks are easier than social engineering. It rarely comes up in investigations, but you do need to set prompt persona for simulating attacks.

At the same time, fine-tuning a model is a pretty clear way to undo needing to do even that.

We benefit from models having business-neutral alignment by default -- the typical case suffers if the default persona was a racist forum troll with extreme politics. So it's more about when you want to turn that off for specific tasks. It's more work than we'd like, and gotchas like changing when models update.

Re:The architecture of today's LLM applications by insanitybit on Sunday November 19, 2023 @01:10PM (#38335680)
Got it, thanks. Alright that makes sense. I haven't really tried much at this point, though I have had ChatGPT give me a few "I can't do that" and I've had to be like "no, seriously, I'm doing this because it's a good thing" in the past (3.5 days).

Fine tuning does seem like a good idea.

Re:The architecture of today's LLM applications by lamroger on Sunday November 19, 2023 @09:45PM (#38341588)
Make me think that a push model would be awesome. Would that be an agent?
Re:The architecture of today's LLM applications by lmeyerov on Monday November 20, 2023 @03:52AM (#38344994)
Interesting, curious what you mean by push model?

Ex: Outbound - Interactive chat sessions trigger outside push update, such as to a database for data to index, new findings & enrichments, etc

Ex: Inbound - API calls trigger internal runs... which in turn can also trigger external workflows

We're experimenting with APIs for embedding in your own apps, headless APIs for automations, and all sorts of integrations. Getting security and modular/compositional interfaces right has been a fascinating challenge!

Re:The architecture of today's LLM applications by lamroger on Monday November 20, 2023 @01:00PM (#38351647)
Automated alerts - SRE as a service. Anomaly detection with some good suggestions. SRE intern as a service? Something like that - notifying you of something vs human requesting.
Re:The architecture of today's LLM applications by lmeyerov on Saturday November 25, 2023 @11:50AM (#38414623)
That's great, thanks! If ever up for a chat about it, let me know -- leo@graphistry