stillsut

I actually wrote a library to do stego with LLM outputs last year and it turned out to be an almost exact implementation of the Anthropic watermark algo.

Repo here https://github.com/sutt/innocuous. It works with last year's llama.cpp. Check out the "Use Cases" and "How it works" sections in the readme if you're interested.

Exactly what I was looking for, thanks.

I've been doing something similiar: aider+gpt-5, claude-code+sonnet, gemini-cli+2.5-pro. I want to coder-cli next.

A main problem with this approach is summarizing the different approaches before drilling down into reviewing the best approach.

Looking at a `git diff --stat` across all the model outputs can give you a good measure of if there was an existing common pattern for your requested implementation. If only one of the models adds code to a module that the others do not, it's usually a good jumping off point to exploring the differing assumptions each of the agents built towards.

I've actually written my own a homebrew framework like this which is a.) cli-coder agnostic and b.) leans heavily on git worktrees [0].

The secret weapon to this approach is asking for 2-4 solutions to your prompt running in parallel. This helps avoid the most time consuming aspect of ai-coding: reviewing a large commit, and ultimately finding the approach to the ai took is hopeless or requires major revision.

By generating multiple solutions, you can cutdown investing fully into the first solution and use clever ways to select from all the 2-4 candidate solutions and usually apply a small tweak at the end. Anyone else doing something like this?

[0]: https://github.com/sutt/agro

I think the easiest explanation is to look at the table here: https://github.com/sutt/innocuous?tab=readme-ov-file#how-it-...

Watch how the "Cumulative encoding" row grows each iteration (that's where the BTC address will be encoded) and then look at the other rows for how the algorithm arrives at that.

Thanks for checking it out!

I've been building something like this, a markdown that tracks your prompts, and the code generated.

https://github.com/sutt/innocuous/blob/master/docs/dev-summa...

Check it out, I'd be curious of your feedback.

Encoding / decoding hidden messages in LLM output.

https://github.com/sutt/innocuous

The traditional use-case is steganography ("hidden writing"). But I see more potential applications than just for spy stuff.

I'm using this project as a case study for writing CS-oriented codebases and keeping track of every prompt and generated code line in a markdown file: https://github.com/sutt/innocuous/blob/master/docs/dev-summa...

My favorite pattern I've found is to write encode implementations manually, and then AI pretty easily is able to follow that logic and translate it into a decode function.

I'm also working on a library to steer the sampling step of LLM's but more for steganographic / arbitrary data encoding purposes.

Should work with any llama.cpp compatible model: https://github.com/sutt/innocuous

You can also earn zaps for pull requests working on Nostr clients.

We've been hosting some bounties like this one here: https://app.lightningbounties.com/issue/615dc5f7-ed91-4ecd-8...

I've got some receipts for what I think is good vibe coding...

I save every prompt and associated ai-generated diff in a markdown file for a steganography package I'm working on.

Check out this document: https://github.com/sutt/innocuous/blob/master/docs/dev-summa...

In particular, under v0.1.0 see `decode-branch.md` prompt and it's associated generated diff which implements memoization for backtracking while performing decoding.

It's a tight PR that fits the existing codebase and works well, you just need a motivating example you can reproduce which can help me you quickly determine if the proposed solution is working. I usually generate 2-3 solutions initially and then filter them quickly based on a test case. And as you can see from the prompt, it's far from well formatted or comprehensive, just "slap dash" listing of potentially relevant information similar to what would be discussed at an informal whiteboard session.

I'm actually working on something similar to this where you can encode information into the outputs of LLM's via steganography: https://github.com/sutt/innocuous

Since I'm really looking to sample the only the top ~10 tokens, and I mostly test on CPU-based inference of 8B models, there's probably not a lot of worries getting a different order of the top tokens based on hardware implementation, but I'm still going to take a look at it eventually, and build in guard conditions against any choice that would be changed by an epsilon of precision loss.

I think the "magic" that we've found a common toolset of methods - embeddings and layers of neural networks - that seem to reveal useful patterns and relationships from a vast array of corpus of unstructured analog sensors (pictures, video, point clouds) and symbolic (text, music) and that we can combine these across modalities like CLIP.

It turns out we didn't need a specialist technique for each domain, there was a reliable method to architect a model that can learn itself, and we could already use the datasets we had, they didn't need to be generated in surveys or experiments. This might seem like magic to an AI researcher working in the 1990's.

"Unstructured data learners and generators" is probably the most salient distinction for how current system compare to previous "AI systems" examples (NLP, if-statements) that OP mentioned.

Makes sense: M-chips, Falcon-9, GPT's are product subsets or the incumbent's traditional product capabilities.

At a meta-level, I wonder if there's this un-talked about advantage of poaching ambitious talent out of an established incumbent to work a new product line in a new organization, in this case Apple Silicon disrupting Intel/AMD. And we've also seen SpaceX do this NASA/Boeing, and OpenAI do it to Google's ML departments.

It seems like large, unchallenged organizations like Intel (or NASA or Google) collect all the top talent out of school. But changing budgets, changing business objectives, frozen product strategies make it difficult for emerging talent to really work on next-generation technology (those projects have already been assigned to mid-career people who "paid their dues").

Then someone like Apple Silicon with M-chip or SpaceX with Falcon-9 comes along and poaches the people most likely to work "hardcore" (not optimizing for work/life balance) while also giving the new product a high degree of risk tolerance and autonomy. Within a few years, the smaller upstart organization has opened up in un-closeable performance gap with behemoth incumbent.

Has anyone written about this pattern (beyond Innovator's Dilemma)? Does anyone have other good examples of this?

Got your shovelware right here...with receipts.

Background: I'm building a python package side project which allows you to encode/decode messages into LLM output.

Receipts: the tool I'm using creates a markdown that displays every prompt typed, and every solution generated, along with summaries of the code diffs. You can check it out here: https://github.com/sutt/innocuous/blob/master/docs/dev-summa...

Specific example: Actually used a leet-code style algorithms implementation of memo-ization for branching. This would have taken a couple of days to implement by hand, but it took about 20 minutes to write the spec and 20 minutes to review solutions and merge the solution generated. If you're curious you can see this diff generated here: https://github.com/sutt/innocuous/commit/cdabc98

Not an expert but my chats with ChatGPT led me to believe async + FastAPI can give you 40x throughput for request handling over non-async code.

The essential idea was I could be processing ~100 requests per vCPU in the async event loop while threading would max out 2-4 threads per CPU. Of course let us assume for either model we're waiting for 50-2000ms DB query or service call to finish before sending the response.

Is this not true? And if it is true, why isn't the juice is worth the squeeze: more than an order of magnitude more saturation/throughput for the same hardware and same language, just with a new engine at its heart?

Hey I made an open source version of this last week (albeit for different purposes). Check it out at: https://github.com/sutt/innocuous

There's lot of room for contributions here, and I think "fingerprinting layer" is an under-valued part of the LLM stack, not being explored by enough entrants.

> we should always consider asking the LLM the same question more than once, perhaps with some variation in the wording. Then we can compare answers,

Yup, this matches my recommended workflow exactly. Why waste time trying to turn an initially bad answer into a passable one, when you could simply re-generate (possibly with different context)

I wrote up an example of this workflow here: https://github.com/sutt/agro/blob/master/docs/case-studies/a...

Not OP but I know it can be difficult to really difficult to measure or communicate this to people who aren't familiar with the codebase or the problem being solved.

Other than just dumping 10M tokens of chats into a gist and say read through everything I said back and forth with claude for a week.

But, I think I've got the start of a useful summary format: it that takes every prompt and points to the corresponding code commit produced by ai + adds a line diff amount and summary of the task. Check it out below.

https://github.com/sutt/agro/blob/master/docs/dev-summary-v1...

(In this case it's an python cli ai-coding framework that I'm using to build the package itself)

I'm rolling my own like this [0]:

.agdocs/

├── specs/ # Task specification files

├── conf/ # Configuration files

├── guides/ # Development guides for agents

└── swap/ # Temporary files (gitignored)

Every markdown file in guides/ gets copied to any agent (aider, claude, gemini) I kick-off.

I gitignore this .agdocs directory by default. I find this useful because otherwise you get into _please commit or stash your changes_ when trying to switch branches.

But I also run an rsync script before each release to copy the .agdocs to a git tracked mirrored directory [1].

[0]: https://github.com/sutt/agro/blob/master/README.md#layout

[1]: https://github.com/sutt/vidstr/tree/master/.public-agdocs/gu...

Yeah, I've been able to stack ~200 AI-generated commits on top of each other. Over 95% of the code is now AI generated.

You can see each prompt and each commit that was generated here: https://github.com/sutt/agro/blob/master/docs/dev-summary-v1...

Since AI-Gen code is such a roll of the dice, the key is to roll the dice a lot - usually generate at least 3 potential solutions from at least two separate providers at the outset - and get good at quickly reviewing the offered solutions, or iterating on the prompt and regenerating.

Just added a fork of aider that does do agentic commands: https://github.com/sutt/agent-aider

In testing I've found it to be underwhelming at being an agent compared to claude code, wrote up some case-studies on it here: https://github.com/sutt/agro/blob/master/docs/case-studies/a...

I've been finding adding context for your external packages is really important when the package is relatively new and or has breaking changes since the model training cut-off date.

Two that stick out this week are google's genai (client for vertex/gemini ednpoints) that is updating methods and moviepy in their 2.x breaking changes (most of the corpus for this library was trained with v1.x).

I wrote about some findings here, and that there's still not a great way to have the models examine only the pieces of documentation that they need for their work: https://github.com/sutt/agro/blob/master/docs/case-studies/a...

Yeah I suspect some of these providers will become Microsoft in the '90s type bully holdouts on implementing the emerging conventions. But ultimately with CLI interface you have workarounds to all the major providers read in your system guidelines. But in an IDE - e.g. like MS had with VisualStudio - you more lock-in potential for your config files.

Yesterday, I was writing about a way I found to pass the same guideline documents into Claude, Gemini, and Aider CLI-coders: https://github.com/sutt/agro/blob/master/docs/case-studies/a...

Last year, I was actually working on a bounty platform for Github PR's.

The low quality human-authored PR's that came in (due to the incentive we offered) combined with the fact that a draft PR could be made for pennies with AI made this concept dead in the water as far as I'm concerned.

The pain point of getting some attention and action on your opensource codebase is really no longer relevant, in fact the pain point seems to be moving to how to optimize the limited reviewer / maintainer bandwidth under the onslaught of proposed suggestions.

To this end I've been experimenting with a framework that builds PR's from the major agents and but with a focus on how to structure the tasks and review process that optimize the review => accept/revise cycle. If you're interested I've been writing up some case studies here: https://github.com/sutt/agro/blob/master/docs/case-studies/a...

I think the Github-PR model for agent code suggestions is the path of least resistance for getting adoption from today's developers working in an existing codebase. It makes sense: these developers are already used to the idea and the ergonomics of doing code reviews this way.

But pushing this existing process - which was designed for limited participation of scarce people - onto a use-case of managing a potentially huge reservoir of agent suggestions is going to get brittle quickly. Basically more suggestions require a more streamlined and scriptable review workflow.

Which is why I think working in the command line with your agents - similar to Claude and Aider - is going to be where human maintainers can most leverage the deep scalability of async and parallel agents.

> is way better than having to set up git worktrees or any other type of sandbox yourself

I've built up a helper library that does this for you for either aider or claude here: https://github.com/sutt/agro. And for FOSS purposes, I want to prevent MS, OpenAI, etc from controlling the means of production for software where you need to use their infra for sandboxing your dev environment.

And I've been writing about how to use CLI tricks to review the outputs on some case studies as well: https://github.com/sutt/agro/blob/master/docs/case-studies/i...

This has repo has a great prebuilt llm docs sections. This seems like a killer feature for clients who want to roll their own customizations.

https://github.com/crbnos/carbon/blob/main/CLAUDE.md

https://github.com/crbnos/carbon/tree/main/llm

Currently working on a python package: agro.

It helps devs build parallel agents from the cli. (think claude code, gemini, aider)

And simplifies the git workflow for testing and accepting the best solution.

Check it out: https://github.com/sutt/agro

I keep a little text file pr.txt and try to link my project to HN / X / other aggregators once a day...

This way I spend a few hours each day coding, and a few minutes each day promoting and refining me message.