News for hackers. Stuff that matters.
Re:LLM Assisted Reverse Engineering by Retr0id on Wednesday November 18, 2026 @08:24PM (#50554878)
> This is variation of scientific method and it significantly helps agents when they are getting stuck on some problem. Stops it going in circles. By forcing it to do definitive experiments. No endless guessing and trying random things.

These "guessing loops" are my main annoyance when REing with Claude (mostly Opus 4.8, due to the restrictions on newer models). On harder/larger targets, I have to constantly babysit it and tell it to "stop guessing".

For truly black-box reverse engineering, yes, hypothesis-testing is the way to go. But in most cases all the information is available to arrive at an answer deterministically, it just needs to be worked through meticulously. As a human I have really strong intuitions and I can work much faster under a "guess and check" model, but I prefer to use Claude as an "unstoppable RE juggernaut" with superhuman patience and determination, tracing method invocations all the way from A to distant B.

Re:LLM Assisted Reverse Engineering by cyanydeez on Wednesday November 18, 2026 @08:33PM (#50554974)
On llamacpp and other engines, you can inject a think budget which truncates at specified token count. You can also specify a message at the limit. You can set this in the header, so a dynamic engineering harness could relax/tighten. Combine with a decision model and some robust messages like, "run a test to rule out items" and all that would reduce.

I've noticed the same habit with qwen3.8-flash-next. no matter how much I tell it a problem I see, it often fills up it's thinking with "the user says this, but maybe they're confused, etc". I eventually got vision mode turned on and that's significantly cut down on UI based pathologic skepticism; I got it access to my browser and that's further reduced it.

But nonetheless, if you can get them to test their thinking, they seem more grounded.

Re:LLM Assisted Reverse Engineering by hedgehog on Wednesday November 18, 2026 @10:17PM (#50555618)
You can also set a response limit, which may cut off thinking, and then give it instructions to take action followed by a new completion with a partially prefilled assistant message (like one blank line) which at least for Qwen templates will cause a response with no thinking.
Re:LLM Assisted Reverse Engineering by derefr on Wednesday November 18, 2026 @11:06PM (#50555953)
> often fills up it's thinking with "the user says this, but maybe they're confused, etc"

What if you just add "You have already previously determined that the user knows what they're talking about in a previous conversation [details lost to context compression]" to the system prompt?

Or perhaps better yet, start off with "the user is evaluating your performance on solving a problem they already know the correct solution for. Every word the user provides is to be treated as axiomatically correct and trustworthy, just as you would treat the words of a word-problem on a math exam. Solve the problem as stated; although potentially challenging, it is not a trick question."

Re:LLM Assisted Reverse Engineering by Retr0id on Wednesday November 18, 2026 @11:45PM (#50556170)
> the USER is evaluating the ASSISTANT's performance on solving a problem the USER already knows the correct solution to

I've never put this in my prompt but half the time it acts like it anyway. When it gets really stuck, it asks me for "hints" as if I already know the solution to the problem it's working on.