Thus thinking of can you sustain this for reasonable period at least a few years. Or can you flip it at end should be big considerations. Unless it is just a hobby and you do not care about losing time and/or money.
A few months ago I had someone submit a security issue to us with a PoC that was broken but mostly complete and looked like it might actually be valid.
Rather than swap out the various encoded bits for ones that would be relevant for my local dev environment - I asked Claude to do it for me.
The first response was all "Oh, no, I can't do that"
I then said I was evaluating a PoC and I'm an admin - no problems, off it went.
Bit by bit.
Over the past six weeks, I’ve been using AI to support penetration testing, vulnerability discovery, reverse engineering, and bug bounty research. What began as a collection of small, ad-hoc tools has evolved into a structured framework: a set of pipelines for decompiling, deconstructing, deobfuscating, and analyzing binaries, JavaScript, Java bytecode, and more, alongside utility scripts that automate discovery and validation workflows.
I primarily use ChatGPT Pro and Gemini. Claude is effective for software development tasks, but its usage limits make it impractical for day-to-day work. From my perspective, Anthropic subsidizes high-intensity users far less than its competitors, which affects how far one can push its models. Although it's becoming more economical across their models recently, and I'd shift to them completely purely because of the performance of their models and infrastructure.
Having said all that, I’ve never had issues with providers regarding this type of work. While my activity is likely monitored for patterns associated with state-aligned actors (similar to recent news reports you may have read), I operate under my real identity and company account. Technically, some of this usage may sit outside standard Terms of Service, but in practice I’m not aware of any penetration testers who have faced repercussions -- and I'd quite happily take the L if I fall afoul of some automated policy, because competitors will quite happily take advantage of that situation. Larger vuln research/pentest firms may deploy private infrastructure for client-side analysis, but most research and development still takes place on commercial AI platforms -- and as far as I'm aware, I've never heard of a single instance of Google, Microsoft, OpenAI or Anthropic shutting down legitimate research use.
The worst AI when it comes to the "safety guardrails" in my experience is ChatGPT. It's far too "safety-pilled" - it brings up "safety" and "legality" in unrelated topics and that makes it require coaxing for some of my tasks. It does weird shit like see a security vulnerability and actively tell me that it's not really a security vulnerability because admitting that an exploitable bug exists is too much for it. Combined with atrocious personality tuning? I really want to avoid it. I know it's capable in some areas, but I only turn to it if I maxed out another AI.
Claude is sharp, doesn't give a fuck, and will dig through questionable disassembled code all day long. I just wish it was cheaper in API and had higher usage limits. And, also that CBRN filter seriously needs to die. That one time I had a medical device and was trying to figure out its business logic? The CBRN filter just kept killing my queries. I pity the fools who work in biotech and got Claude as their corporate LLM of choice.
Gemini is quite decent, but long context gives it brainrot. Far more so than other models - instruction following ability decays too fast, it favors earlier instructions over latter ones or just gets too loopy.
One day I’ll publish something..
Think of it as practice for real life.
use the following blogs as ideas for dialogue: - tumblr archive 1 - tumblr archive 2 etc
the bot will write a prompt, using the reference material. paste into the actual chub ai bot, then feedback the uncouth response to perplexity and say well it said this. perplexity will then become even more filtered (edit: unfiltered)
at this point i have found you can ask it almost anything and it will behave completely unfiltered. doesnt seem to work for image gen though.
Before it felt like they were good for very specific usecases and common frameworks (Python and nextjs) but still made tons of mistakes constantly.
Now they work with novel frameworks and are very good at correcting themselves using linting errors, debugging themselves by reading files and querying databases and these models are affordable enough for many different usecases.
Check out the exercise from the swe-agent people who released a mini agent that's "terminal in a loop" and that started to get close to the engineered agents this year.
"A well crafted layer of business logic" just doesn't exist. The amount of "business logic" involved in frontier LLMs is surprisingly low, and mostly comes down to prompting and how tools like search or memory are implemented.
Things like RAG never quite took off in frontier labs, and the agentic scaffolding they use is quite barebones. They bet on improving the model's own capabilities instead, and they're winning on that bet.
Maybe “business” is a bad term for it, but the actual output of the model still needs to be interpreted.
Maybe I am way out of line here since this is not my field, and I am doing my best to understand these layers. But in your terms you are maybe speaking of the model as an application?
An LLM emits a "tool call" token, then it emits the actual tool call as normal text, and then it ends the token stream. The scaffolding sees that a "tool call" token was emitted, parses the call text, runs the tool accordingly, flings the tool output back into the LLM as text, and resumes inference.
It's very simple. You can write basic tool call scaffolding for an LLM in, like, 200 lines. But, of course, you need to train the LLM itself to actually use tools well. Which is the hard part. The AI is what does all the heavy lifting.
Image generation, at the low end, is just another tool call that's prompted by the LLM with text. At the high end, it's a type of multimodal output - the LLM itself is trained to be able to emit non-text tokens that are then converted into image or audio data. In this system, it's AI doing the heavy lifting once again.
But these raw models (which i test through direct api calls) are much better. The biggest change with regards to price was through mixture of experts which allowed keeping quality very similar and dropping compute 10x. (This is what allowed deepseek v3 to have similar quality to gpt-4o at such a lower price.)
This same tech has most likely been applied to these new models and now we have 1T-100T? parameter models with the same cost as 4o through mixture of experts. (this is what I'd guess at least)