Please write and post this guide.
The trick is calibrating what a "solid generalist" means. I think I'd describe myself that way, but perhaps not among the HN crowd. Would be very interested in being a soundboard for such content, if that's helpful.
What you want to know about a strange binary (barring obfuscation, sandbox escapes, and other nasties) is: who does it talk to (ip addresses, hostnames, sockets, etc), what does it open (files, registry entries, api's, services), when does it do these things (eg. runtime conditions, magic packets, port knocks, triggers, checking for other software), where does it write or read data (directories, filehandles, remote sites, etc), why does it do this given it's stated purpose (why does it have an encrypted section, and where is its key, is it using weird encoding to bypass filters, etc.) and then finally the How it does these all things is the effect of answering those other questions.
I think the hardest part of analysis is having an organized way of knowing what you are looking for because we don't know the right questions to ask and we tend to work at the edge of limited knowledge. Should this rando binary be talking some app hosting site, and why? Why would a developer encode endpoint names in a lookup table that only constructs and returns them at runtime? Why would someone use any of these libraries or data formats on purpose? The harder it is to answer these questions, the more suspicious I get.
If you start with the 5-W's, the How falls out of that a lot faster. If you can answer these questions about a binary, you're easily 50% there in determining whether it behaves as expected. Having an organized goal can take you from zero to basically useful if you answer those questions about it. The rest is just screenshots of menu items in ghidra and maybe cyberchef for purely static extraction.
I feel like I should pile on caveats here about how most malware isn't obfuscated or using novel techniques, a lot of it is just spyware capabilities you clicked through to accept, or a repackaged legit binary with some downloaded RAT attached and some nested compressed libraries. I'm sure someone who is more serious about this will say, "that's misleadingly simple!" but once you have a why and a what, the how is a work problem.
Dynamic debugging and stepping through is the next stage. It's also basic, but when you are goal oriented instead of being able to reproduce all usable code paths, it's more achievable. If you get the IP addresses out of random binary and what protocols it's talking, and maybe what files it accesses, it means you've set up your analysis environment and done the initial checks, and that's valuable grunt work you can pass on to someone with deeper skills.
If you can go from zero to this, that's an afternoon well spent, imo. It's not trivial, as it assumes a lot of knowledge about system architecture and network protocols, but the questions above necessarily have answers, so I can guarantee you can find them with some directed effort. I don't mean to trivialize more advanced analysis, this isn't the same thing, but as an entry point, this is how I would recommend approaching it.
I'm familiar with 'strings' and I've been playing with 'binwalk' to take apart files, but I'm out of my depth when it comes to loading something up in a debugger or whatever (is ghidra a debugger or what's the difference?) and looking at code. I don't speak C, and everything seems to look like C when it's shown in the examples of these things. How do I know if I'm looking at a sensible decompilation with actual runnable code or just gibberish because I'm trying to interpret a jpeg as an executable?
I don't know if that makes me teachable or beyond help, but I'd be an eager student.
If you know how to program you could probably already make sense of a lot of C, and for the rest you could try asking an AI to explain it to you.
> I'm out of my depth when it comes to loading something up in a debugger or whatever (is ghidra a debugger or what's the difference?)
When you hear "debugger", think "breakpoints". It's any tool that lets you do things like set breakpoints and step through code execution. In the C world, that would be tools like GDB and Valgrind. For Java - the IDEs themselves ship with debuggers (Eclipse, Intellij, etc), or jdb. In the frontend world your browser devtools ships with this functionality.
Debuggers can also typically let you view lower level details of a given binary's execution - like machine code for C, bytecode for Java, etc. They won't however decompile binaries or bytecode into the original higher level language.
Ghidra does come bundled with a debugger, but it can also do lots of other stuff (including decompilation).
> I don't know if that makes me teachable or beyond help, but I'd be an eager student.
If you're actually interested in this stuff, it would probably help to get some baseline familiarity with systems programming. Look up the 15-213 course material and exercises and work through it with the help of video lectures and a tutor - sentience agnostic.
You need to run the executable to do this tho so maybe use a VM.
I used frida a few times to do random stuff like making foobar2000 always play the same mp3 regardless of what is in the playlist, and made a game's speed adjustable by intercepting calls to system gettime and changing the value.
Use Ghidra to check what the executable imports and intercept found functions in Frida.
How does an antivirus tell the difference between e.g. TeamViewer and a repackaged app with a RAT?
loved your post. I'm by far a lot less experienced for sure. There is one thing in this sentenced that stood out because my order of what to address first is always what and how.
Only at the end the why (e.g motive) might or might not become visible. It has saved me from jumping to premature conclusion (or attribution) in the past ...
Most "who-dunnit" genre of films are based on making you believe the why is the ultimate goal. For me though the why is a by-product of addressing the what/how and I find things remain smoother and with less rabbit holes to get lost in.
Tried Ghidra for the first time on the weekend to look at some 8051 firmware
Got stuck with disassembly as it seems to be misinterpreting some data sections as code - can see English strings in a hex editor but Ghidra is trying to convert them to asm
Disassembling all sections just in case they contain code is a common conservative policy for disassemblers: even without malicious payload hiding tricks even definitely never executed sections could contain embedded executable code.
It's been a while since I've looked at asm in anger so it's taking me a while to get back into it (plus this is a side project ATM)
Tools like Ghidra et al. merely lay bare the truth you already know.
Re "allowing them to proliferate", the solution for that is flagging. (In case anyone doesn't know: to flag a comment, click on its timestamp to go to its page, then click the 'flag' link at the top. There's a small karma threshold before flag links appear.) And downvoting, of course (the karma threshold for that is higher). If the comment was bad enough to respond the way you did, why didn't you downvote and/or flag it?
What I could absolutely do is sit down and write a long reply about why I think the comment missed the mark, and how it could improve. I have done this in the past too. The problem here is that doing so is a lot of work. My thought process is that most of the people who are posting like this know that they're just posting low quality stuff, and if there's an easy reminder to stop doing that, they will. If they happen to reply with "no, I'm serious, here's why…" then there's no harm done; otherwise it signals (to other people too) that we're looking for something better than that.
Downvoting a bad comment is fine. Not posting is fine. If you want to neutrally let someone know that their comment isn't substantive enough for a good HN post, that's ok as long as you're careful how you do it. Dropping an insult on them is never helpful.
Unless you are talking about obfuscated / virtualized payloads, isn't it common to just "cheat" by running it in an emulator / debugger, then taking the unpacked code section from memory and work from there? It was the approach I took in a CTF task: https://nevesnunes.github.io/blog/2021/10/03/CTF-Writeup-TSG...
I used a REPL to manually do the steps you describe dynamically, but doing it statically means writing a decoder. You really need a proper sandbox to do dynamic analysis becase you don't know what's going to actually detonate, whereas static analysis gives you a whif of how off it seems, and that's sufficient for most security and privacy purposes. It was also common in Android apps several years ago now, not sure what the current state of the art is though. Android isn't my problem anymore.
Officially, I suck at this and I defer to more skilled people because I am a much better writer than hacker, but when they aren't around, you go to war with the army you have. :)
There is no chance I will ever publish original RE research, but it's a handy go-to tool, along with cyberchef, binwalk, and some other breadth-first static analysis tools for hunting specific IoCs. I could probably teach a solid generalist who cared to get to the level of being able to dissassemble something and say, "yeah, this is dodgy" or not in an afternoon.
As an exercise, next time you get a cheap peripheral like a headset or a other usb device, pop the driver installer package into ghidra and click through the call graph just to see what else it does. You may be surprised.