It sounds like they had some software watching for any out-of-the-ordinary queries, which they then manually verified with the apparent person who was executing them. When that person had no idea what they were talking about, that's when the red flags went up.

And unfortunately, I don't think that's so uncommon.

I was the lone technical hire at a startup I was at a couple of years ago. We ran our app on top of postgres, and I used a native macos postgres client to keep an eye on our database (accessed via an SSH tunnel). The startup is very high-touch with clients - we built personal relationships with our clients.

Unsurprisingly our CEO loved looking at the numbers and the data. We could see conversion, and keep an eye on what our users were up to and how they were using our product. I was the sole engineer and I was only working 3 days a week. I didn't have time to build monitoring dashboards and all that; so instead I spent half an hour teaching our CEO the basics of how to do postgres read queries, and how to pull the results of a query out into CSV and into excel. That was probably the most productive half hour of my time at that startup - it was a huge enabler for him, allowing him to explore the data in arbitrary ways. I showed him how to sort, filter, select specific fields and join tables.

If I had instead built a specialised dashboard, it would have taken me away from our product. And I wouldn't have any idea what queries to display - any dashboard I made would have been worse for him than just querying the database directly. I'm still convinced that I made the right call.

(We did eventually build a dedicated admin panel, but months later - only when we actually needed it because some common tasks were too difficult through SQL alone).

I'm not religious about this. My team automates anything we have to do more than a couple of times, and from time to time we do have to manually alter something in a prod db. Do it often/long enough though and someone _will_ screw something up.

That sort of situation is constantly costing the company money for no good reason, it'd be fiscally sound to spell out these costs and contrast them with any estimates you could assemble on making the application start working again. Manually replaying queries is expensive, terrible, terrifying, and can potentially break data-integrity really badly.

Assumes poster works at a company where management cares past "Does it work?"

That's less rare than it used to be, but HN sometimes underappreciates how much of the economy is still governed by companies with CFOs who couldn't tell you the first thing about their enterprise infrastructure.

That said, I wonder if what’s going on here is that the admin account was never supposed to be used directly, so that’s why the alert triggered.

Anyone know of some lightweight foss monitoring tools that are in this arena? (for single server MySQL setups).

"As part of our investigation into the alert, we learned that the individual whose credentials were used had not actually made the query,"

How often do you design a database query to get the count of all rows from a table? It sounds like an uncommon use case, but probably one you could mute from the audit if you do it intentionally.

Any time I paginate a list, so it can show the "x of y pages"?

A search makes more sense here.

Not much, but that doesn't mean folks won't.

https://stackoverflow.com/questions says "17,335,212 questions " and the pagination goes "1 2 3 4 5 … 346705", if you want a real-world example.

I know few companies use it on even fewer DB's but it can really help to identify "rogue access" to your data. It takes (a lot) of time to tone it down to your specific needs and DB access, but once it is up and running it can trigger at the right time (as in the article).

There are different kinds of DAM, inline just reading the network traffic, missing the direct access accounts would have from the DB server itself, and the ones that monitor the DB service memory directly. Of course you could start way cheaper / simpler with some proper logging from the DB software itself.

It is a growing security area that is a nice (once tamed) additional information source.

If your audit system knows that this application never runs such queries, and today it did, it knows something changed and flags it.

Meaning, something that indicated some software was doing queries that were both unusual and somewhat high frequency?