The replies are focused on MySQL (and to a lesser extent, PHP). None of that matters.
The hero here is the logging system serving as seekable stream of collected events. (I’m trying hard here to not say K-fka).
By consuming events from that source and being able to perform time-based seeking, you’ve offloaded (or dare I say, eliminated) your scaling pressures.
Even if you commit the “sins” of no indexes on heavy queries, using MyISAM, or building a giant, memory-intensive associative array… none of it matters.
You can choose to wipe your local dataset, start again in a different language, switch to another data store model, and none of your downstream consumers will know nor care (unless you blog about it).
*This* type of design, that prioritizes consumer-facing resiliency and continuity, is what I look for in systems that scale.
That Comcast prototype sounds amazing. PHP and MySQL can really bring you very far, i use it for nearly everything and it is so incredible easy to develop and “deploy” new versions. I use propel as ORM and it is very close to the DB and I can change schemes fast and just let the autoupdate handle the rest.
A k8s cluster for something like that wouldn't cost anywhere that amount, plus you would get the advantage of being able to deploy your app in multiple availability zones. If you don't need k8s features, there are definitely simpler options that will give you redundancy and won't break the bank.
I have gone past mysql limits a few times. Will you go beyond its capability is a simple measure. Can your hardware write fast enough to keep up? If your hardware maxes out writing to disk 200 megs per second, welp mysql maxes out at 200 megs per second. If you need 400 megs per second of writing, well you need two instances that can write independently. That is when you are fucked. At that point you get into sharding* and doing whacky things mysql isn't meant todo. Reading things from disk, mysql's replication scales just fine.
My go to language of choice is PHP/MySQL. I've never not been able to scale a project. At one point while working at Comcast I prototyped a system for every error code from every cable set top box (150 million) in the country on a MySQL server instance running on a MacBook Pro pulling data from Splunk in real time. All so I could generate some png charts and embed them in Slack.
I know there are limits to MySQL but so far in my career I haven't had any projects hit those limits because hardware has been getting better faster.