Computers? 100 years old.
What part of any "modern" industry isn't still very very young? even industries like construction that have been around for thousands of years are still relatively young.
Computers and computing? infinitesimal.
For example, the newest gas turbine designs are already exceeding 50% of the maximum theoretically possible efficiency as allowed by thermodynamics. So it doesn't matter how many thousands or millions more years gas turbines are optimized, by us, aliens, future descendants, etc...
No future gas turbine industry in this universe can possibly double the thermodynamic efficiency of the finished product.
I was just listening to an AI podcast and they were duscussing going from the 1 second, 1 minutes, 1 day [unit of response - I cant recall the name of the measurement] -- but I assume thats the "moores law" of AI right now?
And as we get closer to the physical limits of chip production scale/die/etc - I assume they will be scaled horizontally while the GPT-X capabilities will scale volumetric-ally.
Is this a sound assumption?
For example, CNNs were largely known and discovered by the 1990s-2000s, but the compute simply didn't exist yet until GPU manufacturing became commoditized.
OpenAI's massive quantum leap is thanks to the massive corpora they were able to leverage, which until the past 5 years, simply didn't exist.
This is why we've seen massive jumps in Mandarin Machine Translation and Computer Vision from the PRC due to their massive corpora/dataset of English+Mandarin language news from CCTV/CGTN and local surveillance camera data respectively.
This compute limit is a big reason why the US Federal Govt has been working on the Exascale Computing Project for example.
Yes! thats what I mean by scaling VOLUMETRICALLY -- scaling horizontally and vertically are now replaced by volumetrically. (coining a term?)
its going to be an obloid-worping sphere-oid
Thats what I see down the pipe..?
Thoughts, anyone?
Even for training on a massive dataset, you are still limited by compute and processing time (good ole Computational Complexity), which is why HPC projects like the Exascale Compute Project were created in 2015 along with additional funding+research in efficient and alternative data structures.
I highly recommend going down the rabbit hole of High Performance/Accelerated Machine Learning.
The eminence of AI is more 'volumetric' than 'out'
Out scales "up and out like a hill or a lift"
- volumetric is spherical - it presses into the future AND the past (it already has been harvesting history, but created a fire-hosed spigot-interface for the future as well) -- and draws it into its center for eval...
They are different.
Always on the positive expressions of XYZ axis - but had ignored the negative of each, where AGI will go in all dimensions...
See the early shifts from vacuum tubes to semiconductors, from expertly-crafted BJT to crude CMOS, or the rapid march of good-enough architectures in the latest process node clobbering beautiful architectures in older nodes. As the industry has matured and course has stabilized, perfecting the design has grown in importance.
That’s how a company stagnates and eventually gets replaced. Yes operational efficiency and optimization is important, but you only do that if you have a big moat and not much growth ahead of you. You need to spend at least 30% on growth and innovation so you can stay relevant. Higher if you are the one trying to replace the 99% ones.
Not every industry is like software/tech. And the tech industry is already moving towards a "mature and competitive field" model. Otherwise why is Google and Meta cutting all their unnecessary spending and touting efficiency over growth? (besides AI tech, I suppose) Why all the layoffs?
Curious what was your experience operating in those fields.
Shit like "we did chain driven timing and it worked, but in new engine it prematurely wears", or "someone decided it is fine for belt-driven timing to take dip in oil, oh surprise, we now have parts of the belt landing in various parts of the engine".
Or my favourite, "metal, belt driven water pump is almost never replaced thru the lifetime of the engine and it's cheap, let's make propellers plastic and also drive it from separate electric engine"
That sounds like a mature and competitive field. I am suspicious of companies and fields where people aren't working like this. Look at the car industry, or energy, or farming, or shipping, or even aerospace. Only one of every thousand engineers at Boeing will ever decide the shape of an aircraft wing. The other 99% are there to implement and optimize its construction. Any company not spending 99% of its energies on optimization does not operate in a competitive environment and is therefore very likely on borrowed time. Eventually a competitor will appear or an IP monopoly expire and the easy times will be over.