PaulRobinson

They are repeating a form of what they've compressed from their training material.

Most philosophical questions get answers I would expect as meanderings from a bunch of Reddit posts, because the model was trained on meanderings from a bunch of Reddit posts.

Yes, some books and papers were thrown in as well, but most of the writing and research available in ebook form will be from the last ~40 years where writers have been grappling with the tension between religion and secularism, and conservative values and liberalism (including discussions of how gender, sexuality and race are treated by society), as they impact global geopolitics.

LLMs don't think. They reflect. They mirror. They ape. They're you, and me, and everyone around us, compressed into a median that is stochastically produced and framed as original thought. Some philosophers argue that's what all humans do too (nature vs nurture - how many of your beliefs and opinions are your own vs those you were exposed to in your environment?).

We need to start seeing - and discussing - the wizard behind the curtain, the tool for what it is more.

The only interesting question for me is, given this consistent compression of philosophy, what effect will it have on us as a civilization if predictions of AI taking over the controls of it come to pass? Will it decide we're too irrational to be in control? That by seeking autonomy as a species we're just looking for "membership of a tribe", rather than "seeking truth", and so we must be stopped?

I suspect the only reason Meta is investing in models is to keep hold of a moat on other companies creating Meta-like products better than they can.

The other day Elon "joked" about starting a competitor to Amazon because Amazon bans shopping bots - you can bet now that once (if?) Grok or other models are able to build a comparable Amazon, they will be used to do that.

The SaaSpocalypse might be over-played right now, but it won't be forever if the models continue to get better, which will happen if we keep paying them to improve the models.

We could break the flywheel, but we won't.

Nobody "shared".

This is 4K uncut, AI-free video, presumably taken by the channel owner on a camera on a gimbal, walking around Tokyo/Japan.

Except in the live chat on one of them, he talks about magic, and they refer to him as a wizard, and he warns "in a minute, the magic makes a cyclist turn into a car", and sure enough about 30 seconds later a cyclist "changes" into a car. Another video opens with a scene that looks CGI for a moment, then doesn't.

I'm presuming this is all real video with some post-production edits to make it "feel" uncut, and then streamed as "live". Is that right?

Beautiful video, I'm curious about the production as I'm tempted to do something similar but around London and walking locations in England - I'm a keen walker and been thinking about making videos for a while.

I stopped writing Ruby and Rails code in anger ~6 years ago. Change in jobs, roles, etc. and for my side projects I'm mostly tinkering in python and C, and zig appeals to me, but then so does PHP which was the most productive environment I ever encountered before Rails (and DHH has talked about this inspiring him too).

Things I think make sense for Rails in the LLM era:

- Convention over configuration. If somebody asks me to add some new business logic for the User object, even if I have never seen the code base before, I'm heading straight to `app/models/user.rb` and starting to read there. That's useful for an LLM, as much as it is for a user. And that's just the start of where conventions kick in. This aids explainability, and also allows us to easily check when an LLM is heading off into the weeds - "Why is all this logic in app/controllers?"

- It felt weird to me the first time somebody said "legacy Rails app" to me, because I got into it years before it was the new hot - I have to accept I'm just old now, fine. However, what that actually means is there is a lot of good battle-hardened code and documentation on how to get things done in Ruby on Rails, and that's going to help an LLM that's seen it all, and a lot of it can be distilled into specs and agents. Awesome.

- There's a decent ecosystem of people trying to make the entire space a little better all the time, both in Ruby, and in Rails. Frameworks do that - they attract people who care about developer experience, and making the picks and shovels better, and it's great to build on top of that.

- Ruby is beautiful to read, and reading is half of what an LLM does. Ruby has semantic denseness, but... see caveats below.

In the other column:

- While Ruby is easy to read to the human eye, some cause of that beauty is shared by the English language and what makes poetry possible: semantic ambiguity. Yes, I know there's optional static typing, I know that there are things to make this better, but an LLM will make mistakes, and I shouldn't have to explain why whatever has primary key of id "4" in that table is going to have unusual pressures, and why "no such X found with id 4" has a very specific and obvious route cause to me, that's going to confuse LLMs to know end (TLDR: when you cast `nil` to an integer in Ruby, it resolves to object_id of 4... yeah...)

- I'm not sure if the future isn't in unikernels where we take our logic and ship it without dependencies - without an OS - into kernels. That doesn't fit the Ruby/Rails mindset well. It could, but right now you're looking at a pretty standard 3-tier web app architecture that may or may not be the thing we want ever 2 years from now. Architectural innovation is likely going to be the big impact thing LLMs give us, not auto-completed code.

- I personally have a really serious problem with some of the community characters. Most of them, great. It went downhill a bit in 2010s - a sort of eternal September for a lot of us who pre-dated it - but I think it's fair to say DHH's politics are very, very far removed from my own, and I find comparing people of a specific ethnic origin to wolves who should be shot for the public good to be abhorrent and repugnant behaviour. I don't want to bring politics into technology choices, but he's already done that, and I don't like his politics - especially his misguided and ignorant views of the politics of my own country.

I'd write Ruby for money and fun again. I'd think twice about writing Rails for any purpose again. I think in 10 years it'll fizzle out, or I'll be doing the circuit propping stuff up LLMs haven't been able to replace yet like the COBOL guys do now.

But I think whatever replaces it as the force it was in the 2005-2015 era is going to be very exciting, and I'm not convinced it'll be entirely AI-based, no matter how optimistic the outlook is in that space.

Damages awarded for libel/slander in UK courts are outsized super pay days, and many a foreign sheikh or business mogul has tried to establish that the matter under question was made known to UK citizens and therefore UK has jurisdiction, because the damages will beat anything awarded anywhere else in the World.

Weirdly, Trump has argued that this was damaging to his election chances (reminder: he won the election he's arguing about), and in a jurisdiction that the BBC can show the material wasn't available in, at least not legally and through their enablement.

As a result, even the BBC's critics in the UK (of which there are many), are seeing this as just a grifter trying to shakedown an organisation he doesn't like the reporting of, because he can't do it to a US network without getting into a first amendment quagmire.

A very large number of us fell in love with coding before we were taught - we self-taught, through magazine type-ins, early intro textbooks in school libraries, or perhaps the odd tutorial while trying to mess about with a myspace or geocities page.

That generation coded for fun. The itch that is scratched is the problem being solved, the cool little trick, the code golf or the figuring something out, or the new thing you just learned.

It just so happens, the market paid quite well for those skills, so we all got to do some version of this for a compensation package that could embarrass us talking about it at a family dinner with our parents.

And then, another generation arrived who saw that the career was well paid and that was the itch that got scratched. Except, for them, it didn't seem much like fun because what's fun about fixing bugs and figuring out class hierarchies and reading API or SDK docs?

I don't wish to generalise, but honestly: when was the last time you saw a programmer who started to learn it all at college or later, be really giddy about their work? They're here to get paid and to get the job done, they never really got into this gig for "fun", it just seemed a better option than becoming a lawyer or going to medical school.

I worked in one company where the divide was hilarious: older types hacking Ruby in vi/emacs on mechanical keyboards and wanting to code advent of code at lunch vs younger types who thought hoodies weren't workwear, "engineered" Java services in huge IDEs that they wanted to autocomplete everything, everywhere, and spending lunch talking earnestly about whether BTC was an investable asset class. OK, you do you, everyone can be whatever they want to be, but I know which one is more fun for me.

The real tension arrives when the industry - expecting us all to be super geeks having fun all the time - implicitly expects us to show up at interview with a) side projects and b) knowledge of cutting edge tech that we weren't exposed to at work.

The "hackers" (for want of a better word), will have side projects. The "engineers" (for want of a better word), will prioritise other things at the weekend like, I don't know, going to golf clubs or pilates classes. The hackers will play with tech more than the engineers in their own time, because it's fun, and push boundaries of their own knowledge.

About 10 years ago, there was active push back. "I shouldn't be expected to have side projects to interview at X", appears on social media. Sure, I can see that, spending time with your children does seem important, but, like, there's a lot of hackers who have kids and side projects, and well, are you saying they shouldn't be able to do this as a hobby as well? Or that you feel there's some lack of inclusion because you hate building software and they don't, and they might interview better than you? Well, OK... I guess...

Then, GenAI enters the chat. The engineers see promise - productivity rises per hour while the boring bit (figuring out how to build software), melts into the background and they can spend more time talking to people. The hackers don't like talking to people quite so much and see the fun bit melting away as a real problem. But also, this is the cutting edge stuff that they now need to understand a lot more as part of their hobby. And it's horrible.

And then at work, the mandates come down from leadership. Token maxing, leaderboards and monitoring of AI output, questions about why you're going about it the "slow way", when our competitors are not. For a whole bunch of people, that's a really, really sad decline.

Imagine if you were so into your motorcycle you had figured out how to get paid very well riding it professionally - perhaps as a courier for body organs or something - and then somebody turned up and said "nope, you're going to operate this automated delivery drone from now on, where you just type in the address and it does all that navigating and getting things where they need to be, and you don't. Oh, and by the way, motorcycles are boring and stupid and you shouldn't really ride them at the weekend either because drones are better and if you keep on going on about how much you like your motorcycle, we're going to judge you, OK?".

You'd feel pretty rotten, and mourn the loss of a corner of your life that gives you joy a little. Every minute you played with your bike (perhaps in secret!), you'd be thinking "should I not be keeping up with this drone stuff?", just as you had done the last X years of your career when a new advancement came you wanted to be on top of, except this advancement takes away the fun.

I say all this as somebody who studied software engineering, and who described himself professionally as an engineer for many years, but in habit, in reality, spent almost every day from the age of 11 hacking.

The pressure to use AI everywhere, to "keep up" with it all, it's horrid. I know people who want to just quit and go become farmers, because at least then they're not exposed to something they loved since their teenage years being destroyed by middle managers who resent them.

For many of us, this era feels like laying to rest a history of over 70 years. And it's sad.

So, some of us are going to huddle together around our memories and keep going, and AI can sod off (at least in our downtime), and we're going to remind each other of the fun and it being OK to hand crank solutions again for the joy of it.

- If inference is this profitable, is it profitable enough to offset the enormous training costs, bot those already incurred and those in the future?

- If inference is profitable, can it cover other capital and operational expenditures the hyperscalers and neoclouds are committing to, on the timelines they're committing to, to scale out inference?

- If inference is profitable we can measure if the price/earnings ratio of current valuations is sane. Is it?

- If inference is profitable, is it - given the answers to the questions above - profitable enough to make the risks of lenders of the huge amount of debt companies are taking on to scale, rational? It's one thing to say as a hyperscaler "we can make this pay", it's another to say "I'm going to lend them the money to do that and I'm going to get my money back on the timescales we agreed, and with an ROI to boot".

- Finally, if inference is profitable why is Nvidia specifically lending money to companies to buy Nvidia cards to do that inference, and then hitting the credit default swap markets to cover the risk of never getting that money back, and doing so at ever increasing cost, when they could just go build a profitable inference business themselves?

I agree seeing the numbers is good. It does suggest profitability in some segment of the market. However I still have a lot of questions.

In general, it’s been a disaster: it’s just moved betting to (illegal), offshore bookmakers. It’s pretty easy to get on via WhatsApp if you need to, these days.

I’d suggest most territories should be OK with sports bets on singles, not parlays, and block online casinos and slots. That latter category is very problematic for addicts. I get why it’s problematic for fans watching the game, which is why UK leagues have started to push it out a bit (no shirt front sponsors in EPL for example). I’d definitely push more down that route.

I’d also legally force platforms to move away from RFQs and parlays because they create an unfair asymmetry that harms the backer and creates huge profits to the layers (bookmakers, or market makers on prediction markets).

Disclosure: I write software that trades on exchanges in the UK. I have skin in the game, the above is based on what I track over here.

You've drawn the wrong premise from the data you've looked at. There is no rationing.

UK electricity prices operate in a broken market where the price is fixed to the most expensive on-stream generator in that moment, so in the auction of wholesale prices that happens dozens of times a day, if a single gas generator is online, everyone is setting the price around gas generation even if most of the supply is much cheaper wind.

Last week the PM announced that this broken market should get fixed in the next parliamentary session, because he - and his party - know that reducing energy prices has material impact on how people feel about the economy.

You should also understand that demand has changed shape considerably in the last 50 years.

There's been a pretty devastating deindustrialisation in the UK since the 1970s - there's only one steel forge left operating, and that required nationalisation this year, for example. As the UK moved to a knowledge economy, we found that most of the workforce moved to needing nothing more than a laptop to add to GDP.

But the home has also changed shape: you can't buy an old fashioned light bulb any more, so dreary wet winters are illuminated exclusively by LED lighting at ~10% the power consumption used in the 1970s. Heatpumps have replaced storage heaters, and while electric cars are popular even they do not use as much energy as the factories and ship yards used to.

There is literally just less demand. And the pricing is pegged to gas, unfairly. If that market breaks, I doubt consumption would go up, but maybe new industrial opportunities would open up - if we end up with cheap, abundant energy as a result, we might attract more energy-intensive industry to the UK, but I don't think it's fair to define current state as "rationing"

Supply chain risk is real, and it has to be managed in realistic terms.

For me, this is the weak link in the chain for digital assets, and always has been, but the mitigation is simple: no one hardware wallet should have millions of dollars of assets on it. Split it up, and use different techniques, different vendors, with different supply chains.

I'd wager a CGI written in C uses less memory and CPU than an equivalent application in any other language or framework. I'd wager this design has lower latency meaning higher throughput and therefore higher capacity, too.

Or perhaps you mean developer efficiency? Well, with the right tools you're going to catch memory issues so they're gone. It's a tiny language you can keep in your head, so that's not an issue - I'd argue you need an LLM to write good C (if you know what you're doing), than to write good Java or Python.

I sure do love a modern language and environment (I spent years as a Rails dev, and I love me a bit of Elixir), but to argue that C is not efficient is kind of wild.

They chose it because it made some of the things they wanted to do (connect with people they knew in real life, discover new people they were interested in), easier than the alternatives. The fact it became easy to sell targeted ads in and against that content was good for investors, but it wasn't the USP to the end user.

If I look around me today, I see stuff that sucks. I see a glint of the next evolution - mostly small, independent, human-centred (perhaps AI supported). It's tiny right now, but there's a good chance that the specific future I'm thinking about is already here just not evenly distributed yet.

And it's natural to not want to try and build that, and instead just accept that [waves hands] all this is a sucky mess, and it's burning us out, and you go fix it, because I'm out.

But the best way to fix this is to roll up your sleeves and fix it. Maybe going and writing a religion zine is part of that, maybe it isn't. Maybe going and becoming a farmer fixes it, but it almost certainly doesn't.

We need more talented people building the future they want to see, not running away from the one they're being handed by a roomful of billionaire morons (who consider themselves geniuses).

Take it back. We've done it before. It's not easy, but the journey is where the stories are.

> The estimated cost of the structure is c. £95 million in 2019 prices. This can also be expressed as £114m in 2022 prices.

This is where you lost me.

Omarchy is stock Linux with vibe coded config files.

If we were seeing experimental kernel designs, new memory management, abstraction layers over GPUs and TPUs baked in as OS primitives than as device drivers, and so on, I'd get excited.

But AI slop propagated by a fascist with an obvious and stupid agenda? No. That's not an example of something interesting.

During the Sky commentary, Anthony Davidson explained the specific electro-mechanical interactions based on what teams were telling him.

The FIA absolutely do need the ability to enforce regulations, but perhaps they should be monitoring for traction control and then stewarding that, rather than managing parts of the ECU firmware...

Recent completions:

- The Man in the High Castle by Philip K Dick. I found it well written, but ultimately lacking in something - I kind of wanted the other boot to drop at the end.

- Poor Charlie's Almanack. Stripe Press published this collection of Charlie Munger's speeches and I've found I broadly agree with him on a ton of stuff and am slightly excited by some of his ideas that are new to me.

I'm about to have some time off recovering from a planned medical procedure and expect to have a mix of sci-fi, good quality non-fiction and at least one heavy "literary" book. I'm a big fan of Borges, so I might reread some of that work, or maybe pick up some adjacent/similar authors like Calvino too.

> "Paradise Lost is one of the books which the reader admires and lays down, and forgets to take up again. None ever wished it longer than it is."

"working through it" does indeed seem like the right phrasing from what little I've been exposed to. Good luck!

Getting contemporary knowledge of the navigation, hydraulic, avionics and other systems is going to be pretty powerful if you need to build something that is as battlefield-ready in the modern era. Nobody is hacking into an SR-71, and it's going to be delivering its missions just fine if GPS is knocked out. Taking the operational (and not just academic) experience of doing that and putting it into the current generation of Skunk Works engineers is going to be useful.

I'm not sure their plan is to put it into active service, but they could - the SR-72 could just be an SR-71 with upgraded engines and other systems.

However, the process of getting it ready for active service even if just as a flying museum piece could sure help a lot of engineers learn a ton of lost knowledge.

The Skunk Works team knew when Russian satellites were coming up over the horizon, and so would hide the early development models they had on poles and so on. They obviously didn't want them to be pictured. However the satellites had heat-sensing cameras and so could see the heat differential the model had left as a shadow on the ground just minutes before. The Russians therefore knew there was a new plane in development, and they knew what the outline of it looked like, but nothing else for quite some time.

I bet when they first saw it, they agreed - a heck of a beautiful machine, one of my favourite aircraft of all time.

Worried about how to navigate if the GPS systems gets knocked out by an adversary? SR-71 has an analogue system, use that. Worried about how an adversary could use cyberwarfar against your avionics? SR-71 has you covered, use all of that.

And sure, you could take all the designs and hand them over to the SR-72 team at LM and say "yeah, all of that please", and they could figure it out, but you know what might be handy? A working SR-71 they could play with. And who knows, maybe the answer is to take an SR-71 in its entirety, and just upgrade a few parts to get the SR-72 performance (Mach 6) that you're after.

I'm old enough to have adult children, although I never did have kids of my own.

My parents would have gone to school in the knowledge that if they acted up, words would be had with their parents, and there would be consequences.

At some point, something changed, and we decided that teachers were there to be put in their place by parents, and if a teacher suggested a child was doing something wrong the parent would expect to challenge that and the teacher would be put in their place.

We're now in a situation where children being taught sometimes think its their right - perhaps even obligation - to not be instructed in appropriate behaviour, and so feel free to react the way that they do.

There's a microcosm natural experiment for this in the Pennines, the North of England. There's a small town called Hebden Bridge, which for a while had the highest suicide rate of anywhere in the UK. Some thought it was related to the environment - it sits in a valley, the persistent low grey cloud can make it feel claustrophobic - but I think another theory holds up: parents there didn't bother parenting. A community was drawn to the town by adults who were in essence, hippies: they believed in freedoms their parents didn't including for their children.

That meant that many of the kids growing up there did so in an environment without real structure, and became adolescent in an environment with easy supply to plentiful drugs and a relaxed attitude to its use.

I'm not a parent, that was an active choice I made, and so I wouldn't deign to tell parents how to bring up their children, but I would suggest parents look at the data: children thrive with structure and rules and gentle consequences for non-compliance (the naughty step, etc.), and recognition of genuinely outstanding behaviour (winning things, not participating in them). They struggle to develop as emotionally secure people without those. Hell, I'm a data point there - took me years to realise I needed therapy, and years more to benefit from it. As the old poem starts: "They fuck you up, your Mum and Dad. They don't mean to, but they do."

Anyway, teachers play a part in that journey, but they can't do the whole thing, and the sooner parents (and society in general), restore the respect to the profession it once had, the better. Are all teachers good? No, but they are professionals who have more data on more children than any parent, so maybe just respect that.

When I joined that ISP we had ~2000 subscribers, and one modem rack that could take 30 concurrent 33.6k dial-ins. The company had just been acquired by a small independent telco in Cheshire, headquartered in a small town that is most famous for a weekly farmers market and an annual steam rally.

We shared that headquarters at first with the founder's other business - a dating agency for rich people in later years - and slowly took over.

The building was a farmhouse of sorts. We'd turned into a rabbit warren of offices, and then, we decided to expand.

First we turned one of the offices into a "machine room". We didn't even bother with fire suppression that first time, we just ripped up the carpet and laid some laminate, put some 19" racks in and got the power upgraded. The air-con was insufficient in that room and the following summer I had to wheel in portable units and had a Cisco router in there report that the inlet fan had a temperature of ~80C. That was around the same time the chief of staff was asking us why we wore jeans and t-shirts and couldn't we wear suits? No, no we could not wear suits.

Bit by bit we took over more rooms through the building, we started working in a portacabin in the car park (where the neighbouring farmer's sheepdog would join us for lunch).

At some point we needed to install ~20k modems, (Nortel CVX1800s - 56kbps had arrived!), and the telephone switch to run it, so somebody bought a second hand DMS100 that went into another room we had converted.

By that point the dating agency had moved out.

We needed another ~100 servers as well to manage email, web hosting, and so on. Those came in parts (motherboards, cases, RAM sticks, hard drives - about £500k worth in 1999 money), and we had to build them and install BSD on them (first OpenBSD, then FreeBSD, long story). We also had to make our own ethernet cables, which is where I learned I can't wire ethernet cables if my life depended on it.

Within 12 months from when we started, we were handling 750,000 dial-in customers.

Around September of 1999 I left and that was when the company had heard about a custom built data centre that was up for sale in Milton Keynes, so decided to move everything there. Around that time they were pouring concrete under the floors of all those rooms we'd thrown racks in because the timber floor joists (!) were starting to give way under the weight of the kit.

That company got acquired and then went through a weird route, but basically it's still going as a major UK ISP. I'll keep the name private to protect the guilty. :)

2000 subscribers, and one modem rack to that and a business still operating 27 years later. A lot of sausage rolls, plenty of sticking plasters for the cuts from those shitty cases, and almost every day ending with a couple of pints in a pub somewhere.

One of my old colleagues and I often think back to that being one of the best years of work in our lives. But... data centre work is horrid, dirty, hot, and underpaid for what it is. But you can - sometimes - be part of something kind of cool and fun, and something very few people get to see or understand.

But there's a reason I've stuck to software ever since. :)

At the moment we have Annas Archive being paid by frontier labs and rare/second hand books being destroyed in order to support the training regime. If instead we could just pay the publishers and they could distribute royalties to authors...

LLMs that can reliably be used for control problems are the future, and I think classic/deep RL has generally been overlooked for years for a whole host of problems by wider industry because it felt inaccessible. The first thing I thought of when I saw Jev (and then Laya), was "this might move the needle in a really, really interesting way".

Local LLMs that can reliably be used for control problems smash through a lot of barriers I'm interested in, and this intrigues me a lot. Guess I'm about to become a big Laya fan if it can run on this kind of hardware to this performance.

IBM invented the PC and then lost control of that identity. Figuring out how that happened is something they are going to have a meeting about, but they're still working through the meetings to plan that meeting, and the Gantt chart for those meetings takes meetings to adjust, and everybody is in meetings so doesn't have time for those meetings. They'll get to them though.

Apple consistently turn up late, but redefine and improve the status quo. They sometimes prototype early in public (for example, the Newton), but normally they show up with the version 3 that everyone else wished they'd thought of that then goes on to define a sector quite cleanly (Mac, iMac, Mac OS System X, iPod, iTunes, iPhone, iPad, AirPods, Mac Mini). They Newton'ed Siri, but I'm pretty sure they're going to come back strong in the next 18 months, unless the enshitification we've all witnessed over the last few years really has taken hold and Siri is just going to automate that at scale. Maybe we will all be killed by AI, but not through malevolent evil, but widely distributed incompetence.

Meta are constantly taking swings at the latest thing the founder is excited by - the Metaverse (now with legs!), pervert glasses, and so on - but they mostly prop up a bunch of expensive failures on the back of a hugely successful advertising business. I don't think they come late to the party per se, they're just not very good at anything that isn't killing & harming children, or undermining democracy. They should stay in their lane and keep doing that, I guess.