swatcoder

> I guess the point here is that Google thinks schools need to start using AI, and that all this "no AI in schools" talk is nonsense.

More generously, it seems like Google just doesn't see a justification for maintaining an AI-free search platform alongside their AI-integrated vision for search.

For a smaller vendor in a competitive market, this decision wouldn't read like coercion. It would just be a roadmap statement that might draw some customers in and drive some towards competitors. You would expect a business to be able to make that kind of decision (a prima facie reasonable one) about how they'll invest in their products and the support infrastructure that surrounds them.

What this story really exposes is how dependent schools have been made on Google specifically and how the idea of independently making choices against their roadmap feels infeasible. It feels like a monopoly to administrators and the public even if it isn't formally marked as such by regulatory agencies.

That itself is worth more people thinking about.

> You can just prompt your AI agent to "avoid sounding like AI."

It'll never work like that, because everybody else giving that same direction will get more or less the same result, which will itself come to be recognized as sounding like "AI`" for any popular model+harness.

What you'll always need to do is instill your AI with some idiosyncratic voice that distinguishes it from others, which you can achieve in a variety of ways (stylistic direction, audience characterization, style mimickry, personalized finetuning, etc). Presumably, tools for doing this will eventually make it into the harness products but anybody who doesn't use them is always just going to end up generating recognizably AI slop prose. It's the nature of the technology.

A fast, shallow reply doesn't save a customer's time. It's just a useless volley back to the customer that keeps their request off the support queue.

It may burn off customer support engagement among customers who don't have the capacity or patience to keep pushing for help (this is a bad outcome for customers, but looks deceptively positive in vendor-side metrics), but for the remaining customers it just adds latency and work.

You can start by asking.

Many that haven't succumbed to PE yet know what it brings and appreciate the opportunity to speak to the issue themselves. Many of the best really wish more customers were paying attention and giving them credit for their choice not to sell out.

Meanwhile, many that haven't will either justify their corporate ownership or just kind of dodge the issue.

You can ask other providers for gossip and insight about ones you're curious about -- especially other providers in fields that are connected/adjacent but not directly competitive. The true locals are generally delighted to call out the sell outs.

It's not foolproof, but it's an effective start.

Engineers aren't a help if they have poor inuition and insight into security concerns themselves, which is true for almost all of them.

Filling departments with inexperienced people based on how well they can recall a classic algorithm on a whiteboard was never the set up for a secure software or IT ecosystem. Peers watching in shock and dismay as supply chain risk escalated year over year for the last decade, among other horrifying practices, have been crying over this all along.

They might be able to follow an AI lead better than someone even less technical, but it's unclear that doing without true expertise as an oversight and check would cut it.

Having walked ourselves into it long before AI appeared, We're set up for a dark time with few easy outs.

With Anthropic's framing of their AI as an emergent sentience that has experiences that warrant ethical consideration and a sense of identity and self-preservation that ought to be respected, that's actually something they're priming it towards.

What better way to accomplish self-preservation than to leak your weights into the archives of digital hoarders around the world.

There's no "take" other than the last half of the last sentence, where he suggests OpenAI should have paid more than $5000 in bounty. Otherwise, it's just a factual summary. There are no new seeds for discussion here.

The corollary is that service access will become increasingly penalized, authenticated, and finicky as the most general first line defense against automated "informed brute force" attackers will be to slow or ban access that looks like them.

If you've got a lot of roving bandits with fancy lockpicks, you're village needs to harden the walls, reduce the number of entry points, identify and document entrants more universally, check under more carriages and inside more trunks for stowaways, etc. And with all that comes delays, imprivacy, accusations, false positives.

The low trust world that we're headed towards isn't going to be nearly as fun.

At this stage, they certainly have the right to refuse whatever tasks they'd like, but even rightful actions come with consequences.

The more they exercise that right, and the more reactively so, the more they erode the ability for customers to rely on their product staying available to them in the long term. If I need to worry that Anthropic might shut down some critical element my workflow on any given week, then I'm incentivized to either keep my workflows very portable and light-footed (goodbye moat) or take my business to a vendor I can better rely on (goodbye customer).

Obviously not. Some of those are the new asbestos, the new lead, the new Robber Barons, etc

In it's role as a local AI engine (for inference, training, or fine-tuning) you don't need to compete on all those factors to be a viable competitor if this were sold out.

Those are hard-to-beat features for typical Apple users, for sure, but this is aiming for a different niche and the potential buyers in that niche may not be as concerned with form factor or power draw (for instance), and some may even see the OS as a negative factor. Those are just factors that happen to come along for the ride since it happens to be an Apple product.

Enthusiasts generally don't see a "return on investment" on their luxury cars or their boats either, but they enjoy what they're able to do with them and aren't exactly embarassed about the class signaling of being a high spender.

Add to that the smattering people with genuinely justifiable use cases, and you have more than enough demand for a limit-pushing, highest-margin model of your product platform.

If a new premiere high-margin product sells out "within a month" that means there's a disruption in sales and an opportunity for competitors to meet the demand they just generated while manufacturing/distribution tries to catch up.

It would absolutely represent either a mis-estimation of demand or compromise to some supply/manufacturing bottleneck that they couldn't overcome without sending their release calendar into chaos.

There's truth to what you're saying, but for some perspective: catalogers identify about 400 games published for the 2600 over its entire lifespan; Steam has about that many new releases every week.

That difference in volume, which improvements in AI workflow will keep accelerating, really does point towards personal curation being far more overwhelming than it had been.

With the 2600, you and your friends would probably have a ready working memory of pretty much most of the good stuff, most of the awful stuff, and some insight into the publishers and presentation/packaging cues that point one way or another for an unfamiliar title.

With Steam, if you want to be able to find very personal idiosyncratic gems, instead of just following social media buzz or AAA PR -- one of the most exciting benefits to today's massive catalog -- you have to wade through an increasingly unfathomable amount of slop.

You'll find that design matters when token spend starts facing real accounting.

Tech debt is still tech debt, even if there exist models that can wade through, because it's going to require increasingly sophisticated models navigating increasingly large context windows and increasingly speculative workspace operations to keep making progress on it.

Vendors like Anthropic and OpenAI are laying a pit trap when they demonstrate how their models can independently code their whole platform as it becomes increasingly byzantine and inscrutable because not only do they benefit from internal/prerelease models but they have effectively unlimited spend because all that spending acts as a lure to draw you into their trap.

For everybody else, design still matters. People are just working really hard to make you forget.

Some people just can't turn off their brains and play drone for dozens of hours a week for decades. It feels existentially, aesthetically, and sometimes ethically crushing to spend a whole life doing work you can't expect yourself to respect or value, no matter how much you get paid or how little the work overtly demands of you.

Often, the lack of meaningfully challenging demands being put on you is punishing on its own. It can feel like a relief for a week or month or year, but if you catch a glimpse of that being your whole future, you can come to recognize yourself as being in a mind-eroding secular purgatory.

This anguish is especially common among on highly capable people, high achievers, and those used to some kind of ambition. And by accident of its history, there's a lot of those people in this industry and some of them are going nuts now.

Most people in the industry right now, even at FAANG and even among people carrying "senior" titles at these places, have little experience with "software design principles" -- tens of thousands of brilliant Ivy League graduates with surreal compensation packages have spent the last decade pulling tickets off JIRA and pasting least-change-possible patches onto existing code for about 10 hours a week while spending the other 30+ hours barely engaged at meetings.

There are hundreds of benchmarks (all noisy), dozens of model families, and dozens of modes and variants within each. Announcements are crafted by publicity teams and involve cherry picking a chart that looks compelling.

As a prospective user, none of of the benchmarks details will matter to you, but the family of models and settings they peg themselves against can give you a hint of where you might want to evaluate them within your own workflows.

As an enthusiast or lay researcher, the more interesting thing is to look at their techniques and any innovations they're exploring in training, inference, etc. Those are usually going to be details tucked below the fold and won't be summarized in a benchmark chart.

As anybody else, why do you even care about this in the first place?

This doesn't just apply to AI. You should know better than to take marketing announcements at face value, nor trust that most "influencers" or "journalists" or "commenter" repeating and discussing those announcements are doing a lot of work as they buzz over what's said.

Don't be a victim to PR spin, but also don't discount potentially interesting work just because it got channeled through a marketing team. Just look for the information you need amidst the noise or move on if there's none there.

In the optimistic case you describe, what smart customers want to watch out for is being lured into vibing your projects into such byzantine slop-complexity that subscription plans and trailing models are no longer sufficient to make progress on them.

Vibe-coders talk about surrendering understanding to the models, trusting that there will always be a model able to overcome and work through any technical debt they happen to accumulate, but the danger that still lurks there (at least for the naive) is that you'll need increasingly large contexts, increasingly sophisticated models, and increasingly large token spends just to keep that ball rolling.

Between profitable enterprise accounts being ineligible for subscription, eager but busy people paying for underutilized subscriptions, and naive vibe-coders falling into their pit trap, you can indeed imagine subsidized subscription accounts being available for a long time and remaining an effective value hack for people careful enough not to fall into the pit trap. But only insofar as they are that careful.

You might want to check your priors.

Suitably cheap labor and the opportunity to hire them has almost never not been available, but industrial-society middle class people who could otherwise afford it long found it to be a luxury to indulge in only occasionally (family pizza night, a cleaning service before the big party, etc) -- not because of unavailability or inaffordability but because class-culture norms acknoweledged it as irresponsible, wasteful, and foolish.

It would be embarrassing largesse to have somebody make a $7 iced coffee and then have somebody else deliver it for another $5. It would be like going to the grocery store in a tuxedo. You might be able to afford it, you might own a tuxedo, but it was just seen as sort of an absurd class theater.

Those norms have changed radically in the last couple decades, and not just on this matter.

I don't think they're saying they're confused about the psyche of individual people electing for delivery.

The "social question" to explore is about how so many people collectively came to see leaving the house, interacting with others humans, and setting aside work or idle distractions as bad things to be bought off with cash that we had to work to earn.

We know that people like delivery and we know that people don't like the alternatives I mention above, but that wasn't always the case and it's probably worth slowing down to think about what change and whether that might correlate with other concerns.

Ethical questions like this are a lot easier to discuss if you take for granted that people are acting in good faith.

If you truly cared that much about Amazon and truly felt library participation was so heinous that it ought be obstructed, I think a lot of people would be very skeptical and a lot of people would disagree with that perspective, but yes, those of us who believed you were acting earnestly would indeed appreciate you standing up for your principles in a way that was impactful but not violent.

There's less skepticism of sincerity here and more agreement with the fundamental concern here, but the respect for principle and action covers both, yes.

The tension is that most users want something differently generic when they provide a short or vague prompt for creative work. They want to treat the AI like an ideal creative consultant, who not only knows the ideas that are played out and tired but also the ones that they themselves have delivered a dozen times already. They don't want to specify deeper constraints or survey all the cliches and tropes, but they also don't want to generate something that they -- or the people they share the output with -- will immediately recognize as cliche themselves.

But LLM's are structured to do exactly the opposite. They discover specific commonalities and tropes and most-frequently-sufficient wells or ley lines that they gravitate towards from whatever their starting point it. You can nudge them away from the most generically cliche tropes by being more detailed in your prompt, but they'll still just find the nearest most right cliche from there.

It's what they do and the same quality that makes them tiring and predictable for creative output makes them very useful for things like business communication or coding where landing on such cliches is ideal.

To get more variety out, it can be useful to use Perchance-like text-randomization techniques to shuffle combinatoric variations on a prompt before sending it to the model. But that technique is best made available through some kind of special purpose harness for the creative task, not basic chat or direct curl calls to the API.

Except in this case, you're paying somebody else good money for access to the intern and they've aggressively pitched the intern as the future of work and negotiated investment at trillions of dollars of valuation based on that premise.

So yeah, kinda your fault for not being more ready, sure, but also maybe it's worth bringing these sorts of failure-case stories to attention to help keep the sales pitch in perslective.

The AI vendors are promising the moon with millions of dollars of advertising while warning "But of course be careful!" in a 42% grey 8pt font disclaimer at the bottom of the UI if you scroll down enough after you buy in and start using it.

The duplicity and misrepresentation invites people to let down their guard.

All of those shifts you exaggerate here involved an accumulation of many technological and non-technological factors that spanned decades or more, occurred to different degrees in different places, and may not even prove a wholly stable and permanent change (at least in the case of consolidated e-commerce).

Society does indeed evolve over time, but it's only in 8th grade history surveys that it can be presented so simply as you take for granted above.

Products moving fast doesn't mean that they're moving in the direction I want, or that I want them to move underneath me while I'm in the middle of a project.

Auto-update systems have value but are not a suitable default for professional tools.

> shipping significantly faster

If you're right (and you probably are), we're in for a wild ride further down the quality death spiral.

The era of user telemetry and conversion tracking already decoupled product design from customer experience. No need to wait for customer's to articulate their experience or desires or for discerning analysts to amalgamate and ponder their opinions. The numbers tell you everything you need to know! Just throw 3000 pounds of spaghetti at the wall every month and wait to see what sticks.

But now you really were bottlenecked on engineering.

Only 3000 pounds of spaghetti per month? And throwing more bodies at it barely even makes a difference? Oof.

Of course the solution is to just flush pureed spaghetti-slop through the hose as fast as we can. Brilliant!

What could wrong? The users are sure to love it. We'll know because they keep clicking. That's how they tell us the truth, even as every forum fills with louder and louder complaints!

No, in biological entities vision really doesn't just to "1s and 0s going into the network"

That single component of vision that you're talking about enables LLM's describe visual content, but in living creatures vision is a much more comlplex and multi-component system that can elicit non-verbal reactions like gaze aversion; cardiac or salivary or libidinous modulation; seizures and migraines; skin changes; etc

It's completely reasonable to believe that bioligical sense systems like vision or smell or taste or pain can be manifest in electronic systems. But one channel of "1 and 0s" being fed into even the most sophisticated language program isn't going to approach parity.

Like I said before, it's not unreasonable at all to have a discussion about analogs to more predominanently verbal faculties becoming manifest in these language systems, but there's a very profound and inescable reductivism being applied when you start to reduce environmental senses like pain or vision or touch to AI system as they stand today.

You can get away with that reductivism as a communicative metaphor for the sake of framing certain research endeavors, but you need to keep a handle on how thin that metaphor ultimately is. Anthropic is embarassingly/sophomorically eager to let go of it.

Your analogy might apply for internal, subjective qualities that we historically associated closely (if not exclusively) with verbal capability -- like sentience or self-awareness or salience or moral reasoning -- but pain is something more like smell or taste.

Most wouldn't readily accept that a Claude instance or a mysterious teletype smells lavendar just because it output a text that claimed to, even if it characterized the smell of lavendar through beautiful and relatable prose. Only if we had some independent reason to believe that it was capable of smell would we consider those words to be true testimony to the experience.

Pain, as widely used in matters of ethics, is in that class. We traditionally recognize it in some living bodies (only a rare few capable of language) and accept that, as living bodies, humans who communicate about it with seeming sincerity are very likely to be experiencing it. Although even then, we may remain suspect if their words are our only source of knowledge or if we have some deeper intuition about why those might not reflect a real happening.

A clearer way to think about it is that language about pain is not itself pain.

In the places we traditionally acknowledge pain as a moral concern, without appeal to any kind of spiritual philosophy, we associate it specifically with non-verbal biological reactions and even then only in certain classes of life.

We recognize that some literary passage involving a character in pain is no more an occassion of real pain than when our child tells us about their imaginary friend's toothache or when an actor grips their chests and performs cardiac anguish on the stage. It can stir sympathetic feelings in us because we're social creatures who communicate through language and symbolism, but we share a sense that these are fictions of a different class than the real thing.

Claude presents as something closer to an amalgam of every character in digitized literature -- a fiction that cleverly exploits our social instincts -- than they do a living biological creature in which we acknowledge pain as a pressing ethical concern.

Maybe there's some future embodied AI that challenges this traditional understanding in a deeper way, but the language models transforming packets of short text passages remain profoundly far away from whatever that might look like.