flatline

It’s written in the first person and does not have many of the AI tells that I’ve grown used to, but a couple paragraphs in, I am inclined to agree that this is largely AI generated. It’s like the written version of a monotone voice: it carries on methodically for way too long without any real build up or character.

And now I wonder if I’m just harping on some poor human who needs to spice up their writing a bit. Was there any one thing that gave it away to you?

Yes and no. DS9 plays with things at the edges, much like Ian Banks’ novels deal with the behavior that lies at the edge of the Culture and is part of what permits a utopian society to exist at all. It cuts directly to the question of when and if martial force —or other, more subversive mechanisms of power — must be deployed for protection against a force that threatens the existence of an otherwise non-violent society. I think these are criticisms of visions like Roddenberry’s without necessarily undermining them or missing the point entirely. I also think it’s been an ongoing commentary on modern civilization as much as speculative fiction.

Some of the other series and movies miss the point more dramatically for me by just becoming space opera. Star Trek was always political.

It's not a store of value by these metrics. If your primary concern is being able to store assets that cannot be seized or frozen by, say, foreign adversaries, it suddenly becomes much more viable. Occasionally up or down 50% is a lot better than down a guaranteed 100%.

I'm really torn between these two perspectives. You are not wrong about monopolies. And yet, when Google and Apple struck that deal I thought, thank god I will not be forced into using whatever Apple has to offer up as a search product.

The tooling around LLMs is still nascent and, in my opinion, quite poor. You may be able to shotgun easily-falsifiable mathematical proofs with raw compute, but get into the messy world of real data, uncertainty quantification, and a host of other things that go into building real world models, and I think that an LLM plus tooling dedicated to the task plus a human in the loop will open up capabilities not available to an LLM in a chat or development centric software environment.

I had a conversation about a month ago with a friend who had gone through menopause and I realized I had no idea what the ovaries did when they stopped producing eggs. She didn’t either! I can see why it isn’t the most obvious target for research when the primary focus is fertility, but I do find it fascinating that women’s bodies stop the reproductive cycle and keep on kicking for decades. So much complexity, it’s quite the evolutionary trait.

MaM actually has one of the higher bars, but they have the most regular signups and the community to support it. That’s a good point of reference from 10 years ago, though.

Seedpool has regular open registrations, no fee, lots of freeleech content. You can pay $10 for a VPS that gives you a TL membership and get 5 TB upload credit with a couple hours of effort. All you have to do is seed and sign in like once a month to show you are still active.

I went the route of handling epub first (really, anything pandoc can convert to epub) on ebookaloud, and it took quite a few iterations to get consistent parsing across a wide corpus of test documents.

My experience is that gathering and testing with that corpus was about 30% of the overall effort. I have not yet gathered the motivation to tackle pdf: it’s more of a layout format than a text format. Not just tables but multi-column layouts and similar oddities leave a pile of imperfect heuristics in their wake.

Multi-lingual support is even worse. I was working with arabic script in pdf years ago and they don’t even capture the encoded text, just rendering instructions due to the high number of ligatures. I don’t know if things have improved since then.

I have no problem tipping for service. Tipping when I pay in advance and get no service beyond being handed my food — what exactly am I tipping for? I don’t think people are opposed to tipping in general, it’s just been applied to everything.

One time I was forced to pay a tip for self checkout at the Austin airport. I was in a rush for my flight, it prompted, and I could not find any way to opt out.

We're sitting on a year of more or less capable coding models and I have not exactly seen a revolution in new software being released. AI is likely a force multiplier for specific subsets of individuals and workflows. Coding is not a bottleneck for a lot of systems.

I suspect that SaaS is silently being eaten alive as an industry.

There are numerous problems with “alignment.” What are “human values” to begin with? He outlines some at the beginning of the post, implicitly: build bigger, better, more powerful things faster without adequate safeguards. We are literally pouring trillions of dollars of value into this enterprise, and I would say this is something that many humans also value in a qualitative sense. Then we have explicit values which in the West are largely rooted in Christian morality. Nietzsche circled this dichotomy two hundred years ago and I feel like what we have gotten since then is an increasingly detailed anatomy of power as the basis for what is normal vs deviant behavior. He who has the power, makes the rules, to be reductive.

I do think this carries some weight from this particular author due to the length of his tenure. I happen to agree with him in spirit, but this is still largely a post revolving around sentiment not substance. Does anyone think that the overriding incentives even leave room for something like this in practice?

Meta, ByteDance, ...

I don't think the discrepancy is in LLM capability improvements over the past year.

Correctness has never been a priority across an industry where rapid iteration and feature delivery drive sales. There's always some opportunity cost to doing things right, at the price of technical debt down the road. If AI is primarily used to produce fragile code, people will be wary of AI solutions. There's also ongoing public debate about AI safety and alignment. Deploying AI in safety critical applications feels riskier than ever in the current environment, even though it doesn't have to be.

SQLite FTS could have done this a decade ago. We've had good local search capabilities for two decades and they either been underused or abandoned (e.g. Google Desktop, Yahoo! Search). This may be a reasonable projection of where AI is headed. You can do a lot locally but there is too much incentive to centralize around cloud infrastructure, then the privacy concerns make that prohibitive and we end up with what feels like a false choice of cloud or bust.

Maybe it's LLM hype that will bring more powerful capabilities to the desktop?

I think that's a bit strong of an assertion. How many people use copilot daily under an enterprise agreement? I don't necessarily disagree in spirit, especially given the questionable data sanitization around the recent Navier-Stokes announcement, but most companies disclose huge amounts of data regularly to hopefully-trusted third parties. I think the internet -- and a good share of the world's commerce -- would grind to a halt if we suddenly stopped. Setting up, securing, and maintaining local models for even a small user base is non-trivial and there is way more demand than supply for that skillset right now.

Exactly. It’s still just software, which someone programmed and deployed to do specifically dangerous/malicious things. I feel like we already have legislation and case law surrounding this.

Does mathematics still have a practical impact without humans in the loop? I don't think there's one single answer to that question, but I think it's worth considering exactly what that impact may be.

Tech companies are as much the topic of this post as AI, I think that's the immediacy.

Isn’t the implication that the vulnerability had already been found and exploited instead of reported to Google, because its value to Google was so low? Whoever found it originally had more incentive to sell it on the grey/black market. Or is the incentive structure truly different for vulnerabilities discovered to be already in the wild?

It’s Foucault’s interpretation of the panopticon. If we know that we may be watched — and punished — we self-police into following the norm, even if we do not know when we are being watched or that someone is necessarily monitoring us individually.

When I was a kid in the 80s none of this was an issue and I was free to roam. Starting in the 90s this became an issue and not only will parents not let their kids roam but the kids themselves have less interest because the norm is to stay in or only go out when supervised. You know you are doing something slightly wrong. It only takes occasional reminders like this to reinforce the rule, once established.

Same, this is like blogspam in the days of old that doesn't add anything to the original content and is just there to generate revenue for this site. The linked posts are better and per HN guidelines the toplink should probably be changed to one of those.

General doesn’t mean it applies to literally everyone. If autism turns out to be a specific condition that is e.g. the result of abnormal neuron pruning during early development, it could have a wide array of symptoms, but that understanding may someday lead to genetic treatments for sever cases, better early childhood interventions, etc.

I do think that the current ASD umbrella as written in the DSM is so broad as to seem nearly meaningless when take in isolation. It’s not meant to be in isolation however, it’s defined first and foremost as a disorder, so the symptoms need to cross some threshold of tolerable. It’s also associated with a growing body of research as to how nervous system development can cause related clusters of symptoms.

Is autism as we understand it scientifically accurate? Probably about as much as anything hovering in the intersection of social sciences and medicine. It does have some predictive power. If you meet the diagnostic criteria for autism you are likely to have certain comorbidities for example. Having certain autistic traits are good predictors of certain outcomes later in life. And we do have better therapies especially for young people than we have in the past.

The fact that people can have non-overlapping clusters of traits and still be autistic does seem problematic. It may not be the right model. Just my $.02, I’m not the autism researcher in this convo, but I am autistic :)

This line of thinking diminishes the real struggles autistic people face. Not everyone is autistic, and it's not just a quirky set of traits. I do believe society could be better arranged for people who for whatever reason don't conform to the overculture norms, so we could talk about disability along the axis of how society privileges or disadvantages certain individuals.

The concern is that autism as a rigorous category may be useless, but that doesn't mean that autistic people are suddenly neurotypical -- a term that is more politically than scientifically oriented. People who are not autistic broadly are allistic, to be precise with language. They may be neurodivergent in other ways.

I’m not sure why you were downvoted. I have to use Windows for work. The barrier to entry is simply lower for many system administrators and users alike. The homogeneity of a Windows rollout is extremely attractive. You are going to have a hard sell getting your accounting team and receptionist onto Linux. Your developers will complain and make do. I love complaining about Windows!

Their sandbox just sucked, the agent found a route through artifactory which was also connected to the internet. After the agent breached it, they put it back in the patched sandbox and it found its way out with another artifactory exploit — which was still connected to the internet.

No attempt was made at an actual air gap as far as I can tell. OpenAI is dangerous and irresponsible and doing a lot of pearl clutching over how their cybersecurity agent is doing cybersecurity things.

I think I agree with you in spirit. I don’t like what these companies are doing, and Anthropic’s actions can for the most part stand on their own. Copyright law is just a special interest of mine, and I do think it’s important to recognize what external incentives exist and what they prioritize. Because other companies will act in similar manners under the same incentive structure, and the problem is going to cascade and magnify if it hasn’t already. There are active court rulings setting precedence for this behavior - take note!

Why would a company keep the hard-copy around at the risk of it being inadvertently given away, resold, etc.? It's a huge outstanding liability given that the illegal copying of works -- the other part of that case -- is what they settled out of court for some huge amount of money. Destruction is the only thing that makes sense.

I'm old enough to have been around when DCMA legislation was under discussion. Many people were dead-set against it and raised concerns over matters exactly like this. In Rainbows End (2006), Vernor Vinge wrote about a similar scenario where a robot went through the university library shredding books, and scanned the shredded pieces to recombined them into a digital archive.

Anthropic may be doing shady things and may have even done this on their own recognizance, we just don't know. As it stand, this is 100% a consequence of US copyright law, much of which was written by large corporations to protect their own assets.

They cannot scan the books then resell them or donate them under current US copyright law. It’s not clear to me that they could warehouse them if they wanted. In the recent Bartz v Anthropic case the judge ruled this destruction as legal, saying

> The print original was destroyed. One replaced the other.

So that there was still only one “copy” of the book. This is in compliance with the DMCA. You can make a personal digital copy of a work but then you cannot resell the hard copy and keep the digital one. Same principle applies here.

This is why we have hierarchies of abstraction. Pretty much every field of mathematics relies on constructing notations, models, and other tools to simplify things in a way that is verifiable. LLMs rely on the same basic technique, they can just pull from a wide variety of these abstractions at once. So far we've been able to understand their proofs just fine. Computer-assisted proofs in the past that relied on brute-force is where we have run into trouble. We cannot reason about millions of possibilities at once, and we had to trust that the computer program that analyzed them was correct, which is a really hard problem and leaves humans fairly unsatisfied. I think we are actually progressing in terms of understandability in computerized proofs.

You deploy an application that ostensibly has the desired capabilities but it fails at scale, or the architecture is not extensible to add new capabilities or fix bugs. Those are expensive mistakes and in safety-critical domains could be deadly.

The thing is I've worked on plenty of code like this written by humans. AI just accelerates the process and puts this type of code within reach of more people. It could be seen as a step backwards in good software engineering practices we've painfully learned over the last two decades. I see it as a cycle repeating itself in a new iteration, with the hope that we'll end up with still better tools and processes in another few years. But I agree it can and will result in real harm in the meantime.