crackalamoo

Yes, this is essentially it.

As a corollary, the output classes can be any set, rather than needing to be set before training.

An raw LLM no, but given code execution it can do a good job. I had my Claude Sonnet write a sonnet in this form after 14 rounds of iteration with Python:

The autumn wind moves slow across the field,

and every falling leaf now yields its fight.

The summer gold at last has ceased to yield,

and short days now give way to longer night.

I walk alone beside a calm and quiet stream,

and watch it slowly carry every fallen leaf.

I think of you as some half-forgotten dream,

and taste again that same old, bitter grief.

The silent stars come out to watch it still,

and pale moonlight falls gently on the hill.

A lonesome owl calls out from past the mill,

and time moves on, unhurried and calm, until

the dawn returns to paint the sky brand new,

and I still recall the love I lost with you.

I love this, easy intuitive explanation

I find it works as focus when the tasks are all related/in the same repo. Not the same as single-task flow state of old style coding, but also still a kind of flow

Interesting, how common is this vs just unit testing? How do you avoid formally verifying something against a spec that could subtly fail in production?

This is super fun! Very elegant idea

Has anyone done this for larger neural nets? Is there a way to extract some kind of pattern or is the image just noise no matter how you construct it? I'd be curious to see something like that

Isn't this kind of the same as an AI copilot, just with higher autonomy?

I think the limiting factor is that the AI still isn't good enough to be fully autonomous, so it needs your input. That's why it's still in copilot form

This seems like a solvable engineering problem. For example, you could have a lightweight subagent with its own context for reading the skills and determining which to use

I also use pure HTML and CSS (and a touch of hand-written JavaScript)

I'm a little skeptical of AEO. What's the point if AI users just ask the LLM to retrieve the information and never visit your blog? I almost never click the links ChatGPT gives me

Maybe it makes sense if you're selling a product or service, but I don't see the appeal of AEO as the new SEO. Maybe I'm missing something?

My two cents: if you're not doing anything too political or controversial, it's fine or even beneficial to mix in the occasional personal essay with the professional.

After all, many of your readers are also human beings with lives, maybe even lives similar to yours based on your professional content. (The rest of your readers are LLMs.) Your readers might appreciate your perspectives on random life things or just getting to see what their favorite blogger is up to.

I make heavy use of the "temporary chat" feature on ChatGPT. It's great whenever I need a fresh context or need to iteratively refine a prompt, and I can use the regular chat when I want it to have memory.

Granted, this isn't the best UX because I can't create a fresh context chat without making it temporary. But I'd say it allows enough choice that overall having the memory feature is a big plus.

Grade inflation is common at many schools. And many difficult technical classes grade on a curve, sometimes to the point where you can get an A with an 85%.

But yeah, I still don't see how an 85% average would be a 4.0.

Why is dependence on LangChain an issue?

Not that I disagree necessarily, just wondering if there's a consensus that LangChain is too opinionated/bloated/whatever for real industry applications, or if there's some other reason.

The recent Apple paper seemed pretty flawed. Anthropic/Open Philanthropy did a rebuttal paper, The Illusion of the Illusion of Thinking.

> Do they reason? No. (Before you complain, please first define reason).

Defining reasoning is the problem. No, they don't reason in the same way as humans. But they seem to be able to go through reasoning steps in some important way. It's like asking if submarines swim.

Not sure how tools like Cursor work under the hood, but this seems like an easy model context engineering problem to fix.

I agree. And if human civilization survives, your concerns about energy and resources will be only short term on the scale of civilization, especially as we make models more efficient.

The human brain uses just 20 watts of power, so it seems to me like it is possible to reach human-level intelligence in principle by using much greater power and less of the evolutionary refinement over billions of years that the brain has.

I agree humans only rarely step outside the circle, but I do have this intuition that some people sometimes do, whereas LLMs never do. This distinction seems important over long time horizons when thinking about LLM vs human work.

But I can't quite articulate why I believe LLMs never step outside the circle, because they are seeded with some random noise via temperature. I could just be wrong.

Yeah, unfortunately I feel like despite all the advances in Unicode tech, my modern terminal (MacOS) still bugs out badly with emojis and certain special characters.

I'm not sure how/when codepoints matter for wcwidth: my terminal handles many characters with more than one codepoint in UTF-8, like é and even Arabic characters, just fine.

Yes, 100% this. And even more so for reasoning models, which have a different kind of RL workflow based on reasoning tokens. I expect to see research labs come out with more ways to use RL with LLMs in the future, especially for coding.

I feel it is quite important to dispel this idea given how widespread it is, even though it does gesture at the truth of how LLMs work in a way that's convenient for laypeople.

https://www.harysdalvi.com/blog/llms-dont-predict-next-word/

I use /foo/bar/ with the trailing slash because it works better with relative URLs for resources like images. I could also use /foo/bar/index.html but I find the former to be cleaner

I don't think this article is paywalled? You can just click "I'll do it later" on the banner

To some extent, AI needs less extensive teaching than the problem-solving skills taught in school because it works in natural language anyways and builds off whatever else you know. You can easily talk to an AI about CS if you know CS, or even use it for coding, but not so much if you only know "AI".

It's not that learning AI isn't important, it's just that school isn't necessarily the place for it: AI is always changing, and can be learned relatively easily online, while properly learning something like linear algebra or operating systems online is pretty unlikely except for a very motivated student.