py4

Clarification: I meant interesting applications of LLMs to improve the “learning process” itself

tnx. can you share more about your game experiment?

I am not sure. Why do you need domain expertise beyond being able to draft a verifier for the problem? Once you have the verifier, it's just a matter of compute. You can argue that being a domain expert allows you to narrow down the search space for the LLM and save time/compute costs. This is partially true, but LLMs are getting better and better at search (they can already do end-to-end performance optimization faster than performance experts at a FAANG company I work at), and compute to maintain the same intelligence level is getting cheaper.

A concrete example: GPU performance optimization for a kernel. This was (and still is) a very niche domain with not many top-notch experts. But kernel performance and characteristics are easily verifiable. You can run the agent in a closed loop for it to improve iteratively (and people are already doing it, coming up with kernels better than human-written ones).

You see Tao's example because:

1. He is curious (so he asks detailed questions, which are not necessarily needed in a closed-loop optimization).

2. Verification in math is harder. Many math tasks used in RL are easily verifiable. But for advanced open conjectures that require long proofs, you cannot trust the proof directly from the LLM (so it's not as easily verifiable as basic math problems or code). The model needs to write it in Lean, and you still need to make sure the Lean implementation correctly captures the specification of the problem. So you still need a human for verification in advanced math. But I don't see why you would need this in domains like performance improvement.

I understand the negative sentiment toward Altman, but just to be fair, how do we define 'lying'? How does his behavior compare to that of the average executive, or even the average human?

I don't particularly like the guy; I'm just curious to see if he is objectively a liar in a way that most engineers, executives, or humans are not, or if this perception is simply a bias stemming from the general negative sentiment toward him.

you wait for it to finish then review or give feedback in the meanwhile? if it's the former, i'm curious about what you still do in the meanwhile

sounds like a plan haha. How is OpenCode vs Claude Code orchestration? have you ever needed to modify the orchestration since it's open source? do you mind sharing more about your experience?

I wrote my experience here: https://pooyam.dev/i-miss-coding

I was saying it was not commoditized a decade ago, but i feel it's getting commoditized *now*. So you seem to be basically saying SWE is over and it's time to move on to something that is primarily based on human-human interaction?

I honestly think a better title would be "how to stay relevant in LLM era"

simple example: Claude Cowork was written entirely with claude code

1. No, implementing well defined requirements were not commoditized a decade ago. You still have to come up with the design and proper (efficient,correct,...) solution that respects the requirements. it was and still is the skill set of a L4/L5 SWE.

2. If you think LLMs cannot help with navigating ambiguity and requirements, you are wrong. it might not be able to 100% crack it (due to not having all the necessary context), but still help a lot.

i'm trying to see what i can do to stay relevant

llm training/inference stack

LLM is not hype. it has made and my colleagues who are NOT working on CRUD, way more productive

Let's say you want to make an architectural change. There are two options:

1. Ask AI to come up with the different options and let you review it

2. You think about the options and ask AI for feedback

#1 is much faster but results in atrophy (you are not critically coming up with the architecture changes)

#2 uses your and AI skills but it's gonna be slower.

which one will you choose? currently i'm doing #1

if it's able to do the interesting/engaging part faster than me, i don't see why i should not outsource to it (The same argument as why use LLM-assisted programming at all, you don't want to miss the productivity boost)

Had missed it. Tnx