larsfaye

I agree, this really isn't surprising. They're pattern interpolators, and so much of what we deal with in the world is rooted in patterns of some type, including neural nets.

IMO, this is a similar behavior that we saw from AlphaFold, which also used a type of Transformer architecture to make its protein predictions.

All I keep hearing about is that coding skills don't matter anymore. Yet, it's these very skills (and my domain expertise), that constantly keep me out of trouble and able to use these AI tools and models the most effectively. It seems that these skills are as relevant as ever, and even the big AI companies won't hire you if you don't have them. In this video I dive into a personal recent experience, along with some interesting findings about what skills OpenAI and Anthropic are looking for.

>And the fact that I'm a generalist is really playing to my strength for the first time

I was just speaking with my wife about this a few days ago, that I am actually happy that I didn't specialize in one specific discipline, or rather spend my time around learning a little bit of everything, which now gives me the foothold into any direction that I want to go.

> It's time to reskill for AI.

I'm just curious, what do you think that looks like for you and for others? AI is not so much a skill as it is an interface, and one that is quite possibly one of the easiest to use that we've ever created. It is an interface for information, essentially. It's the skills that you come to the table with that determine the results that you're going to get back, and whether they're going to be quality or slop.

It sounds like what you're describing is similar to this new course that Theo Browne (ugh) and Kent C Dodds (double ugh) are launching called "mega.dev", and it appears that it's essentially just talking about the systems and tooling that's going to go into utilizing agents, and code is no longer even a topic.

> The long term impact of AI on society is exclusively a net positive if utilised in the correct way.

Hi, I'm Earth, have we met?

This is EXACTLY how I am working with these tools, you summarized everything about why I use this approach perfectly. Zed + OpenCode + Mimo/GLM/Kimi models are incredible. I haven't touched a "frontier model" for months.

I really treat them as "delegation layer". They're systems that sit right alongside your stack, and at this point, at every level in it, like a connective tissue that intersects at all points. I can delegate a spec, a function, a file, a whole feature set, an app, a debugging loop, etc.. My day-to-day isn't all that different, it's just like having the "Ship's Computer" that I can ping anytime I need something. I'm enjoying the benefits that a smart data processing pattern interpolator provides, but I'm not giving up any personal control or agency. My goal isn't to go faster, nor is it to avoid coding, because coding isn't drudgery or below us just because we have new code generation tools.

>I am glad that I live far away from the effects

Are you not on planet earth? Unless you're on Mars, you're not far away from any effects.

I originally found the paper through this video:

https://www.youtube.com/watch?v=HTUh0OO6Kmo

The video seemed to indicate it was a study they ran (she refers to the examples as "our subjects were two of twenty four participants"). It seems they are just citing the study. While it doesn't change the study or the takeaways, I'll update the post for clarification!

It's basically laid out in the article: the reason that you are able to gain so much from LLMs is because of your decades of experience that predated their existence. Ability begets ability, but if that ability doesn't take root, these more advanced toolsets don't yield the same outcome (and are likely to do harm).

To be clear, I feel the same way as you. I have been doing this for about twenty-five years, and I have developed a keen sense of taste and judgment to know how to properly scrutinize and utilize these systems. I definitely feel like I am able to learn things faster, but that is directly correlated to having experienced a lot of friction over the years that cultivated said taste and judgment in the first place.

I imagine it's sort of similar to someone who's just getting started in mathematics, but is introduced to WolframAlpha, versus someone with decades of experience in the field and what they learn with it.

I posted this elsewhere, but that story has a lot more nuance.

Plato (who actually wrote these quotes, because Socrates only spoke) wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.

Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse. When I look around, the world doesn't seem brimming with mindful critical thinkers and there's reasons for that (many reasons, of course).

AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.

May I ask how many years experience you have?

The end result is a total and complete reliance on AI providers. It's not really a secret; that's why I started the article with the direct quote from Altman. They would love to see a world where nobody is able to do any type of development work without an LLM subscription enabled...even though they also admit that these models perform best when they are steered by someone who is highly capable and knowledgeable in the first place. They're powerful, but they are also VERY fallible.

Local AI would thwart this vision (which is why they're desperately also trying to ruin the hardware market at the same time), but the effort it would take to get local models to run as well as these frontier models is an odd thing to pursue just to avoid learning the fundamentals and gaining direct experience.

Plato wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.

Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse.

AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.

That's because the institutional knowledge was passed along through the act of coding. Skills that I learned in Assembly and FORTRAN are applicable to even modern programming languages...but not to "natural language."

Exactly the type of usage I would hope to see advocated for in the future. As I wrote, when leveraged with Socratic methods and a sort of "interactive documentation", they can really boost one's understanding. Of course, this is predicated on the individual having enough information to know the right questions to ask.

Literally all those skills are the result of decades of friction, challenge, and deep domain knowledge...all of which are circumvented when LLMs are utilized early in the process.

CAD is also useless to Mechanical and Electrical engineers who didn't learn fundamental the skills that it streamlines.

Curiosity is really key, and I find that LLMs diminish curiosity in people that might otherwise have it because the allure of the answer is often a lot stronger than the allure of the friction. I guess that's just the distilling of the industry over time, but that could certainly lead to a shortage of expertise.

Do you think LLMs and compilers have a lot of overlap?

Math skills are at their lowest point ever in modern society, and many are still advocating for the reduction in reliance on calculators for this reason.

https://ivyleaguecenter.org/2024/03/12/over-reliance-on-calc...

Could be a doing a whole lot better. Do you think there's an abundance of critical thinking in society? Because math skills aid in that.

Actually, they have for a large majority of the population. And fun fact: that debate is still going on, and the Mathematical Association of America (MAA) enacted a rule prohibiting calculator use in AMC.

https://ivyleaguecenter.org/2024/03/12/over-reliance-on-calc...

Math fosters critical thinking, and we've been in a critical thinking deficit for decades and decades. I don't think we can afford to sink even deeper...

> I don't know how you teach that without coding

That's the neat part: you don't!

Even Robert "Uncle Bob" Martin, who's as AI pilled as they come and doesn't read code any longer, says that junior developers shouldn't even look at AI tooling for the first three years:

https://www.youtube.com/watch?v=RxxxGkFIUJ0&t=1356s

"How lucky I am to have something that makes saying goodbye so hard" - Winnie the Pooh

The extended title is more like:

"AI Coding will Prevent Expertise if We Continue To Focus on Code Generation rather than Socratic and Educational Workflow Design"

...but I had to compress it for brevity! The points and studies I bring up in the essay support what you're saying.

Problem is: I'm seeing very little movement in that direction, but a whole lot around the abdication of responsibility when it comes to AI coding in general.

Yeah...I know I should be massively impressed, and I am to a degree, but isn't this what we should expect? LLMs can pick up on patterns that no human can see. In that sense, they really are a type of "search engine", but I use that term loosely. Tao is using them as a way to sift and sort theories and information that the model has a plethora of training data on. It's like being able to converse with the codex of humanity, and extract data via an algorithm that is highly adept at sorting through it.

Correctamundo. That's EXACTLY what I just wrote about. And how we'll create the next generation of people who know how to evaluate responses when we're creating a generation of people who are increasingly reliant on LLMs to do the work makes for a strange paradox.

https://larsfaye.com/articles/ai-coding-will-prevent-experti...