Buttons840

I hope there's still value to be found in a bunch of developers standing around a whiteboard and talking, and I think this scenario is where their ability and understanding really shows these days.

If we don't value that though--if we don't value understanding the system and just thinking through problems as a group--then let's stop pretending and just not do code reviews. Probably nobody cares and just types "LGTM" after having their own LLM do a spot check to cover their ass.

It's really easy to push developers to stop caring. If management says "go fast fast fast, use those LLMs!", then developers can't keep up, get frustrated, stop understanding the system, and at that point they're just there to collect a paycheck before society collapses. Or, if management is making dumb decisions all the time, same thing.

Hopefully as implementing ideas takes less and less time and resources, smart people who actually care will start building things with other smart people who care.

A lot of top people are going to be having an ego crisis I fear. We're all just human idiots looking to enjoy time with each other while we can. Fortunately, that is enough for a good life.

Why some of us work more and earn less starts to be a little harder to answer though...

This is the "recursive self-improvement" they've been warning about.

I've always laughed at that phrase, thinking "haven't these things been improving themselves since the very beginning? This is all just iterative gradient descent, each step is a 'self-improvement'", but this really is different.

I wonder if we'll ever discover a new branch of linear algebra or something that allows us to find the optimal weights much more efficiently? Maybe AI will help us discover it?

That ended up not happening.

Then let them. If they want to offshore, let them have the associated pros and cons, and free up real estate that American workers can use to start competitors.

Voting is still a thing. You still have to convince people that just voting for what they want is less important than carefully considering the affect their votes will have on billionaires.

Interesting example because there is no lack of Tetris-like games.

You are free to do so. Have fun!

A silver lining is that porting games will serve as a catalyst to help the self-hosted LLM ecosystem grow.

We're already knocking on the door of just clean-room recreating any game or experience in one shot. In such an environment, who really cares about reverse engineering old games?

I mean, from game company perspective, will they really care about their old games getting reverse engineered when their next big title can be clean-room recreated by anyone with the skill to write a sentence? Maybe creating a game similar to the next big release will be something that the AIs refuse to do next? Interesting times.

As an aspiring game developer myself, I guess reducing the accessibility of old games benefits me (but I don't like it)--and hey, if you can't get that old game to run, maybe we can vibe code for a weekend and make a new game that captures the same feeling.

You want to create a human feeling using AI, but AI is not needed for the feelings you want to create. All you really want is a feeling of variety and surprise in the way computer opponents behave. A LLM-level of AI might just do the same thing all the time and become quite boring.

And yet, that argument I just made often bothers me, because I do think there is a place for strong AI in games, and many people just reflexively say "strong AI wouldn't be fun". I think they are partly right, but I also think a strong AI that is purposely dumbed down in fun ways could be really enjoyable.

When it comes to shooters though, those games aren't really about competition and decisions, they are about giving the human player an experience. I do think strong AI is overkill in such situations.

A strong AI for chess absolutely improves chess computer games.

A strong AI for a Magic: The Gathering style card battler would improve the game.

A strong AI for Civilization would improve the game, but I think we're starting to get into a realm where you can fake some things. I do think Civilization games are lacking because of poor AI, but I don't know that we need LLM-levels of AI in Civilization.

Again, in shooters, there's no decision, no illusion of fair play or equal competition based on making decisions, it's all about the human experience, and so LLM-level AIs aren't needed here at all.

If anyone out there wants to pay for my LLM bill, I'll record and send you all the outputs they generate in exchange (perhaps with some exceptions).

I mean, these AI companies have money, and they want data. I have data, and I want money. Maybe we can work something out.

In a world where buying and selling data is central to entire industries, why shouldn't I gather and sell my own data?

I hope to experiment with using classic tabular reinforcement learning in game AI one day. Imagine a framework that makes setting up small Q-learning tables easy, and you can just sprinkle dozens of Q-learning algorithms throughout your traditional AI scripting.

These old basic algorithms often have mathematical proofs that they will converge to optimal behavior, and they do so fairly quickly. Turning the complex world of the game into a Q-table would require some creativity and judgement, but is not that hard I think.

In the end we would have a controllable AI that actually does adapt. That's my hope anyway.

There was a guy in Missouri who pressed F12 (view source) and saw that the state website contained the SSN of all teacher--just right there in plain text in the HTML.

He reported it responsibly and then the governor personally started attacking him and threatening him with a felony.

I checked your link and I see that is a 2022 DoJ policy. How much peace of mind do you think a DoJ policy (not a law) from 2022 gives me?

I read as far as the first sentence: "The policy for the first time directs that good-faith security research should not be charged."

Soooo... the underlying message here is that good-faith security research is technically against the law, but their policy is not to press charges. They clearly consider good-faith security research to be technically against the law, because it wouldn't make sense to say "for the first time, our policy is to NOT charge people who follow the law"--that makes no sense. What they're saying is "we could charge good-faith security researches, but let's not as a matter of policy".

Reading further you see the policy is about NOT prosecuting people who do something like use an alias as their account name. That same paragraph says their focus should be on people who access systems without permission.

And to be clear, I am advocating for red teams that operate without permission. The bad guys are attacking our systems without permission, and I think the good guys should be protected when they do the same for a good purpose.

I'd feel a lot better if I knew some red team security researchers were going hard trying to break into critical national infrastructure, just to ensure it's not too easy, but probably this isn't happening because it would be a felony to even try. The bad guys are free to try though.

All this will tend to get blamed on AI, but AI didn't put these vulnerabilities into place. AI is just a tool that can be used to discover vulnerabilities, and the only question we have to answer is whether or not that tool should be available to regular people.

Some will try to frame the question as being about whether we should allow AI aid in cyber-security, but that's not the question. This technology isn't going away. The bad guys will have it. Nations will have it. Again, the question is, can regular people also continue to have it?

And this is a society where many of the most powerful corporations got started by just blatantly ignoring the laws of their industry. See: Uber, OpenAI, etc.

Is an AI that is operating a browser on my behalf to answer a question I asked 5 seconds ago--is that a bot or a person?

Building on your analogy: Currently we spend a lot of time and effort making sure our walls appear nice and inviting, with lots of convenient entry points where advertisements can be placed. In the future we will build ugly walls with a few well guarded entrances.

What I mean is, look at how much time and effort we spend on making software services beautiful and branded, and so little time spent on security. In the future we'll probably have fewer webpages and more hardened text-based APIs that end users interact with via their AI agents.

Oh yeah, Screeps, that game is fun and I might stick with it this time if I can vibe code and turn my ideas into code without spending a lot of time.

I'm thinking it might be a nice playground to just experiment with different game AI architectures.

I've been thinking of trying to combine extremely simple Machine Learning techniques, like tabular Q-learning, with a variety of scripted action and goals. I want to try mixing a lot of really tiny machine learning algorithms (maybe neural networks, but not necessarily) into classic AI scripting and see if it can be more dynamic, but still predictable and controllable. Screeps would be a great testing ground.

I've always thought an evil Skynet based on language models is kind of a funny idea, because all of it's secret plans are just spelled out in plain text.

That's a strawman argument. I'm just saying that we need to find ways to wrangle the mountains of code coming out of LLMs into something we can trust, as best we can.

We need to build systems where trust in the code is not needed.

I don't trust the JavaScript code I run in my browser, I don't need to.

A language with a strong enough type system and effects system could say, "this function updates the users preferences, and may not access the network, and may not access data from other users", and the compiler can ensure these constraints AT COMPILE TIME! Provably correct code.

"Illegal" and "shouldn't be allowed" are the same thing when it comes to the government.

If something is legal, the government should allow it.

I am saying that it shouldn't be allowed, because it is not in harmony with the laws, and is therefore illegal.

If the government is going to get involved here, I would regulate companies like DoorDash or Uber to require them to provide company vehicles to drivers.

When DoorDash drivers subtract vehicle wear and tear, fuel, depreciation, and maintenance from their earnings, their net hourly pay frequently drops below the standard minimum wage.

DoorDash benefits from desperate people who slowly transform their vehicle into cash by laboring thousands of hours. They could have just sold the car; either way they end up with cash but no car.

People need to pay rent, so they look at their capital, owning a car, and then trade away that capital for less than it's worth, and a few years later they are worse off, but at least they made rent for those months.

Monopolies are not illegal, anti-competitive practices are illegal; there's a subtle difference.

You're right. Duopolies aren't great either and just because two companies are in fierce competition doesn't mean everything is good with the law. Both companies might be engaging in illegal anti-competitive behavior.

The AI should have a tool that it can use to quote an underlying document. The AI itself would not be responsible for filling the quote; that would be done by the interface which deterministically quotes the requested section.

> the guys in charge of who is and isn't a citizen said he isn't.

At least you recognize that who is and isn't a citizen is no longer determined by law and facts, but by "guys".

Can I get a promotion for asking AI to write code for me and the putting it into a PR?

If the service/product requires zero developers to create, then why wouldn't customers just use one of the 10,000 clones that are free?

There should be people whose sole job is to do PR reviews.

The PR reviewers delegate work to programmers, the programmers delegate the work to AI; then, the programmer reviews the AIs work, and the PR reviewer reviews the PR.

Those "architects" or other idea people should be reviewing the PRs and keeping an eye on the whole system. That's their job, right?

Usually the programmers are responsible for their own work, and then also reviewing their own work, which is just weird. The context switching is very draining.

The problem is that PR reviews are viewed as a side-task, and it's a thankless job. The guy who implemented the fancy new feature will be personally thanked by the CEO and taken out to lunch, the PR reviewer who pointed out in the original PR that it would corrupt the database gets a "nice catch" comment, and that's it.