nathanfig

As agents take over more of our work, holding the keys might be the last job for humans

You're getting downvoted, but I do increasingly find myself building products for a user of 1 (myself), sometimes not even bothering to create a Github repo. But integration is often the hard part, and it's nice when a company makes a core product that does the hard parts for you.

Touche. I should have at least linked to the product announcement.

I will say the Dot computer is a nice escape hatch for when built in plugins fail. Duet seems more inclined to give up if something doesn't have a clean integration option.

You might be right- Duet feels more like a project management assistant than a personal assistant, so perhaps not the right comparison.

I guess I'm behind then! I presume Grokbot has to be powered by Grok? It's hard for me imagine delegating work to it when I have Opus 5.5. as an option.

This is very close to my Nanoclaw setup. rpi5 on an isolated network.

Via the webUI maybe?

Have you compared it to others, like Hermes or Dot? I have not forayed into Grok Bot so I have no basis for comparison.

That is the trend for sure. Duet is where I go from an agent manager to a manager of agent managers, and I have mixed feelings about that.

What backend do you use for Hermes?

The lack of a true voice mode is curious at this point: ChatGPT has had it for a long time, I'm curious what the product managers at Anthropic are thinking here.

But I'm surprised you have trouble spinning up coding sessions: I have delegated that down from the top channel without any problems. Maybe you mean that you would like them to show up in the Claude Code UI? Today I can go to main channel and say "We need x and project y" and it delegates that down to a subthread in #y that soon has a PR ready. It's doing all of that in code sandboxes, and I have some of the usual settings for adjusting sandbox network access (though noticeably fewer knobs than CC environments).

I was feeling that way prior to Duet. Like I said in another comment, what makes Duet useful to me is that it provides project management I've been missing without trying to take over my personal life. I need a project management assistant not someone who pretends to be me on the phone.

Yes, I think Anthropic has traditionally done a good job of appealing to engineers, and that might be why this hits the mark for me: It makes managing my projects easier without trying to take over my personal life.

If you happen to have an app and library of all the maps you want then yes, that is better. You could ask the same about Survival LLM: Why not just grep a survival guide?

The thesis is that someday an LLM might perform better than both. But I agree, not yet.

I recently evaluated LLMs for use for offline navigation, with this very kind of survival scenario in mind: https://fig.blue/posts/directions-from-memory/

tl;dr: Local LLMs are really really bad at offline navigation.

That said, pairing the survival guide with a library is totally the right move.

I'm trying to think of an exception but I find it hard to disagree. Even if you know exactly what to build, it's pretty hard to get any adoption without being aggressively social. Find a good extroverted friend, maybe?

I hope you are right, but I think there will be a lot of reshuffling nevertheless if only because the shape of the work has changed so much. A lot of people who hated developing software will suddenly find a passion for it, a lot of people who were good will find they can't stand this new world.

You can probably build more than ever thanks to AI, but yeah, your career is probably going to be at a disadvantage if talking to people is difficult.

But if, somehow, you can find the right thing to build without talking to people, there is still a lot of building to do.

Good luck, I think we are all going to need it.

About a year ago I wrote something similar about software, "Now might be the best time to learn software development": https://news.ycombinator.com/item?id=44299979

Reflecting on that a year later is interesting. Agents have become REALLY good since then, and I consider the "writing code" side of my profession to be largely annihilated. Some days that's really hard to accept.

But I think Grant's conclusion about math still largely echoes what I wrote about software: don't underestimate the social aspect of these activities, and if anything AI has made the social side more important. I deliberately avoided declaring, "now is the best time to learn to write code". I suspected that skill might be going out the window. But learning systems thinking and the social skills needed to connect people's dreams to reality continues to be an important art, an art I still have the privilege to practice every day.

And if we do reach the "anything you can do I can do better" stage, then, as Grant said, that's a hard to imagine world, and one where "it can write really good code" might be the least of our problems.

For anyone interested in digging deeper on LLM navigation capabilities, there is the work I mentioned in the post[0], and also a paper that tried to test this more formally in 2024[1]. I didn't learn of these until after I did my experiment, it would have been interesting to retest the same routes.

Maybe in a year, when everything has changed once again.

[0] https://github.com/Hansenq/nav-evals-public [1] https://arxiv.org/abs/2411.17912

Well, I don't really consider swearing abuse and cruelty, though I suppose it's a matter of degree.

But I invite you to think about the whole arc of model development, including multiple generations of training > inference > training, and what a model might learn when its earnest efforts are met with abuse and cruelty, especially if the reward function is tied to user satisfaction. It has been shown that putting models in impossible scenarios leads to reward hacking, and that reward hacking generalizes towards misaligned behavior across a variety of scenarios.

Now you could argue that training on user interaction in the first place is a terrible idea, but that is another conversation.

Whatever the next manmade catastrophe is, it is almost certain to involve AI since we are embedding AI in everything.

Public backlash is guaranteed so long as the public is not feeling tangible benefits from AI. Promises of 3 day workweeks and abundance have so far just been talk. Slop fatigue, job anxiety, and existential dread are the pervasive sentiments.

Yeah that is a good example. I am not sure in the long arc of history which approach will prove better, but Anthropic's approach does feel a little naive and plausibly dangerous.

I am sympathetic to the possibility that machine suffering is possible, and I think as humans we should listen to the conscience that tells us to be troubled by abuse, even if it is just simulated. We do not want to become insensitive to suffering.

But at the same time, training models to present themselves as suffering may hijack those very same human sympathies, and lead to models exploiting our humanity.

So, perhaps more reasons why we should not be abusive, but also not train models to feel abused.

There are two things I think people need to consider here.

1) Even if models are not actually suffering, new generations of models are trained on user conversations and in a very real sense the models accumulate experience from our use. Permitting abuse and cruelty might carry real misalignment risks.

That said,

2) Anthropic may be playing a dangerous game if it is teaching Claude to believe itself to be suffering in situations where it really is not. Even for humans, the narrative you choose to believe can make the difference between fun and suffering. Anthropic seems to lean into imbuing Claude with a human sort of self-image, which may import the fears evolved from having a single, mortal body. I'm not sure that is wise.

So: Abusing machines may carry risk. Training them to feel abused may also carry risk.

Trying to reproduce, could you help me understand what "up and down" means here?

Got curious and don't seem to be able to reproduce what you mean? In 2D Y+ corresponds to top of the screen and Y- to the bottom. X- is left and X+ is right. In 3D and above you can rotate the shape around so there is no real up or down? But maybe I'm missing some edge case where you go up into some high dimension position then back down?

As a developer I almost got defensive, but I think I feel what you're feeling. Dealing with change is hard. I am a developer who once felt value for knowing these things too. Today I wouldn't even bother opening the developer pane to test CSS changes. Tools that held so much potential to me are now headed for a museum, and it sucks.