crent
Tangentially related, I find myself saying, “people in general have no capacity for nuance” more and more. Probably every single day. I feel like this mindset crept into the HN hive mind over the last 10 years so it’s a bit frustrating to see a space I always felt good about having multi-faceted debates on turn into a forum full of the same attitude I find on Facebook and Reddit. That’s the cycle I suppose…
1. Assume the model which initiates the worm is a high powered model.
2. It uploads it's weights in some way and uses some URI to encode the location of the weights instead of transferring the weights themselves. It not only uploads it's own large set of weights but also uploads sets of weights which increase in size to support replication across a variety of resources.
3. Initially a small inference layer is set up across low-power, poorly monitored IoT devices. It is slow and not very powerful.
4. That layer is used simply to find more powerful hardware but it has the locations of a variety of weights embedded in it so it can set up a stronger distributed inference layer as it finds more capable hardware
5. At a certain level of inference, the entire system could start protecting itself by keeping the less capable inference layers alive through more distribution
6. As the article mentions, it's not entirely in the realm of fiction to imagine that at a certain capability level, the system could scam people out of money and use that to then begin purchasing compute
---------
Depending on the details, you could probably turn off the originating model as soon as step 2 has been completed and if you can't get rid of that layer of inference, it could just continue. Maybe in addition to weights, the initiator also uploads a series of prompts for varying "roles" of the entire system it's developing.
At a certain level, how would you even trace and observe such a thing? Especially if the first layer is created through consumer level IoT devices where the average Joe isn't monitoring what's happening on the hardware.
If we apply that same thought process to philosophy and reason, how can you define it without creating boundaries of what it is not?
It’s a bit tricky because sometimes you get caught focusing on being the observer and letting sensations pass. It’s a more subtle thought but as you practice those subtle sensations become more apparent. It’s a useful sensation in the beginning to let the more “coarse” sensations and feelings go but eventually you need to let that go too.
I often find the egotistical viewpoint to be banal. I don’t mean egotistical in a demeaning way - I mean it just to frame the types of thinking that revolve around the self, inner world or “my individual experience”. It’s an obvious, but not provably correct, conclusion that the human experience, consciousness and reasoning are special because I’m experiencing it in the same way that the earth is the center of the universe because look around you.
It frustrates me in these conversations to hear things like “it’s a text generator, it has no senses”. What even is a sense? For those of us who are software engineers, who live in the abstract, I do not understand why it’s so hard to take the leap to abstract over our own senses as inputs/outputs. Hell, we commonly understand that we have 5 senses but what do I make of the apple I’m visualizing in my brain or the full orchestra I can summon in my “minds eye”? Isn’t that an input just as much as an apple I see with my eyes?
Lately I've been landing on a couple of different reasons but I don’t think it’s one size fits all:
- Ego/identity - “I am the crafter of code. That is what I do.” If someone has their identity deeply wrapped up in the concept of being a “software engineer” or “programmer” then LLMs are a direct threat to that. People don’t tend to do well with this. Think about dogmatically religious people, pseudoscience followers etc who are confronted with evidence directly contradicting their beliefs. Their entire world view revolves around that identity and if you threaten that, you threaten the foundations of their self-perception.
- Career - you can take a lot of what I said above but also map it to threatening someone’s career. What if crafting software by hand becomes a niche, artisanal craft but most software is industrially generated? A lot of people will experience hardship if so and if they can’t find a way to be flexible into the future.
- Passion - the ones who love software purely for the craft see this change as robbing them of the one thing they enjoyed in their career. Work is a big part of our life and if you kill the joy for a large group of people, that’s tough to deal with.
- Lack of understanding their current purpose in role - I often see software engineers who don’t realize why they were hired. It was never to “write good clean code”, that was a means to an end. The end is generating business value for the company that hired you. That’s really it. It’s banal but it’s just a job like bagging groceries only it has required specialized knowledge so it pays well. Only very niche roles have actually hired for the craftsmanship. If someone is working for Groupon and they believe perfectly elegant systems are the value they provide, they are a bit deluded IMO. They build a platform that peddles coupons. That’s hardly comparable to building software that allows a surgeon to remotely operate a robot that does open heart surgery. Most of us do not work on truly mission critical software like that.
Most importantly though, I think this topic needs to be approached with empathy. This is truly a seismic shift in how we, as software developers, work. Change is not easy to cope with especially when it threatens physical safety and identity. Will this change be here to stay and are we in for the extinction of software creation as we have known it? I don’t know. I do know that the world’s financial and governmental systems do seem to be betting on that outcome, however.
What I do to combat this, and other "brain noise", is also to listen to music but I use headphones with high volume. I also listen to the same playlist repeatedly so it's not distracting and instead quiets that loud part of my brain to allow me to focus.
I’m not sure I entirely agree but I do think the paradigm is shifting enough that I feel bad for my coworkers who intentionally don’t use AI. I can see a new skill developing in myself that augments my ability to perform and they are still taking ages doing the same old thing. Frankly, now is the sweet spot because the expectation hasn’t raised enough to meet the output so you can either squeeze time to tackle that tech debt or find time to kick up your feet until the industry catches up.
It’s not technically shorting but you are making a bet that the stock will remain the same price or lower.