NotAnOtter

I think open source models could be vulnerable to these attacks. So long as I sandbox everything then I am as secure as my sandbox which is air gapped from my personal stuff i.e. no models on the same machine I log into my bank.

I know closed source models will have nefarious shit injected into it. It will start modest at first, of course, they need to get past the hyper scrutiny phase. Then it will simply be telemetry, then it will simply be feeding my behaviors into ad targeting, then it will simply be computer vampirism, and so on. One step at a time.

I prefer a gremlin I know I need to keep chained than a seductress I think I can probably trust.

But that's exactly my point?

Most algo's, especially ones made as a 'side project', are actually just directional bets and not true algorithms. And there are cheaper ways to take directional bets thus most of these are actually losing money - if the author just understood their directional bet, they could buy relevant stock/options to have the same risk profile for cheaper. But compiling algorithms back down into their directional components is an entire field in quant finance, akin to something like DS&A for CS.

Which is the joke I was referencing. When you take a directional bet and win, that isn't proof you're algorithm 'worked', it proves your directional bet won. Akin to someone who "vibe codes a website" and then sends you a localhost url. It's proof you're just making shit up and not aware of what it is you're building or how it works.

You think there won't be a job where you're primary duty is to write & maintain software in 10 years?

Trading algo generating revenue in a bull market is a meme among many trading communities.

As models get better and better at doing the stuff users are asking it do to, the more and more they trust it and grant it increased privileges.

Back when AI was new, I needed to check every single output because it was 50/50 on the code being complete junk. Now 99% it's correct first time and 1% close but needs some tweaking. I'm lazier and lazier WRT granting permissions and just trusting it to go nuts.

Mostly trying to think about what career to pivot to, since SWE wages are going to get crushed in the next 5-10 years.

If you owe the bank $10k, that's a you problem.

If you owe the bank $10Mil, that's the banks problem.

If you steal $10Bil from the bank, the government bails out the bank and now it's everyone's problem.

I think there has always been a spectrum between utility and art, with most things falling obviously into one camp or the other. Assuming all else being equal (climate impact, etc), the ability to provide a utility for cheaper is nearly always good.

99.99% of software is a utility. It being cheaper to produce is a net win for society. The LLM-solution isn't an "all else being equal" scenario, there are other obvious downsides. But the part where an in-demand utility got cheaper is very good.

FWIW - I think the small business market is actually very suitable for former SWE's. The capacity to troubleshoot processes, variance, bottlenecks, etc. is what turns a mom& pop shop into a regional chain.

I think you underestimate how many people would be willing to battle a computer if it meant doubling their salaries. Think of all the teachers with STEM degrees being paid $60k, if someone offered them $120k to prompt a computer for a while I'm sure many would come running.

And that's literally one single profession.

Disclaimer; I often feel dread around my career fading and needing to switch to a field I don't like & won't be paid well in.

That said, I think the rational perspective is "things will be fine, basically".

I tried writing a longer reply but organized thought drifted into ramblings about economics I only partially understand. TL;DR I think some opportunities will close and others will open, and current engineers will find a home. Mostly software architecture or as technical advisors for product companies (if everyone writes code, you need some Dude Who Knows It Well to guide the non-engineers when they get stuck. Akin to IT).

I think wages will stagnate but wages are generally inflated, if we're being honest. As much as I love my doctor-level salary, I can honestly say I don't deserve it.

Same thing happened to me!

I had a comment deleted by reddit admin & it banned my account, I appealed and received a message apologizing for the mistake and they unblocked my comment. But my entire account is still banned! Screw the comment I want my old account back. I've appealed many times but of course, no response. It's just being sent into the void.

Seems there is just no reason to use the website anymore. Either you post the most boring hive-mind nonsense, or otherwise just get banned.

Bad faith argument

Not gonna keep going back and forth when you seem to agree but are choosing to be difficult. We agree SO was falling before LLM's hit the market. We agree LLM's accelerated SO's demise.

You seem to think they weren't failing before LLM (simply rapidly losing member activity), which is a narrative that I won't follow.

It's pretty wild. Meanwhile bot farms just shuffle all identifiers.

They are actively harming human users in defense of their toxic mods & botters. Site is dying and they are the murderer.

Then explain the usage chart declining since ~2015?

As I said in my original post, LLM was the final nail in the coffin. I'm not arguing they aren't related. I'm saying they SO was falling long before LLM's took over. This isn't difficult logic.

IMO the community will fracture in two directions. Reddit's differentiator over Instragram, twitter etc. is that it's community based rather than individual based (with the algo making psuedo communities)

I feel some users will leach into platforms that created even more walled gardens, i.e. Discord, or platforms that reduce the sense of walled gardens i.e. Twitter.

> I think Stack Overflow went dead because of AI specifically

This doesn't hold up when looking at usage charts. There is a clear peak around ~2015 with a steady decline through to now. LLM's came to market in their current form in the last couple years, and took a couple years to be broadly adopted. There was a clear and obvious market fall off way before AI / LLM.

> Reddit isn't comparable

I agree with that in isolation; but since I don't agree with the AI premise this isn't especially relevant. I don't think AI will replace Reddit, I think one of the other major platforms will absorb it's users like Reddit / Hackerrank / better documentation / back searching absorbed SO's users through 2015-2021

I feel Quora gave up years ago though.

When you load a random content page, the top 50% of the page is a question, the bottom 50% is an ad that is designed to look like a comment, and the entire right panel is ads. Quora is more ads than content, you have to scroll and decipher what is or isn't an ad based on their greyed 2px font ad disclaimer.

How are concerns about ecological impact misplaced when discussing solutions to ecological problems. It feels pretty relevant to me.

And from everything I've seen/heard, tidal based solutions are just fundamentally incompatible with their product. Keeping sensitive metalic moving parts in saline solution exposed to the sun for years on end - paired with other random things like boating accidents or marine life - it's a non-starter. Constructing these things creates pollution. If it's lifecycle impact is less than oil's, great, I just don't believe we'll ever get to a state where it's better than oil AND (solar/geo/wind) + Batteries.

It's a massive stretch to think one poorly placed meltdown somewhere in the UK would lead to the UK collapsing. I suspect it would be visible on a 10 year GDP chart but not "trending towards 0" levels of economic fallout.

Also I might just be misinformed but I thought nearly all of the radioactive waste from nuclear plants is already collected. It's not a collection problem, it's a storage problem. And a "what do we do when the energy company shuts down and stops maintaining their storage yard" problem.

People are broadly misinformed. Nuclear plants release significantly less radiation than coal based plants, as an example. They do create a lot of waste that we currently don't know how to process, but the quantity is actually shockingly small when discussion a global issue. We can just pick some cave in northern Canada or central sahara and bury it there, it seriously isn't that much.

Yes, lifecycle analysis is the holy grail.

And I'm not qualified to say the tidal based solutions will never beat out Geo/Solar/Win + Batteries. In my informed but non-professional opinion, it seems like this avenue will never ever work at scale.

From everything I've seen, we have the answer, we're just stuck under the boot of old money oil barons. Solar + wind + geo (depending on the geographic area) for the majority of our power generation. Nuclear + batteries to smooth out the duck curve form the bottom, paired with more aggressive demand pricing & thermal regulations to smooth it out from the top. That's the answer. But lobbyist's going to lobby.

I used to throw loads of parties. At somepoint I realized..

1. It's expensive. I never once got a reasonable contribution from my friends. I knew this at the time, but eventually I was over it. Paying $100-250 per event just to deal with all the work and drama that comes with it.. not worth.

2. It's a lot of work. Hours of prep, hours of hosting, hours of clean up after. At the end of the day ~12 hours of effort for ~4 hours of fun is not a good ROI.

3. It frequently was an excuse to get drunk or high. Which is fun, whatever. But as I grew more health conscious, this was less and less appealing. I can drink on my own if I want.

4. There are better alternatives. I don't have to do any of the above options if I just jump on Discord for a while. Or join a rec league sport. Or spend it with my family.

As cool as this sounds, I'm not sure I'm as enthused with stuff our oceans full of more tech, which inevitably will wear down, break, and pollute.

It's better than oil (duh), and something that provides power when solar/wind can't is great (duh). I just wish we would give up on approaches that are basically "If we had a few million of these giga-ton structures all over the ocean, they would provide power equivalent to a few dozen nuclear plants"

I do not need to know if AI therapy is as good as Real Life therapy. It almost certainly is not.

I need to know if using it as AI therapy is actively harmful for some significant percentage of the population, and should be avoided. This arxiv does not discuss that as far as I can tell. LLM therapy is closer to an interactive journal. Journaling, getting your thoughts out, being forced to articulate grief in succinct words and pick out patterns - is all healing.

And most people cannot afford professional therapy.

If the thing gating your from creating a great product was the engineering, then I would call you a great product designer and not an engineer. If you suddenly can create products with a new tool - you didn't change, the tools did.

You're still a great product designer and not an engineer.

>You should remain open to new things in this industry

I'm open to new things. I've seen demo's, attended presentations, and spent a long time toying around with it myself. I have not been convinced there is any meat there, not in it's current iteration. LLM's are designed to make things that "look" like human output and thus are very good at hiding bugs. It's ok at getting the first 20% of the project done, but that was never the hard part. It's always been the last 20%, and modern LLM's simply cannot do it. Not on large scale projects.

New things have come and gone. So far the only thing I'm convinced of is, it's easier to get funding when you can claim you use AI. That's it.

> I have never copy and pasted code into development from an LLM/AI helper

Well that's simply a different reality from what my employer is encouraging. So not relevant. They not only want us to copy-and-paste, they want us to delete otherwise functional code to make it easier to paste in AI generated stuff.

Asking questions is fine, that's much much closer to an augmented search engine than prompt engineering. You're describing something different from what this post is about.

>5 years is not what I would consider a big bargaining chip

I'm not bragging. I'm giving context. If I was 0 yoe or 20 yoe, those would be relevant too. And for what it's worth, I also started in middle school.

>one leadership away from asking their employees the same thing your employer is

Yeah that's probably true