ej88

i think the thread shows:

- most people's perception of art is heavily affected by the framing (and to a lot of people ai = bad, and so they start seeing technical issues with it that could /never/ be made by Monet despite it being a Monet)

- but I think the critique here is more: even if someone recreated a Monet stroke-for-stroke, what's the value of this copy? I think the artist's personal life and context around the painting adds so much more to it compared to just being a pretty painting (perhaps this is the single most important part of what makes a painting interesting and valuable)

Ya, the demos were pretty contrived (feels like a running theme amongst the labs...)

swe bench pro has a public and private test set, where the private eval is from proprietary codebases only

This is cool!

I used to work on post-training & evals. it's really hard to make a good eval set and catch all forms of reward hacking. Excited to see more from poolside!

An omni model seems very useful for real-time human-computer interaction, off the top of my head:

- Voice assistants

- Customer experience

- Gaming

- Meeting assistants

- Real-time coach or user assistant for using software

- Translation

- Real-time work on a computer controlled by voice (frontend / mobile dev, CAD, 3D modeling, etc)

Traditionally a lot of these use cases with LLM agents are higher latency because the model needs to wait for the speaker to finish, then decide to call a tool or respond - if they call a tool they need to process the tool result and decide if they want to call a tool or respond, etc...

i would argue its the opposite

farming hit a ceiling because of demand

software today is heavily, heavily constrained by supply. demand is basically infinite for actually good software that solves problems people have (and people always have problems).

"She rejected several applicants with PhDs and engineering backgrounds, reasoning that their level of education could not compensate for a lack of hands-on specialty coffee experience."

This is depressing.

1. part of the moat is their guardrails and obviously they are audited and tracked. there are agents issuing refunds and more at scale right now so not sure where the skepticism comes from.. you're free to try and jailbreak them

2. another part of the value prop of these companies is figuring out how to construct the proper harness to take advantage of the lower latency of faster models while shoring up the weaker intelligence, how you blend deterministic and non-deterministic behaviors, compliance etc.

its a hard problem which is why f500 is willing to pay up

1&2 are already happening, these startups take on brand liability and trust to do so

3 depends on how companies want to measure it, but lack of user submitting satisfaction score is not a good thing

you can use a model w/o reasoning, + use various tricks to simulate low latency

that's fair, most implementations in the industry are in the early stages and implementing a full powered agent with access to all the tools it needs is hard (very political as you can imagine). i hope over the next year you notice them getting better!

adding some context as someone who works in this space

1. most people (average, non-tech people) reach for the phone to call in for easily solvable problems. Plus, if the agent is integrated deep enough & has tools to interact with crms, you can raise the ceiling on the types of problems it can solve.

You're trying to avoid the bad customer experience of human 1 reading off their script, then they transfer you to some other department who may or may not know how to solve your problem, and the entire interaction cost the company way more than the value created, so the company is disincentivized to help customers.

2. All the companies in this space start with the outsourced BPO market for cx (multi billion market still) but the next market is going to be in revenue generation and churn prevention at scale, i.e. how do you proactively avoid customer issues, how do you upsell and generate revenue instead of reducing cost, how do you keep customers happy?

3. I think more companies will pivot to outcome based pricing on the contrary, makes it so much more measurable than seat-based and protects margins better than usage based. Plus cx is one of the few industries with very well known metrics

4. Kind of? Most companies in this space don't use native voice models which are noticeably dumber, they use transcription + a stronger text model + TTS. The majority of customers can be handled with the latest SOTA text model and you need smart context engineering to handle the long tail of more complicated asks

its a moat vs. other startups and it carried them to multi-B valuation

obviously the product needs to deliver and nrr needs to be good in the long run

true. we'll see how many ai cos become profit printers a few years from now

ai skeptic fanfic evolves in fascinating ways every day

hes board chair of openai and is ex co-ceo of salesforce, ex cto of facebook, can get a meeting with any exec in F500...

their moat is distribution

ime its very implementation dependent

but even a simple impl to answer questions can knock out like 50% of callers who are tech-illiterate at 100x cheaper cost, it's just strictly better economics and better for those customers

It's always interesting seeing how HN reacts to AI CX (as someone who works in this space). Yes, the tech savvy crowd loves to say how they always ask for a human and love old school phone trees

in reality 50-80% of callers come in with easily answerable questions because they don't know how to nav the website and prefer to ask in natural language

The vast majority of callers call in to resolve their issue, and most don't care if they are speaking to a bot because they just want their issue fixed. Agents (if implemented well) are an order of magnitude more effective at resolving issues compared to a call centre worker who is reading off a script and churn within 9 months

There's also the 2nd order effs of making CX cheap. before, there is the perverse incentive of companies trying to keep you off support because each call costs them way more than the value they get. if your cost per call drops 100x you can invest in turning a cost centre into a revenue driver (+ a better experience)

im not sure i understand your reply, but it sounds like you're agreeing with me that yts biggest advantage is the network effect?

Ive been preparing somewhat for this, as someone who knows they aren't a top N% engineer. My current role involves a certain amount of sales and product in addition to SWE (and luckily I find it fun to talk to customers!)

I think it's prudent for a lot of swes to think about what a future looks like where most of the job is managing and unblocking agents.

my main qualm with Ed is his analysis on the financials is decent, but he absolutely refuses to admit that the technology is useful (especially in the hands of competent users), and that all the labs are extremely compute starved due to overwhelming demand.

Theres no viable yt alternative because of network effects not the video hosting

Those same network effects dont exist (yet) on models

sorry, i shouldve defined it better. my point of view is an 'ai pilled' company is one that has a realistic understanding of the benefits and limitations of ai productivity, and leadership + employees are fully bought in, and theres a general high trust environment

if ai has to be enforced (mandatory usage, kpis, training, restrictions on tools) -> clearly the execs think the employees are not bought in

typing every line by hand -> self explanatory

layoffs -> this one is a bit of a stretch, but from what i've seen the best companies at leveraging ai are not laying people off, instead continuing to hire more to capture the market or capitalize on the demand. could be confounding variables though

most are not, e.g. if your company has any of these you're probably not ai pilled

- mandatory ai usage

- ai usage tied to kpis or performance reviews

- trainings on how to use claude code

- restrictions on what tools you can use

- layoffs

- engineers still typing every line of code by hand

"The psychic toll of AI" -- It's sad, but each of these scenarios (barring the AI notetaker, which I haven't found to be an issue personally but ymmv) are indicative more of the culture of the company than the tool itself. From my experience it seems like the most frontier companies have the best AI-use culture.

I work at a very 'AI-pilled' company, but:

- Everyone reads and reviews every PR and leaves human comments

- Documentation is written well and tended to by humans

- There's no 'AI mandate'

- Whether features are possible are first explored by an agent but manually traced by a human through the codebase

You can treat AI like a very powerful tool to augment you and run your agent swarms at the same time.

in these cases arr = annual run rate, commonly used when your revenue is either going vertical (cursor - good) or your revenue is choppy and full of short term projects (mercor - bad)

alpha is your abnormal rate of return

more colloquially it's your 'edge' over your competitors

This is cool!

Seems like Claude is actually building almost like a layered Figma wireframe that you can do fine grained adjustments afterwards (e.g. adjust font size).

Interesting that Canva provided a quote of support. I'm not familiar with the differentiation, but it seems like this will directly siphon customers from Canva, right?

i would say theres more nuance than that (disclaimer: dont have a crystal ball)

software engineers who are comfortable doing business work - managing, working with different stakeholders, having product and design taste, being sociable, driving business outcomes are going to be more desired than ever

likewise, business leads who can be technical, can decompose vague ideas into product, leverage code to prototype and work with the previous person will also be extremely high value.

i would be concerned if i was an engineer with no business acumen or a business lead with no technical acumen (not counting CEOs obviously, but then again the barrier to starting your own business as a SWE has never been lower)

clearly not the same when they were abstracted from the realities of building software and.. directly taking accountability for it!

by semantics, i mean the definition and pool of tasks, responsibilities, and outcomes a job is comprised of is shifting so fast that the borders of what is a 'software engineer' and 'business person' are melding together. software engineers are business people in their own way