News for hackers. Stuff that matters.
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by bravetraveler on Tuesday December 09, 2025 @03:52PM (#46210488)
Not sure if I'd use the same descriptions so pointedly, but I can see what they mean.

It's perfectly fine to link for convenience, but it does feel a little disrespectful/SEO-y to not 'continue the conversation'. A summary in the very least, how exactly it pertains. Sell us.

In a sense, saying: "go read this and establish my rhetorical/social position, I'm done here".

It's like if I got to meet an author I really liked, wanted to talk about their work, and they focused more on the catalog than the shared experience.

Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by tomrod on Tuesday December 09, 2025 @04:33PM (#46210991)
Hell, I would consider myself graced that simonw, yes, THAT simonw, the LLM whisperer, took time out of his busy schedule to send me to a discussion I might have expressed interest in.
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by bravetraveler on Tuesday December 09, 2025 @04:45PM (#46211141)
> send me to a discussion I might have expressed interest in

No, no, remember? Points to the blog you were already reading! Working diligently to build a brand: podcast, paid newsletter, the works.

Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by tomrod on Tuesday December 09, 2025 @11:12PM (#46214010)
I wasn't speaking to this interaction, and my point is genuine. Simonw has done fantastic work in the LLM space
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by bravetraveler on Wednesday December 10, 2025 @07:45AM (#46217090)
... and my point remains: he's fine. Could be better. This is one among many interpretations. If he does grace us, he can choose to bait the hook more effectively. Or not.

This interaction is, effectively, a link dropped with an announcement of an announcement. For what has already occurred. Over-fitting, training? You don't say.

If I wanted to be more of an ass, I'd look to argue about hype generation. But I don't, I appreciate any honest effort, which I believe for Simon.

Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by simonw on Tuesday December 09, 2025 @05:36PM (#46211688)
I've genuinely been answering the question "what if the labs are training on your pelican benchmark" 3-4 times a week for several months at this point. I wrote that piece precisely so I didn't have to copy and paste the same arguments into dozens of different conversations.
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by bravetraveler on Tuesday December 09, 2025 @05:46PM (#46211789)
Oh, no. Does this policing job pay well? /s Seriously: less is more, trust the process, any number of platitudes work here. Who are you defending against? Readers, right? You wrote your thing, defended it with more of the thing. It'll permeate. Or it won't. Does it matter?

You could be done, nothing is making you defend this (sorry) asinine benchmark across the internet. Not trying to (m|y)uck your yum, or whatever.

Remember, I did say linking for convenience is fine. We're belaboring the worst reading in comments. Inconsequential, unnecessary heartburn. Link the blog posts together and call it good enough.

Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by Barbing on Tuesday December 09, 2025 @07:51PM (#46212725)
Surprised to see snark re: what I thought was a standard practice (linking FAQs, essentially).

I hadn’t seen the post. It was relevant. I just read it. Lucky Ten Thousand can read it next time even though I won’t.

Simon has never seemed annoying so unlike other comments that might worry me (even “Opus made this” even though it’s cool but I’m concerned someone astroturfed), that comment would’ve never raised my eyebrows. He’s also dedicated and I love he devotes his time to a new field like this where it’s great to have attempts at benchmarks, folks cutting through chaff, etc.

Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by bravetraveler on Tuesday December 09, 2025 @07:54PM (#46212744)
I'm mainly being snarky about the call for 'same arguments into dozens of different conversations'. The FAQ is a promise to catch them, expect more links. Jump for joy.

Yes, the LLM people will train on this. They will train on absolutely everything [as they have]. The comments/links serve to build engagement more than anything. This is fine, that's not why I'm being critical. The blog-o-sphere motivation, or the belief that this is a common or difficult-to-answer question, perhaps.

I'll readily admit it's silly. That's why I chose snark.

Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by simonw on Tuesday December 09, 2025 @06:24PM (#46212117)
You don't have to convince me the pelican riding a bicycle SVG benchmark is asinine. That's kind of the point!
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by bravetraveler on Tuesday December 09, 2025 @06:25PM (#46212127)
Having read the followup post being linked, I'm even more confused. Commenting or, really, anything seems even less worthwhile. That's my point.

You bring the benchmark and anticipated their... cheesing, with a promise to catch them on it. Cool announcement of an announcement. Just do that [or don't]. In a hippy sense, this is no longer yours. It's out there. Like everything else anyone wrote.

Let the LLM people train on your test. Catch them as claimed. Publish again. Huzzah, industry without overtime in the comments. It makes sense/cents to position yourself this way :)

Obviously they're going to train on anything they can get. They did. Mouse, meet cat. Some of us in the house would love it if y'all would keep it down! This is 90s rap beef all over again

Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by charcircuit on Tuesday December 09, 2025 @06:47PM (#46212290)
If you want a summary you can have your ai assistant summarize the link.
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by bravetraveler on Tuesday December 09, 2025 @06:49PM (#46212303)
Woooooosh, please see if an LLM can help you. I'm not getting paid for this
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by renewiltord on Wednesday December 10, 2025 @07:11AM (#46216832)
It is SEO-y and I’m sure no small impulse is to drive traffic to his website since he’s primarily an AI influencer.

However, there are always people who are “native” to a platform and field. Pieter Levels is native to Twitter and the nomad community. Swyx is native to Twitter/HN and the devtools community. And simonw is native to at least HN and the LLM-interest community. And various streamers and onlyfans creators do the same with theirs.

Through some degree of releasing things that whatever that community values they build a relationship that allows them greater freedom in participating there. It does create a positive feedback cycle for them (and hopefully the community) that most of them will try to parlay into something else: Levels and the OnlyFans creators are probably best at this monetization of reputation but each of them is doing this. One success step for simonw would be “Creator of Pelican LLM benchmark”.

Once you’ve breached some stable point in the community the norms are somewhat relaxed. But it’s not easy to do that. You have to produce some extraordinary volume of things that people value.

I think, tbh, tptacek here could most effectively monetize if he decided to. But he doesn’t appear to want to so he’s just a participant not an influencer so to speak. Whereas someone like Levels or simonw is both.

It’s just creator economy stuff. Meta discussions like this always pop up. But ultimately simonw is past the threshold of trust. There are people who say “wtf? Why is levels making $50k/mo on a stupid vibe-coded flying game?”

It ain’t the game. It’s the following before the game. The resource is the audience.

Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by bravetraveler on Wednesday December 10, 2025 @07:26AM (#46216947)
Thanks for posting, I agree. I regret this being taken so pointedly at Simon, just a player in the game.

The best guy spinning the sign puts some effort in, or more crass, the best strippers make you believe.

Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by dugidugout on Wednesday December 10, 2025 @01:02PM (#46221089)
Well put and thank you for adding much depth here!
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by tptacek on Thursday December 11, 2025 @06:58PM (#46239103)
Wait how do I monetize? Am I leaving money on the table?
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by renewiltord on Friday December 12, 2025 @12:16PM (#46246177)
Not so much leaving money on the table as you have a valuable resource you are underutilizing in this respect. You’d have to cross-cultivate your audience and expand to fill some niche.

That means that instead of dropping facts here you should post it on Substack and link to it here. You’ll be fine with it because you’re trusted here so people will be like “oh it’s fine”.

And then once you have the newsletter going and you have a following there you can do paid placements etc. that don't have to be done by you.

I think you have the highest potential on this front but that doesn't mean there's comparative advantage. It may not be worth it to you. For instance, I would never do it. To keep that wheel spinning you have to keep putting energy into it. And I have other ways of making money, so even with your account it would be a waste.

Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by tptacek on Friday December 12, 2025 @01:16PM (#46246912)
I'm going to start encrypting the bottom 2/3rds of all my comments and charging a subscription fee for the unlocks. I can be my own Substack, dammit.
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by renewiltord on Saturday December 13, 2025 @01:56AM (#46252633)
Hahaha ))
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by tptacek on Saturday December 13, 2025 @02:03AM (#46252661)
U2FsdGVkX1+F16B6AtiHGa1yNbh1HzERm4kYq8UwK4SEbIMUg4N5xZfj