News for hackers. Stuff that matters.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by the_doctah on Saturday October 29, 2022 @07:00PM (#33388812)
What hubris to think that your system is so complex that other engineers couldn't understand it.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by hn_throwaway_99 on Saturday October 29, 2022 @07:39PM (#33389080)
Oh please, that's a ridiculous assessment of what was said. Coding giant distributed web apps is a very different domain than coding for cars. I wouldn't put much stock in Twitter engineers assessing self driving car code either.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by Nicksil on Saturday October 29, 2022 @07:44PM (#33389127)
>Oh please, that's a ridiculous assessment of what was said. Coding giant distributed web apps is a very different domain than coding for cars. I wouldn't put much stock in Twitter engineers assessing self driving car code either.

You don't need to be a part of any particular domain to effectively audit code.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by hn_throwaway_99 on Saturday October 29, 2022 @07:48PM (#33389146)
Bullshit. I'm a web engineer, and I wouldn't be competent to audit code that runs Space X's rockets, nor would anyone I know in my career space.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by Aeolun on Saturday October 29, 2022 @08:12PM (#33389343)
If I see the SpaceX code, and it’s full of thousand line functions with a cognitive complexity of 200 I can confidently say it’s going to be hard to maintain.

Clearly it gets rockets into the sky, but something similar is true for all code that hasn’t failed catastrophically yet.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by arrow7000 on Saturday October 29, 2022 @08:26PM (#33389451)
Is there any reason to suspect that Twitter's code will be of this terrible quality? If not, then again: what's the point of bringing random engineers in to take a look at code from an entirely unfamiliar domain?
Re:Tesla engineers were on-site to evaluate the Twitter staf... by Aeolun on Sunday October 30, 2022 @12:25AM (#33391063)
Since it’s a huge enterprise with hundreds of engineers, I don’t have to guess. It’s just a matter of magnitude.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by XorNot on Saturday October 29, 2022 @09:10PM (#33389818)
Except you would be wrong. Or at least not obviously right: safety critical code does all sorts of weird things in the interests of verifiability because it must be exactly right: it doesn't get "maintained", because once it's flight proven you do not change it without an entirely new verification process.

This also leads to interesting practices at times like favouring repeating code over writing separate functions to ensure the flow of reading doesn't jump around and instead always moves forward.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by georgeg23 on Saturday October 29, 2022 @08:19PM (#33389393)
It's mostly Linux, written in C, a bit of math for GNC, quaternions etc.. nothing too crazy if you have a math + good C background.

They put the flight software through a lot of hardware in the loop tests, simulating launch. That's probably the main novel thing.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by saagarjha on Saturday October 29, 2022 @08:27PM (#33389456)
You can probably read the code. Whether you can accurately assess it for the right set of tradeoffs is generally far harder.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by Nicksil on Saturday October 29, 2022 @07:52PM (#33389173)
>Bullshit. I'm a web engineer, and I wouldn't be competent to audit code that runs Space X's rockets, nor would anyone I know in my career space.

You're still missing the point. It isn't about knowing how rockets work or anything in particular. It's about being able to search for and recognize issues.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by inerte on Saturday October 29, 2022 @07:59PM (#33389212)
At Twitter's scale the issues are far less about "this code smells funny" but how dozens of systems interact together. Ain't nobody gonna figure out problems by simply checking out a handful of repositories, and specially in a few hours / days.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by Nicksil on Saturday October 29, 2022 @08:02PM (#33389245)
>At Twitter's scale the issues are far less about "this code smells funny" but how dozens of systems interact together. Ain't nobody gonna figure out problems by simply checking out a repository, and specially in a few hours / days.

I don't know why you think someone would have to know about Twitter's specific implementation of distributed systems in order to inspect those systems for non-twitter-specfifc issues.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by inerte on Saturday October 29, 2022 @08:08PM (#33389300)
Ack, and thanks for your comment. I am choosing to disengage from this conversation, have fun!
Re:Tesla engineers were on-site to evaluate the Twitter staf... by Nicksil on Saturday October 29, 2022 @08:22PM (#33389422)
>Ack, and thanks for your comment. I am choosing to disengage from this conversation, have fun!

Sorry, what was it about my comment you vehemently disagree with?

Re:Tesla engineers were on-site to evaluate the Twitter staf... by saagarjha on Saturday October 29, 2022 @08:28PM (#33389466)
Issues such as what?
Re:Tesla engineers were on-site to evaluate the Twitter staf... by Nicksil on Saturday October 29, 2022 @08:32PM (#33389503)
>Issues such as what?

Such as those which do not meet the standards defined for the particular audit.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by saagarjha on Saturday October 29, 2022 @08:43PM (#33389607)
I mean, you can do a very general audit of things like “is there CI” or “do they use horribly unsafe serialization everywhere” but I don’t see anything beyond this without a huge amount of effort.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by Nicksil on Saturday October 29, 2022 @08:51PM (#33389679)
>I mean, you can do a very general audit of things like “is there CI” or “do they use horribly unsafe serialization everywhere” but I don’t see anything beyond this without a huge amount of effort.

Do we know what is or isn't being reviewed? Do we know how much effort is being applied in this process or if there's a defined upper and/or lower bound?

Re:Tesla engineers were on-site to evaluate the Twitter staf... by saagarjha on Sunday October 30, 2022 @06:08PM (#33399352)
The upper limit is the point where it makes sense to bring in experts rather than random Tesla engineers.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by moooo99 on Saturday October 29, 2022 @08:04PM (#33389265)
> You're still missing the point. It isn't about knowing how rockets work or anything in particular. It's about being able to search for and recognize issues.

Still, the issues you'll likely encounter inside a car are very very different from the ones you'll see inside a historically grown distributed system serving millions of web, app and API requests. Auditing code is more than analyzing runtime/memory complexity.

Thats not to say it's impossible for Tesla engineers to audit. But I'd imagine it would take quite a bit of time to gather meaningful insight into the landscape and would hardly be an efficient use of the time of senior Tesla engineers.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by Nicksil on Saturday October 29, 2022 @08:11PM (#33389323)
>Still, the issues you'll likely encounter inside a car are very very different from the ones you'll see inside [...]

There are many systems involved in those cars, there's also many people working for those kind of companies who do not solely "work on cars."

>Auditing code is more than analyzing runtime/memory complexity.

I agree.

>Thats not to say it's impossible for Tesla engineers to audit. But I'd imagine it would take quite a bit of time to gather meaningful insight into the landscape and would hardly be an efficient use of the time of senior Tesla engineers.

Take longer than if the team were comprised of Twitter staff or who are already familiar with Twitter's infrastructure and code base? Sure. But that's the case with just about any audit conducted by an outside entity.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by hn_throwaway_99 on Saturday October 29, 2022 @08:00PM (#33389227)
I'm not missing the point. "Searching for and recognizing issues" is not somehow completely independent from the domain of the code in question. More importantly, different domains often have completely different primary concerns. E.g. web app engineering is often concerned with reducing cycle time because the technology means that you can instantly release updates. That's obviously very different from the concerns of launching a rocket, and I would expect them to have very different engineering practices.

In short, experience matters.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by kelnos on Saturday October 29, 2022 @08:37PM (#33389564)
I don't think Musk is looking for an itemized bug list. He's probably more interested in overall code quality, estimates on the level of technical debt, and a qualitative feel about the state of things. While a distributed systems person would have an easier go of that, I think most decently-well-rounded software developers would provide value there.

Also remember that Telsa runs their own distributed systems to support the connected features of their cars. Certainly that's not the same as Twitter, but it's definitely in the same ballpark.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by Nicksil on Saturday October 29, 2022 @08:05PM (#33389276)
>I'm not missing the point. "Searching for and recognizing issues" is not somehow completely independent from the domain of the code in question. More importantly, different domains often have completely different primary concerns. E.g. web app engineering is often concerned with reducing cycle time because the technology means that you can instantly release updates. That's obviously very different from the concerns of launching a rocket, and I would expect them to have very different engineering practices.

>In short, experience matters.

Is this based off real experience in the field of auditing?

Re:Tesla engineers were on-site to evaluate the Twitter staf... by uoaei on Saturday October 29, 2022 @07:58PM (#33389201)
I implore you to consider that auditing code is more than just big-O analyses.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by Nicksil on Saturday October 29, 2022 @07:59PM (#33389210)
>I implore you to consider that auditing code is more than just big-O analyses.

That isn't at all what I said nor implied.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by uoaei on Saturday October 29, 2022 @08:11PM (#33389324)
You seem to have fooled literally every person replying to your post, then. Perhaps consider that your reactionary post was not quite as "rational" as you'd like to believe, or at least that you haven't yet said what you mean.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by Nicksil on Saturday October 29, 2022 @08:25PM (#33389439)
>You seem to have fooled literally every person replying to your post, then.

Fooled? No.

>Perhaps consider that your reactionary post was not quite as "rational" as you'd like to believe, or at least that you haven't yet said what you mean.

What part of any of my comments are reactionary? What have I said that I don't mean? I genuinely don't understand.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by kernal on Saturday October 29, 2022 @08:11PM (#33389334)
> Bullshit. I'm a web engineer

Did you graduate with an engineering degree or do you just like calling yourself a "web engineer"?

Re:Tesla engineers were on-site to evaluate the Twitter staf... by Jach on Saturday October 29, 2022 @08:51PM (#33389673)
It's more polite than "webshit", but the overall comment isn't making the case that the more polite term should be used... I'd like it if people calling themselves web engineers held themselves to a higher standard. And were able to see that even if someone is a webshit now, they might have been something else in the past, and something else again in the future, tech is such a great career space in that it's not incredibly difficult to change what domains you're working on. Even apart from general expertise that allows an engineer to go and review arbitrary code (with full understanding, and immediately? No, but you can get started, and find common areas and boundaries, and find who the main contributors are and who isn't so important, and draw big black boxes over areas that really need a specific expert's look, and there's tons of automated tools that can help too e.g. security audit consulting firms can find issues in huge codebases quite fast), it would be surprising if Tesla didn't already have some former Twitter employees already who could contribute to this review if it makes sense to use them. Let alone former employees from other companies that have systems similar to Twitter's. It'd be surprising if all the engineers selected were just people who have never worked for another company besides Tesla.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by eternalban on Saturday October 29, 2022 @08:10PM (#33389313)
This is true to an extent. At the extremities, a software system can be audited without any previous domain exposure: architecture, and implementation means (language, deployment, etc.)

It can be difficult to critique choices, specially in system architecture, when domain requirements are presented or falsely asserted as 'dictating the choice', but this can be addressed by directing conversations with stakeholders to nail down the conceptual model that has been implemented in architecture style x, and then the conversation is on more equal footing.

Auditing language usage, libraries, development process, etc. are much less domain specific.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by TigeriusKirk on Saturday October 29, 2022 @08:05PM (#33389277)
Really the only way I've been able to make sense of it is if he brought people in to produce an audit of changes made in the last couple months. IE he knows he doesn't know how to get that information, but he knows he has people at Tesla who do.

They wouldn't have to understand any of it, just be able to identify where and when changes occurred, for later review for any last minute shenanigans.

I'm not personally convinced this whole incident even happened, but if it did, this is what I think would make sense.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by bdowling on Saturday October 29, 2022 @08:05PM (#33389269)
"Tesla engineers don't just work on in-vehicle code though. The servers and systems needed to implement Tesla's telemetry and software updates would be in the same ballpark of expertise, albeit with much less scale and different requirements than Twitter."

-- hn_throwaway_99 https://news.ycombinator.com/item?id=33389128

Re:Tesla engineers were on-site to evaluate the Twitter staf... by hn_throwaway_99 on Saturday October 29, 2022 @08:36PM (#33389552)
Yes, both were me. I'll respond since you seem to be implying my responses are incongruent.

Here I was responding to a comment "What hubris to think that your system is so complex that other engineers couldn't understand it." Which is still a bullshit assessment of the argument (which the original comment was about) that you need expertise in specific domains to be able to evaluate code, which I agree with.

My comment you quoted above is that I think that there would be a subset of Tesla engineers (just not the ones that, for example, solely focus on driver systems) that would overlap with Twitter's domain.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by sidibe on Saturday October 29, 2022 @07:05PM (#33388847)
The hubris is thinking Tesla engineers (who are offered worse pay and wlb relative to Twitter) are coding masters of every domain. I won't even snipe at their own software here.

It takes a long time to onboard to a big mature infrastructure that is serving hundreds of millions of users for people moving from a similar company, even longer if your day job is whatever Tesla engineers are doing.

And that's onboarding, not pretending to be an expert which seems to be the role they're taking on.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by MonkeyMalarky on Saturday October 29, 2022 @07:42PM (#33389108)
On one hand, you can usually safely ignore 99% of code in a system that does the plumbing in a system and focus on the few key algorithms. It wouldn't be a stretch at all to take one of the self driving AI guys and have them evaluate say, an ML classifier used to flag spam/abuse content. That would be well inside their domain.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by saagarjha on Saturday October 29, 2022 @08:30PM (#33389486)
Key algorithms to do…what, exactly? Why not just read the documentation on that instead of having people bring you a random sample of their code on paper?
Re:Tesla engineers were on-site to evaluate the Twitter staf... by naasking on Saturday October 29, 2022 @08:45PM (#33389619)
Who said they're evaluating the code? Maybe they're evaluating the people by asking them questions about the code they've just written. That's a great way to find out who's full of shit.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by saagarjha on Monday October 31, 2022 @09:53AM (#33405704)
Why the printouts then?
Re:Tesla engineers were on-site to evaluate the Twitter staf... by naasking on Monday October 31, 2022 @11:04AM (#33406783)
So they can have the code on hand while interviewing them, ask about the code, why they made a certain choice, what other options were available, etc. Have you never had an interview with code printed out or otherwise provided for discussion?
Re:Tesla engineers were on-site to evaluate the Twitter staf... by saagarjha on Monday October 31, 2022 @11:09AM (#33406863)
Usually we both have copies on our computers.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by johnbcoughlin on Saturday October 29, 2022 @07:46PM (#33389136)
> That would be well inside their domain.

Would it?

Re:Tesla engineers were on-site to evaluate the Twitter staf... by Nicksil on Saturday October 29, 2022 @07:49PM (#33389150)
>Would it?

Would it not?

Re:Tesla engineers were on-site to evaluate the Twitter staf... by ceejayoz on Saturday October 29, 2022 @08:49PM (#33389657)
> It wouldn't be a stretch at all to take one of the self driving AI guys and have them evaluate say, an ML classifier used to flag spam/abuse content.

Facebook has a years-old bug with their ML classifier used to flag spam/abuse content in Facebook Groups. Its exact behavior has morphed over time as they flailingly attempt to fix it, but for several straight months that behavior was "posting to a Group via the Graph API fails with 'unknown error' if it includes a dollar sign". This in an API used by tens of thousands of apps and many millions of users, at one of the richest (in both money and programmers) companies on the planet.

ML is often a black box "computer says no" scenario with little meaningful ability to debug.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by Nicksil on Saturday October 29, 2022 @07:34PM (#33389043)
>The hubris is thinking Tesla engineers (who are offered worse pay and wlb relative to Twitter) are coding masters of every domain. I won't even snipe at their own software here.

Who is thinking that?

Re:Tesla engineers were on-site to evaluate the Twitter staf... by hnfong on Monday October 31, 2022 @03:48PM (#33411804)
I think you think that the GP is thinking the Tesla engineers are thinking that :D
Re:Tesla engineers were on-site to evaluate the Twitter staf... by Turing_Machine on Saturday October 29, 2022 @08:17PM (#33389374)
You don't need to be a "coding master of every domain" to recognize shit code.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by arrow7000 on Saturday October 29, 2022 @08:29PM (#33389478)
Is there any reason to suspect that there is shit code at Twitter? Or that that is in any way related to the problems that Twitter has or the "problems" that Elon said he wanted to fix?
Re:Tesla engineers were on-site to evaluate the Twitter staf... by Turing_Machine on Monday October 31, 2022 @06:22PM (#33414139)
There isn't any reason to expect that there isn't shit code, either.

If you spend $44 billion dollars on something, you're going to want to look it over extremely well.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by saagarjha on Saturday October 29, 2022 @08:29PM (#33389481)
Pay and work life benefits are generally independent of technical competence. But I agree with the rest of your comment.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by onethought on Saturday October 29, 2022 @08:38PM (#33389567)
Knowing enough to call bullshit is way lower than knowing enough to contribute meaningfully to a codebase.

Even just having access to code while someone talks I can prove with much higher value questions on what they are saying… because even if I’m asking something dumb it prompts a deeper explanation.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by newZWhoDis on Saturday October 29, 2022 @08:07PM (#33389286)
Also what hubris of the Twitter engineers, who seem to be in this thread, thinking anything they built was that special.

Your video player somehow managed to be worse than Reddit, the perennial joke of web dev.

Not to mention “something went wrong” again and again, maybe by the 4th reload would a linked tweet actually show up for me.

I’m on 25 gig symmetrical fiber with <10ms to my local AWS hub, FWIW

Re:Tesla engineers were on-site to evaluate the Twitter staf... by spoils19 on Saturday October 29, 2022 @08:12PM (#33389341)
Video player works fine for me, and I'm on 20 gig symmetrical fiber with 8ms to my local Azure hub. Maybe check your firewall settings?
Re:Tesla engineers were on-site to evaluate the Twitter staf... by giantrobot on Saturday October 29, 2022 @08:17PM (#33389373)
> Not to mention “something went wrong” again and again, maybe by the 4th reload would a linked tweet actually show up for me.

That seems like a purpose-built feature to punish non-Twitter users to induce them to sign up for Twitter and/or use the Twitter app instead of the web UI. It's not a bug but a misfeature.

Re:Tesla engineers were on-site to evaluate the Twitter staf... by barry-cotter on Saturday October 29, 2022 @09:05PM (#33389792)
No, that happens on the Twitter app too.
Re:Tesla engineers were on-site to evaluate the Twitter staf... by closeparen on Saturday October 29, 2022 @09:30PM (#33389956)
What hubris to think you can wander into a system as complex as a $44bn tech company and draw any meaningful conclusions about it at all in a weekend. I mean, in my world we have some regular "tech reviews" with our Sr. Directors/VPs at a pretty abstract slides-and-diagrams level, and even that takes weeks of prep leading up like 4+ hour presentations. For something with only a glancing relationship to code level reality. I have occasionally seen my work reflected in these reviews, and the game of telephone that produces them is quite lossy.

Any competent engineer can come to understand most any part of the system, but it's going to take minimum several weeks to understand what's really going on in a little 0.01% corner of the engineering shop.