yoelhacks

Ah sorry, you were very clear on the pricing page and I meant 100k LoC, not tokens.

In your explanation here, you mention running it per PR - does this mean running it once? Several times?

$8/100k tokens strikes me as potentially a TON if the idea is that we're going to be running this as part of the iterative local development cycle (or god forbid letting agents run it whenever they decide). As you mentioned, one of the issues with AI generated code is often that it writes too much and needs direction on shrinking down.

I could easily see hitting 10k+ LOC on routine tickets if this is being run on each checkpoint. I have some tickets that require moving some files around, am I being charged on LOC for those files? Deleted files? Newly created test files that have 1k+ lines?

Very interesting, I like that they call out the particular areas where it is strong.

Maybe. Current prices certainly include probability that Tesla is able to pull off major transformation into more than just a car manufacturer (batteries, supercharge network, self-driving software, etc). The implicit assumption is that Musk is helpful for that - which would certainly have been true some years ago. Getting rid of him would be an admission that the loftiest dreams won't happen as fast as some would like, but doesn't necessarily mean they're out of the picture.

Had fun making this visualization of how security works - look right? https://eraser.io/git-diagrammer?diagramId=Gyr7VvuFOGNGsnznQ...

After seeing some really cool tools that build high-level overview diagrams of a repo, we wanted to see if we could create a tool for building any diagrams from a codebase (for the diagrams we support, which are architecture, ERD / schema, flow charts, and sequence diagrams).

Try it out on any public Github repo! A few notes: 1) Works best if you are pretty explicit with the prompt. We're only offering 2 free runs right now because of the expense involved. We have some examples built in and I've left a couple links below. 2) Works best on repos of a few hundred files or low thousands. More than that and it really helps if you can provide some explicit instructions on where to look. 3) The feature that links to specific code files is pretty fun.

Here a some examples:

Supabas real time updates: https://eraser.io/git-diagrammer?diagramId=2XXiOvPfJjtfMbWdM... Subase realtime architecture: https://eraser.io/git-diagrammer?diagramId=nIvgs2dMvarCYJ8zV... Primsa migrations: https://www.eraser.io/git-diagrammer?diagramId=uK7UptrWd4EEY...

I was also unclear what this actually did. Not sure how much this helped, but it is a nice visualization of what's happening: https://eraser.io/git-diagrammer?diagramId=ysiiD14zhpPemvoJO...

I was curious about Mistral so I made a few visualizations.

A high level diagram w/ links to files: https://eraser.io/git-diagrammer?diagramId=uttKbhgCgmbmLp8OF...

Specific flow of an OCR request: https://eraser.io/git-diagrammer?diagramId=CX46d1Jy5Gsg3QDzP...

(Disclaimer - uses a tool I've been working on)

I often see takes on this topic from the engineering side. "It's hard!". "Managers just don't understand".

It feels like as a community, it would be useful to get more articles seeing things from the other side and exploring functional approaches beyond provide-a-worst-case-scenario-estimate.

There's a reason this dynamic is so pervasive. In order for everyone in an organization to do their job well, people do often need a realistic set of estimates. Can sales promise the prospect this integration? Can marketing plan a launch for the new feature? Can the CPO make a reasonable bet on a new line of work?

In my experience, the nuance here is more about handling the mis-estimates. How do we discuss the risks up front? How much work should we put into contingency planning? How do we handle the weeks / months before a deadline when it is clear that we need to limit scope?

Javascript is outperformed by WebAssembly in the sense that it runs faster.

A lot of things that bring a lot of value to a lot of people are still much, much faster to build via the JS / TS ecosystem.

It absolutely makes sense that calculation-heavy workloads will be ported to WASM, but there's a lot more to building an app.

The shift right parts of the article are more interesting...

I used to work on an app that very heavily leaned on Elasticsearch to do advanced text querying for similarities between a 1-2 sentence input and a corpus of paragraph+ length documents.

It was fascinating how much tokenization strategies could affect a particular subset of queries. A really great example is a "W-4" or "W4" Standard tokenization might split on the "-" or split on letter / number boundaries. That input now becomes completely unidentifiable in the index, when it otherwise would have been a very rich factor in matching HR / salary / tax related content.

Different domain, but this doesn't shock me at all.

Choosing which immoral deeds to make illegal is a very central role of government!

At companies I've been at (mostly earlier phase startups, YMMV) there has always been an effort to do some sort of technical vetting.

Designers need to present designs / their portfolio.

Sales people need to do a demo.

Product people need to put together a mock roadmap or pitch a feature.

And so on.

It sounds to me like what you're describing is a single *tool*, not a single model. Is that possible?

I agree that we need multiple views, but it isn't just a matter of filtering part of a single view - it's also different fundamental primitives that are needed to communicate different kinds of ideas or aspects of a given system.

To me, this seems parallel to the evolution of LLMs into multi-modality by default. Different queries will need different combinations of text, heat maps, highlights, arrows, code snippets, flow charts, architecture diagrams, etc.

It's certainly not easy, but it's exciting to consider :)

> (Which is often maintained manually, separate from the repo.)

To me, this is the interesting avenue for investigation.

Rather than go from visualization -> code, how can we take an existing visualization that represents some underlying system (a code base, module dependencies, service architecture, a network topology, etc) and easily update the representation as the underlying system changes...

I think there's a very important real-world nuance here.

What you want with a programming language is to handle granular logic in a very explicit way (business requirements, precise calculations, etc.). What this article posits, and what I agree with, is that existing languages offer a more concise way of doing that.

If I wanted to program in a visual way, I'd probably still want / need the ability to do specific operations using a written artifact (language, SQL, etc). Combining them in different ways visually as a first-class operation would only interest me if it operated at the level of abstraction that visualizations currently operate at, many great examples of which are offered in the article (multiple code files, system architecture, network call).

This is exactly (one of) our use cases at Eraser - taking code or natural language and producing diagram-as-code DSL.

As with other situations that want a custom DSL, our syntax has its own quirks and details, but is similar enough to e.g. Mermaid that we are able to produce valid syntax pretty easily.

What we've found harder is controlling for edge cases about how to build proper diagrams.

For more context: https://www.eraser.io/decision-node/on-building-with-ai

Very interesting to see that they've explicitly codified the role of the system prompt vs. user prompt. Have folks seen improvements by moving meta-task description into system prompt and out of the assistant <> user conversation?

I've noticed a lot of in-depth content on the mechanisms of LLMs themselves, but not nearly as much tactical advice for how to plan, ship, and iterate on differentiated features - especially outside of the world of general prompt engineering techniques. Would love to hear from folks who have other preciously won nuggets of wisdom!