kierangill

Mine as well. Thanks for the callout. It has been added.

My revolt is against the cognitive stress of reading generated text. A trope typically indicates I’m in for an uphill read.

I recently read this William Zinsser quote that inspired a nickname for this: Clotted Claude [1].

> Nobody has made the point better than George Orwell in his translation into modern bureaucratic fuzz of this famous verse from Ecclesiastes:

> > I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favor to men of skill; but time and chance happeneth to them all.

> Orwell's version goes:

> > Objective consideration of contemporary phenomena compels the conclusion that success or failure in competitive activities exhibits no tendency to be commensurate with innate capacity, but that a considerable element of the unpredictable must invariably be taken into account.

> First notice how the two passages look. The first one at the top invites us to read it. The words are short and have air around them; they convey the rhythms of human speech. The second one is clotted with long words. It tells us instantly that a ponderous mind is at work. We don't want to go anywhere with a mind that expresses itself in such suffocating language. We don't even start to read.

[1] https://blog.kierangill.xyz/clotted-claude

I’m happy to see attention in this area of research. I don't see this referenced in the paper, but a related paper worth reading: "The Medium is the Message: How Non-Clinical Information Shapes Clinical Decisions in LLMs" [0].

> Through the perturbation of patient messages, we evaluate whether LLM behavior remains consistent, accurate, and unbiased when non-clinical information is altered. […] Our findings reveal notable inconsistencies in LLM treatment recommendations and significant degradation of clinical accuracy in ways that reduce care allocation to patients. […] Our perturbations reflect realistic patient messages from electronic formatting errors and/or simulate patient groups that would be impacted by a wide adoption of patient-AI systems (female patients, non-binary patients or those who use gender-neutral pronouns, patients with health anxiety, patients with a more dramatic disposition, patients with less technological aptitude, and patients with limited English proficiency, etc.)

We’re all peering down the kaleidoscope of a trillion parameter model. It’s no surprise gentle nudges in inputs (grammar, language proficiency, cultural norms) yield different outcomes, despite the intent not changing. It’s one thing to generate crap code, it’s another to generate crap medical advice.

[0] https://dl.acm.org/doi/10.1145/3715275.3732121

Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages.

For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration.

Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations lead to more frustrating future chats because the LLM misses intent and or context. (Or, the presence of paragraphs and paragraphs of LLM output makes the next token predictor dumber? Unsure.)

I work at a telemedicine company. We’ve benchmarked a few frontier LLMs on public medical imaging datasets. One test included high-quality and high-consensus otoscopic images. We didn’t anticipate the models to do well on something so niche, but what concerned us was how poorly calibrated the models were.

I know you can’t trust an LLM’s self-assessed “confidence” of a prediction, but I’ve found that confidence can at least be directionally correct for some tasks. For our benchmarks, however, confidence was poorly correlated. What’s worse is that binary classification models (“Do you see $diagnosis in this photo?”) highly influenced the LLM to confidently predict $diagnosis.

I’m concerned for those using LLMs for diagnostics, and getting confidently led to the wrong conclusion.

Blueberry Pediatrics | Full-stack Senior or Staff Engineer | REMOTE (US only)

American healthcare is seldom affordable, accessible, or high-quality. We are fixing this for pediatrics. Blueberry is the most affordable option amongst our competitors. We practice the highest quality pediatric telemedicine, as evidenced by our exclusive hiring of board-certified pediatricians and the usage of at-home medical kits. And, we’re accessible 24 hours a day.

Our success is shown in the lives we save, the costs we save our insurers, and our exploding B2B and D2C business.

As you can imagine, pulling off affordable high-quality healthcare is a challenge. It requires a lot of engineering ingenuity, a C-suite aligned with positive patient outcomes above short-term profits, and a great product team.

We use Django, Hotwire Turbo (an HTMX-like framework), Pytorch, Sklearn, and Flutter. Experience in these technologies helps, but what’s more important is general full-stack knowledge, curiosity, and a strong work ethic.

Full-stack engineer: https://jobs.ashbyhq.com/blueberrypediatrics/dc8108f3-34ed-4...

Our homepage: https://blueberrypediatrics.com/

Our engineering blog: https://engineering.blueberrypediatrics.blog/

Blueberry Pediatrics | Full-stack Senior or Staff Engineer | REMOTE (US only)

American healthcare is seldom affordable, accessible, or high-quality. We are fixing this for pediatrics. Blueberry is the most affordable option amongst our competitors. We practice the highest quality pediatric telemedicine, as evidenced by our exclusive hiring of board-certified pediatricians and the usage of at-home medical kits. And, we’re accessible 24 hours a day.

Our success is shown in the lives we save, the costs we save our insurers, and our exploding B2B and D2C business.

As you can imagine, pulling off affordable high-quality healthcare is a challenge. It requires a lot of engineering ingenuity, a C-suite aligned with positive patient outcomes above short-term profits, and a great product team.

We use Django, Hotwire Turbo (an HTMX-like framework), Pytorch, Sklearn, and Flutter. Experience in these technologies helps, but what’s more important is general full-stack knowledge, curiosity, and a strong work ethic.

Full-stack engineer: https://jobs.ashbyhq.com/blueberrypediatrics/dc8108f3-34ed-4...

Our homepage: https://blueberrypediatrics.com/

Our engineering blog: https://engineering.blueberrypediatrics.blog/

Agreed here. A key theme, which isn’t terribly explicit in this post, is that your codebase is your context.

I’ve found that when my agent flies off the rails, it’s due to an underlying weakness in the construction of my program. The organization of the codebase doesn’t implicitly encode the “map”. Writing a prompt library helps to overcome this weakness, but I’ve found that the most enduring guidance comes from updating the codebase itself to be more discoverable.

I applied in early 2021. Getting rejected from any company carries a sting, but I was grateful to have gone through the process.

I didn’t realize at the time, but Oxide’s application process was the best form of interview prep I’ve done. The process forced me to thoroughly document my values and career accomplishments. In later non-Oxide interviews, I effectively recited what I had written my materials. In that way, it has felt less one-sided than every other company application process I’ve gone through. I was able to take away an artifact from the experience, versus being filtered out via a coding challenge. It’s also been rewarding to reflect on my submission from years ago to see how my mindset and skills have evolved.

If you have any interest in working in the pediatric telemedicine space, I encourage you to email me your application. We accept Oxide materials. I’m happy to provide feedback as a hiring manager. My email and our company website are in my bio.

Past the basic setup, what are people doing to grease the tracks for these tools?

That is, how are people organizing their context and codebase to help the tooling guide themselves to the right answer?

I have some half-baked thoughts here [1], but I know there are more (better) methodologies to be discovered.

[1]: https://blog.kierangill.xyz/oversight-and-guidance

This is buried in a footnote, but worth clarifying: "How Life Works" offers much more than just the history of our understanding of biology. Oxide and Friends has good coverage on the book here https://oxide-and-friends.transistor.fm/episodes/bookclub-ho...

Blueberry Pediatrics | Full-stack Senior or Staff Engineer | REMOTE (US only)

American healthcare is seldom affordable, accessible, or high-quality. We are fixing this for pediatrics. Blueberry is the most affordable option amongst our competitors. We practice the highest quality pediatric telemedicine, as evidenced by our exclusive hiring of board-certified pediatricians and the usage of at-home medical kits. And, we’re accessible 24 hours a day.

Our success is shown in the lives we save, the costs we save our insurers, and our exploding B2B and D2C business.

As you can imagine, pulling off affordable high-quality healthcare is a challenge. It requires a lot of engineering ingenuity, a C-suite aligned with positive patient outcomes above short-term profits, and a great product team.

We use Django, Hotwire Turbo (an HTMX-like framework), Pytorch, Sklearn, and Flutter. Experience in these technologies helps, but what’s more important is general full-stack knowledge, curiosity, and a strong work ethic.

Full-stack engineer: https://jobs.ashbyhq.com/blueberrypediatrics/dc8108f3-34ed-4...

Our homepage: https://blueberrypediatrics.com/

Our engineering blog: https://engineering.blueberrypediatrics.blog/

Blueberry Pediatrics | Full-stack Senior or Staff Engineer | REMOTE (US only)

American healthcare is seldom affordable, accessible, or high-quality. We are fixing this for pediatrics. Blueberry is the most affordable option amongst our competitors. We practice the highest quality pediatric telemedicine, as evidenced by our exclusive hiring of board-certified pediatricians and the usage of at-home medical kits. And, we’re accessible 24 hours a day.

Our success is shown in the lives we save, the costs we save our insurers, and our exploding B2B and D2C business.

As you can imagine, pulling off affordable high-quality healthcare is a challenge. It requires a lot of engineering ingenuity, a C-suite aligned with positive patient outcomes above short-term profits, and a great product team.

We use Django, Hotwire Turbo (an HTMX-like framework), Pytorch, Sklearn, and Flutter. Experience in these technologies helps, but what’s more important is general full-stack knowledge, curiosity, and a strong work ethic.

Full-stack Senior engineer: https://jobs.ashbyhq.com/blueberrypediatrics/dc8108f3-34ed-4...

Our homepage: https://blueberrypediatrics.com/

Our engineering blog: https://engineering.blueberrypediatrics.blog/

> but why does this have to be the case?

From the essay’s context, I take it to mean “is benefitted by” rather than “must absolutely”. Maybe my world view is distorted by Zinsser but I see this as an authorism.

A writer can choose to trade off vigor for nuance by hedging. They can preempt arguments with “it is my opinion that” and “one ought to”. But, it is my opinion that, exhaustive disclaimers are not fun to read. I know it’s his opinion — this is posted to his Internet Blog, not a textbook.

> Don't be kind of bold. Be bold.

I’m a sucker for when the form serves as an example for the author’s idea.

> If it were, it wouldn't be good, because the rhythm of good writing has to match the ideas in it, and ideas have all kinds of different shapes. Sometimes they're simple and you just state them. But other times they're more subtle, and you need longer, more complicated sentences to tease out all the implications.

From William Zinsser’s On Writing Well:

> The growing acceptance of the split in-finitive, or of the preposition at the end of a sentence, proves that formal syntax can't hold the fort forever against a speaker's more comfortable way of getting the same thing said—and it shouldn't. I think a sentence is a fine thing to put a preposition at the end of.

Another from the same book:

> CREEPING NOUNISM. This is a new American disease that strings two or three nouns together where one noun—or, better yet, one verb-will do. Nobody goes broke now; we have money problem areas. It no longer rains; we have precipitation activity or a thunderstorm probability situation. Please, let it rain.

> Today as many as four or five concept nouns will attach themselves to each other, like a molecule chain. Here's a brilliant specimen I recently found: "Communication facilitation skills development intervention." Not a person in sight, or a working verb. I think it's a program to help students write better.

Love this post. Writing on programming languages has changed how I think about _programming_ in general.

I often think about this quote from TAPL. This framing of “safety” changed how I design systems.

> Informally, though, safe languages can be defined as ones that make it impossible to shoot yourself in the foot while programming.

> Refining this intuition a little, we could say that a safe language _is one that protects its own abstractions_.

> Safety refers to the language's ability to guarantee the integrity of these abstractions and of higher-level abstractions introduced by the programmer using the definitional facilities of the language. For example, a language may provide arrays, with access and update operations, as an abstraction of the underlying memory. A programmer using this language then expects that an array can be changed only by using the update operation on it explicitly—and not, for example, by writing past the end of some other data structure.

https://www.cis.upenn.edu/~bcpierce/tapl/

Blueberry Pediatrics | Full-stack Senior or Staff Engineer | REMOTE (US only)

American healthcare is seldom affordable, accessible, or high-quality. We are fixing this for pediatrics. Blueberry is the most affordable option amongst our competitors. We practice the highest quality pediatric telemedicine, as evidenced by our exclusive hiring of board-certified pediatricians and the usage of at-home medical kits. And, we’re accessible 24 hours a day.

Our success is shown in the lives we save, the costs we save our insurers, and our exploding B2B and D2C business.

As you can imagine, pulling off affordable high-quality healthcare is a challenge. It requires a lot of engineering ingenuity, a C-suite aligned with positive patient outcomes above short-term profits, and a great product team.

We use Django, Hotwire Turbo (an HTMX-like framework), Pytorch, Sklearn, and Flutter. Experience in these technologies helps, but what’s more important is general full-stack knowledge, curiosity, and a strong work ethic.

Full-stack Senior engineer: https://jobs.ashbyhq.com/blueberrypediatrics/dc8108f3-34ed-4...

Our homepage: https://blueberrypediatrics.com/

Our engineering blog: https://engineering.blueberrypediatrics.blog/

Also, if you haven’t read the linked Oxide RFD#3 [1], I recommend doing so. It has shaped much of our hiring process.

> But how does one assess candidates for such positions? This is an age-old question without a formulaic answer: designing, building, selling, and supporting computing systems is itself too varied to admit a single archetype.

> In terms of evaluation mechanism: using in-person interviewing alone can be highly unreliable and can select predominantly for surface aspects of a candidate’s personality. While we advocate (and indeed, insist upon) interviews, they should come relatively late in the process; as much assessment as possible should be done by allowing the candidate to show themselves as they truly work: on their own, via their creations.

[1] https://rfd.shared.oxide.computer/rfd/3