CompleteSkeptic

we've been public about this before the category was even created

of all comments, "enterprise" taste is among those I'd least expect

intentional to make it fun (:

typesafe ai | ONSITE in SF | basically all forms of engineering (front-end / back-end / infra)

help make jev!

typesafe.ai/careers

yes and can do many of those in parallel

unfortunately all hand-written :( my chief-of-staff does unironically handwrite em dashes though

the edit is right - jev would be cheaper, faster, and more self-consistent (in general)

we actually use astra (and fable) in this way for our evals: evals.typesafe.ai

someone on the team cooked hard on that and it shows example traces comparing our model to opus/sol

1. yes a general model 2. no training at all 3. but it is focused on "System 1" tasks (more human judgment, less math reasoning)

strings (and all sequential data structures) are not allowed at all - this is how we make sure all outputs can be computed in parallel (thus no output token cost)

They don't like adding stealth startups :(

1. I am extremely on the same page 2. I do think that subconscious is not only much smarter than we give it credit for, but also much more robust than the "jagged frontier" of current LLMs

(shilling my blog post on that jaggedness: https://www.completeskeptic.com/p/lies-damned-lies-and-bench...)

very accurate!

the one nuance I'd get into is I'd call it "zero-shot" over "instruction-tuned" (the latter often implies a particular distribution), but very safe for sharing

> Type safety is not factual correctness.

I very much agree with this and want to hone in on where do actually disagree. Would you say a linear classifier hallucinates?

inputs are structured program state. there is an example at around second 30 of the doom demo

(though ideally everyone gets off the waitlist and can try it out for themselves )

we hope so! the bigger hope is to not just eat LLM market share, but to allow for people to use AI much more in the inner loop of software

the hard part for coding is actually state engineering (e.g. getting your dependencies in context) - we haven't even tried it yet (because my philosophy is we should automate the easy tasks before the hard and we've been working on getting the model smart on the former)

we do think there's a lot of potential though and do want coding themed releases soon

thanks a ton!

constrained decoding (OpenAI-style structured outputs) make models dumber unfortunately - the short+dense version is that simply masking logits is insufficient because if ever a model was assigning probability to an invalid token, the model is by definition confused. you'd be better off erroring IMO

architecture is close to the chest for now, but we have talked about writing a paper

I don't want to shill my blog too much, but I will say data is probably far most interesting than architecture: https://www.completeskeptic.com/p/the-bitterest-lesson

love that you love the manifesto! letting the first batches off the waitlist now, but we do have some early users describing their experience (https://x.com/danshipper/status/2099947471518474522)

that's right, but because these models are probabilistic, it's also possible to be confidently wrong (and all future models will be smarter still and still have that possibility)

I'm biased but I wouldn't call it misleading - generating text is super awesome and flexible, (we describe that in the blog post - and I personally use string models all the time) but it's true you pay a high tax for autoregressive generation

> Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.

that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do

exactly right!

definitely not AI-generated - this is my real wardrobe

we also thought the voice at the end was AI-ish, but apparently that's a real voice actor but slightly sped up

we have played with this! the fascinating thing we've found so far is that adversarial examples for our model are quite different from that of LLMs so that they work even better together

it is just a model, no harness yet ;)

it is a structured data model, but technically not a language model (it doesn't generate language)

it's our output tokens that are free (under the system one / jev column)

you could, but it the model is not optimized for text

this is complex, but generating text is highly complicated and requires mode dropping to make long cohesive text

CEO here - that is right!

I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable).

But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)

constrained decoding tends to make models dumber - this is why it's rarely used