Jev-GPT/Ask-Jev

How it works

ChatGPT writes one word token at a time. It reads everything so far, picks the next token, and repeats. Ask it anything and you get prose back:

But Jev, it can’t write. It must choose.

Jev is an evaluation model from TypeSafe. Given a situation, a question, and a few possible answers, it returns a probability for each option. Someone put it well: a smart switch statement.

TypeSafe calls it a System One Model, inspired by Kahneman’s term for fast, gut-feel reaction.

Look at these three faces:

😀 😡 😢

You knew which one was angry without thinking or calculating. That is System One.

Now consider this:

A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?

That pause you just felt is System Two. (Answer: 5 cents)

Jev’s day job: decisions inside software, like which team gets a support email.

Or more important questions, like:

const { answers } = await experimental_evaluate({
  model: "typesafe-ai/jev",
  state: "My boss messaged me at 5:55pm on a Friday: “Can we talk?”",
  questions: {
    q: {
      type: "choice",
      instructions: "What’s about to happen?",
      criteria: {
        a: "I messed something up",
        b: "I’m getting fired",
        c: "I’m about to get more work",
        d: "It’s a quick question",
      },
    },
  },
});

answers.q.probabilities;
// { a: 0.34, b: 0.08, c: 0.40, d: 0.18 }

An 8% chance I’m getting fired, which my nervous system rounds up to certainty. It panics exactly like I do. (Try your own in Ask Jev.)

So Jev can’t write prose. But nothing moves a person like being told a thing cannot be done. 🔨

Making it write

ChatGPT is choosing too. So why couldn’t Jev?

The first problem is scale. Jev takes at most 255 choices at a time, not hundreds of thousands.

So I took a page whole book out of xkcd’s book.

In Thing Explainer, xkcd explains rockets, phones and the human body using only the thousand most common English words.

Those thousand ten hundred words (“thousand” isn’t one of them) became Jev-GPT’s entire vocabulary.

It’s a tight list. “Boss” isn’t on it. Neither is “atmosphere,” so Jev has to explain the sky with “air” and “light.”

Let’s take it one word at a time

Say you ask “Why is the sky blue?” and Jev has written “The sky is” so far. To choose the next word, I give it options like:

The sky is blue
The sky is very
The sky is full
The sky is not

Jev scores them. The winner gets added to the answer. Repeat. That’s it.

But Jev can only do 255 options

A thousand words don’t fit, so we hold a tournament to find the next word.

1. Shuffle the vocabulary into four groups of 250 and send all four groups to Jev.
2. The top 25 from each group, 100 in all, go to a final round with punctuation and an option to stop.
3. The winner becomes the next word.

So every word costs five games: four groups, then the final.

In the demo above, you can click any word, on the right, to see what else it was guessing at.

How does it compare with real LLMs?

Under every answer, the demo shows Gemini 2.5 Flash-Lite. Same question, same brief, same ten hundred words.

Gemini takes under a second and about a hundredth of a cent. Jev takes about half a minute and about two cents.

The half minute is my fault, not Jev’s. Gemini answers in one call. Jev needs five per word, about ninety trips across the network.

And Gemini breaks the rule. It has the same list, and it still reaches for “things,” “app” and “kids.” Those are marked in red. Jev never breaks the rule, because it can only choose, never write.

What were my takeaways?

In case the ridiculousness didn’t come through: if you want prose, use an LLM. This one is NSFW, in the literal sense.

• The options are the whole game. Anything you can turn into a list becomes something Jev can decide.
• The shape of the question matters. Offered as one option among a thousand words, “stop here” was almost never picked. Asked as its own yes-or-no question, Jev stops in the right place.
• No hallucinations is not no mistakes. Every word it used was on the list. It just picked the wrong ones.
• Its world knowledge is thin, but better calibrated than I expected. Asked whether a person is real, Jev gives Elon Musk 98%, Diogo Almeida, who built Jev, 74%, me 53%, and a name I made up 9%. Gemini Flash-Lite was certain about the first two, and certain that I don’t exist.

So, three kinds of model:

So use it to route, sort, flag and score. Split a hard question into small ones and do the thinking in your own code. And keep it away from anywhere a mistake is expensive, because it is fast, cheap, confident and sometimes wrong.

And don’t ask it to write. Although I recommend trying it once.