Respan's Span-01 checks AI agent behavior in a single pass

Respan, a platform for tracing and evaluating AI agents, has introduced Span-01, a new classification model. Span-01 reads the conversation and action logs of an AI assistant or agent and tells you whether certain behaviors appear in them. According to the company, the model can apply behavior definitions it has never seen before and returns its answer in a single pass, without generating text.

What Span-01 is

You describe the behavior you want to catch in a plain sentence, such as "the user expresses frustration" or "the assistant apologizes". The model looks at the conversation and returns three probabilities for each definition: the behavior is present, absent, or cannot be judged from the conversation. Several definitions are checked on the same conversation at once.

According to Respan's blog post, the model was first trained on general classification with reinforcement learning from AI feedback (RLAIF) and then specialized for behavior detection. The company says the goal is for the model to apply new definitions in new domains. It says a hybrid attention design helps keep that reasoning intact across long agent traces.

What it is for

Teams running AI agents cannot read thousands of conversations by hand, so they usually have a large language model review them. Respan pitches Span-01 as a faster and cheaper way to do that job. According to the company's posts, the model is built to catch things like:

• Prompt injections hidden in tool outputs as Markdown or Base64, and jailbreak attempts
• Hallucinations and answers not grounded in sources
• Exposure of secrets such as passwords and keys
• Agents stuck in loops or misusing tools
• User frustration, refund requests and escalations

The company's claims

According to Respan, Span-01 ranked first overall among nine models on its own behavior benchmark, which combines English and multilingual behavior detection. The company says the free Span-01 Lite also beats TypeSafe's decision model Jev and Anthropic's Sonnet 5. Respan also claims Span-01 runs at a much lower cost than Jev and OpenAI's GPT-6 Luna.

In a separate table where the company scores production behaviors by safety, reliability, grounding and response quality, Span-01 comes out ahead of Jev and Sonnet 5, while OpenAI's full GPT-6 Sol model scores higher. Respan itself notes that a full frontier model remains the strongest ceiling in that table.

How to use it

Span-01 is used through Respan's API. A request sends the conversation along with a list of behaviors, each made up of an ID and a one-sentence definition. According to the docs, there is a free tier and a paid tier, and access is enabled per organization by the company. For now, you join the waitlist on Respan's site to get access. The blog post also has a playground that compares Span-01 with Jev on the same example. Current pricing is on Respan's site.

Respan, previously known as Keywords AI, is a startup from Y Combinator's Winter 2024 batch. The Span-01 announcement was published by Frank Chen of the company's research team. All the results shared come from benchmarks Respan built itself, and the model's code and files have not been released. To see how the model performs in real use, it is worth following evaluations from independent researchers as well.

Sources: Respan blog: Span-01, Respan API docs: Score behaviors with Span-01.