The Jev Revolution: What Engineers Are Already Building with System One Models

Today at the AI Engineer Code Summit, the main theme was system one models — non-generative models that return structured choices, scores, or probabilities. They’re also typically much faster and cheaper than reasoning LLMs (a.k.a. system two models).  And who better to give an inspirational rallying cry for system one models than Diogo Almeida, co-founder and CEO of TypeSafe AI. Almeida was lead creator of Jev, the product that kicked off this trend less than two months ago. Jev was an instant revelation for AI engineers when it was released in September. In a podcast with Latent Space on launch (which quickly became our most popular episode of the year), Almeida explained the core problem with making LLMs the heart of software: reliability.  In his keynote at AIE Code today, he took it a step further by claiming that Jev is “reliable intelligence” that should be used for “just automating everything.” Integrating system one models into code Of particular interest to AI engineers is Almeida’s point that system one models should become integrated with code. As he put it in his keynote, “code is the language of automation, therefore you need to integrate with code.” Latent Space founder swyx interprets this as a call to “liberally sprinkle ‘System One’ model calls everywhere INSIDE application code to make smarter software instead of increasingly convoluted forms of ‘concat strings together and hope for the best" AI engineering.’” The term “smart software” was used throughout Almeida’s presentation. It sounds like the kind of term used by companies like Microsoft or IBM during Web 2.0, but in the AI era it does actually signify a harsh reality about LLM-driven software: sometimes it’s just not very smart. As Almeida quipped during his presentation, “you get Einstein 95% of the time, but Mr. Bean 5% of the time.” “And if you can't tell which is which, how the fuck are you going to automate anything,” he added. Almeida ended with an impassioned plea to “automate everything.” Like an AI-pilled, pink-haired Steve Jobs, he called out to “the dreamers and the people who don't want to be limited by everything that has happened so far,” promising that TypeSafe AI wants to “really fight for you, and we want this to be as reliable as a fourth logic gate.” How Glean uses Jev With the unenviable task of immediately following Almeida on stage, Chinmay Goyal from Glean presented a more prosaic version of Jev usage. He explained that system one models are used at Glean alongside LLMs and other types of classifiers, and that Jev is especially suited for “zero-shot flexibility.”  “As the taxonomy of the decision space changes, we don't have to fine-tune these models again and again,” he said. In a short interview after his presentation, Goyal told Latent Space that their customers don’t actually need to know which type of model they’re using. “Jev powers a significant number of flows [but] the user just needs to have a product that works; that is always our aim with building anything. So we want to hide away all the complexity of whether we are using Jev or we are using a full-blown LLM.” In the conclusion of his presentation, Goyal offered some rules on when to use Jev: Jev patterns and prototypes Next up at AIE Code were Chibi Vikram and Kevin Bernstein from UiPath, a business orchestration and automation platform. One of their examples showed a test of Jev ranking document chunks. They held a “tournament” where Jev picked the most relevant chunks from small groups, then the winners advanced to another round, and so on. This whole process took under a second, even with thousands of Jev calls. “We realized that Jev is a building block that we can adjust,” said Vikram about this and other experiments, “and we have to find these new strategies where we will be able to apply this building block. It might look less intelligent on its own, but with these strategies and patterns, we can get more intelligent — with even the current Jev model that we have.” Scott Roberts, a Principal Engineer at Workday, finished the ‘system one’ session at AIE Code with some prototypes his team has run with Jev. In one, an interface starts to fill in a time-off card while the user is still typing. It was an interesting showcase of Almeida’s idea that a fast model call can change what the software itself does. Roberts described it as going “from predicting what you want to say to predicting what you want to do.” Like the early internet The biggest learning I took from the System One presentations today is that a model like Jev can open up surprising new ways to build software. I was sitting next to a conference attendee who heads up AI at his company. He told me that in his spare time, he used Jev to create a website that sorts and ranks news about fantasy football. But the final word has to go to Almeida, who likened the new opportunities opened up by system one models to the experimentation of the early internet. “I think that there's going to be an explosion of what software can do, and we will want people to be exploring those boundaries of software. Not necessarily because it's valuable, but because they can, because we have new frontiers to explore, and this is what the early internet was like.”