AI Decision Models: The Difference Between Them and LLMs

Decision models have recently garnered attention as a distinct approach within the AI landscape. One notable example frequently discussed is Jev, developed by TypeSafe AI. Unlike traditional models, Jev generates structured choices, scores, and confidence estimates rather than plain text.

Although conventional AI models have long been used for predictions and classifications, Jev is specifically designed to make structured decisions that software can utilise directly. This methodology offers an alternative pathway for developing AI applications, particularly when a system needs to assess information and determine subsequent actions.

How does Jev function?

In a typical LLM interaction, you submit a prompt and receive generated text. Even when requesting JSON output, the application must still parse the response and validate its contents before it can be used.

Jev operates differently. You supply the relevant context, define the specific questions you wish to answer, and specify the expected answer type. In response, Jev returns structured results that your application can process immediately.

Jev supports three primary question types:

• Choice: Selects an option from a predefined list, such as billing, technical support, or sales.
• Score: Rates an item against an ordered scale, such as low, medium, or high priority.
• Yes or No: Estimates the probability that a statement is true, for example, determining whether a support ticket requires human attention.

It is possible to query multiple aspects of the same input within a single request. For instance, a support application could classify a ticket, assess its urgency, and determine if it warrants human review in one go. Jev returns these results in a structured format, allowing the application to determine the next course of action.

Decision models vs. LLMs

LLMs and decision models serve distinct purposes. An LLM is capable of explaining decisions, generating content, writing code, or reasoning through complex tasks. Jev, however, is engineered for focused decisions with clearly defined outputs.

The distinction is not that LLMs are incapable of making decisions; they can certainly classify information and return structured data. However, Jev is built specifically around this task, eliminating the need to generate a free-form response that an application must subsequently interpret.

This makes it particularly useful for repeated, well-defined decisions. For tasks requiring detailed explanations, creative output, or complex multi-step reasoning, an LLM remains the more appropriate tool.

What are confidence scores?

A key feature of Jev is that its outputs include probabilities and confidence metrics. These values assist developers in gauging how strongly the model favours a particular answer.

For example, a customer support application might query whether a ticket should be escalated. If the model indicates a high probability that escalation is required, the application can route the ticket to a senior support team. Conversely, if the result is uncertain, it can be directed for human review.

It is crucial to note that confidence does not guarantee accuracy. A model may be highly confident yet still arrive at the wrong decision. Developers should therefore test results against real-world examples and select thresholds that align with the consequences of each decision.

Additionally, it is important to differentiate the probability of an answer from a separate confidence score. The specific interpretation depends on the output type. Applications should adhere to the model’s documentation rather than assuming all returned numbers represent the same metric.

Where can decision models be beneficial?

Classification and routing

Many applications require sorting incoming information prior to processing. A support platform, for instance, might classify requests by topic and direct them to the appropriate team. Similarly, a business application could categorise documents, emails, or customer feedback.

Jev can return a category from a predefined list, enabling the application to route items without the need to parse a generated paragraph.

Scoring and evaluation

Decision models are also capable of evaluating information against a defined scale. An application could, for instance, rate a support ticket by urgency, score a document for relevance, or assess whether a response meets specific criteria.

These results can be leveraged to rank items, prioritise workflows, or identify cases requiring further scrutiny. The quality of these decisions, however, remains dependent on the model’s capabilities and how accurately the criteria describe the task.

AI agents and tool selection

AI agents utilise models to determine which actions to take. An agent might select a tool, classify a request, or decide if a task requires additional information.

While a general-purpose LLM can make these decisions as part of its broader reasoning, a decision model can handle a narrower choice prior to the agent acting.

For example, a custom workflow using Hermes Agent could employ Jev to classify an incoming task before passing it to the agent. It could also assist in determining which workflow should handle a specific request. This would necessitate a custom integration rather than relying on assumed native support.

Decision models should not be relied upon as standalone security controls. If an agent has the capability to execute commands or modify files, permissions and deterministic rules must enforce which actions are permitted. While a model’s classification can inform this process, it should not replace it.

Alternatives to Jev

Jev is part of a broader category of models designed to produce structured decisions. OpenJev and Laya are two other projects worth exploring when comparing different approaches.

OpenJev

OpenJev is an open project associated with the Jev approach. It is worth examining if you wish to explore an implementation outside of a hosted service environment. Before implementation, however, you should verify its current availability, hardware requirements, supported features, and licensing terms.

Laya

Laya is an open-source decision model from Convai Innovations. It focuses on tasks such as selecting from predefined options, rating information, and answering yes-or-no questions. It may be relevant if you wish to evaluate a model that can be executed within your own environment.

These projects should be compared based on actual performance rather than just their stated capabilities. Use consistent test data to measure accuracy, latency, resource requirements, and how well their confidence estimates align with real-world outcomes.

Can you run decision models locally?

Running a model locally grants greater control over the software environment and data processing location. However, deployment options vary by project. Jev is offered via an API, whereas open-source alternatives may provide options for running inference on your own hardware.

Hardware requirements depend on model size, memory usage, and throughput. Some models can operate on a CPU, while others benefit from a GPU, particularly when handling high volumes of requests.

If you wish to experiment with local AI models, DaDesktop provides cloud desktops with dedicated GPU options. This allows for development and testing without the need to purchase a high-end GPU for your local machine.

Learn more about running local LLMs or explore AI agent environments.

When should you use a decision model?

Decision models are advantageous when an application requires a defined answer that can trigger a specific next step. They can complement LLMs by handling classification, scoring, and routing, while the generative model focuses on language and complex reasoning.

The optimal choice depends on your specific application. Test the model against your own data, compare it with conventional classifiers and LLMs, and measure accuracy, latency, and cost. For critical decisions, include human review and safeguards where appropriate.

Want to discuss your use case? Contact us and we will point you in the right direction.