Engineers often focus on model accuracy, but technical success fails if users can't interpret outputs. By bridging the gap between raw predictions and business context, teams can build AI systems that drive measurable outcomes rather than just adding to dashboard fatigue.
By Ali Nahvi, Akshit Behera, Yvonne Fan, and Melissa Ramey.
Your models are accurate. Your pipelines are healthy. Your dashboards are working. And your system is still failing its users. Imagine thousands of dashboards and more than 20 applications continuously producing machine learning predictions, alerts, scores, and recommendations. Nothing is technically broken. Yet the people consuming those signals still have to determine which ones matter, what they mean, and what to do next.
That was the problem Salesforce faced in early 2025. Sellers had roughly 12,000 dashboards competing for their attention alongside more than 20 additional apps and tools. The models could be improved. The dashboards could be consolidated. Even more accurate predictions could be generated. None of those changes answered the question that exposed the real failure: if your system knows what is happening but your user still does not know what to do, what exactly have you built?
From 12,000 dashboards and 20+ notification apps to a single conversation a seller already has open.
Consider a churn model that returns a risk score of 0.83. The model has done its job. But what has the user actually learned? Is 0.83 high enough to act on? Why is the account at risk? Should the seller prioritize it over another renewal? And even if the prediction is correct, what move should the seller make?
The team had spent years building conventional machine learning models for problems like seller performance, pipeline health, skill gaps, attrition risk, product adoption, and onboarding completion. Models such as XGBoost and gradient boosting could be remarkably good at assessment. They could identify what was happening and where attention belonged. But their output often ended there. A score on its own has no context, recommended move, or understanding of what good looks like inside a particular business. Put that score on a dashboard and you have created another accurate piece of information that someone must find, interpret, and translate into action.
The model has not failed. The system is simply asking its output to perform a job it was not designed to do. That led to the distinction that changed how the system was designed: treat every model output as a signal, not an answer. Apply the same test to your own models. Take any prediction and ask what decision a user can make from that output alone. If someone must supply additional interpretation before acting, the model is producing a signal, regardless of how accurate that signal is.
Once you call 0.83 a signal rather than an answer, another question appears: what turns that signal into something a person can actually act on? Suppose a pipeline model identifies a stalled opportunity. That is useful, but it does not tell you how your organization prioritizes the opportunity, what a successful response looks like, or what an experienced seller would recognize about the situation.
Closing that gap required three inputs. The first was the signal: output from AI and machine learning models covering performance predictions, pipeline health, skill gaps, attrition risk, product adoption, and onboarding completion. Signals are particularly effective at assessment, identifying what is happening and what deserves attention. But knowing what deserves attention does not tell you what your organization wants done about it. That requires business logic: thresholds, priorities, operating rules, and current business motions that determine how a prediction should be interpreted.
Even those two ingredients leave something unresolved. A rule can tell you an opportunity deserves attention. It cannot necessarily tell you how an experienced seller would respond. That requires knowledge: institutional and field expertise about what good actually looks like in context. Combining the three allowed the Next Best Action layer to produce something fundamentally different from a score: a recommendation containing the action, the account it applies to, the reasoning behind it, and the expected impact.
The Next Best Action layer fuses signals, business logic, and knowledge into one recommendation.
The reusable pattern is the separation of assessment from action, not the particular layer itself. When you find this gap in your system, ask three questions: what can the model reliably assess? What operating rules determine how to interpret it? What knowledge is still required before someone can act? Those questions define what the layer after your model actually has to do.
Now imagine three renewals are simultaneously flagged as risky. The models have done exactly what you asked. Business rules may even establish that all three deserve attention. Which customer should the seller contact first? What if the apparent blocker is not the real blocker? How should the seller change a discovery call based on what this particular customer just said? Those answers depend on contextual judgment.
The team began thinking about information as a spectrum. Common sense sits at one end, followed by documented content and practical know-how. At the far end is knowledge: tacit, contextual, experience-based judgment. You can document a discovery process, but knowing how to respond to a particular customer’s organizational dynamics requires something different. That exposes an important constraint. Signals scale cheaply. Knowledge does not. It accumulates in people over years, is expensive to extract, and can lose value when flattened into generic best practices.
So when a recommendation still requires a person to interpret the situation, do not immediately reach for another model. Ask what that person knows that your system does not. The bottleneck may be knowledge rather than prediction. But even after you have synthesized the signal, business logic, and knowledge into a useful recommendation, you still have to get it to an AI agent reliably.
The information spectrum: We focus on the tail end, where expertise is hardest to capture and most dependent on context.
How does an agent know which capabilities exist, which ones to call, and when the underlying system simply has no answer? The team used Model Context Protocol (MCP), an open standard originally published by Anthropic, as the contract between the agent and the recommendation layer. MCP exposes callable tools with defined input schemas and return types. Rather than hard-coding every possible path, the agent can determine at runtime which available tools it needs based on the user’s request.
For example, an agent might determine that answering a question requires both a Next Best Action record and KPI history. If a tool returns nothing because there are no open renewals or active pipeline, the agent can recognize the absence rather than manufacture a result. If the requested capability does not exist within its available tool surface, it can decline instead of guessing. Adding an agent does not eliminate the requirement for trust. It makes the contracts surrounding the agent more important.
The transferable decision is not whether you use MCP specifically. Ask instead whether your agent has explicit contracts for what it can retrieve, what each capability returns, and what absence looks like. Conversational fluency can otherwise hide an unreliable system. Once multiple teams start exposing those capabilities, however, another question becomes unavoidable: who should own them?
Suppose several domains have data and capabilities your agent may need. Should they feed one centralized MCP layer, or should teams closest to each domain expose and maintain their own capabilities? A centralized topology provides one contract across a consolidated foundation. A decentralized topology lets domain teams own their respective capabilities and contribute them to a shared agentic surface.
The decision repeatedly came back to ownership and data governance: who can hold which data, who maintains each contract, and how much coordination is practical. The lesson was not that one topology was universally better. It was that architecture often follows ownership. Before choosing your topology, map ownership first. Who can hold the data? Who can maintain the capability? Who can be responsible for its contract? Those answers may resolve the architecture more clearly than debating centralized versus decentralized design in the abstract.
But now the architecture can do something dangerous: work perfectly and recreate the original problem. If users have to visit yet another application to access the agent, have you not simply built dashboard number 12,001?
Even a well-designed recommendation is useless if users have to hunt for it. Sellers were already working in Slack, where deal conversations, manager interactions, and customer follow-ups were taking place. Building another destination would have recreated the problem the team was trying to eliminate.
So the agent answers on request. A seller can ask it to “Plan my day” and receive prioritized accounts, the signals behind those priorities, their potential impact, and relevant knowledge for acting on them. The interaction is pull rather than push. No additional broadcast, morning digest, or unsolicited notification is required. A system that began with thousands of dashboards and more than 20 tools demanding attention instead waits until the seller asks.
That gives you one final design test: after improving the intelligence behind a decision, are you forcing users to adopt another destination to receive it? If so, you may have solved the prediction problem while preserving the attention problem. The goal was to stop making people go somewhere else to turn insights into decisions, not to create a better place for insights.
Now look at your own system. Perhaps your model produces a risk score, anomaly, forecast, ranking, or prediction. Before asking whether you need a newer model, a better LLM, or another interface, ask a more revealing question: what happens between this model producing a prediction and a person knowing what to do? If the answer is “the user figures it out,” you have found an engineering gap.
Audit the path from signal to decision. Identify what the model can reliably assess, the business logic required to interpret that assessment, and the contextual knowledge needed to act. Then determine how an agent should retrieve the result, who should own those capabilities, and where the user should encounter them. Wherever a human is silently supplying missing rules, context, or judgment, you have found part of the architecture your system has not captured yet.
The most important shift is simple: treat every model output as a signal, not an answer. Once you do, the engineering problem changes. You are no longer asking how to display a prediction. You are designing the missing layer between prediction and action.
The post From Prediction to Action: How to Turn AI Outputs Into Decisions appeared first on Salesforce Engineering Blog.
Continue reading on the original blog to support the author
Read full articleStandard RAG pipelines often fail in production because they strip semantic structure from complex documents. This framework helps engineers move beyond basic text retrieval to build trustworthy AI systems capable of handling real-world enterprise data with high accuracy.
Standard RAG misses complex logic spread across documents. GraphRAG uses knowledge graphs to link entities, enabling agents to find hidden exceptions through multi-hop retrieval. This improves accuracy for conditional business decisions where simple similarity search fails.
As AI agents gain agency to modify production systems, traditional text-based metrics fail. Engineers must implement outcome-based evaluations that verify state changes and tool usage to ensure reliability, safety, and correctness in automated workflows.
Manual bug triage at scale is slow and inconsistent. By combining custom ML with LLMs, teams can automate complex engineering judgments, preserving institutional knowledge while reducing months of manual effort to days, significantly accelerating product quality improvements.
