Manual bug triage at scale is slow and inconsistent. By combining custom ML with LLMs, teams can automate complex engineering judgments, preserving institutional knowledge while reducing months of manual effort to days, significantly accelerating product quality improvements.
By Priya Sethuraman, Abhishek Ghose, Lovish Agarwal, and Aditya Pandey.
In our Engineering Energizers Q&A series, we highlight the engineering minds driving innovation across Salesforce. Today, we spotlight Priya Sethuraman, Director of Software Engineering. Priya leads the Sales Cloud BugWiser initiative, an AI-powered system that standardizes customer bug classification and transforms customer bug signals into faster engineering action. By combining a custom machine learning model with a large language model (LLM), BugWiser reduced customer bug triage time from more than 300 person-days of manual effort to less than a week while generating trusted release insights at scale.
Explore how Priya’s team transformed years of engineering expertise into a trusted AI system that standardizes bug classification, accelerates root cause analysis, strengthens engineering productivity, and helps Salesforce close the loop with customers faster.
As Salesforce accelerated feature delivery, customer feedback became more valuable and more difficult to process consistently. Every customer bug became a signal that could reveal regressions, missed use cases, accessibility issues, and broader product quality trends. The challenge wasn’t simply fixing individual bugs, rather, it was transforming those customer signals into timely engineering action while continuing to ship innovation.
The mission became shortening the time required to understand customer-reported bugs while enriching every issue with the engineering context needed to accelerate root cause analysis. Historically, engineers manually reviewed every customer bug, analyzed associated investigations, determined the likely root cause, classified each issue, and documented their findings before meaningful work could begin. That process produced valuable insights but required significant manual effort and depended heavily on engineering expertise. Leadership also needed broader release insights, including which product areas generated the most customer issues, which classifications became dominant across a release, and where engineering investments would have the greatest impact. Producing those insights often required days of manual analysis.
BugWiser changes that operating model. By standardizing customer bug classification and enriching every investigation with trusted engineering context, engineering teams can identify trends faster, improve engineering productivity, prioritize investments, and quickly close the loop with customers.
The hardest part wasn’t reducing manual effort; it was encoding years of engineering judgment into a system that could apply it consistently across Salesforce. Every customer bug required engineers to review investigations, determine the root cause, classify the issue, and document their reasoning. Because those decisions relied on individual expertise, similar customer issues could be classified differently across engineers and clouds.
Instead of asking AI to replace that expertise, the team focused on teaching a system to think the way subject matter experts already did. The model was trained using years of historical engineering decisions, so instead of relying entirely on generalized AI reasoning, it learned how Salesforce engineers had historically evaluated customer bugs. That preserves institutional engineering knowledge while applying it consistently across thousands of investigations.
Standardization became just as valuable as automation. Engineering teams now share a common framework for bug classification, giving leadership clearer visibility into dominant problem areas, recurring customer issues, and the engineering actions with the greatest customer impact.
Applying AI to bug reports wasn’t the difficult part, rather, building a system engineers would trust was. General-purpose AI could generate classifications, but the goal was not to replace subject matter experts. It was to teach the system how those experts already approached customer bug investigations.
That led to building a custom machine learning model trained specifically on Salesforce’s historical bug classifications. The model learns from years of engineering decisions, producing recommendations that reflect established engineering practice rather than simply generating plausible answers. Every prediction includes a confidence score so engineers understand how certain the system is before acting, and it explains the evidence behind each recommendation so reviewers have greater confidence in the outcome.
When engineers disagree with a prediction, they do not simply reject it. Instead, they provide the correct classification, which continuously retrains the model and ensures it evolves alongside engineering expertise. Rather than asking engineers to trust AI, the team built an AI system that continuously learns from engineers. That philosophy, combining machine learning with human expertise to deliver trust by design, became one of the defining principles behind BugWiser.
Bridging custom model training with enterprise LLMs.
One of the biggest engineering decisions was recognizing that no single AI model excelled at every part of the workflow. The custom machine learning model performs customer bug classification using Salesforce’s historical engineering decisions to deliver fast, deterministic predictions with confidence scores and explainability. LLMs like Anthropic’s Opus complements that by synthesizing grounded engineering context and generating richer summaries for engineers.
The decision was not about avoiding large language models. Instead, it was about choosing the right AI model for each task. While LLMs excel at reasoning across unstructured information, the smaller, specialized machine learning model proved better suited for deterministic bug classification, confidence scoring, and explainability. By assigning each AI model to the work it performs best, the team achieved the right balance of accuracy, speed, flexibility, and trust across engineering workflows.
The biggest challenge was learning an entirely new discipline. Like many engineering teams, the team began with large language models before realizing they couldn’t provide the deterministic behavior, confidence scoring, and explainability the bug classification workflow required.
That realization meant learning how to identify the right model for the use case, train it using years of historical engineering decisions, evaluate its performance against general-purpose LLMs, and continually improve it as new engineering knowledge emerged. Building that expertise meant moving beyond simply consuming AI tools to developing an AI system tailored to a specific engineering problem. Operationalizing the solution introduced another challenge. Unlike LLMs, smaller custom machine learning models required defining how they would be hosted, retrained, monitored, and integrated into Salesforce’s engineering workflows.
The investment proved worthwhile. Beyond delivering a production-ready AI system, the team gained a deeper understanding of one of AI engineering’s most important lessons: the best AI model is not always the largest model. It is the one best suited for the problem you’re trying to solve.
Trust by design became a core engineering principle. Every recommendation had to provide enough evidence for engineers to confidently act on it or intentionally override it. Every prediction includes a confidence score and explains the evidence behind each bug classification, giving engineers the transparency needed to trust or question the recommendation.
Human review remains central to the workflow. High-confidence predictions can be accepted automatically, while lower-confidence recommendations receive manual review. When engineers disagree, their corrections become new training data that continuously improves future predictions. Across the initial Sales Cloud and Service Cloud deployments, approximately 90% of predictions were accepted without modification, while only a small percentage required human correction.
Success isn’t measured simply by how quickly customer bugs are classified. It’s measured by how effectively customer signals are transformed into trusted engineering action. By combining a specialized machine learning model, an LLM, and continuous human expertise, BugWiser helps engineering teams standardize bug classification, accelerate root cause analysis, improve engineering productivity, generate richer release insights, and close the loop with customers faster.
The post How AI Reduced Customer Bug Triage from Nearly a Year to Less Than a Week appeared first on Salesforce Engineering Blog.
Continue reading on the original blog to support the author
Read full articleScaling localization for global software is a major bottleneck. This approach replaces manual workflows with a multi-stage AI orchestration pipeline, maintaining quality across 34 languages while reducing costs by up to 90% and handling massive volume growth without extending release windows.
This approach transforms tribal knowledge from code reviews into executable system requirements. By automating feedback loops and multi-tier evaluations, engineers can reduce repetitive manual reviews and ensure AI-generated artifacts consistently meet organizational standards.
This approach reduces 'reasoning tax' by pre-processing messy documentation into synthesized context. It enables faster onboarding for humans and provides AI agents with a reliable, low-cost knowledge layer, improving the efficiency of RAG systems without complex infrastructure.
This article demonstrates how generative AI can eliminate manual bottlenecks in ETL processes. It provides a blueprint for transitioning from fine-tuned models to foundation models while maintaining reliability through grounding and validation.
