An organization running large-scale regression testing required an intelligent system to analyze test failures, classify issues, and streamline defect management. The existing process relied heavily on manual effort to review logs, compare outputs, and create defect tickets.

The objective was to build a reliable and scalable system that could process regression outputs, identify meaningful issues, and generate actionable insights while maintaining accuracy and auditability.

• The client faced significant delays due to manual triage of regression failures, especially when handling large volumes of test data.
• Regression outputs contained noisy and unstructured data such as logs, timestamps, and environment-specific variations, making it difficult to identify real issues.
• There was no standardized mechanism to classify failures into categories like regression issues, environment issues, data-related problems, or unknown.
• Duplicate defect creation in Jira was common due to lack of historical reference and tracking.
• The absence of a structured workflow made it difficult to ensure consistency, traceability, and audit readiness.

• To address these challenges, an Automated Regression Triage Framework (ARTF) was designed using a state-driven multi-agent architecture with a centralized supervisor.
• The system begins with a data ingestion layer where regression artifacts (base output, current output, and diff files) are processed, cleaned, and transformed into a high-signal dataset.
• A Supervisor (orchestrator) manages the workflow and routes tasks between specialized agents based on the current state of execution.
• The Analysis Agent classifies failures into meaningful categories such as regression issues, environment issues, and data issues, while also grouping similar errors and mapping them to business requirements.
• A Human-in-the-Loop (HITL) step ensures that all AI-generated recommendations are reviewed and validated by a human, who makes the final decision to confirm or override the classification.
• The Jira Agent creates or updates defect tickets with structured data, logs, and summaries, while avoiding duplicate issue creation.
• The Reporting Agent generates a consolidated summary of the regression run, highlighting key insights, patterns, and impacted areas.
• A Learning Agent continuously improves the system by storing past issues as embeddings and leveraging historical data to identify similar patterns in future runs.

• The framework significantly reduced manual effort involved in regression triage and defect management.
• Automated classification and grouping improved the accuracy and consistency of issue identification.
• Integration with Jira streamlined defect tracking and eliminated duplicate ticket creation.
• Human validation ensured reliability while enabling continuous learning from past decisions.
• Overall, the system improved processing speed, enhanced visibility into regression health, and ensured a more structured and audit-ready workflow.

• Multi-Agent Systems
• LangGraph (State-based orchestration)
• Vector Databases (for learning and similarity search)
• Azure Sevices
• Jira Integration APIs
• Event-driven Workflow Systems