Most engineering leaders are drowning in data but starved for insight. We have dashboards full of metrics, but they often create more questions than answers and rarely tell us what to do next. In the age of AI, where development velocity is accelerating at an unprecedented rate, this problem is only getting worse. Shipping code faster than you can fix it is a real risk, and a dashboard that doesn't lead to action is just a distraction.
This guide reframes the conversation around engineering metrics, moving beyond simple definitions to an actionable framework. We'll cover the metrics that matter in an AI-driven world, how to group them for clarity, and most importantly, how to use them to create a culture of continuous improvement.
The four pillars of engineering metrics
Generic lists of metrics are often overwhelming and unhelpful. Instead of an alphabetical list, a more effective approach is to group metrics into four key pillars that map directly to business outcomes. This is how you move from observation to action.
Velocity: These metrics measure the speed and throughput of your development process. In the age of AI, it’s not just about how fast you can ship, but how efficiently your process is running. Key metrics include Deployment Frequency, Lead Time for Changes, and PR Cycle Time.
Reliability: As velocity increases, it’s critical to track the stability of your systems. These metrics help you understand the impact of your speed on customer experience. Key metrics include Mean Time to Acknowledge (MTTA), Mean Time to Recovery (MTTR) and Change Failure Rate (CFR).
Quality: This pillar focuses on the health of your codebase and the effectiveness of your development practices. Key metrics include Code Coverage and Time to Open (PRs).
AI Impact: A new and essential category. As teams adopt AI, it’s crucial to measure its actual impact on both productivity and risk. Key metrics include PRs Merged per Author and Incidents per PR. You can track these and more in Cortex's AI Impact Dashboard.
DORA metrics: The gold standard for DevOps
The DORA metrics are a great starting point for any team serious about performance. They provide a balanced view of both speed and stability. The five key DORA metrics are:
Deployment Frequency: How often you successfully release to production.
Lead Time for Changes: The time it takes for a commit to get into production.
Change Failure Rate (CFR): The percentage of deployments that cause a failure.
Mean Time to Recovery (MTTR): How long it takes to recover from a failure in production.
Reliability: The ability of your team to meet your own reliability and availability targets.
DORA provides an essential, system-level view of engineering performance. Many teams complement DORA with other frameworks to add a more qualitative, developer-centric perspective. For instance, the SPACE framework incorporates developer satisfaction, while DX’s Core 4 focuses on the direct developer experience. These frameworks offer useful insights into how teams are working, providing valuable human context to DORA's system-level data.
To go deeper on DORA, you can watch our on-demand webinar or enroll in the DORA course in Cortex Academy.
How to move from metrics to meaningful change
The real value of metrics comes from using them to drive a culture of continuous improvement. The most effective leaders we work with don't just track metrics; they use them to run experiments. Here’s the pattern we've seen work:
Start with a hypothesis, not a metric. Instead of asking "How can we improve code coverage?", a better question is "What do we believe will happen if we improve code coverage?" This frames the work around a specific outcome (e.g., "We believe improving code coverage by 10% will reduce our change failure rate by 5%").
Use Scorecards to turn a standard into a story. A metric is just a number; a standard is a commitment that tells a story about what your team values. Use Cortex Scorecards to codify your best practices and track your progress. This is how you turn a metric into a lever for change.
Create a plan and measure the impact. With a Scorecard in place, you can create a clear plan to improve your target metric. Use Cortex’s dashboards to monitor your progress over time and, more importantly, to see if your original hypothesis was correct. Did improving code coverage actually affect the change failure rate?
Share the results and iterate. Whether the experiment succeeded or failed, sharing the results helps build a culture of learning. The goal isn't just to hit a number; it's to build a system of continuous improvement. Once you've completed one experiment, use the same framework to tackle the next.
Monitoring metrics with Cortex
Manually collecting and tracking metrics is a recipe for failure. It's time-consuming, error-prone, and it doesn't scale. Cortex is designed to solve this problem by providing a single pane of glass for all your engineering data.
With Scorecards, you can automatically track dozens of metrics from the 50+ tools Cortex integrates with. And with Initiatives, you can set time-bound goals to drive focused improvement on your most important metrics.
Interested in learning more about how Cortex can help you turn metrics into meaningful change? Schedule a demo today.
