Most engineering leaders are drowning in data but starved for insight. We have dashboards full of metrics, but they often create more questions than answers and rarely tell us what to do next. In the age of AI, where development velocity is accelerating at an unprecedented rate, this problem is only getting worse. Shipping code faster than you can fix it is an existential risk, and a dashboard that doesn't lead to action is just a distraction.
This guide reframes the conversation around engineering metrics, moving beyond simple definitions to an actionable framework for measuring engineering organizational effectiveness in the age of AI. We'll cover the metrics that matter in an AI-driven world, how to group them for clarity, and most importantly, how to use them to create a culture of continuous improvement.
The five pillars of engineering metrics: the DRIVE framework
Generic lists of metrics are often overwhelming and unhelpful. Grouping metrics into pillars that map to business outcomes helps you move from observation to action. Cortex organizes them with the DRIVE framework, a five-pillar frame built to measure how the whole organization operates: Delivery, Reliability, Initiatives, Vigilance, and Efficiency.
Delivery: These metrics measure the speed and sustainability of your development process. In the age of AI, it's important to measure both the structural and human capacity to sustain high performance: the humans doing (or orchestrating) the work, and the actual movement of code through the system. Key metrics include Deploy Frequency, Lead Time for Changes, and On-call Pager Volume. Secondary metrics include PR Cycle Time, Change Lead Time by Stage, and Flaky Build of Test Rates.
Reliability: As velocity increases, it’s critical to track the stability of your systems from the lens of your customer's experience and expectations. These metrics help you understand the impact of your speed on customer experience. Key metrics include Curated List of Functional Service-Level Objectives (SLOs) as pass/fail and Sev0/Sev1 Incident Count. Secondary metrics include Mean Time to Recovery (MTTR), Change Failure Rate (CFR), SLO Coverage, and Error Budget Burn Rate.
Initiatives: This pillar asks whether your most important non-product engineering work is moving, and whether your Operational Excellence reviews meant to drive the initiatives are actually effective. Key metrics include Tier 1 Milestone Completion Rate and OpEx Action Item Completion Rate. Secondary metrics include Tier 2 and Below Completion Rate and Age of OpEx Action Items.
Vigilance: Measures how well you are actively defending your systems and managing acceptable risk. Key metrics include Fixable Critical and High Common Vulnerabilities and Exposures (CVEs) Past SLA, Assets Below Compliance Baseline, and Orphaned and Unverified Assets. Secondary metrics include Open Medium and Low CVEs and Count of SLA Breaches Across All Severities.
Efficiency: This pillar asks, "Are we allocating resources to the right problems?" AI is changing the equation; token spend is exploding and engineering teams are still optimizing costs for AI products, introducing a new risk area. Key metrics include Actual Cloud Spend vs. Budget, AI/LLM Token Costs, and Percent of Capacity Spent on Innovation. A secondary metric to include is Infrastructure Utilization Rate.
This framework measures how effectively the entire organization operates, not how busy any individual engineer looks. Access the full DRIVE framework, including rollups and red/green thresholds, here.
DORA metrics: An excellent starting point for DevOps
The DORA metrics are a strong foundation for any team serious about delivery. They give a balanced view of speed and stability, which is why they are incorporated into the Delivery and Reliability pillars above. The current DORA model tracks five metrics:
Deployment Frequency: How often you successfully release to production.
Change Lead Time: How long it takes for a commit to reach production.
Change Fail Rate (CFR): The percentage of deployments that cause a failure in production.
Failed Deployment Recovery Time: How long it takes to restore service after a failed deployment, renamed from Mean Time to Recovery (MTTR).
Deployment Rework Rate: How often deployments require unplanned follow-up work.
Earlier versions treated reliability as a guardrail metric rather than one of the core five. Any of these metrics can be gamed in isolation, which is why it's important to view them together, within a broader frame.
DORA provides an essential view of engineering delivery. Many teams complement DORA with other frameworks to add a more qualitative, developer-centric perspective. For instance, the SPACE framework captures satisfaction, performance, activity, communication, and efficiency, adding human context that system metrics miss. DX’s Core 4 focuses on the direct developer experience. Both are strong at what they measure: individual and team productivity.
That is also their limit. In the AI era, individual productivity is the wrong atomic unit. When AI assistants write a growing share of the code, the question that matters most is how effectively the whole organization delivers, stays reliable, and manages cost and risk.
To go deeper on DORA, you can watch our on-demand webinar or enroll in the DORA course in Cortex Academy.
Avoid the vanity-metric trap
Goodhart's Law states that "when a measurement becomes a target, it ceases to be a good measure." Track PRs merged per author as a goal and engineers will split work to inflate the count; the number climbs while nothing improves.
Choose metrics that expose tradeoffs rather than ones that flatter, and measure differently for different audiences. Teams need diagnostic metrics they can act on this week, while executives need outcome trends tied to reliability, cost, and risk. Reporting the same raw dashboard to both is how metrics lose meaning on the way up.
How to move from metrics to meaningful change
The real value of metrics comes from using them to drive a culture of continuous improvement. The most effective leaders we work with run a recurring Operational Excellence review to drive change: a weekly or biweekly leadership meeting built around the DRIVE metrics, the same discipline behind the AWS, Stripe, and Google ops reviews. The cadence turns dashboards into decisions, and allows for teams to run experiment loops, outlined below:
Start with a hypothesis, not a metric. Instead of asking "How can we improve code coverage?", a better question is "What do we believe will happen if we improve code coverage?" This frames the work around a specific outcome (e.g., "We believe improving code coverage by 10% will reduce our change failure rate by 5%").
Use Scorecards to turn a standard into a story. A metric is just a number; a standard is a commitment that tells a story about what your team values. Use Cortex Scorecards to codify your best practices and track your progress. This is how you turn a metric into a lever for change.
Create a plan and measure the impact. With a Scorecard in place, you can create a clear plan to improve your target metric. Use Cortex’s dashboards to monitor your progress over time and, more importantly, to see if your original hypothesis was correct. Did improving code coverage actually affect the change failure rate?
Share the results and iterate. Whether the experiment succeeded or failed, sharing what happened builds a culture of learning. Aim for a system that keeps getting better, and the numbers will follow. Once you've closed one experiment, use the same framework to tackle the next.
Monitoring metrics with Cortex
Manually collecting and tracking metrics is a recipe for failure. It's time-consuming, error-prone, and it doesn't scale. Cortex is designed to solve this problem by providing a single pane of glass for all your engineering data.
With Scorecards, you can automatically track dozens of metrics from the 50+ tools Cortex integrates with. And with Initiatives, you can set time-bound goals to drive focused improvement on your most important metrics.
Interested in learning more about how Cortex can help you turn metrics into meaningful change? Schedule a demo today.
Frequently Asked Questions (FAQs)
What is an Operational Excellence (OpEx) review?
The OpEx review is a recurring meeting where engineering leaders interrogate the state of the organization using the DRIVE framework, prioritize action items, and re-allocate resources to where they matter. It's the practice that turns measurement into improvement. AWS, Stripe, and Google SRE have run versions of this for years; in the AI era, it's the structured human-in-the-loop mechanism that keeps the AI software factory accountable.
Does DRIVE replace DORA metrics?
No. DRIVE incorporates DORA rather than replacing it. DORA metrics give a balanced view of speed and stability, which is why they live inside the Delivery and Reliability pillars of DRIVE. DORA answers how well your delivery pipeline performs; DRIVE answers a broader question about how effectively the whole organization delivers, stays reliable, and manages cost and risk. Teams already using DORA can keep those metrics and roll them up into the wider operational picture DRIVE provides.
How is DRIVE different from SPACE and Core 4?
DRIVE measures at a different altitude. SPACE and the DX Core 4 measure individual and team productivity, DRIVE measures how effectively the whole organization operates. In the AI era, individual productivity is the wrong atomic unit for that question. When AI assistants write a growing share of the code, what matters most is whether the organization as a whole ships reliably, defends its systems, and controls cost.
How do you turn engineering metrics into real improvement?
Run a recurring Operational Excellence review as the human-in-the-loop governance mechanism. The most effective leaders hold these reviews weekly or biweekly with a designated facilitator and assign tangible action items that include an owner and a due date.
