Measure and improve developer productivity

Knowing how productive your development team is is essential to planning your software delivery timeline and releasing high-quality software to the end-user. However, there exist several misconceptions about what developer productivity is and how it can be increased. Developer productivity is not a single metric you can measure, like the number of pull requests each developer in the team is making in a given development time. Instead, it comprises a variety of interconnected aspects that can be measured by different metrics and improved over time.

In this article, we take a look at developer productivity: why you should measure it and how you can do so despite the challenges it poses. We also offer some recommendations for ways to improve it consistently and sustainably over the long term.

What is developer productivity?

Developer productivity is a measure of the speed and efficiency with which developers and programmers build reliable and quality software. This measure of productivity is usually based on certain criteria and expectations or is measured within a specific timeframe. An organization can define a baseline for the relevant metrics and measure productivity against those. Both the software and its development life cycle are directly dependent on the productivity of its developers, so this is a measure that software engineering managers, leadership, and investors alike display significant interest in.

Measuring developer productivity is not merely a numbers game. The amount of work that a developer gets done is not necessarily a good software development productivity metric. A developer putting in more hours of work, for example, may, in reality, be a result of poor planning on the part of the team. Relying on metrics such as lines of code written, hours worked, or tasks completed can instead result in inaccurate and incomplete information about developer productivity. It is meaningful metrics, then, that make up developer productivity.

In order to push developers to do their best work and produce quality software, it is worth understanding what does and does not work for them in their existing development environment. Because developer productivity is a complex notion that is influenced by a multitude of factors, a single metric is incapable of offering deep insight into it. Even then, each developer or team is plagued by their specific struggles, so the metrics that make sense to one team may not be as useful to another.

In that sense, the measurement of developer productivity encompasses a wide variety of metrics. These can include the number of open pull requests and how long it takes the developers to close them, or the amount of focused time developers are getting on their calendars. The velocity of your JIRA tickets is another example of a metric that reveals how productive developers are in your organization.

Why should you measure developer productivity?

Why is the measurement of developer productivity essential to an organization’s growth? To begin with, any kind of visibility into both your outcomes and your processes will give you insights into what is and is not working. Only when you are aware of the shortcomings of your existing workflows and the struggles faced by developers can you take steps collectively to improve the situation. When you are aware of the bottlenecks in the developers’ workflows and what is required to fix them, you can also make informed decisions about resource allocation and create high-performing teams.

Analyzing such information over time is also helpful to understand progress over the long term, which is necessary for sustainable growth that extends beyond a few weeks or months. Solid documentation practices also strengthen these efforts.

When it comes to developer productivity, it is equally important to get your team members nto care about those metrics as it is to track them against these. Emphasize why these outcomes matter and how they are going to help your startup. For example, improved MTTR is going to mean that your on-call rotations are less noisy. This will subsequently result in your developers being able to spend more time on the actual features of the product. At an organizational level, such practices help you build a culture of visibility into these outcomes. This leads to your teams being more productive and efficient, resulting in a culture of reliability.

Why is measuring developer productivity challenging?

Owing to its complexity and the inutility of a one-size-fits-all approach to developer productivity, measuring it is not always the most straightforward process. The following are a few common challenges you may run into or mistakes you may make when first attempting to measure developer or team productivity.

Only focusing on individual productivity

More often than not, it makes more sense to measure developer productivity at a per team level. Although the contributions of individual developers are not to be undervalued, it is the work that teams do together and their ways of collaborating that are ultimately telling of the levels of productivity and efficiency in building software that they are able to reach on a regular basis. The relations between developers are as important to their productivity as their individual effort.

There is also the problem of standardization. It is difficult to make sense of a standard for productivity when each developer contributes differently to a product. Some developers might actively work on new functionalities while others might be tasked with more reviewing and testing-related tasks. As a result, it makes more sense to measure the collective productivity of a team.

Incentivising developers the wrong way

By excessively pushing productivity and metrics onto developers who are doing their best, you run the risk of pressurizing them. Under mounting pressure, developers might resort to quick fixes or shortcuts that show up as improving metrics but do not improve the output or process quality in any meaningful way. Overemphasizing the need to review a large amount of code in less time, for example, can result in surface-level reviews that are not thorough or effective. This directly impacts code quality.

Some metrics are easier to circumvent, but they may still be valuable measures of developer productivity. In this case, both managers and developers will benefit from stronger and more open communication efforts such that developers care about the metrics and are not overburdened.

Measuring counterproductive metrics

Some metrics, like pull request counts, yield nothing of business value. Instead, they disincentivize developers from working on harder, more time-consuming issues as these slow down the count. Similarly, equating more lines of code with higher productivity is not as straightforward as it may initially appear. Longer code risks containing more bugs, and it can often be difficult to read. Processing larger chunks of code takes more computing power and is more time-consuming. It is not, then, a guarantee of good-quality code.

Missing out on the context

Very few metrics are helpful by themselves. Understanding developer productivity through a combination of relevant metrics is recommended because together these metrics offer context. A more significant proportion of activity is also not captured by these metrics at all. Use the metrics to dive deeper into the underlying reasons for how they are showing up on your dashboards. Neglecting the context of these metrics is akin to sailing past an iceberg with no knowledge that most of the structure is floating underwater. At the end of the day, it is the context that gives meaning to your metrics and allows you to understand developer productivity to the extent possible.

Not following up

Developer productivity begins at measurement, it does not end there. Thinking through the possible reasons for why your team is doing well on certain metrics and not on others is essential to figuring out what actions you can take next. Then only when you actively change certain practices or guidelines will there be any scope for improvement. Visibility into the outcome metrics is only useful when you do something with it.

How can you effectively measure developer productivity?

Before you start tracking any metrics, the first step is to figure out why you are trying to measure developer productivity. The reasons can range from you trying to get more developers to improving developer happiness and motivation to be productive so that they don't have to deal with as many blockers. Or, you may be trying to improve a specific part of the developer lifecycle that is currently being bottlenecked. Your motivations can be a combination of these two. Knowing why you are concerned about developer productivity within your team will not only help you identify which metrics specifically are worth measuring but streamline and focus your efforts such that you are on track to improve along those lines in particular.

Another question worth asking at this stage is whether it is developer productivity you are trying to measure or if there is another second-order metric that is, in reality, more relevant to you. In many cases, developer productivity is not as valuable. For example, you may be trying to track developer productivity as a proxy to DORA metrics, or as a proxy to understand how frequently you are deploying. If that is the case, you are better off tracking the metrics more relevant than productivity. Instead, think about how you can track developer outcome metrics.

Often, developer outcome metrics bleed into developer productivity. So if measuring these outcome metrics is more suitable for your team’s progress, figure out which ones you are interested in examining in-depth. Let’s say you are trying to improve mean time to resolve (MTTR), i.e., how long it takes for developers to respond to incidents. That is an example of developer productivity. Another consideration to keep in mind is that the outcome metrics important to each team may be different. So for one team, the time-to-close for JIRA tickets may be useful and another team may care more about their MTTR if their alerting mechanisms are falling short. For a third team, the MTTR may be important, but because their time-to-close is not up to the mark and they are unable to look at each ticket. For these reasons, you need to identify the purpose of tracking developer productivity or outcome metrics and choose the ones most relevant to improving your team’s workflows.

After identifying the relevant developer outcomes you care about, build automation around them. Create reports that are visible to everyone on the team on those outcomes. If, for example, you are looking at MTTR, create reports that are automated from the data you have on it and bring visibility to the performance of your developers on that metric. Put that information front and center to inform everybody involved about your performance trends and existing bottlenecks concerning MTTR.

Give engineering leadership visibility into their team's MTTR and, consequently, their team's outcome metrics. When they are able to see which teams are struggling with which outcomes, they can brainstorm as to why that is the case. When such analysis does not come from internal or engineering leadership, it is harder to emphasize it across the organization. For this reason, it is important to make sure that engineering leadership has visibility into outcomes on both aggregates as well as per team levels.

At the end of the day, the teams are best equipped to define the visibility metrics that are valuable to them. Perhaps your team is tracking MTTR, pull requests, JIRA tickets, and on-call in a single place. The team can then ask themselves where it is that they are struggling. At this point, an engineering project management personnel or a tech lead can highlight the need to improve a particular part of the process. So, the importance of giving teams visibility at a granular level cannot be denied and it is here that the process metrics could potentially come in. It is also worth emphasizing that these process metrics are more for developers to help themselves and figure out what they want to improve. It is not what they will be tracked against.

After both leadership and the teams have clarity on the reasons that they are struggling, give teams the resources they need to improve on those particular metrics. Being proactive at this stage is of the essence so that you do not lose the momentum and can work on improving developer productivity and outcomes from the get-go.

In addition to understanding why you are interested in tracking certain metrics, automation and aggregation at the team level are both essential to measuring developer productivity. When your measurement practices are robust and put into place, you are more likely to succeed in improving developer productivity in a way that is sustainable.

Additionally, if you are looking for more structure to guide your measurement efforts, OKR and SPACE are two specific frameworks that your organization may benefit from. Let us discuss them briefly.

OKR framework

The OKR framework relies on objectives and key results. It allows you to define certain goals and what the tangible results would look like if the goal is met. This is an effective way of encouraging developers and teams to define what is important and to focus on it. When everyone is on the same page about what success looks like, it is easier to identify when the objectives have been realized. Such transparency increases accountability.

Developers are also able to better manage their tasks when they have clarity on what is expected of them. You can also use it to identify differences between developers working towards similar goals, which can lead to trying to understand why the differences exist.

This method, however, has two drawbacks. Operating under the OKR framework can lead to developers jumping to new tasks without first getting rid of existing technical debt. It can also cause undue stress to work with a fixed baseline every week as there is little room for emergencies, innovation, or consideration of the fact that some issues take longer or more effort to fix than others.

SPACE framework

Many swear by the SPACE framework to effectively measure developer productivity in their organizations. It takes a multidimensional approach, covering aspects of satisfaction and well-being, team and individual performance, activity, collaboration and communication, and efficiency and flow. Let’s examine each of these to understand why they are worth taking into account when thinking about productivity.

Satisfaction and well-being

Your developers’ emotional, mental, and physical well-being plays a significant role in their productivity at the workplace. If they are consistently not satisfied with the work assigned to them, or if they are dealing with multiple stressors on a daily basis, their productivity is likely to take a hit. To this end, keep an eye out for symptoms of burnout as well as indications that your employees are happy and healthy.

To measure the degree of satisfaction and the well-being of your software engineering team, it is important to first understand what is important to them. For example, some developers might prioritize working on software projects that are meaningful to them, while some might value the opportunities they get to collaborate with their peers.

Performance

While we do not recommend relying solely on output metrics without any context or in isolation, you can instead benefit from taking a look at outcomes in conjunction with other metrics. Measuring team and developer performance is far from being the end goal. Use the results you get on these metrics to get to the underlying reasons and conditions that motivate that level of performance in the team.

Focus on measuring both developer-related and business-related performance metrics. This is because developer output by itself is not particularly useful for understanding productivity. Use business metrics to instead contextualize and add meaning to your results on metrics like the quality and health of your software.

Activity

Like performance metrics, activity metrics must also be contemplated with a grain of salt. These are simple measures of the work that is completed by your team. The limitations of activity metrics are more obvious: it is, for instance, nearly impossible to measure all the tasks and activities a team collectively does in a single day and to decide whether these were high-impact activities. Activity metrics usually measure tasks like pull requests and commits, deployment frequency, and code reviews.

Despite its limitations, measuring activity is a crucial component of the overall framework of software development work. When examined in conjunction with performance or job satisfaction, for example, the activity metrics are valuable pieces of the puzzle and can help inform your productivity improvement strategy.

Communication and collaboration

A team’s success is contingent on the ease and clarity with which developers communicate amongst themselves and with engineering leadership. The software development process relies heavily on feedback, without which there is little to no scope for improvement in the product. Similarly, building an elaborate product goes beyond the capabilities and work of individual developers if it is to be of high quality. For these reasons, in addition to the positive impact of effective communication on the overall developer experience, it is important to measure the quality of communication and opportunities for collaboration within your team.

No two teams work alike owing to the differences in both the developers that comprise the teams and how they communicate and collaborate with one another. Spend time understanding how your software development team, in particular, comes together to identify any gaps in communication practices or opportunities for collaboration that the team could benefit from paying attention to. The quality of feedback loops in the team is another robust metric to consider.

Efficiency and flow

Developer productivity is also a direct result of the amount of uninterrupted quality work the team can accomplish. For developers, it also results in greater satisfaction with their work. While it may be difficult to identify concrete metrics to measure the efficiency with which the developers are working, you can more easily identify any obstacles or aspects of the workflow that are doing more harm than good. Locate instances of wastage of time and resources and look for any patterns. Surveying developers to better understand what they need to achieve efficiency without excess pressure can also be helpful.

All in all, the SPACE framework not only identifies the key contributors to developer productivity but also emphasizes the need for studying them together. Even if your team is doing better on some metrics than on others, only when you approach them together can you understand how they impact one another and subsequently identify the specific areas that you need to focus on.

How can you sustainably improve developer productivity?

Measurement is the first step in making productivity-related changes in your teams. Effectively measuring developer productivity simplifies the process of subsequently improving it. Here are a few healthy and effective practices you can incorporate in your efforts to improve developer productivity.

Prioritize efficiency

It goes without saying that it is not worth prioritizing efficiency to the point that your developers experience high amounts of stress. However, employing a few different techniques to change the workflows across the team can improve productivity in the long run without sacrificing the well-being of your team. For example, continuously attempting to reduce technical debt pays off over the long term and can significantly increase developer productivity.

Also, pay attention to the productivity tools your developers are using on a regular basis. These should support your developers in their work and not act as hindrances in any way. If you find that this is the case, discuss your developers’ needs and possible alternatives with them.

Center developer experience

Developer productivity, as much as it is a consideration in the overall software development life cycle, is also important to developer experience. Focusing on improving the quality of the developers’ day-to-day work life can go a long way in ensuring that their well-being is not severely affected. Any changes you choose to make to improve developer productivity must take this into consideration.

Similarly, ensure that you do not burden your developers with excessive workloads or demands of multitasking. This can generate stress and a lack of focus, which are both detrimental to their productivity. What may seem like a push towards productivity is likely to backfire in the long term.

Make gradual changes to workflows

Your developers already have existing tools, frameworks, and processes that they are accustomed to. Adapting to the inclusion of a new style of working or a new tool will inevitably impact their productivity, even if for a short period. For this reason, avoid completely overturning their present workflows. Introduce change in short doses and gradually, and only when required. By adding or incorporating them into the existing workflows, your team is more likely to not feel stressed and be more open to the changes. 

The benefits of using Cortex

If you are interested in tracking outcome metrics, consider using a product like Cortex scorecards. These are a great way of automating the process of tracking the metrics that matter to your team. Acknowledging that teams may wish to incorporate data from custom sources, you can also use Cortex Query Language (CQL) to do so in the scorecards. The scorecards also integrate with third-party tools that your developers may already be using in their workflows.

Scorecards also facilitate the process of automatically reporting at a per team level and communicating the relevant findings to engineering leadership. With this information, you can take stock of the current situation and establish standard best practices that will help improve productivity levels in the team. As a result, it becomes considerably easy to incentivize efficiency and productivity and push people to do their best.

At Cortex, we help you define the metrics you care about, measure them with accuracy and with a well-defined purpose, and then improve on them across the board. With end-to-end support for improving developer productivity over the long term, you can be assured that your business outcomes and software quality will also experience an increase.