Measuring developer time spent on platform tasks
Reducing developer time spent on platform tasks is one of the core missions of a platform team. Developer time is expensive and should be spent on product work.
This post is about how to measure developer time spent on platform tasks. Without measurement, it is hard to know whether a platform investment actually reduced the work developers had to do.
What counts as a platform task?
A platform task is any work related to the systems that build, test, deploy, and monitor code.
Examples:
- Writing a GitHub Actions workflow to run the test suite on every PR.
- Debugging a failed pipeline that turns out to have been caused by platform-wide test runner instability.
- Writing Kubernetes manifests.
Ways of measuring
There are different ways to measure time spent on platform tasks, but we generally think that surveys are a good way to do it.
However, the survey needs to be done right. A common way of conducting platform surveys is to send a questionnaire to all developers at the end of a quarter. One of many questions is how much time they spent on platform tasks per week. But people's estimates of how much time they spent on something over the last quarter are unlikely to be very accurate. For instance, estimates might be heavily biased by the experience they had in the last weeks of the quarter ("recency bias").1
A better survey method is to ask a random subset of developers (say, 10%) each day how much time they spent on platform tasks that day. Ideally, you send them an automated direct message with buttons for the possible answers so that the developer only has to make one click to complete the survey. While developers may not reliably estimate time spent on platform tasks over the last quarter, they can reliably estimate that time over the past day. Aggregating all responses over the course of a month gives a good estimate of average time spent on platform tasks per developer per day.
In addition to surveys, we want to highlight two other ways of measuring the time developers spent on platform tasks:
Ticket tracking. Ask developers to tag platform tasks in their issue tracking system. If they also track time spent on each ticket, this can give you good data.
Shadowing. Sit with a developer and watch how they bring a new service into production, without helping them.
Both of these techniques allow you to assess not only the total time developers spent on platform tasks, but also what tasks they spend that time on. This is valuable input for platform roadmap decisions.
I'm already tracking X, why should I measure time spent on platform tasks, too?
You might already collect metrics such as change lead time. Why start tracking time spent on platform tasks, too?
We suspect that many metrics you might be tracking are not strongly correlated with time spent on platform tasks. So, measuring it gives you new information. Below are some example metrics and why we think they usually do not correlate strongly with time spent on platform tasks.
Change lead time. For many code changes, no platform tasks are needed. CI and deployment config were set up once, and shipping a new feature does not require setting them up again. Changes in lead time are often due to changes in manual testing, code review, or approval chains. These are distinct from platform tasks.
Time to first change. How long does it take a new developer to get a trivial change into production? In our own experience, access provisioning, local setup, and other tasks account for the majority of the time to first change. These are one-off tasks that increase and decrease independently of time spent on platform tasks.
Caveat: This skepticism about the correlation between time spent on platform tasks and the metrics above is based on our own anecdotal experience. If you have ever measured multiple of these metrics, we'd be keen to hear what correlation coefficients you saw.
What to do with this
Platform teams can have a large business impact by reducing the time developers spent on platform tasks. This is difficult to do without a way to measure that time.
Luckily, to recognize trends, you don't need a perfect measure, as long as your measure is strongly correlated with time spent on platform tasks. We think daily surveys on a small, random sample of your developers aren't perfect, but should correlate strongly enough.
Heplon is all about reducing time spent on platform tasks. Since developers use Heplon to perform platform tasks faster, troubleshoot issues, and escalate problems to human support, Heplon can produce insights into which platform tasks take the most time. And since Heplon connects to your communication platform, it can also do daily surveys to randomly sampled developers to track time spent on platform tasks.