What’s wrong with Accelerate and pursuit of DORA metrics?

By Andrew Park | 2024-06-18

The ultimate goals for any company should be great products and delighted users. Achieving this requires technical teams to develop strong design skills to create high-quality products, defined by achieving the ideal mix of attributes better than competitors. High-quality products like Apple iPhones, Tesla vehicles, and Nest thermostats did not achieve their status by optimizing DORA metrics. For software organizations aiming to develop top-tier products, the focus on DORA statistics is misguided. This emphasis on incremental delivery impedes great design, which requires integrating great designers with technical teams, rather than merely driving a continuous stream of incremental features.

The Origins of DORA, DORA Metrics, and “Accelerate”

The DevOps Research and Assessment (DORA) project, its metrics, and the book Accelerate emerged in the early 2010s from the collaboration of Nicole Forsgren, Jez Humble, and Gene Kim. DORA published the annual State of DevOps Reports, based on surveys that aimed to identify practices distinguishing high-performing organizations. These reports provided the foundation for the DORA metrics: 

1. Lead Time for Changes, 

2. Deployment Frequency, 

3. Mean Time to Restore (MTTR), and 

4. Change Failure Rate,

In their 2018 book Accelerate, Forsgren, Humble, and Kim claimed these metrics assess software delivery performance. But there are 2 major problems of focusing on optimizing “delivery performance.”

Problem #1: DORA Metrics Lead to Greater Engineering Labor Waste and Operations Costs

Focusing on DORA metrics through a DevOps approach results in increased waste of engineering team labor and higher operational costs. Our financial analysis of the “high performing companies” praised by the authors of “Accelerate” shows that these companies, while meeting DORA’s criteria, consistently fall below the industry average in engineering investment efficiency.

Pursuing DORA metrics drives engineering teams to adopt microservices architectures, because breaking a monolithic software program into 50 microservices leads to a 50x improvement in DORA metrics, despite delivering the same overall functionality as the original monolithic program. This illustrates how DORA statistics can misrepresent technical team productivity.

When organizations rearchitect their monolithic software into a plethora microservices, they often experience a major increase in computing costs. This is because a microservices API call is 10,000 times more expensive than a method call within the same service in terms of latency and computing cycles. This explains why the Amazon Prime team was able to reduce their operational costs by 90% when they moved from a microservices architecture to a monolithic architecture in 2023.

The companies featured in Accelerate enjoy near-monopoly status in their fields and can afford the high labor and operational cloud costs associated with their DevOps approaches. However, this approach is financially unsustainable for 90% of organizations. 

Problem #2: Mismatch Between DORA Metrics and Designing Great Products

The ultimate goals for any company should be great products and delighted users. Achieving these goals requires product managers and designers to fully develop their vision before involving technical teams. DORA metrics, while useful for measuring software delivery performance, prioritize speed and incremental delivery over comprehensive design, leading to misalignment with product quality goals.

Great products like Apple iPhones, Tesla vehicles, and Nest thermostats achieve their status through exceptional design, not by optimizing DORA metrics. Agile, Scrum, and DevOps often push for early implementation, disrupting the design process and wasting technical resources. A more effective approach starts with allowing Product Management to present a clear product vision before technical implementation, ensuring alignment and reducing wasted technical team effort. For top-tier products, integrating designers with technical teams is crucial, focusing on achieving the best design rather than continuous incremental features.

In summary, DORA metrics can hinder the design process needed for high-quality products. A comprehensive design approach that allows product visions to mature before technical work begins is essential for creating products that truly delight users.

Conclusion

In summary, the pursuit of DORA metrics and the principles outlined in Accelerate are fundamentally misaligned with the process of creating high-quality products that delight users. While DORA metrics can offer insights into software delivery performance, they lead to increased waste of engineering resources and higher operational costs, particularly when driving teams toward inefficient microservices architectures. More critically, these metrics emphasize speed and incremental delivery over the comprehensive design process required for great products. High-quality products like Apple iPhones, Tesla vehicles, and Nest thermostats achieve their success through exceptional design, not by optimizing DORA metrics. For most organizations, adopting a product management approach that allows for a fully developed product vision before technical implementation is essential. This ensures that technical efforts are aligned with design goals, ultimately leading to superior products and delighted users.