Category: Impact

How To Measure Real Success In Software Engineering

How To Measure Real Success In Software Engineering

Recently, while attending The Business Show in London, I engaged in a conversation with a CXO of an upcoming Fintech company. The discussion began with cybersecurity implementation—a topic close to my heart—but quickly veered into the realm of engineering throughput. What followed was an incoherent rant by the CXO, a frustrating narrative about firing their Delivery Director for refusing to scale the engineering team to meet deadlines for the company’s next shiny event. Despite my best efforts to pull this gentleman out of his rabbit hole, my time and reasoning seemed to fall on deaf ears.

Reflecting on this interaction over the past month, I’ve realized this episode was emblematic of a larger issue: the prevalent fallacy among CXOs that more engineers equals faster and better output. Surprisingly, this misconception thrives in part because of the silence of engineering leaders—CTOs, VPs, and Directors of Engineering—who often fail to push back against flawed assumptions at the executive level.

Inspired by my recent association with the Information Security Group (ISG) at the Royal Holloway University of London, I decided to don my “academic specs” and examine this fallacy more critically. The result is a deeper dive into the myths of scaling engineering teams, the science behind team efficiency, and a call for a cultural shift in how organizations measure productivity.

The Scaling Myth: Why More Isn’t Always Better

At the heart of this fallacy is a simplistic assumption: more engineers means more features, delivered faster. While this notion seems logical, it is disproven by Price’s Law, a principle that exposes the diminishing returns of team scaling.

Rediscovering Derek J. de Solla Price: From Antikythera to Engineering Efficiency

My journey to understanding Price’s Law began with a fascination for the Antikythera Mechanism —an ancient Greek marvel of engineering and astronomy. It was through this mechanism that I first encountered the work of Prof. Derek J. de Solla Price, a British physicist and historian whose curiosity and intellect extended far beyond antiquities. Inspired by the ingenuity of the Antikythera Mechanism, I was drawn to explore the origins of Damascus and Wootz steel, and its roots in the south-western peninsula of India (as detailed in Aayutha Desam by R. Mannar Mannan). (More about that in another post!)

But it was Price’s insight into the uneven distribution of productivity in groups that struck a chord with my work in software engineering. His principle, now widely known as Price’s Law, asserts that in any team, 50% of the work is accomplished by the square root of the total number of participants.

  • In a team of 10 engineers, approximately 3 contributors (√10) are responsible for half the output.
  • In a team of 100 engineers, only 10 individuals (√100) produce as much as the remaining 90 combined.

This principle highlights a counterintuitive but vital truth: as team size grows, the proportion of high contributors decreases, leading to inefficiencies that compound over time. This isn’t just an academic curiosity—it’s a critical insight for engineering leaders tasked with scaling teams and delivering results.

Price’s Law challenges a long-standing assumption in engineering leadership: that scaling teams proportionally scales productivity. By understanding this principle, CTOs, VPs, and engineering managers can rethink strategies for achieving efficiency and delivering value, even with constrained resources.

The Myth of Highly Motivated Teams

Some self-proclaimed visionary leaders advocate for hiring only highly motivated individuals, often overlooking how teams function in practice. In any organized group, work typically falls into three categories:

  1. Drudgery Work (Low Impact, High Intensity): Routine tasks like debugging or documentation, essential but unappealing.
  2. Intermediate Work (Medium Impact, Medium Intensity): Feature upgrades or system integrations, vital for sustaining operations.
  3. Challenging Work (High Impact, High Intensity): Complex, high-stakes initiatives that highly motivated individuals prefer.

The Problem

Highly motivated individuals often prioritize high-impact projects, leaving routine and intermediate work neglected. This creates:

  • Operational Bottlenecks: Accumulating technical debt and system fragility.
  • Imbalanced Workloads: Overburdened team members handling routine tasks.
  • Team Friction: Reduced cohesion and potential burnout.

The Solution: Balance Over Ambition

Effective teams thrive on diversity in skill sets and balanced task allocation. Leaders must:

  • Distribute Work Strategically: Ensure all types of work are addressed.
  • Value Contributions Equally: Recognize the importance of routine and intermediate tasks.
  • Foster Team Cohesion: Avoid over-prioritizing high-stakes projects at the expense of operational stability.

Conclusion: A truly visionary leader grounds ambition in pragmatism, creating teams that excel not just in high-impact projects but also in sustaining the essentials of day-to-day operations.

Implications for Team Expansion

For CTOs, VPs, and engineering managers, this dynamic presents a counterintuitive challenge: merely expanding the team does not guarantee proportional gains in productivity. Doubling headcount often introduces:

  1. Communication Overhead: Larger teams require more coordination, which consumes valuable time and resources.
  2. Dilution of Accountability: As teams grow, individual contributions become harder to track, potentially reducing ownership and engagement.
  3. Coordination Complexities: Increased interdependencies among team members can slow down decision-making and implementation.

To achieve a twofold increase in productivity, Price’s Law suggests that you may need to quadruple the team size, a move that is often impractical and financially untenable. Instead, engineering leaders must rethink productivity beyond the simplistic metric of team size.

Shifting Focus: Outcomes Over Outputs

Traditional productivity metrics, such as the number of features released or lines of code written, focus on outputs—tangible deliverables produced by the team. However, outputs do not inherently translate into value. Consider the distinction:

  • Outputs: Metrics like features delivered or tickets closed.
  • Outcomes: Measurable changes in user behaviour that drive business results, such as increased user retention or reduced churn.

Relying solely on outputs creates a misleading picture of productivity. A feature-rich application that fails to address user needs or business goals is ultimately unproductive. Instead, outcomes—which capture the real-world effectiveness of engineering efforts—offer a better lens to measure success.

Outcome vs. Impact

While outcomes focus on immediate effects (e.g., increased sign-ups from a new feature), impact delves deeper into long-term consequences. For example:

  • An outcome may be an increase in user sign-ups after a feature launch.
  • The impact would be sustained revenue growth and user satisfaction resulting from the feature’s value over time.

Engineering teams must aim for outcomes that align with strategic goals while keeping an eye on their long-term impacts.

Counterproductive Paradigm: The Threat Surface of Excessive Outputs

Emphasizing outputs over outcomes can be counterproductive, leading to what can be described as an expanding threat surface:

  1. Defects and Bugs: Adding more features often introduce unintended issues that require additional resources to resolve.
  2. Maintenance Burden: More code increases the risk of technical debt, making future development slower and more complex.
  3. Conflict Resolution: Larger teams fixing bugs or implementing features in parallel can inadvertently cause regressions, especially when the main sprint continues uninterrupted.

This vicious cycle diverts focus from strategic initiatives, tying up engineers in a continuous loop of fixes. Instead of scaling output indiscriminately, teams should focus on ensuring that every deliverable contributes to meaningful outcomes.

Focusing on Impacts and Outcomes: A Leadership Imperative

For engineering leaders, the shift from outputs to impacts and outcomes is transformative. This approach emphasizes:

  1. Defining Clear Objectives: Establish measurable outcomes (e.g., reducing churn by 10%) that align with business goals.
  2. Prioritizing High-Impact Work: Evaluate tasks based on their potential to deliver meaningful results.
  3. Empowering Teams: Foster a culture where engineers understand and contribute to broader business objectives rather than just completing tickets.
  4. Continuous Feedback Loops: Regularly assess whether engineering efforts are driving intended outcomes.

This shift not only enhances productivity but also aligns engineering work with the organization’s mission, fostering a sense of purpose within teams.

Conclusion: Redefining Productivity in Software Engineering

Price’s Law reminds us that productivity does not scale linearly with team size. Engineering leaders must navigate this reality by focusing on outcomes and impacts rather than outputs. This paradigm shift requires a cultural and strategic overhaul, but the rewards—greater efficiency, alignment, and value delivery—are well worth the effort.

By embracing this approach, organizations can ensure that their engineering efforts contribute directly to their strategic goals, transforming software development into a driver of sustainable business success.

References

  1. Sundarakalatharan, R. (2022). How to measure Engineering Productivity?. Retrieved from https://nocturnalknight.co/how-to-measure-engineering-productivity/
  2. Bohrmann, N. (2022). How Price’s Law Applies to Everything. Retrieved from https://nielsbohrmann.com/prices-law/
  3. LeadDev. (2022). Focus on outcomes over outputs. Retrieved from https://leaddev.com/velocity/focus-outcomes-over-outputs
  4. Monday Mornings. (2023). Productivity and Price’s Law. Retrieved from https://mondaymornings.madisoncres.com/productivity-and-prices-law-1
  5. TechRadar. (2023). Outcomes versus outputs: the real measure of developer productivity. Retrieved from https://www.techradar.com/pro/outcomes-versus-outputs-the-real-measure-of-developer-productivity
  6. Royal Holloway Information Security Group. (2024). https://pure.royalholloway.ac.uk/
  7. Wikipedia. (2024). Antikythera Mechanism. Retrieved from https://en.wikipedia.org/wiki/Antikythera_mechanism
  8. Wikipedia. (2024). Derek J. de Solla Price. Retrieved from https://en.wikipedia.org/wiki/Derek_J._de_Solla_Price
  9. Wikipedia. (2024). Wootz Steel. Retrieved from https://en.wikipedia.org/wiki/Wootz_steel
  10. Purple Book House. (2024). Aayutha Desam by R. Mannar Mannan. Retrieved from https://www.purplebookhouse.co.uk/product-page/aayutha-desam-book-type-katturaigal-history-by-r-mannar-mannan
Bitnami