How To Measure Real Success In Software Engineering

By Ramkumar Sundarakalatharan | December 12, 2024 | Comments 0 Comment

Recently, while attending The Business Show in London, I engaged in a conversation with a CXO of an upcoming Fintech company. The discussion began with cybersecurity implementation—a topic close to my heart—but quickly veered into the realm of engineering throughput. What followed was an incoherent rant by the CXO, a frustrating narrative about firing their Delivery Director for refusing to scale the engineering team to meet deadlines for the company’s next shiny event. Despite my best efforts to pull this gentleman out of his rabbit hole, my time and reasoning seemed to fall on deaf ears.

Reflecting on this interaction over the past month, I’ve realized this episode was emblematic of a larger issue: the prevalent fallacy among CXOs that more engineers equals faster and better output. Surprisingly, this misconception thrives in part because of the silence of engineering leaders—CTOs, VPs, and Directors of Engineering—who often fail to push back against flawed assumptions at the executive level.

Inspired by my recent association with the Information Security Group (ISG) at the Royal Holloway University of London, I decided to don my “academic specs” and examine this fallacy more critically. The result is a deeper dive into the myths of scaling engineering teams, the science behind team efficiency, and a call for a cultural shift in how organizations measure productivity.

The Scaling Myth: Why More Isn’t Always Better

At the heart of this fallacy is a simplistic assumption: more engineers means more features, delivered faster. While this notion seems logical, it is disproven by Price’s Law, a principle that exposes the diminishing returns of team scaling.

Rediscovering Derek J. de Solla Price: From Antikythera to Engineering Efficiency

My journey to understanding Price’s Law began with a fascination for the Antikythera Mechanism —an ancient Greek marvel of engineering and astronomy. It was through this mechanism that I first encountered the work of Prof. Derek J. de Solla Price, a British physicist and historian whose curiosity and intellect extended far beyond antiquities. Inspired by the ingenuity of the Antikythera Mechanism, I was drawn to explore the origins of Damascus and Wootz steel, and its roots in the south-western peninsula of India (as detailed in Aayutha Desam by R. Mannar Mannan). (More about that in another post!)

But it was Price’s insight into the uneven distribution of productivity in groups that struck a chord with my work in software engineering. His principle, now widely known as Price’s Law, asserts that in any team, 50% of the work is accomplished by the square root of the total number of participants.

In a team of 10 engineers, approximately 3 contributors (√10) are responsible for half the output.
In a team of 100 engineers, only 10 individuals (√100) produce as much as the remaining 90 combined.

This principle highlights a counterintuitive but vital truth: as team size grows, the proportion of high contributors decreases, leading to inefficiencies that compound over time. This isn’t just an academic curiosity—it’s a critical insight for engineering leaders tasked with scaling teams and delivering results.

Price’s Law challenges a long-standing assumption in engineering leadership: that scaling teams proportionally scales productivity. By understanding this principle, CTOs, VPs, and engineering managers can rethink strategies for achieving efficiency and delivering value, even with constrained resources.

The Myth of Highly Motivated Teams

Some self-proclaimed visionary leaders advocate for hiring only highly motivated individuals, often overlooking how teams function in practice. In any organized group, work typically falls into three categories:

Drudgery Work (Low Impact, High Intensity): Routine tasks like debugging or documentation, essential but unappealing.
Intermediate Work (Medium Impact, Medium Intensity): Feature upgrades or system integrations, vital for sustaining operations.
Challenging Work (High Impact, High Intensity): Complex, high-stakes initiatives that highly motivated individuals prefer.

The Problem

Highly motivated individuals often prioritize high-impact projects, leaving routine and intermediate work neglected. This creates:

Operational Bottlenecks: Accumulating technical debt and system fragility.
Imbalanced Workloads: Overburdened team members handling routine tasks.
Team Friction: Reduced cohesion and potential burnout.

The Solution: Balance Over Ambition

Effective teams thrive on diversity in skill sets and balanced task allocation. Leaders must:

Distribute Work Strategically: Ensure all types of work are addressed.
Value Contributions Equally: Recognize the importance of routine and intermediate tasks.
Foster Team Cohesion: Avoid over-prioritizing high-stakes projects at the expense of operational stability.

Conclusion: A truly visionary leader grounds ambition in pragmatism, creating teams that excel not just in high-impact projects but also in sustaining the essentials of day-to-day operations.

Implications for Team Expansion

For CTOs, VPs, and engineering managers, this dynamic presents a counterintuitive challenge: merely expanding the team does not guarantee proportional gains in productivity. Doubling headcount often introduces:

Communication Overhead: Larger teams require more coordination, which consumes valuable time and resources.
Dilution of Accountability: As teams grow, individual contributions become harder to track, potentially reducing ownership and engagement.
Coordination Complexities: Increased interdependencies among team members can slow down decision-making and implementation.

To achieve a twofold increase in productivity, Price’s Law suggests that you may need to quadruple the team size, a move that is often impractical and financially untenable. Instead, engineering leaders must rethink productivity beyond the simplistic metric of team size.

Shifting Focus: Outcomes Over Outputs

Traditional productivity metrics, such as the number of features released or lines of code written, focus on outputs—tangible deliverables produced by the team. However, outputs do not inherently translate into value. Consider the distinction:

Outputs: Metrics like features delivered or tickets closed.
Outcomes: Measurable changes in user behaviour that drive business results, such as increased user retention or reduced churn.

Relying solely on outputs creates a misleading picture of productivity. A feature-rich application that fails to address user needs or business goals is ultimately unproductive. Instead, outcomes—which capture the real-world effectiveness of engineering efforts—offer a better lens to measure success.

Outcome vs. Impact

While outcomes focus on immediate effects (e.g., increased sign-ups from a new feature), impact delves deeper into long-term consequences. For example:

An outcome may be an increase in user sign-ups after a feature launch.
The impact would be sustained revenue growth and user satisfaction resulting from the feature’s value over time.

Engineering teams must aim for outcomes that align with strategic goals while keeping an eye on their long-term impacts.

Counterproductive Paradigm: The Threat Surface of Excessive Outputs

Emphasizing outputs over outcomes can be counterproductive, leading to what can be described as an expanding threat surface:

Defects and Bugs: Adding more features often introduce unintended issues that require additional resources to resolve.
Maintenance Burden: More code increases the risk of technical debt, making future development slower and more complex.
Conflict Resolution: Larger teams fixing bugs or implementing features in parallel can inadvertently cause regressions, especially when the main sprint continues uninterrupted.

This vicious cycle diverts focus from strategic initiatives, tying up engineers in a continuous loop of fixes. Instead of scaling output indiscriminately, teams should focus on ensuring that every deliverable contributes to meaningful outcomes.

Focusing on Impacts and Outcomes: A Leadership Imperative

For engineering leaders, the shift from outputs to impacts and outcomes is transformative. This approach emphasizes:

Defining Clear Objectives: Establish measurable outcomes (e.g., reducing churn by 10%) that align with business goals.
Prioritizing High-Impact Work: Evaluate tasks based on their potential to deliver meaningful results.
Empowering Teams: Foster a culture where engineers understand and contribute to broader business objectives rather than just completing tickets.
Continuous Feedback Loops: Regularly assess whether engineering efforts are driving intended outcomes.

This shift not only enhances productivity but also aligns engineering work with the organization’s mission, fostering a sense of purpose within teams.

Conclusion: Redefining Productivity in Software Engineering

Price’s Law reminds us that productivity does not scale linearly with team size. Engineering leaders must navigate this reality by focusing on outcomes and impacts rather than outputs. This paradigm shift requires a cultural and strategic overhaul, but the rewards—greater efficiency, alignment, and value delivery—are well worth the effort.

By embracing this approach, organizations can ensure that their engineering efforts contribute directly to their strategic goals, transforming software development into a driver of sustainable business success.

References

Sundarakalatharan, R. (2022). How to measure Engineering Productivity?. Retrieved from https://nocturnalknight.co/how-to-measure-engineering-productivity/
Bohrmann, N. (2022). How Price’s Law Applies to Everything. Retrieved from https://nielsbohrmann.com/prices-law/
LeadDev. (2022). Focus on outcomes over outputs. Retrieved from https://leaddev.com/velocity/focus-outcomes-over-outputs
Monday Mornings. (2023). Productivity and Price’s Law. Retrieved from https://mondaymornings.madisoncres.com/productivity-and-prices-law-1
TechRadar. (2023). Outcomes versus outputs: the real measure of developer productivity. Retrieved from https://www.techradar.com/pro/outcomes-versus-outputs-the-real-measure-of-developer-productivity
Royal Holloway Information Security Group. (2024). https://pure.royalholloway.ac.uk/
Wikipedia. (2024). Antikythera Mechanism. Retrieved from https://en.wikipedia.org/wiki/Antikythera_mechanism
Wikipedia. (2024). Derek J. de Solla Price. Retrieved from https://en.wikipedia.org/wiki/Derek_J._de_Solla_Price
Wikipedia. (2024). Wootz Steel. Retrieved from https://en.wikipedia.org/wiki/Wootz_steel
Purple Book House. (2024). Aayutha Desam by R. Mannar Mannan. Retrieved from https://www.purplebookhouse.co.uk/product-page/aayutha-desam-book-type-katturaigal-history-by-r-mannar-mannan

If You Can Do Your Job From Home, Be Scared. Be Very Scared!

By Ramkumar Sundarakalatharan | July 2, 2023 | Comments 0 Comment

Sometimes, It seems like most people have added “Remote Work” to a long list of taboo topics that no one should discuss at work. Topics like Religion, Politics, Sexual Orientation, Medical Issues, etc.

Most people know that remote work is a horrible idea for organisations of any size, but they are afraid to call it out because they don’t want to appear out of touch with their employees/colleagues. Others are afraid to be viewed as hostile figures who wish to create tension with employees or be perceived as a manager who doesn’t trust their employees.

So, employers stay silent, and now everyone thinks remote work is the best business idea of the last 200 years.

Most of these shenanigans use personal anecdotes to defend the benefit of remote work. They say they are more productive at home, with fewer distractions and more time for daily (personal) chores. I do not agree or disagree with that statement completely, as it could be dependent on specific roles.

When Remote Work Could be Beneficial:

If you are an individual contributor and mostly work without the need to collaborate in real time, you can be productive in remote work. For example, If you are a finance analyst who dives through ledgers, borough thru tonnes of CSVs and crunch numbers all day, you may be more productive in Home.

When Remote work could be less than Optimal

If you are a creative or knowledge worker (Designer/Developer), chances are you’d NEED to collaborate with, break down/delegate, get feedback etc. In these roles, it will be almost impossible to get a calendar from 6 different people to drive consensus. Whereas if all of your stakeholders are in the office, it is a mere “Shout” or “Wave” and 3 mins conversation following it.

Google has officially changed its mind about remote work

Google leadership publicly admitted that remote work no longer works for them, and that’s the reason they want all of their employees back behind their desks.

Last week, Fiona Cicconi, Google’s chief people officer, wrote an email to the entire company stating, “Going forward, we’ll consider new remote work requests by exception only.” This is horrible news for employees and some companies who believe that remote work is the greatest idea since the invention of the internet.

Let me declutter the above statement for you.

Google is the biggest tech company in the world that created 100s of resources and tools to enable employees to work remotely, admitting defeat.

Despite the release of many communications tools that enabled workers across the globe and all industries to work remotely, it is finally saying that remote work doesn’t work. Let that sink in. Google’s remote employees are unhappy, but Google’s leadership rarely pay attention to the feeling of their employees (after Larry left Alphabet inc). They only pay attention to the stock price and what is recommended at the shareholder’s meeting.

Google is not alone. Apple, Microsoft, Facebook, and Amazon also laid off most remote employees.

Remote work destroyed the most profitable industry

As you have seen above, Google leadership is no longer willing to offer their employees 100% remote jobs. They want their employees in the office and productive.

You might ask, why is this happening?

It is happening because you witnessed the destruction of the Tech industry in the last two years. The tech industry crumbled because most Leaders were afraid to tell their remote employees to come to the office, so they did the next best thing: they lost money and laid remote employees off (mostly).

According to the Layoffs tracker, more than 200,000 people were laid off in 2023, and over 164,000 employees were laid off in 2022. This is a clear message to anyone who works in the tech industry, stop working remotely or start interviewing soon.

This is exactly what the most innovative CEO in the world did when he bought Twitter.

When Elon Musk bought Twitter, he found the company in tatters. Musk gave them the same two options he gave his Tesla employees, “If you do not return to the office, you cannot remain at the company. End of story.”

That was the end of the story for many of Twitter’s employees, Musk fired anyone who refused to show up at the office, and his company is more productive and innovative than ever.

Musk ended remote work at Twitter, and most people hated him.

Not every executive had the guts to do what Elon Musk did, but now most wish they did because every company needs to lay off their unproductive employees.

Remote employees are less visible. When it comes to your value to your company, you have to be visible. If you think visibility is not important, ask Kayne West.
Remote employees don’t have a strong connection to other employees or their companies. Relationships are extremely important. I established my professional credibility and earned my colleagues’ respect by connecting with them face-to-face, not through a computer screen.
Remote employees are less invested in their work or career. Since most employer link visibility and ability together, I understand why more than 3000 HR managers believe that.

These reasons led Google, Microsoft, Amazon, Facebook, and Zoom to lay off their remote employees first. This is a horrible trend for people who want to work remotely, especially since most remote workers say it could take more than six months to find a new job

In conclusion, I’d like to recommend Richard Baldwin’s video

Is the myth of a “10X Developer” Real?

By Ramkumar Sundarakalatharan | June 29, 2023 | Comments 0 Comment

If you’re a software engineer, manager or leader, I am sure you have heard the term ‘10x developer’ used in discussions. It refers to developers who are purportedly 10 times more productive, or capable, than their peers, while it is a hotly contested category. Some refer to it very liberally, others deny that it even exists. In the last 40+ years, the ‘10X developer` has become a ‘Loch Ness` of the tech world, fueled by the hype associated with Silicon Valley.

I’m not about to delude myself into thinking that writing a blog on it to pass my verdict will put these theories to rest, but the question has gained enough traction that it deserves a little articulation.

Do 10x developers really exist, and if so, how would we distinguish them?

Framing the Issue

All of us can acknowledge that the range of skills in most human activities can be extensive. A marathon runner can cover roughly 10 times the distance that an untrained person could, while a professional chef can cook a 5-course meal in 1/5th the time it takes an average person to do a 2-course meal.

Coding, which is a hugely complex field unencumbered by physical limitations, should naturally show differences in the skill that vary by orders of magnitude. Thus, if by 10x developer we simply mean a person whose skill level is in a different league compared to someone else, then clearly they exist.

How could anyone argue otherwise?

Here’s the rub though, in the data-driven and lexically precise world of modern tech, that’s not what 10x developer means. Instead, a 10x developer is supposed to be someone who genuinely outperforms others by 10 times or more on some quantifiable scale. That ‘quantifiable scale’ is where the problems start.

Where did the term actually come from? Enter Coding War Games.

Coding War Games

Tom DeMarco and Tim Lister have conducted the “Coding War Games” since 1977. This is a public productivity survey in which teams of software implementors from different organizations compete to complete a series of benchmarks in minimal time with minimal defects. They’ve had over 600 developers participate. Its results are publicly available and is very informative, to say the least. Jeff Lester published a wonderful piece on the origins of the 10X developer here.

The top findings from these are,

1, Get your working environment Right

The overriding observation from this study is that quiet, private, dedicated working space with fewer interruptions led to groups that performed significantly better.

2, Remove the Net Negative Producing Programmer

Some developers are “net negative producing programmers” (NNPP), that is they produce so many defects that removing them from the team increases productivity. This is the opposite of a 10X developer, these people are the ones that make the team productivity go from bad to worse.

The Problem with Measuring ‘Skill’

Even where skill can vary wildly, differences will not necessarily be quantifiable. A talented artist may know how to create a painting that teleports you to a whole other world compared to an average artiste who can transport you to the scene.

But the question is, can you attach numbers to that painting’s beauty?

The work of a developer isn’t nearly as abstract, but not all of it can be reduced to metrics either, and definitely not the programming skill itself.

A less glamorous approach may be to judge a 10x dev not in terms of skill but in terms of productivity. Someone who can write 500 lines of code when it takes others to write 50 would then fall in that category.

If you know anything about programming, however, you’ve probably already spotted the problem with this line of thinking. Longer code isn’t necessarily more efficient, and for most people there tends to be a positive correlation between how quickly one works and how many bugs one creates.

This is not to say that programmers can’t produce bug-free code much faster than their peers. Where this statement proves fallacious though, is in trying to peg that difference to a single metric. There are myriad factors at play that will affect a developer’s productivity outside of their skill, including their team and the environment they find themselves. In fact, depending on the situation, the 10x tag may be inaccurate because a developer could be programming well over 10 times as much as another and still produce 1/10th of the ‘Outcomes’!

What should be our conclusion? There can be no doubt that the field of programming has its own Mozarts and Vincent Van Goghs, and few would object if these people were described as being ‘orders of magnitude’ better than the rest. But it is important to recognize that this is only a figure of speech, and not something meant to be used according to its precise quantitative meaning.

I can’t presume to speak for the tech industry as a whole, but I for one have noticed a worrying tendency to read the expression ‘10x developer’ literally.

Ultimately this does more harm than good, as it spreads the myth that there is some universal metric whereby every programmer’s value can always be quantified.

Important qualities like creativity, client focus and teamwork are entirely omitted in this way of thinking, which is why my final suggestion is to stop worrying about lofty 10x developers and whether you are, aren’t, or may or not become one.

Simply focus on being the best developer you can be. That will always be enough.

References & Further Reading

Origin of a 10X developer

https://medium.com/ingeniouslysimple/the-origins-of-the-10x-developer-2e0177ecef60

https://gwern.net/doc/cs/algorithm/2001-demarco-peopleware-whymeasureperformance.pdf

https://news.ycombinator.com/item?id=22349531

The 10x Programmer Myth