Random Acts of Architecture

Software development is rife with overtime. Driven by passion and perfectionism, many developers throw everything into it, reveling in a culture of coding in darkened rooms late at night. Coupled with customers pushing for more or larger deliverables with fewer staff and a legal exemption from overtime pay in many circumstances, developers often shoulder the burden. Unfortunately, consistently working late nights and weekends is not sustainable but many seem unable to break free of the rut.

To put this in perspective, overtime is not unique to software development or even IT – just ask accountants around the end of financial year or salespeople in danger of missing their quota. There is also nothing wrong with flexible working hours and arrangements. Nevertheless, a work/life balance is important particularly for those with families or other commitments outside work.

It is easy to blame overtime on poor management. While true, saying “manage better” does not solve the problem. Indeed, there are many causes of software development overtime, including: (1) poor estimation or underestimation, (2) too few or under-skilled developers, (3) perfectionism, (4) redoing work due to past mistakes or (5) changing requirements/poor communication with other parts of the business. Specifically:

Poor estimation or underestimation: Few cling to the Newtonian beliefs that estimation is merely substituting initial measurements into well understood formulas to predict the delivery date.  The issue is one of negotiation – experienced software developers often know how much features will really cost in time and resources but software developers are often outclassed by customers or management used to arguing their case and, once agreed, deadlines are traditionally immutable, like a salesperson’s quota.

Too few staff or under-skilled developers: No development team is has “enough” developers and with the current trend of outsourcing and offshoring, developers working on project may be under-skilled or unfamiliar with the business problem. However, “you have to go to war with the army you have” as they say but the Dunning-Kruger effect and poor planning rarely take this (and Murphy’s Law) into account.

Perfectionism: Unfortunately, software developers can be their own worst enemies. Well architected, readable, tested software is the goal but “production quality code” is really a tautology – real developers ship. Include time for refactoring but time box it. Include time for design but realize it will change. Include time for testing but test important features first.

Correcting development mistakes: Unfortunately, mistakes happen but software developers must take responsibility for their own mistakes. To put this in context, at least development has the option of working late to fix their mistakes, unlike sales or management.

Changing requirements or poor communication: Many developers think of this as an “us versus them” situation – the “villainous” business “demands” changes and the developers are “helpless victims.” Instead, work on negotiation and trade-offs. After all, if the project is not delivered, everyone suffers, not just development. Make it a business problem or a business risk.

Note the steady addition of new requirements is a smell, possibly indicating the release cycle is too long. If possible, consider multiple smaller releases rather than a single large one.

There is no silver bullet to prevent overtime. Software development is a profession and there will always be work to do. However, it is possible to reduce it. Apart from focusing on soft skills, developers need to:

1. Improve negotiation skills. For example, treat estimates as immutable. Trade features rather than agree to reduce estimates. Otherwise the estimates lose credibility. Treat everything else (deadlines, delivery mechanisms, support, documentation, maintenance releases and so on)  as negotiable.
2. Improve estimation. Each development team faces its own challenges in this area but a few suggestions include:
   1. Improve prioritization. Many customers want everything but estimating the revenue or customers for each feature (or loss if not implemented) is usually the best prioritization strategy. Deprioritize features that have no customer impact (the lean software development definition of waste), too.
   2. Estimate with 10-20% fewer resources and less time. Customers often spot padded estimates (reducing their credibility leading them to press for reductions) but reducing the resources and time instead is usually seen as prudent.
   3. Present estimates using the least accurate unit of measure. Instead of saying “172.3 hours”, say “about 1 month”. Defer more detailed estimates until the problem is better understood later in the project.
3. Track the previous work estimates against actual time and resources. Use that to support current estimates and improve future estimation and negotiation.
4. Focus on delivering. Many developers love learning new things and striving for the ultimate solution. There is nothing wrong with this but a critical project is usually not the place for it. Temper or time box this experimentation and perfectionism.

Agile development methodologies might also be useful but suggest them tentatively. Many have presented ill-defined “agile development” practices as a panacea, thus some consider the term “agile” a less than credible buzzword. Agile development methodologies require a wider organizational change, like the appointment and recognition of single product owners in scrum, and may require more trust before allowing this.

The challenge with much of this is not sacrificing status of development within the organization or being seen as an impediment. The business may expect overtime and changing this perception can be hard. Metrics that demonstrate how things get worse as overtime increases, like as lines of code produced and bugs introduced, might help particularly against those that espouse the “pressure makes diamonds” fallacy.

Reducing it requires developers to recognize their part in the problem. There is nothing wrong with developers striving to create the best solution and putting in the extra effort when they want to but required and expected overtime is problematic. Ignoring extreme cases like those described by EA_Spouse, the best solution is usually a mutually constructive one, particularly with developers improving their negotiation skills.