Random Acts of Architecture

Tales of an architect trying to bring order to the chaos that is modern information technology.

Category Archives: Software Development

Software Development: An Overtime Culture

Software development is rife with overtime. Driven by passion and perfectionism, many developers throw everything into it, reveling in a culture of coding in darkened rooms late at night. Coupled with customers pushing for more or larger deliverables with fewer staff and a legal exemption from overtime pay in many circumstances, developers often shoulder the burden. Unfortunately, consistently working late nights and weekends is not sustainable but many seem unable to break free of the rut.

To put this in perspective, overtime is not unique to software development or even IT – just ask accountants around the end of financial year or salespeople in danger of missing their quota. There is also nothing wrong with flexible working hours and arrangements. Nevertheless, a work/life balance is important particularly for those with families or other commitments outside work.

It is easy to blame overtime on poor management. While true, saying “manage better” does not solve the problem. Indeed, there are many causes of software development overtime, including: (1) poor estimation or underestimation, (2) too few or under-skilled developers, (3) perfectionism, (4) redoing work due to past mistakes or (5) changing requirements/poor communication with other parts of the business. Specifically:

Poor estimation or underestimation: Few cling to the Newtonian beliefs that estimation is merely substituting initial measurements into well understood formulas to predict the delivery date.  The issue is one of negotiation – experienced software developers often know how much features will really cost in time and resources but software developers are often outclassed by customers or management used to arguing their case and, once agreed, deadlines are traditionally immutable, like a salesperson’s quota.

Too few staff or under-skilled developers: No development team is has “enough” developers and with the current trend of outsourcing and offshoring, developers working on project may be under-skilled or unfamiliar with the business problem. However, “you have to go to war with the army you have” as they say but the Dunning-Kruger effect and poor planning rarely take this (and Murphy’s Law) into account.

Perfectionism: Unfortunately, software developers can be their own worst enemies. Well architected, readable, tested software is the goal but “production quality code” is really a tautology – real developers ship. Include time for refactoring but time box it. Include time for design but realize it will change. Include time for testing but test important features first.

Correcting development mistakes: Unfortunately, mistakes happen but software developers must take responsibility for their own mistakes. To put this in context, at least development has the option of working late to fix their mistakes, unlike sales or management.

Changing requirements or poor communication: Many developers think of this as an “us versus them” situation – the “villainous” business “demands” changes and the developers are “helpless victims.” Instead, work on negotiation and trade-offs. After all, if the project is not delivered, everyone suffers, not just development. Make it a business problem or a business risk.

Note the steady addition of new requirements is a smell, possibly indicating the release cycle is too long. If possible, consider multiple smaller releases rather than a single large one.

There is no silver bullet to prevent overtime. Software development is a profession and there will always be work to do. However, it is possible to reduce it. Apart from focusing on soft skills, developers need to:

  1. Improve negotiation skills. For example, treat estimates as immutable. Trade features rather than agree to reduce estimates. Otherwise the estimates lose credibility. Treat everything else (deadlines, delivery mechanisms, support, documentation, maintenance releases and so on)  as negotiable.
  2. Improve estimation. Each development team faces its own challenges in this area but a few suggestions include:
    1. Improve prioritization. Many customers want everything but estimating the revenue or customers for each feature (or loss if not implemented) is usually the best prioritization strategy. Deprioritize features that have no customer impact (the lean software development definition of waste), too.
    2. Estimate with 10-20% fewer resources and less time. Customers often spot padded estimates (reducing their credibility leading them to press for reductions) but reducing the resources and time instead is usually seen as prudent.
    3. Present estimates using the least accurate unit of measure. Instead of saying “172.3 hours”, say “about 1 month”. Defer more detailed estimates until the problem is better understood later in the project.
  3. Track the previous work estimates against actual time and resources. Use that to support current estimates and improve future estimation and negotiation.
  4. Focus on delivering. Many developers love learning new things and striving for the ultimate solution. There is nothing wrong with this but a critical project is usually not the place for it. Temper or time box this experimentation and perfectionism.

Agile development methodologies might also be useful but suggest them tentatively. Many have presented ill-defined “agile development” practices as a panacea, thus some consider the term “agile” a less than credible buzzword. Agile development methodologies require a wider organizational changelike the appointment and recognition of single product owners in scrum, and may require more trust before allowing this.

The challenge with much of this is not sacrificing status of development within the organization or being seen as an impediment. The business may expect overtime and changing this perception can be hard. Metrics that demonstrate how things get worse as overtime increases, like as lines of code produced and bugs introduced, might help particularly against those that espouse the “pressure makes diamonds” fallacy.

Reducing it requires developers to recognize their part in the problem. There is nothing wrong with developers striving to create the best solution and putting in the extra effort when they want to but required and expected overtime is problematic. Ignoring extreme cases like those described by EA_Spouse, the best solution is usually a mutually constructive one, particularly with developers improving their negotiation skills.

A Clash of Development Cultures

Software developers often consider themselves perfectly rational. After all, their jobs consist of programming deterministic, rational machines (computers). However, anyone who has worked with developers realises this is completely untrue. In particular, leaders that direct or influence different groups of developers need to understand what motivates them, especially the more vocal and influential developers and technical leads.

There are two types of influential developers. The first are the “craftsmen” (or the gender non-specific “craftspeople”). Craftsmen are motivated by the respect of their peers and want to be known as “good” developers. Craftsmen tend to:

  • Want to work with cutting edge tools, such as the latest version of an IDE or library, and techniques, such as agile development methodologies.
  • Think “good” code not only satisfies the requirements but is clear, maintainable and is an example to others. They are more likely to refactor code, follow (or create) coding  style guidelines and create automated tests, particularly when they have advance warning their code is going to be reviewed.
  • Thoroughly research and experiment with different ways of solving a problem. They are often critical of designs or direction from others, particularly those removed form the code.
  • Prefer a more egalitarian, flatter management and team structure. They are happy to work in a group of peers, each with their own area of responsibility.
  • Code in their spare time, discuss technical issues over lunch and read or write technical blogs. They hold examples of collaboration, such as open source software, in high regard. They tend to look past certifications and degrees to focus on what was studied or produced.
  • Hold technical opinions strongly. They may argue weakly because they feel their points are “obvious”.
  • Be more common in western cultures like in the US, UK and Australia.

The second are “success-oriented“. They see software development as a relatively well paid and respected stepping stone, usually to management.  They tend to:

  • Prefer familiar tools and techniques, so long as they are well compensated and have promotion opportunities.
  • Think “good” code meets requirements. Other facts are less important.
  • Seek help for problems when there is no immediate, obvious solution.
  • Look to seniority or management for guidance and direction. Indeed, if they are in a senior position, they will often demand the right to design or give direction.
  • Respect and seek “authoritative” measures of achievement such as degrees, certifications and training. The goal is usually to decrease the time to the next promotion, enter the industry at a higher position or be paid more.
  • Hold technical opinions weakly. They may argue strongly because it their training and education emphasized it.
  • Be more common in eastern cultures like in India and China.

Clearly, these groups are not well-defined or mutually exclusive and are generalizations. Craftsmen still want promotions and pay rises. Success-oriented developers may still be interested in technology. They key difference is that the software written by craftsmen is part of their identity so they work to make it as good as they can. They grok languages rather than just learn them.

Craftsmen are not better developers than success-oriented developers and vice versa. For example, craftsman may get distracted by perfectionism or experiments but may drive the organization to newer and (hopefully) better processes. On the other hand, success-oriented developers often use well-understood paths to complete tasks sooner and are happy to maintain older software using older tools.

Success-oriented developers are not more likely to provide business value than craftsmen. Although success-oriented developers are less dismissive of non-technical users and focus more on pleasing management, they commonly implement the requirements verbatim without additional understanding.

Although team composition and work assignment are often the domain of team leaders and management, it benefits everyone in the team to understand others’ motivations. For example, pointing out a better solution to success-oriented developers may frustrate them since their solution is no longer “successful” and requires additional work. Point out the simplicity, time-saving aspects of the new solution or how marketable the new skills required are to motivate or influence success-oriented developers. By comparison, pointing out a better solution to craftsmen may embarrass or make them feel threatened. Point to relevant code, articles or recommendations of industry thought leaders to motivate or influence craftsmen.

Also be aware the type of developer when soliciting feedback. Craftsmen generally give better feedback. Many success-oriented developers are happy to be directed by those more senior or want to direct those they perceive as junior. That said, craftsmen are likely to push newer designs and techniques while success-oriented developers look at things in the context of existing practises.

Moreover, having both developer types working together without self-awareness can create friction, such as when a success-oriented senior developer or manager asserts authority over a craftsman developer pushing what he or she considers a better solution, perhaps backed up by blogs or articles. Outsourcing or mergers/acquisitions often cause or exacerbate this.

The challenge is that developers have weak soft skills and many consider meetings and discussions unproductive time. They rarely focus on interacting with each other, sometimes oblivious to the problems it creates. Unfortunately, developers are not perfectly rational but understanding their motivations can solve common problems or avoid them completely.

(Thanks to Bryan O’Donovan for the helpful review and comments.)

A Glimpse at Stack Overflow

As any software developer that has searched for anything on the Internet probably knows, the Stack Exchange web sites contain an increasingly large body of useful knowledge. This initially included sites like stackoverflow.com for programming issues but has expanded into diverse areas like cooking, poker and lego. There is even a Stack Exchange site for questions about Stack Exchange.

This popularity is largely fuelled by gamification, where game design and theory is applied to ordinary tasks to make them more engaging. Stack Exchange does this primarily through reputation, similar to the XBox’s Gamerscore. Participants gain reputation primarily for asking or answering popular (up voted) questions. As participants gain reputation, they gain access to more privileges, eventually being able to effectively moderate the site.

There are also badges, akin to Xbox achievements. A bronze badge is given for simple things like completing your user profile or answering your first question. Silver and gold badges are awarded for harder things like asking or answering questions with many up votes.

Interestingly, Stack Overflow has reportedly been used by some to demonstrate competency when applying for jobs and Stack Overflow now has a careers site that requires a minimum reputation to enter. This made me wonder how difficult it would be to gain Stack Overflow reputation, say 1000, by genuinely trying to help others and participate in the intended spirit.

I registered on stackoverflow.com at 2:39pm on Thursday 30th August 2012 (Sydney, Australia time) and exceeded 1000 reputation at 10:44am on Wednesday 5th September with 850 reputation (tracked separately) on programmers.stackexchange.com, a site dedicated to software engineering and architecture rather than low-level programming issues. Indeed, Stack Exchange makes this “flair” available showing my very modest reputation and badges:


Although it was only six days, it was a significant time sink. Many hours were spent each day looking for questions to answer and formulating responses. Reputation (in the form of up votes) and answers often go to the first to answer so people learn quickly to provide a simple response then add more detail (and preferably sample code) later. Another strategy is seeking older, unanswered questions but the easy ones are probably already answered and such answers are unlikely to receive many up votes.

Answering questions on Stack Overflow is a lesson in humility. People often give better answers more quickly than you can type them, even in areas you consider yourself an expert, or radically different solutions. A diligently researched and prepared answer may be ignored or a negative comment may incite down votes. This can be frustrating but it can make Stack Overflow a great learning experience because the penalties for being incorrect are minor compared to the rewards of success. It is worth having a guess and let the more knowledgeable correct any mistakes.

Indeed, the reputation system provides surprisingly few opportunities for abuse. For example, someone could up vote a question they have answered (to put it back near the top of the list, looking for other’s up votes) and down vote others’ answers but up votes and reputation from up votes are capped per day. (Please note I didn’t try it. As I said above, the intention was to participate in the intended spirit.)

However, Stack Exchange is far from perfect. It does not guarantee high quality questions. Many are answered by a simple Internet search or are poorly expressed but they usually disappear quickly as the good questions are up voted. Questions on niche technologies may also languish unanswered and, when they are, lack a critical or interested population to up vote them. Like any Internet community, it is also cliquey with long established members.

Overall, I was impressed at the depth and breadth of the community and the site’s reward system. I doubt I will have the patience and dedication to reach the 400,000+ reputation some have earned but the temptation to refresh the page to see if there’s one more question you can answer is there. Answers marked correct by the question author are usually (but not always) thorough and correct and other answers or comments often contain good supporting information.

Many years ago, a potential employer could only verify one’s expertise by contacting old employers or limited technical questioning. Although not a substitute for these, Stack Exchange is one opportunity to demonstrate ability outside previous employment or education. The most telling aspect is the career’s page lets you link your best answers and highlights technologies you have participated in rather than list reputation and badges. My modest stack overflow reputation will not be appearing on my CV but maybe I’ll try to get to the next level, err, get more reputation on the stack overflow MMORPG.

Of Patents and Putters

The recent victory of Apple over Samsung has brought the arguments for and against software patents to the forefront once again. With many of the big technology companies engaged in patent litigation, many ask whether patent law is a distraction that stifles competition. Many technical luminaries, blogs and articles are openly anti-patent so should software developers be actively filing patents in organizations willing do so?

Patents can be beneficial for the developer. Some companies offer financial rewards for filing patents and researching whether something is truly novel, required for filing a patent, may reveal other solutions and techniques. However, patents’ biggest benefit is demonstrable innovation. A filed patent has been reviewed by the organization’s internal process and the patent office. This is rarely perfect but far more review than many other achievements.

That said, filing patents can distract developers from working on products and can be expensive for the organization. Similarly, if a developer files patents on a product then moves to a different organization working on similar products, the developer must carefully avoid infringing those patents at the new organization.

Some, such as the Electronic Frontier Foundation (https://www.eff.org/patent), argue that filing software patents endorses or exacerbates the negatives of patents. Many prefer the open source movement ideals where information and software should be available to be extended and used by many. Much of modern computing builds on concepts and algorithms that, if originally patented, may have stalled or prevented much of what IT takes for granted.

However, is this being disingenuous? Could these concepts and algorithms have developed despite patents? Consider the case of VisiCalc, the first spreadsheet. It was not patented, allowing Lotus 1-2-3, Microsoft Excel and other competitors to copy, extend and eventually surpass it. If it had been patented, would Lotus, Microsoft and others created an even better solution to the original problem than a spreadsheet? (as per Russ Krajec’s blog post at http://www.krajec.com/blog/what-if-visicalc-was-patented)

There are many more arguments for or against patents. However, even if all the anti-software patent arguments were proved and pro-software patent arguments disproved, for example, there is still no guarantee the software patent system would be dismantled immediately or at all. Legal and political systems move slowly at best.

Indeed, the technical or ideological pro- and anti-software patent arguments are all academic. Organizations file patents to protect their intellectual property (IP) in a world where patent litigation is increasingly common and IP is becoming organizations’ greatest asset.  It is a strategic business decision.

If an organization chooses to file patents, a software developer refusing to file patents makes about as much sense as a golfer refusing to use a putter. The golfer may lobby for a rule change banning putters but, in the meantime, the golfer is only inflating his or her handicap. This is not a “two wrongs make a right” scenario. If the organization is successful, it will inspire copy cats and litigation from its competition. Depending on the industry and product, filing patents may be pragmatic.

Software developers often see themselves only as potential victims of software patents, the expense and difficulty of filing them and the misplaced assertion that developers do not producing anything novel fuelling circular logic. If the organization is willing to bear the costs of patent management, why not patent it? There are undoubtedly smarter people out there that can solve the problem more effectively if needed. The organization can always choose not to assert the patent over open source or use it defensively, too.

Some argue that nothing in software development is new and most work is merely reinvention but saying that modern software developers are all less capable, intelligent and imaginative than their forebears is clearly false. This is not to say that developers produce new things all the time. Similarly, while many patents are obvious in retrospect, it is also much easier to judge when their ramifications are in everyday use.

What developers can do is ensure filed patents cover only products the organization intends to produce. Much of the consternation around patents from developers revolves around “non-practicing entities” (NPEs or “trolls” as they are derogatorily known), where the focus is on licensing existing patents and not using them to produce products. Research NPEs are the exception, despite Ars Technica’s Fox News-worthy rant (http://arstechnica.com/tech-policy/2012/04/how-the-aussie-government-invented-wifi-and-sued-its-way-to-430-million/).

Irrespective of whether an organization chooses to file patents, developers should keep easily accessible evidence of design documents and design discussions, such as E-mails and written notes. Even if an organization chooses not to file patents or the organization is not building anything novel, the organization may still be the victim of a patent lawsuit and demonstrable prior art can refute or diminish a patent’s novelty.

Software developers are very good at arguing things on technical merits. Unfortunately, the software patent issue has never been a technical or even ideological issue. These are merely rationalizations. This is as economic and political issue, and most software developers are not economists or politicians. Some protection for inventions is required and patents are the current, imperfect implementation of this. Until we come up a better system, software developers are only hurting themselves by not using all the rules of the game to their advantage.

In Defense of Documentation

Anyone who has spent a modicum of time working with software developers learns that developers hate creating designs, use cases, “readmes”, code samples and similar technical documentation. Agile proponents point to the “working software over comprehensive documentation” principle and Lean proponents claim documentation is “waste” since it does not directly contribute to the finished product or customer value.

They have a point. Writing clear, understandable documents is hard and it takes time away from coding. Keeping them up-to-date when inevitable changes occur is similarly time-consuming, particularly large changes or redesigns. Consequently, many stop asking for documentation, exacerbating poor or absent documentation. Many developers argue “code is documentation”. It is always up to date and developers had to work to understand it. Why shouldn’t everyone else?

However, saying “code is the only documentation” is a fallacy. Well written and designed code and indicative automated tests can be good documentation but every project includes code the developers are not proud of. Software developers often think of themselves coding like athletes lining up for a race at the Olympics. They have spent years training and are lining up to compete under perfect conditions. However, frequently code is written when developers are tired, stressed or unfamiliar with the problem or tools and this produces less correct and readable code.

Also, and arguably more important, developers are not the only audience for technical documentation. QA (testers), localization, management and documentation writers (those that write the manuals and non-technical documentation) all need technical information about the product and usually cannot read code, at least not well enough to get the information they need time efficiently. Developers new to the product also need a starting point.

Part of the problem is that documentation, particularly in larger organizations, consists of word processed documents or E-mail. E-mail in particular is a poor choice since it is only available to the recipients and often gets lost. An internal Wiki is a better choice, since most maintain a change history, have centralized storage making the latest version easier to find, support access control, provide auditing and can notify users of changes. Wikis are not perfect, however, since their storage formats are often proprietary and they often lack features of modern word pressing tools.

Wikis also encourage splitting documentation from large, monolithic documents into smaller sections. Care must be taken to ensure consistency and information is easy to find but most Wikis support free text searches. This fits better into the Agile and Lean methodologies, where smaller documents are delivered when needed rather than complete documentation delivered up front.

Writing documentation forces developers to identify and empathize with their customers or other developers, encouraging them to simplify the product’s design, user interface, installation and/or configuration. Some techniques, such as Documentation Drive Development and Readme Driven Development, encourage starting from the documentation for this reason.

It seems that literacy has become a skill in the software development industry. We have fooled ourselves into thinking that developers are only productive if they are writing code and poor documentation is acceptable. As Albert Einstein said “If you can’t explain it simply, you don’t understand it well enough.” Writing documentation forces developers out of their warm, cosy technical silos and back into the user world, reaquainting themselves with the problems the software is trying to solve.

%d bloggers like this: