Random Acts of Architecture

Tales of an architect trying to bring order to the chaos that is modern information technology.

Tag Archives: High-level design

Should Software Architects Write Code?

Much has been written and debated on whether software architects should write code. Many argue the more architects understand the language, tools and environment they are designing for, the more effective they are and this is best achieved by implementing some or all of the design. Non-coding architects, sometimes called “PowerPoint architects”, “astronaut architects” or “ivory tower architects”, may use archibabble and talkitecture to convince non-technical stakeholders of their expertise while delegating the unsolved, real problems to developers, so much so that it has become an organizational pattern (“Architect Also Implements“) and corresponding anti-pattern (“Architects Don’t Code“). Others argue that architects responsible for implementing their architectures lose focus on the bigger issues and longer term vision. Understanding does not necessarily require knowledge of the minutiae and, as systems scale up and diversify, implementing it requires too much time or spreads the architect too thin. Therefore, should software architects write code?

As with many difficult questions, the problem starts with the question itself. “Should a software architect write code?” can mean “Should a software architect always prototype or implement their own architectures?”, “Should a software architect write production code most of the time?” or “Should a software architect be able to write code?”. It could also mean “Is coding the best or only way to become a software architect?” or “Can non-coders be good architects?” but that is best left to another blog post.

It also depends on the definition of “software architect”. The Canadian architect (of buildings rather than IT) Witold Rybczynski wrote in his 1989 book “The Most Beautiful House in the World“:

“For centuries, the difference between master masons, journeymen builders, joiners, dilettantes, gifted amateurs, and architects has been ill defined. The great Renaissance buildings, for example, were designed by a variety of non-architects. Brunelleschi was trained as a goldsmith; Michelango as a sculptor, Leonardo da Vinci as a painter, and Alberti as a lawyer; only Bramante, who was also a painter, had formally studied building. These men are termed architects because, among other things, they created architecture — a tautology that explains nothing.”

This is exactly the same issue for software architects. Without a clearly defined and segregated role, anyone designing software or IT related systems can rightly be called an architect, including many developers and technical leads. For the sake of argument, this post uses Simon Brown’s definition, where software architects are responsible for high level design, non-functional requirements and technical vision.

Should a software architect be able to write code? Architects should be able to read and write code because it:

  1. Verifies the code written by developers matches the design and identifies deviations.
  2. Helps the architect learn about changes or new features. If the architect has been assigned to a new project, he or she can learn the product sooner by looking at the code, too.
  3. Allows the architect to write a proof of concepts or prototype. A working demo is much more convincing than an architecture diagram and will usually facilitate better estimates. Care must be taken to prevent non-technical stakeholders attaching too much credibility to it, as with any prototype however.
  4. Provides another pair of capable hands during the project crunch periods.
  5. Makes the architect more forgiving of bugs because the architect has likely made similar mistakes in the past. At least, the architect should have a better understanding what types of issues to expect.

Writing code may help earn the architect respect of the developers. Developers can be notoriously dismissive and a software architect producing some of his or her own code, even if it is just a proof of concept, or providing good feedback from a code review can make the developers feel like the architect is one of them. Having a working development environment and access to source code means the architect can try out new versions without waiting for a build or release. Any significant build, development environment or source code control issues also become apparent to the architect.

Note that code reviews do not replace talking to developers because regular discussions between developers and software architects can help build mutual respect. Otherwise, developers may see the architect as a constraint or threat that must be circumvented. Also, developers often know or can find problem areas faster than the architect reading the code but there needs to be a balance between architect self-sufficiency and squandering developer time.

Software architects are often required to settle disputes between developers, such as when one team discovers a better way of solving a problem or that the proposed design will be harder to implement than first thought. Software architects are also sometimes mentors or coaches for developers or may be used as internal consultants to examine process, quality, automation or similar issues. Understanding code means the architect can use his or her judgment more effectively rather than rely on which developer is more persuasive.

Should a software architect write production code most of the time (usually implementing their own architecture)? If a software architect implements their own architectures, this ensures the design is implementable with the tools and environment used. This can lead to new insights, improved designs and more accurate estimations. It is also implemented by the person most familiar with the design so minimizes miscommunication.

However, architects may jump to implementation (depth thinking) before exhausting other solutions (breadth thinking). Existing implementations may overly influence the architect or the architect become attached to his or her code, fighting against needed improvements. It can distract the architect from higher level tasks such as longer term planning, communicating with stakeholders and reviewing other developers’ code.

Also, part of the role of an architect is to fight for reuse, security and other non-functional requirements. Being forced to prototype or implement their design may encourage compromises that the team need not make. It is not that an architect does not make compromises – design is the art of compromise as many have said – more that it is the architect’s job to make the right compromises rather than those the architect makes creating the initial implementation. The developers will likely rewrite much of the architect’s code, anyway.

Indeed, the more an architect focuses on communication, requirements analysis, stakeholder management and non-technical activities, the more the development skills of software architects may atrophy. As long as the architect is providing value via other means this is not an issue. However, an architect should maintain his or her development skills; whether by extensive research, working on their own projects or contributing where they can; but needs to focus on capabilities, limitations and edge cases rather than speed or a complete understanding.

Problems may occur when organizations promote their strongest developers into a software architect role rather than good communicators that are capable of working at higher levels of abstraction. Friction arises when these architects try to “lead from the front” by implementing their architecture rather than facilitating others to do so. Organizations should promote a developer that has better soft skills, instead.

Many confuse not writing code with a lack of feedback. An unprototyped architecture, hypothetically, may be difficult to implement or problematic. However, a senior developer or technical lead can prototype the architecture if required. This also allows architects and developers to work together and ensure the design is communicated well. Alternatively, the architecture can be shared with others that have implemented similar systems previously or architects or developers working on integrating products. Requiring an architect to implement their own architecture beyond a proof of concept also does not scale well, particularly for large or complex products.

Similarly, many confuse an architect not writing code with a lack of accountability. Architects must produce designs that not only are approved (whether formally or informally) by stakeholders but also developers and developers should not approve a document that does not meet their needs. Issues or errors in the designs should be noted. Some change is expected but major or expensive errors should be attributed to the architect. An architect implementing their architecture in code does not guarantee an issue-free project.

With increased use of agile development methodologies, architects are no longer creating an architecture and “throwing it over the wall” to developers. Even previously ivory tower architects are more involved with lower level issues since less critical decisions are deferred until later in the process and design is iterative. For example, architects in organizations using Scrum should attend at least the planning, review and retrospective meetings. (Some architects may move to other projects or otherwise not see the project through, the “Architects Play Golf” pattern. This is an organizational issue and unrelated to whether architects code.)

Many developers also look down on “PowerPoint architectures”. However, many forget the role of a software architect is as much communication as development and completed, implemented architecture does not help non-technical stakeholders, QA, localization, documentation writers and so on. Of course, these stratospheric PowerPoint architectures are not substitutes for high-level designs developers can implement but the architect represents the developers and products to outsiders and developers often feel any time not spent developing is unproductive. Ultimately, PowerPoint architectures do have their place but developers are as much the architects’ customers as the stakeholders.

Should architects write code? The question is loaded and should be determined by the team on a case by case basis. Architects may prototype high risk projects, experiment with new libraries or try out now tools. Architects may completely delegate the design and implementation of well understood, low risk components. The real question is “How does an architect be successful?”. It is a question of managing and mitigating risk. Architects are often good coders but good coders are not necessarily good architects.

Update: There is a large discussion about this post on the IASA (International Association of Software Architects) LinkedIn group: http://www.linkedin.com/groups/Should-Software-Architects-Code-1523.S.188454845

An Architect’s Place in Agile

Scrum, the most common implementation of the Agile development methodology, has many well-defined roles. Those that contribute directly to the sprint (a unit of work usually lasting 2-4 weeks) are called “pigs”. Those that consult or assist only are “chickens”, the “scrum master” coordinates the sprint and the “product owner” prioritizes work and ensures the customer needs are met.

So where does the software architect fit in? The architect is not a pig if he or she does not write production code. Is he or she a chicken? The architect needs to be driving his or her features in the sprint and be more involved than a chicken. The architect is not responsible for team organization and a customer representative is usually the product owner.

Going back to basics, why is a software architect needed? Architects are rarely needed in projects with small, co-located teams full of senior developers working on well-defined requirements or well-understood problems. They can usually design and cooperate well enough to produce the desired results. However, large, distributed teams full of junior developers working on vague requirements or complex problems need coordination and direction. This is where architects are most useful.

One way of looking at it is Scrum is a software development methodology, not a productization methodology. Software development is one part of producing a product but there are many other parts, particularly for commercially sold software, such as business case design, marketing, licensing, documentation and localization. The architect could deliver non-functional requirements and high-level designs outside sprints like the other non-development tasks.

However, the architect need not deliver a monolithic document for the high-level design. In keeping with the Agile manifesto, as well as the Lean principle of making decisions as late as possible, the architect only needs to produce enough of the design to unblock the next sprint. The architect will still need a high-level design and identify non-functional requirements initially but Agile recognizes that design is as much a process as a product. Designs for subsequent sprints can be fleshed out in parallel with the development team, minimizing design rework as the team learns more about the problem and finds better solutions.

Could a software architect use Scrum to create the high-level design, either separate to or in parallel with the development teams? This can work if the architect has easy access to the resources he or she needs, such as customers to help understand the business problems, architects from other teams to discuss integration and development managers to check resource estimates. This cannot be guaranteed, particularly with larger, distributed groups – the cases where architects are most useful. However, it will occur in practice if the architect is providing designs for the start of each sprint.

Indeed, if the product owner is remote or often unavailable, an architect fits best into Scrum as a stand-in product owner. This breaks the Scrum rules of only having one product owner. However, different time zones, large projects and multiple commitments mean a single person cannot scale, as a former colleague of mine explained.

Development management may baulk at the perceived loss of control by making an architect a product owner. However, the word “owner” in “product owner” does not mean control of the product, merely creating, prioritizing and clarifying tasks, which architects often do anyway. Architects may not be customers but are judged whether the product meets the requirements or creates business value, just like product owners. They also know the product strategy and have spent time with the customer understanding the problem so are well-suited for this role, using their judgment to determine whether to escalate each question to the product owner.

Moreover, I think the question is not “Where does the architect fit into Agile?”, it’s “How can architects leverage Agile to better perform their role?”. For example, the architect can gain more visibility into the development team’s progress and status (through the backlog and burn down charts). The architect can present the design and gain consensus at the planning meeting that starts a sprint and (hopefully) see it working in the hand-over meeting at the end of a sprint.

Most importantly, architects must be in control of their performance rather than victims of process. A lot of smart people have worked very hard on Agile and Scrum and developers new to Scrum are advised to follow it as written, at least initially, because the reasons behind its nuances are often unclear. However, no development methodology can handle every case, and software architects are one of those things that can fall into the gaps.

Handling Challenges to Architectural Decisions

One of the most critical aspects of a software architect’s job is the initial high-level design. Based on customer requirements and the architect’s own non-functional requirements, this is where development starts from. It is also written when the development organization knows the least about the problem and so issues often arise as the development team works it.

Challenges are inevitable. It is important is not to panic and not to take it personally. The architect’s goal is to get the best product, not necessarily his or her product. Therefore, first get the other person to explain their issue and reasoning. Common scenarios include:

Different understanding of requirements:  All major analysis and design discussions around the product should involve the architects. (If not, then this indicates possible broader organizational issues.) Explain the decision’s rationale and ask if that matches the other person’s understanding. One or both of you will probably learn something in the ensuing discussion.

Proposes a better solution: Has the person raising the issue tried it before? Did they use a different tool or technique? Do some research to see whether others have had similar problems and their solutions. A small proof of concept is almost always helpful. Non-functional requirements, such as scalability or integration with other systems, are often the deciding factors.

Use an existing solution: This is often a difficult issue. If the person raising the issue was involved in the “better” solution, he or she may take this personally. Explain why it was not selected and work through the decision-making process. It is easy for developers (and architects) to apply their previous work beyond its design goals, even when it is no longer appropriate. However, existing solutions may save time and effort, help standardize products and are politically safer choices.

Resourcing or deadlines: The architect should understand the current and near future resource allocations, such as looking at the Scrum backlog, and do his or her own estimate of resource requirements because this is often an excuse given when the development team feels a task is not in their best interests. The important question here is prioritization rather than total work, so the architect should work with the customer to deprioritize tasks if needed.

Non-technical reasons: Sometimes decisions have to be made quickly, such as to meet changing market requirements or an unexpected opportunity, or people have conditions imposed upon them, such as head count reassignments or reductions. Those in senior positions do not have to justify their decisions but an explanation (usually discretely from a trusted source) may be invaluable. Politics is a fact of life and organizational awareness is important for architects, even if they choose not to “play the game”.

Before considering a solution, what is the impact of the change? Are other components affected? If so, do those teams have the bandwidth to make the changes? Also, does the change use a familiar, proven technology or is it speculative. Each organization has it’s own level of risk tolerance. Clearly, the larger or riskier a change, the more likely to go through a formal change control process. Assuming it is not clearly in that area and a change is warranted, solutions include:

Accept Change: You are only one person and there are always people smarter than you, people more familiar with the problem than you or just luckier. Assuming the impact is acceptable, make the change, communicate it and move on. In the long run, admitting mistakes will earn you more respect than defending the indefensible. If the change is small or low risk, even if it is a worse solution, it may be easier just to accept the change and move on because it gives others a more personal investment in the design.

Compromise: The solution is what neither of you are proposing or bits of both. Although almost cliche, finding a solution that keeps the important aspects of the original design and the person raising the issue happy is best. The architect must be open-minded even if others are not.

Delay: Sometimes the issue raised requires a change that is simply too large to accomplish with the remaining time and resources. In that case, add it to the requirements for the next version.

Escalate to Customer: If there is a disagreement about customer requirements, often the best solution is to take the question to the customer, such as the Scrum product owner. Remember to put this in terms the customer can understand, emphasizing the impact on users. The area may not have been covered in detail and so require more investigation.

Escalate to Management: Perhaps a team is demanding changes that would make their component easier but have a great impact to the rest of the product. Perhaps an external team is demanding changes without an adequate business purpose or return on investment. In this case, it is best to escalate this to the appropriate level of management if no compromise is possible.

Note that infrequent escalation to management or the customer is not a sign of failure. Indeed, involving them in the project can give them much-needed updates and let them know problem areas.

Lastly, do not forget to capture and communicate details of any changes. This may be a short announcement at the next team conference call, entering something into a change log and/or updating the original high level design documents.

No matter how thoroughly an architect researches and presents the high-level design, there will always be those who disagree. Sometimes, it is important to distribute the high-level design to air decisions and elicit feedback. Sometimes, deadlines dictate a incomplete design must be used. Sometimes, someone just wants to prove they know better than the architect. Remember that “architect” is a role, not a rank, and reacting well to challenges may speak more to an architect’s ability than the architecture itself.

Priming for Effective Feedback

Ignoring software architects’ code, the primary deliverables of architects are technical vision and high level designs. These are rarely complete and correct in the first draft and getting feedback from stakeholders and developers is vital to their success. Indeed, the best software architects are those that consistently extract the best feedback.

The key to effectively receiving feedback is for the software architect to change his or her attitude to feedback, particularly criticism. A software architect’s job is to own the problem, not a solution and providing the best solution, even one that contains elements from others, is the goal. When people give feedback, it means they are invested enough to care about the topic. By comparison, a lack of feedback or dismissive, detail-lacking response, such as “Looks good”, usually means the document has not been read or the reviewer is not invested in its success.

First, identify why feedback is being sought on the document, presentation or similar. Be more specific than just “standard procedure”. For example, is this a customer facing document that must not include company confidential material? Meeting minutes where important details and decisions need to be captured? A radical, new, partially-developed idea to create discussion and spark other ideas? This will dictate the audience and direct their attention.

Then identify what sections or details the reviewers should focus on, particularly for larger documents. Otherwise, some will focus on “big picture” issues, spelling and grammatical errors or otherwise get distracted. If uncertain, start with a high level introduction then focus on potentially controversial aspects such as assumptions, integration points and major decisions. If a document has multiple audiences, consolidate information for different audiences into separate sections for clarity and provide context if needed.

Given the above, many send out the document as an E-mail attachment or link, cynically expecting no response. There are two issues with this. First, typing responses can take time, be prone to misunderstanding and some people just do not communicate well in this medium. Second, without an articulated deadline, many will forget about it.

The solution is to still send document as E-mail attachment or link around but organize one or more meetings or conference calls, the latter preferably with document sharing software, to discuss it. Some will still respond with questions or comments before the meeting. However, the meeting gives people a deadline to review the document by and allows those that prefer verbal feedback to do so. Not everyone will find all issues with the document and a meeting allows feedback to be shared by all members of the group and not just the reviewer, potentially sparking additional ideas. For long or detailed documents, multiple short meetings are preferable to fewer longer ones. It avoids fatigue and fits in better with busy schedules.

Indeed, feedback is a conversation, not a download. The software architect, or whomever is providing the reviewed document, must provide a timely response and to a wider audience if warranted. If the architect is not detail-oriented, enlist the aid of someone who is to help keep track. Disagreements or contradictory feedback are learning opportunities and usually the result of missing information, assumptions or different experiences. In this case, either identity the facts that all parties agree on then build up from there or use an external expert or trusted authority.

Honest feedback requires safety. Architects are often senior developers with lots of experience and can be intimidating, particularly to junior developers. Remind others that you welcome feedback and emphasize other’s contributions to the document. Paraphrase feedback to confirm understanding. Humor, if culturally acceptable, can be a great tool if not overused, too.

Feedback requires mutual respect. An architect that disrespects a reviewer is likely to dismiss their feedback without adequate explanation or justification, impacting safety as mentioned above. If the experience or knowledge gap between reviewers is high, separate the reviewers into groups of similar capabilities then adapt the document or meetings for each audience. Often those with the best feedback are those that failed at something similar because they generally know why they failed better than those that succeeded know why they succeeded.

Similarly, reviewers that disrespect the architect may use it as a point scoring opportunity. An executive or customer may dictate rather than discuss. In these cases, paraphrase the points made to ensure understanding then agree to delve into more detail later in a dedicated meeting. Strong emotion may also be symptomatic of underlying issues probably beyond the scope of the document being reviewed.

What is Technical Vision?

Few things are so error prone and political as the architects’  “technical vision”. Technical vision identifies longer term technical goals (such as add iPad support or upgrade to the new database version) and describes how to reach them, perhaps over multiple releases. Clearly, this is only useful for larger, ongoing or technically complex projects.

Many architects aim for isolated technical improvements in their first technical vision. Unshackled from the constraints of time and resources, these are often ambitious, large-scale changes intended to make anticipated subsequent changes easier by adopting new technologies, creating reusable components and standardizing formats and protocols.

The first issue with this approach is that little is tied back to customer or business requirements. Otherwise, the architect risks an “ivory tower” situation, recommending or enforcing designs often blamed for inflated resource requirements and missed deadlines.

The best way to avoid this and gain credibility is to interview stakeholders, identify changes they want made and incorporate them into the vision. Non-business stakeholders like testers, localizers and documentation writers often want product improvements, too. Perceptive architects prioritize stakeholder requests that echo their own thoughts and, once the architect has demonstrated delivering these in products, architects will gain the credibility and latitude to add some of their own ideas.

The second issue is with component reuse and standardization. While the architect’s high level view is good for identifying opportunities for these and prioritizing them, links back to business requirements are difficult to articulate, particularly to non-technical stakeholders. Developers hold code written by others to a higher standard than their own (particularly documentation, quality, scalability and security), will use such components in ways never originally envisaged or intended and backwards compatibility requirements can limit improvements. However, the biggest danger with writing reusable components and standardization is the tendency to focus on them and not the problems they solve.

Writing reusable components and standardization are organizational decisions as much as architectural ones. Bugs or enhancement requests need tracking, prioritizing, fixing in a timely manner and distribution. Good approaches for reusable components include starting with small, non-critical components or creating internal “open source” projects where anyone can contribute, subject to review. Avoid forking components if possible.

Standardization requires a middle ground between forgoing features (“lowest common denominator” approach) and spending additional effort (the “kitchen ink” approach) and organizations must resist doing too much of the latter. It is best to start with often used interfaces and include security, scalability, performance, error behaviour and similar non-functional requirements.

Lastly, new technologies are often attractive, exciting and claim to solve problems more effectively, reducing development time and cost. However, adopting new technology requires an understanding of the whole system and not just isolated problems, lest it cause more problems than it solves. Once again, there is an organizational cost with retraining developers, purchasing new tools and integrating it with old technologies. New technologies are best trialled first with a proof of concept.

Good technical visions are known to and have the backing of both stakeholders and developers, which is easier if creating it involved them. Technical visions need not be detailed or specific, at least not initially and good technical visions are accessible, unambiguous and concise. If others cannot read them or understand them, it cannot be implemented or implementations will digress from the intention. As mentioned before in this blog, it is the architects’ responsibility to communicate the vision effectively.

Technical visions can change over time, too. This reflects changing business requirements, technology landscape and development techniques. It also reflects expressing the intention more effectively. Constructive criticism and discussion is vital, both to improve it and, as also mentioned before in this blog, incorporating others’ feedback gives them ownership and ensures it is current.

%d bloggers like this: