Random Acts of Architecture

Wisdom for the IT professional, focusing on chaos that is IT systems and architecture.

Tag Archives: Design

The Power of Alternatives

Leave a comment Posted by on July 12, 2021

For most of us, smartphones replaced paper maps long ago. You enter your destination, and it presents you with multiple potential routes. Given one route is often fastest, why bother?

The smartphone may not have all the information, such as weather or traffic. The phone does not know that an alternate route may be faster.

Perhaps the environment changes after navigation starts. Having a prepared alternate makes switching routes easier.

Speed may not be the crucial attribute. Maybe the driver wants the psychological safety of a familiar but slower route. Maybe one route is more enjoyable and scenic. Maybe the driver wants to shop on the way.

The fastest route may also have variability or risk. Maybe a football game at the local sports ground will finish soon. If time is tight, the time of a slower route may be more predictable and, therefore, better.

These reasons also apply to IT architectures at any level, whether technical/component, solution or enterprise.

The problem is most IT architects come from an implementor background, like software developers or network engineers. Good implementors build complex and deep mental models of their systems. These models allow implementors to both isolate issues and plan small to moderate changes quickly and effectively.

Organizations incentivize implementors to make changes quickly and with low risk. Having a good mental model facilitates that. The sooner an implementor can envisage and choose a solution, the sooner they can implement it and the sooner it can ship. This speed inspires confidence and provides technical leadership.

However, architects need to think differently to implementors. They still need higher-level mental models but should think more strategically (“Are we solving the correct problems? Is the solution complete?”). IT architects also need to think politically (“How do I convince stakeholders of the solution’s value and my value?”).

An architect should own all technical solutions in a business problem space. Providing a single solution implies the architect owns the solution, not the problem space. Subsequent design changes, even improvements, may diminish the architect’s credibility.

Focusing on a single solution alters how people justify them. The more assured people are that a solution is best, the less strongly they argue for it. They lose empathy with others to whom the solution’s merits are less clear. They often think beyond the design and evaluation stage and are frustrated when yanked back. Alternatives contrast and identify the solution’s pros and cons.

Providing multiple solutions helps generate discussion. Stakeholders may have differing preferences. Presenting different solutions, like playtesting a game’s rules or A/B testing a website, can draw out these preferences and derive the best solution. Stakeholders are customers.

Creating good alternatives and fighting an implementor’s instincts is difficult. The trade-offs are situation- and stakeholder-dependent.

That said, most organizations treat IT purely as a cost. Therefore, the biggest concern in any IT system is cost, including staff, time, and money. Create alternatives that minimize one or more of these by dropping, substituting or minimizing features.

Never sacrifice quality when creating alternatives. Stakeholders, particularly executives, are often not accountable for maintainability, security, availability, and the like. If they are not accountable, they do not care. For those that are accountable, quality is hard to quantify or demonstrate, so executives almost always delegate it.

Another alternative source is the “shortest path to value” (SPV). SPV identifies small projects within large ones with the biggest “bang for buck”, embodying the Pareto principle or 80:20 rule. SPV reduces otherwise massive projects that are hard to scope or have high schedule risk, making them more concrete and predictable.

Consider implementing the project using different technologies (tools or frameworks) or teams. Another team, even if only hypothetical, may take a different approach. Using or avoiding the technology de jour also opens possibilities. A greenfield project has advantages and disadvantages over non-greenfield projects.

Re-examine constraints or “bad ideas”. Even “hard” constraints are sometimes malleable. People often shun anything close to bad ideas. However, unrecognized good ideas often surround bad ideas.

Providing a single “best” solution undermines the architect’s credibility and removes agency from stakeholders. Like smartphone navigation, an IT architect often lacks full knowledge, may not grasp all requirements or environment, or solutions may provide unexpected opportunities for stakeholders. The biggest barrier is frequently identifying good alternates.

However, the biggest reason why alternatives are so powerful is IT architects need to differentiate themselves and sell their role. IT architects are not accountable like managers or responsible like implementors. Their value proposition is technical insights and good designs. Creating, evaluating and comparing the alternatives provides those insights and demonstrates the superiority of that design.

Image is from https://pixabay.com/illustrations/arrows-alternatives-many-direction-3438123/.

Architecture , , , , ,

Requirements and leadership, not design, are the keys to architecture

Leave a comment Posted by on May 8, 2018

Listen Understand Act

Many IT engineers aspire to be architects. They want to dictate the course of their products or services, leading their fellow engineers. To do so, they focus on designing the best and largest systems, learning all about design patterns, notations and understanding technology from top-to-bottom.

However, if such a thing can be said to exist, even the best design is wasted if it does not solve the right problem. Architects should start here, instead.

Depending on the organization, requirements are often supplied by product management, business analysts or management. During requirements analysis, architect validation identifies ambiguities, omissions, estimated time and resource costs and likely tradeoffs. The resulting requirements and priorities may differ substantially from the original as trade-offs and discoveries are made.

Requirements often present the business understanding of what technology should do, not the most impactful or beneficial things technology can do. Architects are in the best place bridge the gap, driving technology from the bottom-up instead of the top-down.

Business-supplied requirements often lack quality attributes or non-functional requirements like availability, performance and security. These are either assumed or difficult for non-technical people to articulate and architects are the best equipped to specify these.

Architects need to listen more than they talk, learning as much as they can about the business context of their work and its business value. Drilling into requirements is a good start, helping to understand requirements’ context, assumptions and priorities. There is no point where an architect understands everything, only a process to continually learn.

While it is tempting for a newly appointed architect to focus on their pet technical problems, ensuring they have a good pipeline of requirements helps architects to align their efforts to solve others’ problems, not just the ones they perceive. They also need to ensure the business outcomes are met, not just the technical enhancemnts.

Looking at it another way, a design is not just a model (approximation) of the implementation. A design is the requirements for the implementation. Like requirements gathering, design is iterative and may change through the review or implementation process. Like requirements gathering, design is a trade-off. Like requirements gathering, it is an abstraction, leaving some details to implementers. If an architect cannot understand or provide good requirements, their designs are going to be misunderstood, at best, or ignored, at worst.

Moreover, architects are leaders. Not leaders in the management sense but leaders by collaboration, communication and example.

While the technical leadership of architects is well understood, good architects move out of their comfortable technical conversations and into the less comfortable business conversations. As mentioned above, some requirements sit between the technical and business and stakeholders need assurance the system will meet their needs. No design pattern or notation will achieve this.

Architects should focus on outcomes and end-to-end systems, not the minutiae of their designs, particularly in agile environments where just-in-time design occurs or where component responsibility is delegated to teams. Trusting implementors by giving them clear interfaces, scope and direction is the best way to foster their trust in architects.

Architects must own their communication. The responsibility for implementors and stakeholders understanding the design and vision rests with the architects. A design or vision that is not communicated is not understood and an architect producing designs no one understands has zero business value.

An architect must also facilitate communication between teams, particularly when design changes ripple through other teams’ work.

Architects must be accountable for systems they architect. They need to listen to implementors to understand their challenges and how to mitigate them in current or future designs. They need to accept criticism from stakeholders when requirements are not met. They also need to be applauded when their projects or systems succeed.

While designs are the architect’s deliverables in many projects, an architect’s success is driven by their ability to ensure they are solving the right problems and assure people of that direction. Good architects look down toward the technical detail and ensure it is correct. Great architects also look up and around to understand how they can best provide value to the business, sometimes better than the business can.

Image from https://www.flickr.com/photos/highersights/6231641551. Used under creative commons license.

Architecture, Software Architecture , , ,

Architect/Stakeholder Inversion

7 Comments Posted by on August 28, 2013

Stakeholder Architect InversionArchitect/stakeholder inversion occurs when non-technical stakeholders tell software architects how a system should work, not what it should do. Without the “what”, software architects are left trying to guess or reverse engineer it. The resulting system may not solve the customer problem or may bloat with features attempting to do so.

Architect/stakeholder inversion is not a stakeholder wanting to move a system into the cloud to reduce costs. It is not wanting a mobile app to reach a different, younger market or offer a better user experience. It is not marketing pushing for a better analytics tool. They have business justifications.

Architect/stakeholder inversion is wanting two products integrated without saying what data to share or tasks to provide. It is creating a report engine without knowing the reports it will run. It is any framework created solely to handle nebulous requirements.

Architect/stakeholder inversion occurs due to one of three reasons. First, non-technical stakeholders feel they need to give low-level, technical requirements. Usually a sign of inexperience or frustration, the stakeholder bypasses discussion with technical details.

Alternatively, software developers may be used to implementing what they are told. This is common in environments with many ancillary roles (user experience, visual design, business analysis, copy writing, solution architecture, application architecture, agile coach, project manager, scrum master, team leader, etc) and stakeholders may take advantage of this.

Second, stakeholders often make technical assumptions and present those assumptions as solutions. They may not even realize they made assumptions.

Technical people may miss the business impacts of technical choices. However, non-technical stakeholders may miss technical impacts of business choices, too. For example, while the ongoing costs of moving to an “Infrastructure as a Service” (IaaS) or “Platform as a Service” (PaaS) provider may be lower, non-technical stakeholders may not consider the transition cost and impacts on compliance, security, jurisdiction, privacy, bandwidth and latency. The stakeholder might not have considered other benefits, such as elasticity (rapid scale up or scale down), built-in monitoring and management tools and cheap creation of test and staging environments, either.

Stakeholders with technical backgrounds may exacerbate the problem. While the technical solution requested may be good, the business context is still needed. Software architects are part of the checks and balances for the business requirements and stakeholder technical knowledge does not negate this.

Third, stakeholders may not yet know the business goals of the system. This may be driven by schedule (“We need to start coding now so that we will hit the deadline”), a misunderstanding of agile processes (“We will work it out as we go”) or a lack of preparation.

Architect/stakeholder inversion is usually solved by highlighting assumptions or providing alternate solutions. Forming these into questions (“Have we considered doing X instead of Y?”) and prototypes/spikes are effective. However, if software architects are on a “need to know” basis, stakeholders set direction solely by intuition instead of evidence or stakeholders take offence at challenges or questions, there may be wider organizational problems.

Architects and stakeholders should cooperate and respectfully challenge each other, providing greater understanding to both sides. Software architects can make better informed design decisions and glean insight into wider and future direction. The stakeholder can get a better understanding of and confidence in the solution.

That said, there are no sides here. Both the stakeholders and architects are working toward the same goal. If the organization has appointed stakeholders and architects, it realizes the value of each. Architect/stakeholder inversion contradicts this and produces a lower quality product.

Update: This post is featured in a discussion in the International Association of Software Architects (IASA) group on Linkedin.

Software Architecture , , , , , , ,

Big Design Up Front versus Emergent Design

4 Comments Posted by on July 8, 2013

BDUF vs Emergent Design(This post is in response to Hayim Makabee’s posts on emergent design and adaptable design along with some of the follow-up discussions, such as the thread in the “97 Things Every Software Architect Should Know” Linkedin Group and Gene Hughson’s post on emergence vs evolution.)

One argument software architects regularly encounter is that time spent designing systems is wasted. Many say that “big design up front” is not the agile way and “emergent design” is more effective. This cuts straight to the value proposition of an architect. If up front design has no place in the Agile world, are architects redundant?

To most people, “big design up front” (BDUF), sometimes called “big up front design” (BUFD), means a lengthy, detailed design created at the start of a project. It works on three assumptions. First, one can create requirements for a project. Second, one can create a design to meet those requirements. Third, the design’s suitability for meeting the requirements can be evaluated without implementing it. In other words, there can be good designs and bad designs.

Meanwhile, emergent design means minimal or no design up front (NDUF). It works on the assumption both the requirements and design must be deduced so the team starts developing the product and iterates as they learn more about the problem and the solution. The process finishes at a predetermined time or when “good enough” requirements and design “emerge”.

By inference, emergent design assumes designs are often highly problem/solution specific. Adapting existing designs may create more work than they save. It also moves the focus from providing value to following the design.

Emergent design is quite popular among Agile and Lean practitioners. They argue emergent design reduces some waste (unnecessary work) by not creating lengthy documents that people may never read. Of the read design documents, few are updated as changes are made. Many developers are so cynical they refuse to read documentation and jump straight to the code to answer questions.

Big design up front may encourage over design. Unnecessary features may be added (violating the YAGNI principle) or the system may be unnecessarily complex (violating the KISS principal). Emergent design, particularly when coupled with Test Driven Design (TDD), can produce the minimum code required to meet a requirement and no more.

Big design up front may create an illusion the team knows more than they do. This may prompt decisions when the team knows the least about the problem, meaning big design up front can become big commitment up front. Meanwhile, a team that delays making decisions until necessary may discover different features are needed.

Big design up front’s assumptions are also not always true. Every project has a goal but it may not be clear how to get there. Most startups do not have quantifiable requirements, for example, where coding is more experimenting than implementing. New technologies may supersede old techniques or require new ones, meaning designs are either too difficult to create or cannot be evaluated without implementation.

However, proponents of big design up front point out that designing is often more useful than design documents. The design exercise validates and challenges requirements, explores edge cases and discovers mistakes. Without it, developers often dive straight into low level details and even a short time thinking about the problem can expose assumptions or alternate solutions they would otherwise miss.

Emergent design assumes change is cheap. A lot of effort and attention has been directed to this. Continuous integration and continuous delivery aim to make releasing easy. Test Driven Design (TDD) and automated testing aim to find regressions quickly. Agile methodologies like Scrum provide visibility and guidance on how to manage change.

However, not every change is cheap. Hardware can be difficult or impossible to change once manufactured. Network infrastructure changes need to be scheduled to minimize impact to others. Engaging external vendors may require lengthy contract negotiations. Legacy code may lack sufficient automated unit test coverage. Aspects like security, compliance and scalability are difficult to retrofit.

Similarly, software development must be accountable to the organization. Required skills and teams must be hired or contracted. Budgets must be determined. Progress is usually tracked against milestones and must be approved based on return on investment (ROI) estimations. Risks must be identified and mitigated. Early designs (as part of architectures) can help drive all of this.

Designs present abstracted views of the system, emphasizing important decisions and removing noise. This means designs can also be reviewed by others before the more expensive and time consuming implementation to find weaknesses or suggest improvements. Designs of notable projects can teach others, either by following or avoiding them.

The problem with comparing big design up front against emergent design is it usually devolves into straw man arguments. Neither are absolutes. Good big design up front recognizes some design and details are filled in during development. Good emergent design must start with some idea of how the system will work.

Both big design up front and emergent design can be done badly. Poor big design up front can miss important factors, provide a poor solution or communicate good ideas badly. Poor emergent design can waste time rewriting code, introduce regressions and impede governance. Both can create a big ball of mud. However, big design up front need not be change averse. Emergent design need not be chaotic and unpredictable.

Big design up front and emergent design are process agnostic. Big design up front originated in waterfall processes. As mentioned above, emergent design is common with agile development methodologies. However, emergent design can be used within a waterfall design phase (prototyping) or for defined components during development (spikes). A team using agile development methodologies may do some design inside, outside or between iterations.

Both approaches can be combined. For example, adaptable design is a technique where parts of the system that anticipate change, such as unknown or changing requirements, are designed to accommodate them.

Looking at the comparison from a different angle, what does “design” mean? Is it thinking about how to approach the system or is it documenting and communicating it? A small system may be something a developer can completely understand and describe in a few sentences. It has an implicit, undocumented design and can be iterated over time using emergent design. However, a large or complex system using a mix of legacy and new components whose development is split across different teams may need a different approach. In other words, the benefit of up front design increases as the system complexity increases.

Different approaches require different skills. Big design up front requires thinking about a system in abstract terms. It is skill that not every developer has, requiring breadth rather than depth, and is often why democratizing design fails. By contrast, emergent design embraces a detail and code focus, particularly with the focus on unit testing and small, incremental changes. This is one reason emergent design is more attractive to software developers.

Both big design up front and emergent design are tools a software development team can use. Rather than being excluded, software architects are in a unique position. They can help determine which approach is best for a situation. The real challenge for a software architect is knowing the right amount of design for a system and when to do it.

Software Architecture , , , , , , , , , , , , , , , ,

An Architect’s Place in Agile

3 Comments Posted by on August 19, 2012

Scrum, the most common implementation of the Agile development methodology, has many well-defined roles. Those that contribute directly to the sprint (a unit of work usually lasting 2-4 weeks) are called “pigs”. Those that consult or assist only are “chickens”, the “scrum master” coordinates the sprint and the “product owner” prioritizes work and ensures the customer needs are met.

So where does the software architect fit in? The architect is not a pig if he or she does not write production code. Is he or she a chicken? The architect needs to be driving his or her features in the sprint and be more involved than a chicken. The architect is not responsible for team organization and a customer representative is usually the product owner.

Going back to basics, why is a software architect needed? Architects are rarely needed in projects with small, co-located teams full of senior developers working on well-defined requirements or well-understood problems. They can usually design and cooperate well enough to produce the desired results. However, large, distributed teams full of junior developers working on vague requirements or complex problems need coordination and direction. This is where architects are most useful.

One way of looking at it is Scrum is a software development methodology, not a productization methodology. Software development is one part of producing a product but there are many other parts, particularly for commercially sold software, such as business case design, marketing, licensing, documentation and localization. The architect could deliver non-functional requirements and high-level designs outside sprints like the other non-development tasks.

However, the architect need not deliver a monolithic document for the high-level design. In keeping with the Agile manifesto, as well as the Lean principle of making decisions as late as possible, the architect only needs to produce enough of the design to unblock the next sprint. The architect will still need a high-level design and identify non-functional requirements initially but Agile recognizes that design is as much a process as a product. Designs for subsequent sprints can be fleshed out in parallel with the development team, minimizing design rework as the team learns more about the problem and finds better solutions.

Could a software architect use Scrum to create the high-level design, either separate to or in parallel with the development teams? This can work if the architect has easy access to the resources he or she needs, such as customers to help understand the business problems, architects from other teams to discuss integration and development managers to check resource estimates. This cannot be guaranteed, particularly with larger, distributed groups – the cases where architects are most useful. However, it will occur in practice if the architect is providing designs for the start of each sprint.

Indeed, if the product owner is remote or often unavailable, an architect fits best into Scrum as a stand-in product owner. This breaks the Scrum rules of only having one product owner. However, different time zones, large projects and multiple commitments mean a single person cannot scale, as a former colleague of mine explained.

Development management may baulk at the perceived loss of control by making an architect a product owner. However, the word “owner” in “product owner” does not mean control of the product, merely creating, prioritizing and clarifying tasks, which architects often do anyway. Architects may not be customers but are judged whether the product meets the requirements or creates business value, just like product owners. They also know the product strategy and have spent time with the customer understanding the problem so are well-suited for this role, using their judgment to determine whether to escalate each question to the product owner.

Moreover, I think the question is not “Where does the architect fit into Agile?”, it’s “How can architects leverage Agile to better perform their role?”. For example, the architect can gain more visibility into the development team’s progress and status (through the backlog and burn down charts). The architect can present the design and gain consensus at the planning meeting that starts a sprint and (hopefully) see it working in the hand-over meeting at the end of a sprint.

Most importantly, architects must be in control of their performance rather than victims of process. A lot of smart people have worked very hard on Agile and Scrum and developers new to Scrum are advised to follow it as written, at least initially, because the reasons behind its nuances are often unclear. However, no development methodology can handle every case, and software architects are one of those things that can fall into the gaps.

Software Architecture , , , , , , , , , , , ,