Random Acts of Architecture

Tales of an architect trying to bring order to the chaos that is modern information technology.

In Defense of Documentation

Anyone who has spent a modicum of time working with software developers learns that developers hate creating designs, use cases, “readmes”, code samples and similar technical documentation. Agile proponents point to the “working software over comprehensive documentation” principle and Lean proponents claim documentation is “waste” since it does not directly contribute to the finished product or customer value.

They have a point. Writing clear, understandable documents is hard and it takes time away from coding. Keeping them up-to-date when inevitable changes occur is similarly time-consuming, particularly large changes or redesigns. Consequently, many stop asking for documentation, exacerbating poor or absent documentation. Many developers argue “code is documentation”. It is always up to date and developers had to work to understand it. Why shouldn’t everyone else?

However, saying “code is the only documentation” is a fallacy. Well written and designed code and indicative automated tests can be good documentation but every project includes code the developers are not proud of. Software developers often think of themselves coding like athletes lining up for a race at the Olympics. They have spent years training and are lining up to compete under perfect conditions. However, frequently code is written when developers are tired, stressed or unfamiliar with the problem or tools and this produces less correct and readable code.

Also, and arguably more important, developers are not the only audience for technical documentation. QA (testers), localization, management and documentation writers (those that write the manuals and non-technical documentation) all need technical information about the product and usually cannot read code, at least not well enough to get the information they need time efficiently. Developers new to the product also need a starting point.

Part of the problem is that documentation, particularly in larger organizations, consists of word processed documents or E-mail. E-mail in particular is a poor choice since it is only available to the recipients and often gets lost. An internal Wiki is a better choice, since most maintain a change history, have centralized storage making the latest version easier to find, support access control, provide auditing and can notify users of changes. Wikis are not perfect, however, since their storage formats are often proprietary and they often lack features of modern word pressing tools.

Wikis also encourage splitting documentation from large, monolithic documents into smaller sections. Care must be taken to ensure consistency and information is easy to find but most Wikis support free text searches. This fits better into the Agile and Lean methodologies, where smaller documents are delivered when needed rather than complete documentation delivered up front.

Writing documentation forces developers to identify and empathize with their customers or other developers, encouraging them to simplify the product’s design, user interface, installation and/or configuration. Some techniques, such as Documentation Drive Development and Readme Driven Development, encourage starting from the documentation for this reason.

It seems that literacy has become a skill in the software development industry. We have fooled ourselves into thinking that developers are only productive if they are writing code and poor documentation is acceptable. As Albert Einstein said “If you can’t explain it simply, you don’t understand it well enough.” Writing documentation forces developers out of their warm, cosy technical silos and back into the user world, reaquainting themselves with the problems the software is trying to solve.

An Architect’s Place in Agile

Scrum, the most common implementation of the Agile development methodology, has many well-defined roles. Those that contribute directly to the sprint (a unit of work usually lasting 2-4 weeks) are called “pigs”. Those that consult or assist only are “chickens”, the “scrum master” coordinates the sprint and the “product owner” prioritizes work and ensures the customer needs are met.

So where does the software architect fit in? The architect is not a pig if he or she does not write production code. Is he or she a chicken? The architect needs to be driving his or her features in the sprint and be more involved than a chicken. The architect is not responsible for team organization and a customer representative is usually the product owner.

Going back to basics, why is a software architect needed? Architects are rarely needed in projects with small, co-located teams full of senior developers working on well-defined requirements or well-understood problems. They can usually design and cooperate well enough to produce the desired results. However, large, distributed teams full of junior developers working on vague requirements or complex problems need coordination and direction. This is where architects are most useful.

One way of looking at it is Scrum is a software development methodology, not a productization methodology. Software development is one part of producing a product but there are many other parts, particularly for commercially sold software, such as business case design, marketing, licensing, documentation and localization. The architect could deliver non-functional requirements and high-level designs outside sprints like the other non-development tasks.

However, the architect need not deliver a monolithic document for the high-level design. In keeping with the Agile manifesto, as well as the Lean principle of making decisions as late as possible, the architect only needs to produce enough of the design to unblock the next sprint. The architect will still need a high-level design and identify non-functional requirements initially but Agile recognizes that design is as much a process as a product. Designs for subsequent sprints can be fleshed out in parallel with the development team, minimizing design rework as the team learns more about the problem and finds better solutions.

Could a software architect use Scrum to create the high-level design, either separate to or in parallel with the development teams? This can work if the architect has easy access to the resources he or she needs, such as customers to help understand the business problems, architects from other teams to discuss integration and development managers to check resource estimates. This cannot be guaranteed, particularly with larger, distributed groups – the cases where architects are most useful. However, it will occur in practice if the architect is providing designs for the start of each sprint.

Indeed, if the product owner is remote or often unavailable, an architect fits best into Scrum as a stand-in product owner. This breaks the Scrum rules of only having one product owner. However, different time zones, large projects and multiple commitments mean a single person cannot scale, as a former colleague of mine explained.

Development management may baulk at the perceived loss of control by making an architect a product owner. However, the word “owner” in “product owner” does not mean control of the product, merely creating, prioritizing and clarifying tasks, which architects often do anyway. Architects may not be customers but are judged whether the product meets the requirements or creates business value, just like product owners. They also know the product strategy and have spent time with the customer understanding the problem so are well-suited for this role, using their judgment to determine whether to escalate each question to the product owner.

Moreover, I think the question is not “Where does the architect fit into Agile?”, it’s “How can architects leverage Agile to better perform their role?”. For example, the architect can gain more visibility into the development team’s progress and status (through the backlog and burn down charts). The architect can present the design and gain consensus at the planning meeting that starts a sprint and (hopefully) see it working in the hand-over meeting at the end of a sprint.

Most importantly, architects must be in control of their performance rather than victims of process. A lot of smart people have worked very hard on Agile and Scrum and developers new to Scrum are advised to follow it as written, at least initially, because the reasons behind its nuances are often unclear. However, no development methodology can handle every case, and software architects are one of those things that can fall into the gaps.

Handling Challenges to Architectural Decisions

One of the most critical aspects of a software architect’s job is the initial high-level design. Based on customer requirements and the architect’s own non-functional requirements, this is where development starts from. It is also written when the development organization knows the least about the problem and so issues often arise as the development team works it.

Challenges are inevitable. It is important is not to panic and not to take it personally. The architect’s goal is to get the best product, not necessarily his or her product. Therefore, first get the other person to explain their issue and reasoning. Common scenarios include:

Different understanding of requirements:  All major analysis and design discussions around the product should involve the architects. (If not, then this indicates possible broader organizational issues.) Explain the decision’s rationale and ask if that matches the other person’s understanding. One or both of you will probably learn something in the ensuing discussion.

Proposes a better solution: Has the person raising the issue tried it before? Did they use a different tool or technique? Do some research to see whether others have had similar problems and their solutions. A small proof of concept is almost always helpful. Non-functional requirements, such as scalability or integration with other systems, are often the deciding factors.

Use an existing solution: This is often a difficult issue. If the person raising the issue was involved in the “better” solution, he or she may take this personally. Explain why it was not selected and work through the decision-making process. It is easy for developers (and architects) to apply their previous work beyond its design goals, even when it is no longer appropriate. However, existing solutions may save time and effort, help standardize products and are politically safer choices.

Resourcing or deadlines: The architect should understand the current and near future resource allocations, such as looking at the Scrum backlog, and do his or her own estimate of resource requirements because this is often an excuse given when the development team feels a task is not in their best interests. The important question here is prioritization rather than total work, so the architect should work with the customer to deprioritize tasks if needed.

Non-technical reasons: Sometimes decisions have to be made quickly, such as to meet changing market requirements or an unexpected opportunity, or people have conditions imposed upon them, such as head count reassignments or reductions. Those in senior positions do not have to justify their decisions but an explanation (usually discretely from a trusted source) may be invaluable. Politics is a fact of life and organizational awareness is important for architects, even if they choose not to “play the game”.

Before considering a solution, what is the impact of the change? Are other components affected? If so, do those teams have the bandwidth to make the changes? Also, does the change use a familiar, proven technology or is it speculative. Each organization has it’s own level of risk tolerance. Clearly, the larger or riskier a change, the more likely to go through a formal change control process. Assuming it is not clearly in that area and a change is warranted, solutions include:

Accept Change: You are only one person and there are always people smarter than you, people more familiar with the problem than you or just luckier. Assuming the impact is acceptable, make the change, communicate it and move on. In the long run, admitting mistakes will earn you more respect than defending the indefensible. If the change is small or low risk, even if it is a worse solution, it may be easier just to accept the change and move on because it gives others a more personal investment in the design.

Compromise: The solution is what neither of you are proposing or bits of both. Although almost cliche, finding a solution that keeps the important aspects of the original design and the person raising the issue happy is best. The architect must be open-minded even if others are not.

Delay: Sometimes the issue raised requires a change that is simply too large to accomplish with the remaining time and resources. In that case, add it to the requirements for the next version.

Escalate to Customer: If there is a disagreement about customer requirements, often the best solution is to take the question to the customer, such as the Scrum product owner. Remember to put this in terms the customer can understand, emphasizing the impact on users. The area may not have been covered in detail and so require more investigation.

Escalate to Management: Perhaps a team is demanding changes that would make their component easier but have a great impact to the rest of the product. Perhaps an external team is demanding changes without an adequate business purpose or return on investment. In this case, it is best to escalate this to the appropriate level of management if no compromise is possible.

Note that infrequent escalation to management or the customer is not a sign of failure. Indeed, involving them in the project can give them much-needed updates and let them know problem areas.

Lastly, do not forget to capture and communicate details of any changes. This may be a short announcement at the next team conference call, entering something into a change log and/or updating the original high level design documents.

No matter how thoroughly an architect researches and presents the high-level design, there will always be those who disagree. Sometimes, it is important to distribute the high-level design to air decisions and elicit feedback. Sometimes, deadlines dictate a incomplete design must be used. Sometimes, someone just wants to prove they know better than the architect. Remember that “architect” is a role, not a rank, and reacting well to challenges may speak more to an architect’s ability than the architecture itself.

Priming for Effective Feedback

Ignoring software architects’ code, the primary deliverables of architects are technical vision and high level designs. These are rarely complete and correct in the first draft and getting feedback from stakeholders and developers is vital to their success. Indeed, the best software architects are those that consistently extract the best feedback.

The key to effectively receiving feedback is for the software architect to change his or her attitude to feedback, particularly criticism. A software architect’s job is to own the problem, not a solution and providing the best solution, even one that contains elements from others, is the goal. When people give feedback, it means they are invested enough to care about the topic. By comparison, a lack of feedback or dismissive, detail-lacking response, such as “Looks good”, usually means the document has not been read or the reviewer is not invested in its success.

First, identify why feedback is being sought on the document, presentation or similar. Be more specific than just “standard procedure”. For example, is this a customer facing document that must not include company confidential material? Meeting minutes where important details and decisions need to be captured? A radical, new, partially-developed idea to create discussion and spark other ideas? This will dictate the audience and direct their attention.

Then identify what sections or details the reviewers should focus on, particularly for larger documents. Otherwise, some will focus on “big picture” issues, spelling and grammatical errors or otherwise get distracted. If uncertain, start with a high level introduction then focus on potentially controversial aspects such as assumptions, integration points and major decisions. If a document has multiple audiences, consolidate information for different audiences into separate sections for clarity and provide context if needed.

Given the above, many send out the document as an E-mail attachment or link, cynically expecting no response. There are two issues with this. First, typing responses can take time, be prone to misunderstanding and some people just do not communicate well in this medium. Second, without an articulated deadline, many will forget about it.

The solution is to still send document as E-mail attachment or link around but organize one or more meetings or conference calls, the latter preferably with document sharing software, to discuss it. Some will still respond with questions or comments before the meeting. However, the meeting gives people a deadline to review the document by and allows those that prefer verbal feedback to do so. Not everyone will find all issues with the document and a meeting allows feedback to be shared by all members of the group and not just the reviewer, potentially sparking additional ideas. For long or detailed documents, multiple short meetings are preferable to fewer longer ones. It avoids fatigue and fits in better with busy schedules.

Indeed, feedback is a conversation, not a download. The software architect, or whomever is providing the reviewed document, must provide a timely response and to a wider audience if warranted. If the architect is not detail-oriented, enlist the aid of someone who is to help keep track. Disagreements or contradictory feedback are learning opportunities and usually the result of missing information, assumptions or different experiences. In this case, either identity the facts that all parties agree on then build up from there or use an external expert or trusted authority.

Honest feedback requires safety. Architects are often senior developers with lots of experience and can be intimidating, particularly to junior developers. Remind others that you welcome feedback and emphasize other’s contributions to the document. Paraphrase feedback to confirm understanding. Humor, if culturally acceptable, can be a great tool if not overused, too.

Feedback requires mutual respect. An architect that disrespects a reviewer is likely to dismiss their feedback without adequate explanation or justification, impacting safety as mentioned above. If the experience or knowledge gap between reviewers is high, separate the reviewers into groups of similar capabilities then adapt the document or meetings for each audience. Often those with the best feedback are those that failed at something similar because they generally know why they failed better than those that succeeded know why they succeeded.

Similarly, reviewers that disrespect the architect may use it as a point scoring opportunity. An executive or customer may dictate rather than discuss. In these cases, paraphrase the points made to ensure understanding then agree to delve into more detail later in a dedicated meeting. Strong emotion may also be symptomatic of underlying issues probably beyond the scope of the document being reviewed.

The Other Side of Software Architect Innovation

As stated previously in this blog, software architects must be innovators. They are responsible for creating a technical vision to apply new technologies to further the customers’ goals and/or reduce development costs and for high level designs. Many books, lectures, blogs and so on describe the process of innovation and are not repeated here but, when confronted with a problem, most developers and architects either work out their own solution or lazily Google something similar. However, software developers and architects, in particular, need not “reinvent the wheel”.
 
Ignoring structured learning, such as degrees and certificates, there are two ways architects can learn ways others do things. The first is to proactively listen for new information.Most software architects will already read favourite blogs or web sites, follow twitter luminaries, subscribe to magazines, read textbooks, attend conventions and so on. However, the key with information sources is efficiency – five sources providing predominantly relevant information is better than one hundred poor sources.
 
Aim for a mix of information sources, including some technical, some design, some industry commentary and some aggregators (those that provide interesting links to other sources that may only occasionally have relevant information). Review sources over time because the architect’s requirements, knowledge and expectations will change over time. Experiment with new things and plan to either remove “old favorites” or check them less often when they are no longer being consistently useful.
 
The second is to research topics when the need arises. Competitor’s products are good places to start but, unless this is a major feature, information is often sparse. Internet searches or dedicated development sites are useful but iterating synonyms is time-consuming.
 
Software architects often neglect academic texts or papers, which are particularly useful if the group is using more complex algorithms. Use Google Scholar or consider joining ACM or IEEE to access their academic paper repositories. Such papers may be dry or assume too much knowledge initially but persisting may reveal better approaches, establish common terminology and point out less explored areas suitable for potential patents or the architect’s own academic papers if he or she is so inclined.
 
Some would recommend looking at open source software. Assuming the product is chosen with care, open source software or other people’s code in general is a great source of information from obscure language features to new algorithms. However, the software architect must not copy any code verbatim or infringe patents or licenses.
 
If the problem is not solved or not solved sufficiently by prior art and the organization has a process in place, software architects should create and file patents. Arguments over whether software patents encourage or discourage innovation are irrelevant. As they say, “You cannot win the game from the sideline”. The IT industry is pursuing patents aggressively and companies that do not, such as Google with Android, now face legal challenges and hurdles. For the software architects, patents are one of the few peer-reviewed, universally available measures of creativity available. Many organizations also offer financial incentives for them, too.
 
Beyond the software architect’s products and patents, avenues for innovation include open source software, apps in app stores, writing a technical blog, conferences or academic papers. Approval may be required from the organization and social networking activity will need to comply with the organization’s policy but this can help the software architect’s personal brand and help find or precipitate conversations with like-minded individuals.
 
The single most important thing about innovation is to start. A software architect must be a technical leader but cannot lead without producing and cannot produce without starting. The first attempts may be unremarkable or unsuccessful but persistence will bear fruit.

Creating an Innovative Environment

“Innovation” appears to have joined words like “synergy” in the buzzword nomenclature but is central to a software architect’s role, whether it be creating and driving either customer impacting or cost saving innovations. Unfortunately, many software architects and developers get frustrated when new ideas, features and improvements continuously are ignored, pushed to a nebulous future release or get anemic implementations. Innovation needs an environment and culture that allows it to succeed.
 
Many organizations feel that the greatest barrier to innovation is time. After all, many software development organizations are known for time specifically devoted to innovation. Google, for example, has “Google Time” where all employees spend a day a week working on their own projects. Atlassian, an oft quoted example, has “FedEx” days  where all the development staff spend 24 hours on their own projects and present them to the group afterwards. In both cases, this is where many new products or product features originate.
 
Unfortunately, organizations focusing solely on the time allocate it, expect developers to produce wonderful, insightful and customer worthy new products and features then scrap the exercise after initial disappointments or during the next crunch time. These organizations miss three important points.
 
First, these organizations put ideas created by developers (not just product management and support issues) into their product. This means that developers need the communication and exposure to customers or be customers themselves. Atlassian writes software for developers and they use their own products (or “Eat their own dogfood” as many people say) as does Google but most organizations lack this luxury. Software architects and developers need to create communication channels where they can get this information time efficiently. For example, start with requiring developers to use the product for basic tasks when they first join the team then periodically do manual testing in realistic environments (in addition to that done by QA) and have regular lunches with support.
 
Ideally, have architects and senior developers talk with customers directly, such as attending a support call or a conference. Development teams often receive a skewed view of the product consisting of support incidents and requirements from a few important, vocal customers only. Developers and architects ask different questions than non-technical people and hear the same answers with different ears. Even simple act of architects and developers watching customers use the product can reveal incorrect UI assumptions.
 
Build opt-in telemetry into the product, too. Two of the biggest issues with successful innovation are the self-supporting bias, where people over-estimate their own contributions to successes and under-estimate their part in failures, and the confirmation bias, where people interpret facts to support their own existing conclusions. Telemetry does not replace product management’s vision or support’s intuition but effective telemetry provides the most accurate information about what features are used, how often by when and by whom, particularly during A/B testing.
 
Second, innovative organizations have easily extensible or modifiable products, usually by people outside the development team responsible for that feature. No commercial software is ever perfect but the biggest hurdle to innovation execution is getting product quality to a manageable level and stopping endless “fire fighting”. This is particularly difficult on products with a small, maintenance staff or those with overly ambitious deadlines and usually requires extensive automated unit testing, refactoring, documentation and possibly new tools.
 
The team must also have room in the backlog (scheduled modifications or enhancements, some beyond the current release, for those unfamiliar with the agile term). Even brilliant ideas will have difficulty competing if the team has years of work outstanding or over-committed to customers. The team needs to focus, create a more effective and transparent prioritization mechanism and reign in the sales organization.
 
Third, innovation, when it occurs, must also be communicated and shared. If only the immediate manager knows about it, much of its value is lost. Similarly, innovation must be expected and rewarded with goals set, tracked and judged like other performance goals.
 
Software architects and development are not the only source of innovation, either. The best customer impacting innovations come from those that understand both the customer and the technology or product. Software architects are in a prime position to drive product and technology information to others that may innovate such as technical product management, professional services, support leads, architects from other products.
 
Software architects and developers must take responsibility to drive change throughout their organization. Organizations’ inertia often makes this a challenge but innovation does not spontaneously occur in a vacuum. As usual, start small, advertise successes and persist.

Working Remotely and Successfully

A software architect is a difficult role at the best of times, particularly when working in a different office, time zone or country to the development team, other architects or management. In this case, architects miss the “water cooler” and “hallway” conversations and, unfortunately, “out of sight” is often “out of mind”. Much of what follows applies to non-software architect positions, too.

To be successful, remote architects must first create an environment where they are involved. A regular 1am team meeting on Saturday, for example, is neither sustainable nor fair. The team and its management need to adjust. Share minutes or notes for those that cannot attend meetings, too.

Remote architects must use meeting time effectively. They must know what information they need, who they are going to ask and follow it up afterwards. Regularly ask what team members are doing, such as training, travelling or talking to customers. Make time for small talk and rapport building, too.

Apply this to wider communications, too. Move away from E-mails for distributing documents and move to a common repository, like SharePoint or a Wiki so everyone can see what others are working on in addition to helping with versioning, auditing and knowing where the latest version is, and use an approved instant messaging system. When communicating, use a communication mechanism that targets the widest appropriate audience, such as a Wiki, and send notifications or links via traditional mechanisms, like E-mail.

Video conferencing is not as useful as many would claim. Skype and similar products suffer from the lower bandwidth and higher latency of international Internet connections. Professional telepresence systems like HP Halo are like being in the same room but can be difficult with people in different time zones or those that live or work away from the office.

Ensure the team travels to a central location for face-to-face meetings several times a year, too. If the organization cannot afford to bring important team members together for face-to-face meetings, an organization cannot afford to have remote employees. The critical aspect is not the meetings but the lunches, the drinks in the bar afterwards and relationship building, something video conferencing lacks.

During this “non-work” time, identify common interests with other team members, such as sport or similar aged children, and share them. Add the human element back into the team. As a previous workmate of mine once said, “You need to build a relationship before you can do business“.

Remote architects must recruit allies. Architects should talk regularly with their management, ensuring they understand their wants and needs, including up the management chain. Even the best communicators chronically under communicate with remote peers or subordinates, so remote team members must accept that their immediate manager cannot provide them all the information they require. They should talk to their peers and others outside their team, advertising their contributions and strengths, and create mutually beneficial relationships

Remote architects should do something interesting, professionally or not, and share it. Write up a few paragraphs and E-mail it to the team after attending conferences. Work on an open source project, join a local interest group or tutor a local college class.

Remote architects should make an effort to help others within the organization. For example, if someone wants a document proof read or reviewed, take the time to send intelligent comments through and carbon copy (CC) the rest of the team for visibility. Publicly give credit and thanks to others who help you with internal organizational reward systems, if they exist. Laugh at and remember other people’s jokes. Making people feel included will include the remote architect in the process.

Remote architects should get involved in projects outside their area or team. Many large organizations have discussion groups or committees, such as strategy or  review committees. Participating introduces the architect to new people, exposes his or her ideas to wider audiences and, once again, builds relationships.

Most importantly, remote architects should make an effort not to fall behind. They should do  training or otherwise become known as  experts in a field or a product, drawing them  into discussions about this area or product. Volunteer for new projects, particularly high-profile ones, and work hard to make them successful.

Ultimately, the success of a remote software architect is the architect’s responsibility. not the manager or team. This is the ultimate example of “managing your manager” and, if done right, allows an architect to enjoy a compromise of life style and professional success.

What is Technical Vision?

Few things are so error prone and political as the architects’  “technical vision”. Technical vision identifies longer term technical goals (such as add iPad support or upgrade to the new database version) and describes how to reach them, perhaps over multiple releases. Clearly, this is only useful for larger, ongoing or technically complex projects.

Many architects aim for isolated technical improvements in their first technical vision. Unshackled from the constraints of time and resources, these are often ambitious, large-scale changes intended to make anticipated subsequent changes easier by adopting new technologies, creating reusable components and standardizing formats and protocols.

The first issue with this approach is that little is tied back to customer or business requirements. Otherwise, the architect risks an “ivory tower” situation, recommending or enforcing designs often blamed for inflated resource requirements and missed deadlines.

The best way to avoid this and gain credibility is to interview stakeholders, identify changes they want made and incorporate them into the vision. Non-business stakeholders like testers, localizers and documentation writers often want product improvements, too. Perceptive architects prioritize stakeholder requests that echo their own thoughts and, once the architect has demonstrated delivering these in products, architects will gain the credibility and latitude to add some of their own ideas.

The second issue is with component reuse and standardization. While the architect’s high level view is good for identifying opportunities for these and prioritizing them, links back to business requirements are difficult to articulate, particularly to non-technical stakeholders. Developers hold code written by others to a higher standard than their own (particularly documentation, quality, scalability and security), will use such components in ways never originally envisaged or intended and backwards compatibility requirements can limit improvements. However, the biggest danger with writing reusable components and standardization is the tendency to focus on them and not the problems they solve.

Writing reusable components and standardization are organizational decisions as much as architectural ones. Bugs or enhancement requests need tracking, prioritizing, fixing in a timely manner and distribution. Good approaches for reusable components include starting with small, non-critical components or creating internal “open source” projects where anyone can contribute, subject to review. Avoid forking components if possible.

Standardization requires a middle ground between forgoing features (“lowest common denominator” approach) and spending additional effort (the “kitchen ink” approach) and organizations must resist doing too much of the latter. It is best to start with often used interfaces and include security, scalability, performance, error behaviour and similar non-functional requirements.

Lastly, new technologies are often attractive, exciting and claim to solve problems more effectively, reducing development time and cost. However, adopting new technology requires an understanding of the whole system and not just isolated problems, lest it cause more problems than it solves. Once again, there is an organizational cost with retraining developers, purchasing new tools and integrating it with old technologies. New technologies are best trialled first with a proof of concept.

Good technical visions are known to and have the backing of both stakeholders and developers, which is easier if creating it involved them. Technical visions need not be detailed or specific, at least not initially and good technical visions are accessible, unambiguous and concise. If others cannot read them or understand them, it cannot be implemented or implementations will digress from the intention. As mentioned before in this blog, it is the architects’ responsibility to communicate the vision effectively.

Technical visions can change over time, too. This reflects changing business requirements, technology landscape and development techniques. It also reflects expressing the intention more effectively. Constructive criticism and discussion is vital, both to improve it and, as also mentioned before in this blog, incorporating others’ feedback gives them ownership and ensures it is current.

Privacy

Privacy is one of those oft misused terms that people throw around, particularly related to discussions of the SOPA or PROTECT-IP acts in the US. This is an ethical question and, unfortunately, arguments on either side usually degenerate into straw man arguments about “big brother” versus “pirates, criminals or terrorists”.

Taking a step back, privacy can mean one of three things in the context of information technology. First, privacy can mean anonymity, where people can contribute to discussions or other activities without having those comments attributable to them directly. Outside large organizations, this is usually accomplished by adopting a new identity such as a forum user or an online game character. Inside organizations, anonymity is rare. Authority and accountability require real names to be used and identity can be centrally managed even if single sign on or federated authentication are still rare.

Many arguments over anonymity descend into questions of what information can individually identify people, called Personally Identifiable Information (PII)? For example, is the IP address you use to access the Internet PII? If you are the only person accessing the Internet from that IP address and you use it for a long time, it may be. However, if you are behind a NAT, firewall or similar measure this may not be the case.

The problem is these discussions often consider potential PII in isolation. For example, my company regularly performs employee surveys. As the only Australian employee in my business group, if I select my country or office, I immediately lose anonymity. Few will argue that someone’s country is PII but it is more complicated in this case. Add that to easy inference and access to analytics and the situation becomes even more complicated.

Second, privacy can mean confidentiality, where people want to restrict access to information. This is usually enforced by access control (e.g. file permissions) or encryption (controlling who has access through protecting and distributing keys). A common example is a person’s medical records being available to medical professionals treating them but not to others. These records may be available to a wider audience of medical professionals for research or statistics as long as PII is removed.

However, confidentiality alone is not sufficient for privacy. Continuing the medical example, just because you want your doctor to see your medical records does not mean you want him or her to send them to a local newspaper to print potentially embarrassing stories about you. The doctor is permitted to use the records for treating you only. In other words, privacy can mean restricting information use, usually defined via laws or consent from the subject (the person the information describes or identifies).

Privacy in all three forms is clearly important for software dealing with external customers, particularly in areas with heavy legislation such as the medical or financial industries. Information on these could fill novels, is usually jurisdiction specific and is better covered elsewhere.

Some would argue enterprise software targeted at employees is less concerned with privacy. Most organizations’ policies state that employees using organization provided computers or systems submit to scanning for malware, logging of actions, indexing and retention for later retrieval and so on and complying to these policies is usually a condition of employment. Some countries’ governments also require access to otherwise confidential information, such as the recent issues with Blackberry devices being too secure.

However, this is not the case everywhere. Many countries, Europe in particular, have strong privacy laws. These benefit the subject by restricting the collected information’s use to that consented at the time of collection. However, well-meaning privacy legislation can impact IT in unintended ways. For example, if an application logs the path to a file “c:\users\joe\my documents\doc.txt” (Windows) or “/home/joe/Documents/doc.txt” (Mac, *nix) for usage statistics or supportability, has it inadvertently captured the user’s name (clearly PII) in the path and should the application remove or obfuscate that directory in the path?

Many countries also limit the movement of PII across country borders. Consent can permit it but the consent must be specific and prior. This creates challenges when aggregating data across and enterprise, systems with ad hoc reporting systems or those that share data. This is particularly challenging with cloud based systems where the location of data is unclear or data is replicated across multiple locations for redundancy.

The target market of software architect’s products influence or dictate its privacy needs.Indeed, as the individual responsible for non-functional requirements, software architects should understand which form(s) of privacy apply and for whom. They need not be experts – legal departments are for that – but knowing what to work around and work with are important, particularly for bigger sales. Indeed, if SOPA/PROTECT-IP is passed, software architects may have even more to learn and apply.


What Makes a Good Software Architect?

Of course, a software architect individual needs to know the product’s languages, frameworks and tools – it is one of the main things that separates him or her from a business analyst or program manager. If they have previously been a senior developer on the team, that helps with credibility. However, your best developers are not always the best architects. Promoting one of them to an architect position means they are not working with the code day-to-day and this may not be what either the developer or the team wants. A software architect is more interested in the what and why rather than how.

The most important skill a software architect can have is good communication skills. This is not just because they need to talk to a variety of audiences. They need to sell the vision and high level design to non-technical business managers and VPs and build confidence and credibility. They need to impart critical decisions to senior and junior developers and QAs. They need to gather requirements from product managers or customer advocates and demonstrate to them their vision and design meets those requirements. They need to help documentation writers, UI designers and localizers contribute effectively to the product. A software architect may also need to defend implementation decisions to other products’ architects or agree on product interactions and integrations. These are topics I will deal with in later posts.

Software architects need good communication skills because they are responsible for creating a vision and high level design of a product and, if the software architect’s vision and design cannot be communicated to those implementing it, the software architect has no vision or design. Holding the implementors responsible  for understanding and implementing the architect’s vision is like holding patients responsible when they do not forewarn their doctor of them being sick. If developers or QA have misunderstood requirements, the software architect is in the best position to identify and correct that.

Second, a software architect must be able to recognise a good vision, design or decision. Yes the architect should be able to produce a good vision and design but there will always be cases where someone has a better idea or solution. As Helmuth von Moltke said, “no battle plan ever survives contact with the enemy” and this is true of anything an architect produces. The key point is accepting and incorporating good ideas from others.

Indeed, having the confidence to accept changes from vocal stakeholders or implementors can make them feel like they own the product. The design and vision ceases being just the architect’s and starts being the group’s and this collective ownership is not to be underestimated.  The job of an architect is not to produce the best vision or design but to ensure the product gets the best vision and design.

Third, a software architect should be motivated to learn as much about customers as possible. Product managers and other customer advocates can help with this but it is no substitute for original research and experience, such as attending conferences or reading magazines. The difficulty here is doing this effectively. For example, attending support calls may be easy but often give a vision skewed toward vocal and important rather than typical customers.

Last, a software architect has to be willing to roll up his or her sleeves and do “grunt work” when required. For example, if the architect is recommending a third-party library, the architect has to fill out the forms and attend the meetings with the legal department to get this approved. If the development team has daily or regular meetings, the architect should attend these, even if just to listen. If the development team is going in the wrong direction (“We’ll never need indexes on that table containing millions of rows!”), the architect may need to create a proof of concept showing a better one. After all, the architect is just as responsible as the rest of the team for shipping a good product.

%d bloggers like this: