Random Acts of Architecture

Tales of an architect trying to bring order to the chaos that is modern information technology.

Monthly Archives: November 2011

Privacy

Privacy is one of those oft misused terms that people throw around, particularly related to discussions of the SOPA or PROTECT-IP acts in the US. This is an ethical question and, unfortunately, arguments on either side usually degenerate into straw man arguments about “big brother” versus “pirates, criminals or terrorists”.

Taking a step back, privacy can mean one of three things in the context of information technology. First, privacy can mean anonymity, where people can contribute to discussions or other activities without having those comments attributable to them directly. Outside large organizations, this is usually accomplished by adopting a new identity such as a forum user or an online game character. Inside organizations, anonymity is rare. Authority and accountability require real names to be used and identity can be centrally managed even if single sign on or federated authentication are still rare.

Many arguments over anonymity descend into questions of what information can individually identify people, called Personally Identifiable Information (PII)? For example, is the IP address you use to access the Internet PII? If you are the only person accessing the Internet from that IP address and you use it for a long time, it may be. However, if you are behind a NAT, firewall or similar measure this may not be the case.

The problem is these discussions often consider potential PII in isolation. For example, my company regularly performs employee surveys. As the only Australian employee in my business group, if I select my country or office, I immediately lose anonymity. Few will argue that someone’s country is PII but it is more complicated in this case. Add that to easy inference and access to analytics and the situation becomes even more complicated.

Second, privacy can mean confidentiality, where people want to restrict access to information. This is usually enforced by access control (e.g. file permissions) or encryption (controlling who has access through protecting and distributing keys). A common example is a person’s medical records being available to medical professionals treating them but not to others. These records may be available to a wider audience of medical professionals for research or statistics as long as PII is removed.

However, confidentiality alone is not sufficient for privacy. Continuing the medical example, just because you want your doctor to see your medical records does not mean you want him or her to send them to a local newspaper to print potentially embarrassing stories about you. The doctor is permitted to use the records for treating you only. In other words, privacy can mean restricting information use, usually defined via laws or consent from the subject (the person the information describes or identifies).

Privacy in all three forms is clearly important for software dealing with external customers, particularly in areas with heavy legislation such as the medical or financial industries. Information on these could fill novels, is usually jurisdiction specific and is better covered elsewhere.

Some would argue enterprise software targeted at employees is less concerned with privacy. Most organizations’ policies state that employees using organization provided computers or systems submit to scanning for malware, logging of actions, indexing and retention for later retrieval and so on and complying to these policies is usually a condition of employment. Some countries’ governments also require access to otherwise confidential information, such as the recent issues with Blackberry devices being too secure.

However, this is not the case everywhere. Many countries, Europe in particular, have strong privacy laws. These benefit the subject by restricting the collected information’s use to that consented at the time of collection. However, well-meaning privacy legislation can impact IT in unintended ways. For example, if an application logs the path to a file “c:\users\joe\my documents\doc.txt” (Windows) or “/home/joe/Documents/doc.txt” (Mac, *nix) for usage statistics or supportability, has it inadvertently captured the user’s name (clearly PII) in the path and should the application remove or obfuscate that directory in the path?

Many countries also limit the movement of PII across country borders. Consent can permit it but the consent must be specific and prior. This creates challenges when aggregating data across and enterprise, systems with ad hoc reporting systems or those that share data. This is particularly challenging with cloud based systems where the location of data is unclear or data is replicated across multiple locations for redundancy.

The target market of software architect’s products influence or dictate its privacy needs.Indeed, as the individual responsible for non-functional requirements, software architects should understand which form(s) of privacy apply and for whom. They need not be experts – legal departments are for that – but knowing what to work around and work with are important, particularly for bigger sales. Indeed, if SOPA/PROTECT-IP is passed, software architects may have even more to learn and apply.


What Makes a Good Software Architect?

Of course, a software architect individual needs to know the product’s languages, frameworks and tools – it is one of the main things that separates him or her from a business analyst or program manager. If they have previously been a senior developer on the team, that helps with credibility. However, your best developers are not always the best architects. Promoting one of them to an architect position means they are not working with the code day-to-day and this may not be what either the developer or the team wants. A software architect is more interested in the what and why rather than how.

The most important skill a software architect can have is good communication skills. This is not just because they need to talk to a variety of audiences. They need to sell the vision and high level design to non-technical business managers and VPs and build confidence and credibility. They need to impart critical decisions to senior and junior developers and QAs. They need to gather requirements from product managers or customer advocates and demonstrate to them their vision and design meets those requirements. They need to help documentation writers, UI designers and localizers contribute effectively to the product. A software architect may also need to defend implementation decisions to other products’ architects or agree on product interactions and integrations. These are topics I will deal with in later posts.

Software architects need good communication skills because they are responsible for creating a vision and high level design of a product and, if the software architect’s vision and design cannot be communicated to those implementing it, the software architect has no vision or design. Holding the implementors responsible  for understanding and implementing the architect’s vision is like holding patients responsible when they do not forewarn their doctor of them being sick. If developers or QA have misunderstood requirements, the software architect is in the best position to identify and correct that.

Second, a software architect must be able to recognise a good vision, design or decision. Yes the architect should be able to produce a good vision and design but there will always be cases where someone has a better idea or solution. As Helmuth von Moltke said, “no battle plan ever survives contact with the enemy” and this is true of anything an architect produces. The key point is accepting and incorporating good ideas from others.

Indeed, having the confidence to accept changes from vocal stakeholders or implementors can make them feel like they own the product. The design and vision ceases being just the architect’s and starts being the group’s and this collective ownership is not to be underestimated.  The job of an architect is not to produce the best vision or design but to ensure the product gets the best vision and design.

Third, a software architect should be motivated to learn as much about customers as possible. Product managers and other customer advocates can help with this but it is no substitute for original research and experience, such as attending conferences or reading magazines. The difficulty here is doing this effectively. For example, attending support calls may be easy but often give a vision skewed toward vocal and important rather than typical customers.

Last, a software architect has to be willing to roll up his or her sleeves and do “grunt work” when required. For example, if the architect is recommending a third-party library, the architect has to fill out the forms and attend the meetings with the legal department to get this approved. If the development team has daily or regular meetings, the architect should attend these, even if just to listen. If the development team is going in the wrong direction (“We’ll never need indexes on that table containing millions of rows!”), the architect may need to create a proof of concept showing a better one. After all, the architect is just as responsible as the rest of the team for shipping a good product.

AISA National Conference 2011

I attended the Australian Information Security Association (AISA) National Conference for 2011 on Wednesday 9th November. For an event free to AISA members, the speakers were excellent and I recommend this to anyone involved with or interested in IT security in Australia. What follows is a summary of important or interesting points and how I believe they relate to software architecture.

Using anti-virus, firewalls, intrusion detection systems and so on is best practise but attackers are always finding new attacks that circumvent these controls. IT security is slow to adapt and, as Einstein said, “doing the same thing over and over again and expecting different results” is the definition of insanity. With IT increasingly forced to deliver sensitive data to users via convenient rather than secure methods, such as via social networking or on consumer owned mobile devices, the focus needs to shift from protecting devices and systems to protecting information. As Michael Jones from Google pointed out, organizations are being forced to be both open and private and to “deliver sensitive information to the edge”.

Moreover, as John Stewart from Cisco said, (paraphrased because I missed the exact quote) “there are three types of organizations: those that have been hacked once, those that have been hacked twice and those that don’t know (hacked three or more times)”. IT security needs to move from a preventative model, aimed at stopping all intrusions, to a containment model, one which recognises that compromises occur and aims to preemptively contain or localize the damage. Ruth Marshall from Novartis compared IT security to healthcare and that sometimes “the cure [IT Security and the constraints it places on users] can be worse that the cause [the security issues mitigated]” from the user’s perspective and that maybe a “palliative care” model is better.

Indeed, this second point highlighted the security disconnect between organizations and individuals. Individuals care little for security, as shown by the continued use of social networking and similar sites despite recent privacy and security concerns. Organizations do care and, instead of holding ill-equipped individuals accountable, assign this responsibility to the CISO and the IT security department. However, individuals then blame IT security when it does its job, such as pointing out the lack of regulatory compliance and privacy with commonly used social networking sites or the lack of control when consumers user their own devices for work (as mentioned above).

As non-functional requirements, the software architect is often  responsible for security and privacy. The software architect also influences or dictates where data is stored and how it is consumed. There is no accepted formula or standard for what security measures should be used in a product. One approach is to offer two or three choices for the most sensitive parts of the product; complete with the pros and cons of each, the architect’s recommendation and references to requirements (particularly integration, compliance or regulation). Work with the product manager and director on a solution and use it as a guide through the rest of the feature or product. This increases communication and spreads the responsibility, too.

Another interesting point raised at the conference concerned the potential introduction of mandatory Internet filtering in Australia. None of the presenters were in favour of this because it simply will not work and Bruce Schneier argued that technical groups, such as IT Security, need to become more politically vocal. However, he argued these groups need to argue the technical pros and cons while leaving the policy details to others. Technical groups have expertise and experience, therefore credibility, in technical areas (“Technical reasons A, B and C mean this will not solve the problem”). Adding judgement or policy to this (“Problem X is not the real problem” or “Problem X sucks anyway”) weakens this stance because listeners may assume technical details are manipulated to support a political view. Vice versa is also true.

Software architects are often required to defend their decisions or critique others’ for non-technical people. Sticking to the technical details and drawing technical conclusions from that is a very credible way at presenting or establishing your point of view. Also, coming form an engineering-style background, many new software architects view decisions as black or white affairs where the correct answer is “obvious”. Taking time to build a case with supporting facts can avoid miscommunication and frustration while opening the architect’s mind to other solutions. This is not saying that a software architect cannot or should not care for non-technical issues, more than he or she needs established credibility to do so.

Who Needs a Software Architect?

Small teams of senior people working on a well understood problems with known tools and little integration or dependencies on other applications do not need architects. This covers a significant proportion of line of business applications written today. However, there comes a point where cracks appear. There is no well understood threshold for a team size or a product’s complexity to require a software architect – no two individuals, teams, products or deadlines are the same, so what works well for one may not work for the other. However, there are four indicators or “smells” that a software architect is needed.

First, the product either lacks or does not meet non-functional requirements such as scalability, performance or security. Product Managers (PMs) or other customer advocates rarely articulate such requirements and, when they do, usually articulate them poorly or give them too low a priority. A software architect can balance the customer or business requirements with technical ones and provide the technical foundation to meet the customer requirements.

Second, the product lacks technical vision. Enterprise applications rarely exist in a vacuum. They need to integrate with other applications, often share components and need to lay the foundation to make likely future changes easier. Useful new components, frameworks or libraries may become available that allow the team to focus on business value and not solved problems, especially for older products (i.e. 10 years+). The industry may trend toward different platforms (e.g. mobile) or business models (e.g. cloud). Development teams are often too focused on the current release of the current product to take a step back and few others have the technical knowledge required, unlike a software architect.

Third, the development group has poor interteam communication and collaboration. Effective communication is easy for small teams of three to five people. This is much harder if the group has several hundred members in different continents and time zones. Effective cross team management may be difficult because individual team leads are usually focused on their own teams and the common management point may be a director or even an executive who has bigger problems on his or her mind and lacks the required technical focus. A software architect can divide the work and act as a coordinator between teams.

Communication with development teams outside the group is also critical. A development group often needs someone with broad rather than deep knowledge to evangelize their product(s) or identify integration points or sharable code. Once again, team leaders are usually too focused on their teams,  directors are often too removed from the technical details and individual developers find the higher, longer term views too distant from actual code. A software architect usually has better communication skills, the right breadth of product knowledge and understands its technical vision. Similarly, non-technical but development-related teams like user interface designers, documentation writers or localization require someone with a broad technical knowledge and a customer focus.

Fourth, the group has unclear requirements or problem domains. Relatively few teams boast many subject matter experts or developers that use their software like their customers, particularly in junior teams, remote teams or those with a high turnover. PMs are often focused on dealing with customers or support issues rather than communicating detailed requirements to every developer. They also may not understand what the developers do not know and so provide too little or too detailed information. This process is also ongoing due to turn over and changing market conditions. Lastly, requirements also change as the project progresses, whether due to changing customer requirements, competitive landscape or just learning more about the problem.

These are traditionally areas where business analysts (BA) rather than software architects are used and can be used in tandem with a software architect. However, one exception is where business requirements may have unclear technical ramifications and feasibility, such a compliance with industry standards or regulations. Federal Information Process Standard (FIPS) 140-2 or Common Criteria require changes and documentation at many levels of detail which an architect is better suited to than a BA.

Note that software architects do not solve all problems with development. In particular, they highlight or exacerbate political or organizational problems when they try to bring stakeholders together or reconcile conflicting decisions and requirements. This may be intentional (to drive change) but is not sustainable in the long term.

%d bloggers like this: