Random Acts of Architecture

Wisdom for the IT professional, focusing on chaos that is IT systems and architecture.

Tag Archives: Communication

Separating the Wheat from the Chaff: Unpacking Professional Opinions

Fueled by the advent of social media, opinions now surround us. Seemingly everyone has them and broadcasts them to the broader world. Untethered from context and expertise, such expression usually leads to validation or ridicule instead of healthy debate. Uncritical opinion sharing can drive tribalism, becoming a foundation of identity as adherents one-up each other with more extreme views.

Professional environments face different challenges with opinions. They are smaller and simpler than worldwide social media and rarely emphasize extremes. However, organizations frequently need to make quick decisions on incomplete or uncertain information. For example, to solicit feedback on a proposal or design.

Opinion gathering can be a helpful tool. Requesting opinions can reveal similar initiatives, valuable metrics and new perspectives. It can also extract contradictory, irrelevant, ill-informed or otherwise not valuable input from the wrong people.

Effective opinion gathering requires preparation then focusing on relevant motivation, experience and facts.

Preparation involves asking the questions you should ask before seeking feedback.

Creating and sharing a succinct document or presentation clarifies the problem, sets its scope and answers common questions. It gets better quality opinions sooner.

Defining the audience ensures the audience is complete and not excessive. It also helps set an end to opinion gathering.

Preparation should be proportional. Not every issue needs documentation and a plan. However, taking a few moments to think is almost always beneficial.

Unfortunately, opinions are sometimes unsolicited or not on topical problems. On topic opinions may be outside the scope or imply impractical changes.

While perfect alignment is rare, the most useful and actionable opinions come from those with benign motivations. Opinions from those with different views or incentives need careful consideration.

For example, salespeople are often commission-motivated. They are likely to favour whatever gets them their next sale. However, they also have frequent close contact with customers, giving unique insights.

In IT, non-technical stakeholders are not accountable for warranty (e.g. speed, security, reliability, maintainability), only for utility (e.g. correctness, completeness). They will often neglect or downplay quality issues. However, they may also represent a broader or long term view than just the immediate technical details.

Stakeholders may have conflicting motivations. Managers have to weigh short-term targets against long-term goals. The decision-fatigued or overworked may suggest the easiest path over the best one.

Even well-intentioned opinions can be problematic. For example, aspiring experts may give unsolicited ideas, trying to appear knowledgeable in the guise of assistance. Some merely repeat others’ opinions they found compelling. Confidence and credibility are different things.

Potentially the most dangerous opinions are those with an unknown motivation. Actual dishonesty is rare in professional settings. However, self-promotion and self-preservation evolve from helpful to essential as you climb the management ladder.

The opinion giver’s experience is also significant. It prevents foreseeable mistakes and imparts acquired wisdom.

However, there is relevant and less relevant experience. “The last time we did this” may have been under different market conditions, finances or staff.

Experience does not imply expertise. Seemingly capable people may have knowledge gaps, poor tools or incomplete awareness.

People may not adapt or improve, repeating poor practices or old mistakes. Sometimes years of experience are just repeating the same experience.

Like motivation and experience, saying facts are pertinent is an understatement. These include measurements, dates, budgets, quotes and anything that most agree is hard to dispute. They are more effective at forming or changing opinions in professional settings than on social media.

Often the source’s trustworthiness is the most crucial factor in disputes. However, referencing “The Lean Startup” by Eric Ries, anything auditable, actionable, and accessible should be convincing.

Facts should be complete and, like experience, relevant. Statistics or definitions can ignore inconvenient data. Averages imply an often inaccurate or oversimplified modal distribution.

Fact-producing experiments are generally more effective than expertise. They consider local conditions and capabilities. Repeatable experiments are even better because they can become goals to achieve or metrics to track success or failure.

Changing your organization or peers to give better opinions is challenging, particularly given the self-entitlement enabling social media that surrounds us.

However, you can change the way you give and receive opinions. While everyone is entitled to opinions, not all opinions are equal. Thinking critically about those given and received helps professionally and in the broader world.

Image from https://www.thebluediamondgallery.com/handwriting/o/opinion.html. By Nick YoungsonCC BY-SA 3.0 from Alpha Stock Images.

Twitter will never outsource its algorithm

Elon Musk sparked controversy with this recent attempt to take over Twitter. Many support him, citing Twitter’s relatively poor revenue and Musk’s success in turning seemingly unprofitable ventures, like electric vehicles and space exploration, into successes.

However, his recent Twitter poll caught my interest, where 82% of over one million responses voted that Twitter should open source its algorithm.

Musk explained further during interviews at the TED 2022 conference. The “Twitter algorithm” refers to how tweets are selected then ranked for different people. While some human intervention occurs, social networks like Twitter replace a human editorial team’s accountable moderation with automation. Humans cannot practicality and economically manage and rank the estimated 500 million tweets sent per day.

By “open source”, Musk means “the code should be on Github so people can look through it”. Hosting software code on github.com is common practice for software products. Third parties can examine the code to ensure it does what it claims to. Some open sourced products also allow contributions from others, leveraging the community’s expertise to collectively build better products.

Musk says “having a public platform that is maximally trusted and broadly inclusive is extremely important to the future of civilization.” People frequently demonize social networks for heavy-handed or lax “censorship”, depending on their side in a debate. Pundits claim social networks limit “free speech”, conveniently forgetting “free speech” means no government intervention. Pundits cite examples of the algorithm prioritizing or deprioritizing tweets, authors or topics. They also cite account suspensions and cancellations, sometimes manual and sometimes automated.

Musk assumes that explaining this algorithm will increase trust in Twitter. He called Twitter “a public platform”, implying not just public access but collective ownership and responsibility. If people understand how tweets are included and prioritized, the focus can move from social networks to the conversations they host.

Unfortunately, understanding and trust are two different things. Well understood and transparent processes, like democracies’ elections or justice systems, are not universally trusted. No matter the intentions or execution of a system, some people will accuse it of bias. These accusations may be made in ignorant but good faith, observe real but rare failures or be malicious and subversive.

Twitter’s algorithm is not designed to give equal exposure to conflicting perspectives. It is designed primarily to maximize engagement and, therefore, revenue. It is not designed to be “fair”. Social networks are multibillion dollar companies that can profit from the increased exposure controversy brings. Politicians alienate few and resonate with many when they point the finger of blame at Twitter.

Designing an algorithm for fairness is practically impossible. You can test for statistical bias in a numeric sample set but not across the near entirety of human expression. Like the philosophers opposing the activation of Deep Thought in Douglas Adams’ Hitch Hiker’s Guide the Galaxy, debating connotation, implication and meaning across linguistic, moral, political and all other grounds is an almost endless task.

Assuming transparency can assure trust and fairness is possible, open sourcing the Twitter algorithm assumes the algorithm is readable and understandable. The algorithm likely relies on complex, doctorate-level logic and mathematics. The algorithm likely includes machine learning, which uses no defined algorithm. The algorithm likely depends on custom databases and communication mechanisms, which may also have to be open sourced and explained.

This complexity means few will be able to understand and evaluate the algorithm. Those that can may be accused of bias just like the algorithm. Some may have motivations beyond judging fairness. For example, someone may exploit a weakness in the algorithm to unfairly amplify or suppress a tweet, individual or perspective.

Musk’s plan assumes Twitter has a single algorithm and that algorithm takes a list of tweets and ranks them. It is likely a combination of different algorithms, instead. Some work when tweets are displayed. Some run earlier for efficiency when tweets are posted, liked or viewed. Different languages, countries or markets may have their own algorithms. To paraphrase, J.R.R. Tolkein, there may not be one algorithm to rule them all.

Having multiple algorithms means each must be verified, usually independently. It multiplies the already large effort and problems of ensuring fairness.

Musk’s plan also assumes the algorithm changes infrequently. Once verified, it is trusted and Twitter can move on. However, experts continue to improve algorithms, making them more efficient or engaging. Hardware improves, providing more computation and storage. Legal and political landscapes shift. Significant events like elections, pandemics and wars force tweaks and corrections.

Not only do we need to have a group of trusted experts evaluating multiple complex algorithms, they need to do so repeatedly.

Ignoring potentially reduced revenue from algorithm changes, open sourcing Twitter’s algorithm also threatens Twitter’s competitive advantage. Anyone could take that algorithm and implement their own social network. Twitter has an established brand and user base in the West, but its market share is far from insurmountable.

There are other aspects to open sourcing. For example, if Twitter accepts third party code contributions, it must review and incorporate them. This could leverage a broader pool of contributors than Twitter’s employees but Twitter probably does not need the help. Silicon Valley tech companies attract good talent easily. Some contributions could contain subtle but intentional security flaws or weaknesses.

If the goal is to have a choice of algorithms, is this choice welcome or does it place more cognitive load on people just wanting a dopamine hit or information? TikTok succeeded by giving users zero choice, just a constant stream of engaging videos.

Evaluating an algorithm’s effectiveness is more than just understanding the code. It requires access to large volumes of test data, preferably actual historical tweets. Only Twitter has access to such data. Ignoring the difficulty of disseminating such a huge data set, releasing it all would violate privacy laws. Providing open access to historical blocked or personal tweets would also erode trust.

Elon Musk has demonstrated an uncanny ability to succeed at previously unprofitable enterprises like electronic vehicles and space travel. Perhaps there is more to Elon’s Twitter plan than is apparent. Perhaps he is saying what he needs to say to ensure public support for his Twitter takeover.

While open sourcing Twitter’s algorithm appeals to the romantic notion that information is better free, increased transparency will not create a “maximally trusted and broadly inclusive” Twitter. Social networks like Twitter coalesce almost unbelievable amounts of data almost instantly into our hands. They have difficulty with contentious issues and, therefore, trust because they reflect existing contention back at us. It is easier to blame the mirror than ourselves.

The image is a cropped version of https://commons.wikimedia.org/wiki/File:Programming_code.jpg. Used under Creative Commons Attribution-Share Alike 4.0 International license.

If it were easy, we would not need good IT architects

It is hard not to find advice on being a great IT architect. Beyond modelling and technical details, good advice includes effective persuasion, people skills and persistence. However, the need for non-technical skills is often left as self-evident or explained superficially. It also glosses over important details and distinctions.

Traditionally, IT has two types of roles. The first role is the implementor, such as a software developer, network engineer or QA. They produce non-delegatable outcomes. The second role is a manager or executive, someone accountable when something goes well or poorly.

IT architects fit into neither role. Their value becomes apparent when the project or organization gets big, complex or conflicts with team objectives. They are not more intelligent or more capable than implementors or management. Instead, they solve an emergent need.

IT architects are internal consultants, helping team members to execute and management be successful. Enterprise and senior solution architects increasingly provide executive decision support instead of design, almost becoming a separate role.

However, being a consultant has two fundamental but often missed implications for IT architects. The first implication is an IT architect has no intrinsic authority. Their authority stems from the stakeholders, usually management or executives, that their architectures enable or support.

Whose authority is usually evident in client-facing product teams who share a reporting structure. However, this is often unclear in projects spanning multiple reporting lines, such as large or siloed organizations or internal IT projects. Shared responsibilities between business and IT muddy this further.

The second implication of an IT architect being consultative is people often incorrectly think IT architects, like consultants, are paid for their opinions. Yes, implementors consume the artifacts produced by IT architects, like designs and principles (“opinions”). However, neither stakeholders nor implementors blindly accept them.

Instead, an IT architect should prioritize and justify their outputs as stakeholders do to their superiors. This approach consists of three pillars.

The first pillar is financial. How do the IT architect’s actions make money, save money or lose less money? Start with understanding how the organization funds projects. For example, if maintenance and operation costs are separate, the IT architect may need to justify up-front efforts to reduce each to different stakeholders. A move to IaaS cloud hosting is easier if the organization favours OpEx over CapEx.

The second pillar is metrics. Executives and managers often have objectives, measures or other targets they need to reach. While they are rarely perfect, an IT architect should understand them and reference them. For example, if cost-saving is a metric for a critical stakeholder, ensure the design reduces net cost.

The third pillar is language. The IT architect should learn and use the terms used by executives and managers. Such language varies between organizations and shifts over time but talking like decision-makers helps an IT architect seem credible.

Another question is how much authority the IT architect holds. There is no definitive answer. Less technical management and executives often dismiss the question as an IT issue. IT management may lack the strategic incentives to avoid cutting corners. Organizations rarely question the level and nature of authority when successful. However, they can scramble when things go downhill. 

Aligning expectations and processes beforehand, both individual IT architects and the team, is a practical first step. How is development tracked against the architecture? Is there a process for changing or disregarding parts of an architecture? How do you differentiate poor execution from a good architecture or vice versa? 

Once an IT architect establishes this, IT architects can then differentiate the value of their project from their value as IT architects. The accountability for a project rests with the appropriate executives and management, not the architect. The IT architect needs to focus on their part in it.

Selling an architecture or design and selling the value of an IT architect are related but different activities. Selling a design, for example, requires helping the customer meet their own goals more effectively. It happens periodically, usually once per project. It makes you authoritative for that project. Selling an IT architect role and understanding stakeholder needs are both ongoing, continuous processes.

IT architects are a bit like CEOs. CEOs’ primary customers are the investors or owners of the organization. While ignoring actual customers or staff seems cynical, good investors care about the organization as a whole and beyond just customers’ short term needs.

An IT architect’s “investors” are their stakeholders. Like CEOs, IT architects will rarely keep everyone happy. Not every team will have every need met. Every change involves winners (profit, less work) and losers (more work, more responsibility).

The success of an IT architect relies on their ability to identify and sell their value to the correct set of stakeholders. Selling is challenging and complex. Stakeholders are rarely technical, at least from an architect’s viewpoint, and require empathy.

For example, non-IT management wants justification in business terms, usually financial. They often regard “good design practices” as an unnecessary expense. While organizations are using more technology, technology is rarely a core business competency or differentiator. The onus is on the IT architect to justify anything beyond a minimal expenditure.

Having to justify technology use constantly is frustrating. IT architects often evolve from technologists who “look up” or “see the big picture”. However, the big picture is not really about technology. It is about the application of technology to broad, often ill-defined and contradictory goals. If it were easy, we would not need good IT architects.

Image credit: Creator: Sergey Nivens | Credit: Getty Images/iStockphoto

The Problem with Agile Transformations

It seems every organization wants to transform itself to become more agile. They want to respond to opportunities quickly and cost-effectively. They want to adapt faster than their competition in the Darwinian corporate landscape. COVID-19, for example, required a quick shift to remote working for their employees and remote interactions with clients and suppliers. Geopolitical changes alter supply chains and increase legislation. Cloud, IoT and similar internal IT trends accelerate.

An agile transformation often starts with a consultant-led agile processes adoption. These range from methodologies like Scrum at the small scale and Scaled Agile Framework (SAFe) at the highest. Despite some negativity in the agile community aimed at higher scale methodologies, these methodologies help by providing structure, vocabulary, and expectations.

However, following an agile process is the least important part of being agile. An organization attempting to increase agility solely by adopting a new process usually creates superficial changes that foster change fatigue, at best, or failure, at worst. This failure is often then incorrectly attributed to the process without any more profound and helpful introspection, leading to the negativity mentioned earlier.

An agile organization needs agile systems. “Agile systems” does not refer to using a tool like JIRA to track work. Agile systems, both IT and business processes, are easily changed and provide feedback for timely validation or correction.

Modern software development practices are a good example. Frequent demos to users and stakeholders and automated testing and deployment, for instance, ritualize change and create tighter feedback loops.

However, an organization’s systems are not agile if only their software development teams are agile. For example, consultants often build organizations’ back office and operational systems by customizing third-party tools. Organizations then maintain them with skeleton teams and, therefore, usually lack the skills or the environment to change them safely and cost-effectively.

Product Owners are not the answer. Product Owners can shield their team’s backlog to encourage agility. However, systems are more extensive than just the team or teams that maintain or implement them. Product Owners are also usually experts in the business area and handling internal politics. They are rarely also design experts and incentivized to think strategically.

Agility extends to data. Organizations are under increasing pressure to collect and monetize data. Agility requires knowing where that data is (“systems of record”), ensuring its quality and integrating it with other systems (“systems of engagement” or “systems of transformation”).

Governance systems must adapt. Regulatory, financial, legal, risk, IT security, privacy and similar teams must work in smaller batches. A lengthy, formal review of a completed system is often too late.

Unfortunately, articulating the benefits of agile systems is difficult. Pruning teams to their minimums has clear short-term financial benefits. Executives often overstate their systems’ capabilities through ignorance or self-promotion. Product teams may fall into the trap of a short-term, sales-oriented focus under the guise of being customer-focused. A system’s agility often depends on an executive’s skill at shielding budgets or convincing stakeholders.

These problems may extend to company culture. Business processes sometimes lack an agreed, empowered business owner. IT is sometimes “seen and not heard”. Insufficient executive representation or ownership means minimal focus and support, making agility practically impossible.

An agile organization also needs an agile structure. This statement may sound tautological. However, an otherwise agile organization that cannot leverage and benefit from that agility wastes that effort.

A good example is what SAFe terms the “network” versus the “hierarchy”. Most organizations structure themselves hierarchically around similar skills for ease of management. For example, an organization often has a legal department for lawyers, a sales department for salespeople and an IT department for developers and system administrators.

However, work frequently requires people across different teams to cooperate, called the “network”. Identifying value chains then building and supporting these multidisciplinary teams to implement and enhance them increases flow, ownership, and individuals’ agency. Effective use of “networks” requires different management techniques and incentivizing outcomes, not throughput. These often fail without executive support.

Organizations are rarely homogeneous. Some teams or systems may be more agile than others, such as through mergers/acquisitions or pockets of conscientious staff. Tracking and aligning these is vital. Otherwise, some teams may adopt inconsistent processes or tools, be omitted from the transformation or optimize at others’ expense.

This all assumes the organization’s bottleneck is the lack of agility. Organizations frequently want to decrease “friction” but cannot articulate it beyond the highest level. Defining friction by actionable, measurable metrics is a prerequisite. Increasing agility also only highlights any poor prioritization or lack of focus.

The “transformation” concept is also a misnomer. Agile thinking encourages constant self-evaluation and improvement, and this “transformation” does not end when the consulting engagement does. From the executive viewpoint, this can scarily move the fulcrum of structural control into the middle and lower levels of the organization.

The chosen agile methodology’s principles best guide agile transformations. Unfortunately, people easily gloss over these in favour of the easily implemented and more prescriptive processes. However, these are often what the more prescriptive parts are derived from, not vice versa. If you want to be agile, internalize the principles!

Agile transformations are more than superficial changes. Focusing too much on process changes instead of systems and structures often stalls or blocks agile transformations. Instead, these transformations require people to move outside their comfort zones, particularly executives. The problem with agile transformation is it is hard but increasingly necessary.

The image is from https://digileaders.com/webinar-shape-agile-organisation/, used under Creative Commons license.

Talking Non-Tech

IT architects often pride themselves on their technical knowledge. Tasked with designing a system from end-to-end and taking responsibility for that design, they need to ensure the details are right. They also need to demonstrate technical prowess to earn respect from technical developers and engineers.

However, as discussed in previous posts, architects also have to talk to non-technical people to gather requirements, understand the business context and assure them that a design will meet their needs. For people used to delving into the technical details, this context and mindset switch can be challenging.

First, understand the value the architect’s proposed changes bring to stakeholders and organization. Understand not just what each stakeholder has asked for but how that stakeholder’s performance is measured and describe the impact of proposals in those terms.

Taking an operations manager as an example, describe how this will reduce incident frequency or severity. For a salesperson, relate this to imminent or key deals. For any management, ensure they understand how the changes relate to KPIs, long-term objectives or organizational policies.

A quick way to do this is to describe a technical change then ask “So what?”. Relate it to each stakeholder in a sentence or two then invite questions. Take note of anything asked and ensure it is covered next time.

Sometimes non-technical people suggest technical solutions. While most IT architect’s immediate reaction is to dismiss these as ill-informed, a better response is to understand the reasons behind it. Did this suggestion work last time? Is the relative cost for the asker small? Is a suggested tool the only one the asker is familiar with?

A better response from an architect is to evaluate suggestions and provide quicker, cheaper and/or better alternatives. Sometimes, however, it is important to buy-in by using elements of their suggestion, even if it is technically suboptimal.

Unless the organization has prescribed formulas or a culture of doing so, avoid trying to express impacts in financial terms. Chances are architects will get it wrong. Be careful using jargon or discipline-specific terms, too. Technical people cringe when non-technical people misuse technical terms. It happens the other way around, too.

Describe the context of a technical change in both technical and business terms. What existing systems or processes are impacted? What can we do now that we could not before? What can we do better or cheaper? What additional work is required or what work is saved?

No system exists in a vacuum and there are always flow-on effects for every change. If an architect cannot articulate these, chances are the requirements were not fully understood or analysis was lacking.

Describe the impact constraints have on the design or team implementing the design. Do not just list them (“We only have three engineers”). Say how this impacts the solution (“Option A is a better solution but, because we have a small team and a tight deadline, we are going for option B”).

Everyone in the organization has to deal with constraints. Sharing them helps build trust across teams. It also invites stakeholders, who sometimes have more experience, to suggest better ways of dealing with them.

Produce and use good quality communication. Consider using multiple views of a solution for different audiences, emphasizing different aspects. Use aesthetically pleasing diagrams with consistent use of symbols and colour. Do not be afraid of detail – it gives the audience the impression you have a deep understanding of the problem and solution – but ensure the communication is broad, covering the value and context as outlined above, instead of deep. Provide overviews or summaries to help time challenged people understand important points.

Communication should also cover solutions that were not selected or implemented. The implemented design or change will be evident. Understanding what alternatives were considered is often forgotten, particularly for trade-offs or others’ suggestions.

Beyond communicating better with non-technical people, these practices help architects understand the impact of technical changes on the organization beyond the immediate. It raises questions about larger impacts and exposes gaps in the architects understanding. It also helps build relationships.

Ultimately, being able to communicate effectively with non-technical people makes the architect a better architect. IT architects are more than just designers. They are collaborators and evangelists and they cannot do this if they can only talk and think like engineers. Architects are often the face of the team, department or company and the impression the architect needs to make a good impression.

Moreover, an architect’s technical solution exists within the organization’s social and political environments, not just technical. The architect is responsible for their work’s political and organizational success, too.

Image Credit: http://www.thebluediamondgallery.com/wooden-tile/v/value.html under Creative Commons 3 License (CC BY-SA 3.0)

Requirements and leadership, not design, are the keys to architecture

Listen Understand Act

Many IT engineers aspire to be architects. They want to dictate the course of their products or services, leading their fellow engineers. To do so, they focus on designing the best and largest systems, learning all about design patterns, notations and understanding technology from top-to-bottom.

However, if such a thing can be said to exist, even the best design is wasted if it does not solve the right problem. Architects should start here, instead.

Depending on the organization, requirements are often supplied by product management, business analysts or management. During requirements analysis, architect validation identifies ambiguities, omissions, estimated time and resource costs and likely tradeoffs. The resulting requirements and priorities may differ substantially from the original as trade-offs and discoveries are made.

Requirements often present the business understanding of what technology should do, not the most impactful or beneficial things technology can do. Architects are in the best place bridge the gap, driving technology from the bottom-up instead of the top-down.

Business-supplied requirements often lack quality attributes or non-functional requirements like availability, performance and security. These are either assumed or difficult for non-technical people to articulate and architects are the best equipped to specify these.

Architects need to listen more than they talk, learning as much as they can about the business context of their work and its business value. Drilling into requirements is a good start, helping to understand requirements’ context, assumptions and priorities. There is no point where an architect understands everything, only a process to continually learn.

While it is tempting for a newly appointed architect to focus on their pet technical problems, ensuring they have a good pipeline of requirements helps architects to align their efforts to solve others’ problems, not just the ones they perceive. They also need to ensure the business outcomes are met, not just the technical enhancemnts.

Looking at it another way, a design is not just a model (approximation) of the implementation. A design is the requirements for the implementation. Like requirements gathering, design is iterative and may change through the review or implementation process. Like requirements gathering, design is a trade-off. Like requirements gathering, it is an abstraction, leaving some details to implementers. If an architect cannot understand or provide good requirements, their designs are going to be misunderstood, at best, or ignored, at worst.

Moreover, architects are leaders. Not leaders in the management sense but leaders by collaboration, communication and example.

While the technical leadership of architects is well understood, good architects move out of their comfortable technical conversations and into the less comfortable business conversations. As mentioned above, some requirements sit between the technical and business and stakeholders need assurance the system will meet their needs. No design pattern or notation will achieve this.

Architects should focus on outcomes and end-to-end systems, not the minutiae of their designs, particularly in agile environments where just-in-time design occurs or where component responsibility is delegated to teams. Trusting implementors by giving them clear interfaces, scope and direction is the best way to foster their trust in architects.

Architects must own their communication. The responsibility for implementors and stakeholders understanding the design and vision rests with the architects. A design or vision that is not communicated is not understood and an architect producing designs no one understands has zero business value.

An architect must also facilitate communication between teams, particularly when design changes ripple through other teams’ work.

Architects must be accountable for systems they architect. They need to listen to implementors to understand their challenges and how to mitigate them in current or future designs. They need to accept criticism from stakeholders when requirements are not met. They also need to be applauded when their projects or systems succeed.

While designs are the architect’s deliverables in many projects, an architect’s success is driven by their ability to ensure they are solving the right problems and assure people of that direction. Good architects look down toward the technical detail and ensure it is correct. Great architects also look up and around to understand how they can best provide value to the business, sometimes better than the business can.

Image from https://www.flickr.com/photos/highersights/6231641551. Used under creative commons license.

Rebranding Corporate Politics

politicsThe term “corporate politics” conjures up images of sycophantic, self-serving behavior like boot-licking and backstabbing. However, to some IT professionals’ chagrin, we work with humans as much as computers. Dismissing humans is dismissing part of the job.

The best way to “play” corporate politics is solve big problems by doing things you enjoy and excel at.

“Big problems” means problems faced not just by your team but by your boss’s boss, your boss’s boss’s boss and so on. If you don’t know what they are, ask (easier than it sounds). Otherwise, attend all hands meetings, read industry literature or look at your leaders’ social network posts, particularly internal ones.

This is not just for those wanting promotions into management. Individual contributors still want better benefits and higher profile or challenging projects. These come easiest to those known to be providing value and not the strict meritocracy some IT professionals think they work in.

Start by solving small problems as side projects. Choose something impacting more than your own team and minimize others’ extra work. Build up to bigger problems once you have demonstrated ability and credibility.

You need not be the leader. Assisting others making an effort can be just as effective. You can own part of it or bask in the halo effect. If not, recognize those that are. This creates a culture of recognition that may recognize you in the future.

While some IT professionals solve big problems everyday, communicating and evangelizing their work “feels” wrong. This what salespeople do, not IT professionals. Many also think their work is not interesting.

Being successful requires people knowing what you do. This may be as simple as a short elevator chat, a brown bag talk or a post on the corporate social network. It also helps get early feedback and build a like-minded team. Others will be interested if you are working on the right things.

What about the potentially less savory aspects of corporate politics like work social events, sharing common interests with management, supporting corporate charities and so on? These are as much an art as a science. Focus on common goals and building trust, internally and externally. People like to deal with people at their level and contact builds familiarity.

However, this is no substitute for solving big problems. If you are delivering value, interactions with senior decision makers and IT professionals with similar goals should occur naturally. Build on that.

Be aware that problems change over time. Problems get solved by others. The market changes. Competitors come and go. Understanding organizational goals is an ongoing process.

Also realize decision makers are human. They make mistakes. They want to emphasize their achievements and not their failures, just like software developers’ fundamental attribute error bias for their own code and against others’.

However, if your organization makes decisions regularly on “political” grounds, leave. Culture is rarely changed from the ground up and many organizations are looking for good IT staff.

Ignoring the worse case scenario and IT professionals’ bias against self evangelism, the biggest problem with “corporate politics” is actually its name. The concepts behind “agile” and “technical debt” came into common usage once the correct metaphor was found. Corporate politics needs rebranding from something avoided to a tool that IT professionals use to advance themselves. It badly needs a dose of optimism and open mindedness.

Image credit: http://thebluediamondgallery.com/p/politics.html. Usage under CC BY-SA 3.0.

InfoSec: Not just for hackers

everybody-needs-a-hackerI recently read Troy Hunt’s blog post on careers in information security. Troy makes good points about information security as a potential career and the benefits of certifications like the Certified Ethical Hacker. Hackers are getting increasingly sophisticated, requiring specific knowledge to counter, and cryptography is hard. We need more information security specialists.

However, one criticism of the post, indeed the information security industry, is its implication hacking is the sole information security career path. This binary viewpoint – you are either a security person or not and there is only one “true” information security professional – does more harm than good.

Hacking is technology focused. However, security’s scope is not just technical. Information security needs people that can articulate security issue impact, potential solutions and their cost in terms non-security people can understand. This requires expertise and credibility in multiple disciplines from individual contributor level to management to boardrooms.

Security solutions are not just technical. We live in societies governed by laws. These can be standardized government security requirements as FedRAMP or IRAP. These can be contractual obligations like PCI-DSS, covering credit card transactions. These can hold organizations accountable, like mandatory breach disclosure legislation, or protect or privacy, like the European Union’s Data Protection laws. Effective legislation requires knowledge of both law and information security and the political nous to get it enacted.

We are also surrounded by financial systems. Financial systems to punish those with weak security and reward those with good security will only evolve if we (consumers and investors) value security more. Cyber insurance has potential. Cryptographic technologies like bitcoin and block chain algorithms are threatening to disrupt the financial sectors. Information security has and will continue to impact finance.

The list goes on. Law enforcement needs to identify, store and present cybercrime evidence to juries and prosecute under new and changing laws. Hospitals and doctors want to take advantage of electronic health records..

The security technology focus drives people away non-technology people. In a world crying out for diversity and collaboration, the last thing information security needs is people focusing solely inward on their own craft, reinforcing stereotypes of shady basement dwellers, and not on systems security enables.

Bringing this back to software, many organizations contract or hire in information security experts. Unfortunately, the OWASP Top 10 changed little from 2010 to 2013 and some say is unlikely to change in the 2016 call for data. According to the Microsoft Security Intelligence Report, around half of serious, industry wide problems are from applications. Developers make the same mistakes again and again.

Education is one solution – security literate developers will avoid or fix security issues themselves. A better solution is tools and libraries that are not vulnerable in the first place, moving security from being reactive to proactive. For example, using an Object-Relational Mapping library or parameterized queries instead of string substitution for writing SQL.

Unfortunately, security people often lack skills to contribute to development and design beyond security. While information security touches many areas, information security expertise is not development (or networking or architecture or DevOps) expertise.

Information security needs different perspectives to succeed. As Corey House, a Puralsight author like Troy Hunt says in his course Becoming an Outlier, one route to career success is specialization. Information security is a specialization for everyone to consider, not just hackers.

Image credit: https://www.flickr.com/photos/adulau/8442476626

Priming for Effective Feedback

Ignoring software architects’ code, the primary deliverables of architects are technical vision and high level designs. These are rarely complete and correct in the first draft and getting feedback from stakeholders and developers is vital to their success. Indeed, the best software architects are those that consistently extract the best feedback.

The key to effectively receiving feedback is for the software architect to change his or her attitude to feedback, particularly criticism. A software architect’s job is to own the problem, not a solution and providing the best solution, even one that contains elements from others, is the goal. When people give feedback, it means they are invested enough to care about the topic. By comparison, a lack of feedback or dismissive, detail-lacking response, such as “Looks good”, usually means the document has not been read or the reviewer is not invested in its success.

First, identify why feedback is being sought on the document, presentation or similar. Be more specific than just “standard procedure”. For example, is this a customer facing document that must not include company confidential material? Meeting minutes where important details and decisions need to be captured? A radical, new, partially-developed idea to create discussion and spark other ideas? This will dictate the audience and direct their attention.

Then identify what sections or details the reviewers should focus on, particularly for larger documents. Otherwise, some will focus on “big picture” issues, spelling and grammatical errors or otherwise get distracted. If uncertain, start with a high level introduction then focus on potentially controversial aspects such as assumptions, integration points and major decisions. If a document has multiple audiences, consolidate information for different audiences into separate sections for clarity and provide context if needed.

Given the above, many send out the document as an E-mail attachment or link, cynically expecting no response. There are two issues with this. First, typing responses can take time, be prone to misunderstanding and some people just do not communicate well in this medium. Second, without an articulated deadline, many will forget about it.

The solution is to still send document as E-mail attachment or link around but organize one or more meetings or conference calls, the latter preferably with document sharing software, to discuss it. Some will still respond with questions or comments before the meeting. However, the meeting gives people a deadline to review the document by and allows those that prefer verbal feedback to do so. Not everyone will find all issues with the document and a meeting allows feedback to be shared by all members of the group and not just the reviewer, potentially sparking additional ideas. For long or detailed documents, multiple short meetings are preferable to fewer longer ones. It avoids fatigue and fits in better with busy schedules.

Indeed, feedback is a conversation, not a download. The software architect, or whomever is providing the reviewed document, must provide a timely response and to a wider audience if warranted. If the architect is not detail-oriented, enlist the aid of someone who is to help keep track. Disagreements or contradictory feedback are learning opportunities and usually the result of missing information, assumptions or different experiences. In this case, either identity the facts that all parties agree on then build up from there or use an external expert or trusted authority.

Honest feedback requires safety. Architects are often senior developers with lots of experience and can be intimidating, particularly to junior developers. Remind others that you welcome feedback and emphasize other’s contributions to the document. Paraphrase feedback to confirm understanding. Humor, if culturally acceptable, can be a great tool if not overused, too.

Feedback requires mutual respect. An architect that disrespects a reviewer is likely to dismiss their feedback without adequate explanation or justification, impacting safety as mentioned above. If the experience or knowledge gap between reviewers is high, separate the reviewers into groups of similar capabilities then adapt the document or meetings for each audience. Often those with the best feedback are those that failed at something similar because they generally know why they failed better than those that succeeded know why they succeeded.

Similarly, reviewers that disrespect the architect may use it as a point scoring opportunity. An executive or customer may dictate rather than discuss. In these cases, paraphrase the points made to ensure understanding then agree to delve into more detail later in a dedicated meeting. Strong emotion may also be symptomatic of underlying issues probably beyond the scope of the document being reviewed.

Creating an Innovative Environment

“Innovation” appears to have joined words like “synergy” in the buzzword nomenclature but is central to a software architect’s role, whether it be creating and driving either customer impacting or cost saving innovations. Unfortunately, many software architects and developers get frustrated when new ideas, features and improvements continuously are ignored, pushed to a nebulous future release or get anemic implementations. Innovation needs an environment and culture that allows it to succeed.
 
Many organizations feel that the greatest barrier to innovation is time. After all, many software development organizations are known for time specifically devoted to innovation. Google, for example, has “Google Time” where all employees spend a day a week working on their own projects. Atlassian, an oft quoted example, has “FedEx” days  where all the development staff spend 24 hours on their own projects and present them to the group afterwards. In both cases, this is where many new products or product features originate.
 
Unfortunately, organizations focusing solely on the time allocate it, expect developers to produce wonderful, insightful and customer worthy new products and features then scrap the exercise after initial disappointments or during the next crunch time. These organizations miss three important points.
 
First, these organizations put ideas created by developers (not just product management and support issues) into their product. This means that developers need the communication and exposure to customers or be customers themselves. Atlassian writes software for developers and they use their own products (or “Eat their own dogfood” as many people say) as does Google but most organizations lack this luxury. Software architects and developers need to create communication channels where they can get this information time efficiently. For example, start with requiring developers to use the product for basic tasks when they first join the team then periodically do manual testing in realistic environments (in addition to that done by QA) and have regular lunches with support.
 
Ideally, have architects and senior developers talk with customers directly, such as attending a support call or a conference. Development teams often receive a skewed view of the product consisting of support incidents and requirements from a few important, vocal customers only. Developers and architects ask different questions than non-technical people and hear the same answers with different ears. Even simple act of architects and developers watching customers use the product can reveal incorrect UI assumptions.
 
Build opt-in telemetry into the product, too. Two of the biggest issues with successful innovation are the self-supporting bias, where people over-estimate their own contributions to successes and under-estimate their part in failures, and the confirmation bias, where people interpret facts to support their own existing conclusions. Telemetry does not replace product management’s vision or support’s intuition but effective telemetry provides the most accurate information about what features are used, how often by when and by whom, particularly during A/B testing.
 
Second, innovative organizations have easily extensible or modifiable products, usually by people outside the development team responsible for that feature. No commercial software is ever perfect but the biggest hurdle to innovation execution is getting product quality to a manageable level and stopping endless “fire fighting”. This is particularly difficult on products with a small, maintenance staff or those with overly ambitious deadlines and usually requires extensive automated unit testing, refactoring, documentation and possibly new tools.
 
The team must also have room in the backlog (scheduled modifications or enhancements, some beyond the current release, for those unfamiliar with the agile term). Even brilliant ideas will have difficulty competing if the team has years of work outstanding or over-committed to customers. The team needs to focus, create a more effective and transparent prioritization mechanism and reign in the sales organization.
 
Third, innovation, when it occurs, must also be communicated and shared. If only the immediate manager knows about it, much of its value is lost. Similarly, innovation must be expected and rewarded with goals set, tracked and judged like other performance goals.
 
Software architects and development are not the only source of innovation, either. The best customer impacting innovations come from those that understand both the customer and the technology or product. Software architects are in a prime position to drive product and technology information to others that may innovate such as technical product management, professional services, support leads, architects from other products.
 
Software architects and developers must take responsibility to drive change throughout their organization. Organizations’ inertia often makes this a challenge but innovation does not spontaneously occur in a vacuum. As usual, start small, advertise successes and persist.