Random Acts of Architecture

Tales of an architect trying to bring order to the chaos that is modern information technology.

Category Archives: Software Development

Rebranding Corporate Politics

politicsThe term “corporate politics” conjures up images of sycophantic, self-serving behavior like boot-licking and backstabbing. However, to some IT professionals’ chagrin, we work with humans as much as computers. Dismissing humans is dismissing part of the job.

The best way to “play” corporate politics is solve big problems by doing things you enjoy and excel at.

“Big problems” means problems faced not just by your team but by your boss’s boss, your boss’s boss’s boss and so on. If you don’t know what they are, ask (easier than it sounds). Otherwise, attend all hands meetings, read industry literature or look at your leaders’ social network posts, particularly internal ones.

This is not just for those wanting promotions into management. Individual contributors still want better benefits and higher profile or challenging projects. These come easiest to those known to be providing value and not the strict meritocracy some IT professionals think they work in.

Start by solving small problems as side projects. Choose something impacting more than your own team and minimize others’ extra work. Build up to bigger problems once you have demonstrated ability and credibility.

You need not be the leader. Assisting others making an effort can be just as effective. You can own part of it or bask in the halo effect. If not, recognize those that are. This creates a culture of recognition that may recognize you in the future.

While some IT professionals solve big problems everyday, communicating and evangelizing their work “feels” wrong. This what salespeople do, not IT professionals. Many also think their work is not interesting.

Being successful requires people knowing what you do. This may be as simple as a short elevator chat, a brown bag talk or a post on the corporate social network. It also helps get early feedback and build a like-minded team. Others will be interested if you are working on the right things.

What about the potentially less savory aspects of corporate politics like work social events, sharing common interests with management, supporting corporate charities and so on? These are as much an art as a science. Focus on common goals and building trust, internally and externally. People like to deal with people at their level and contact builds familiarity.

However, this is no substitute for solving big problems. If you are delivering value, interactions with senior decision makers and IT professionals with similar goals should occur naturally. Build on that.

Be aware that problems change over time. Problems get solved by others. The market changes. Competitors come and go. Understanding organizational goals is an ongoing process.

Also realize decision makers are human. They make mistakes. They want to emphasize their achievements and not their failures, just like software developers’ fundamental attribute error bias for their own code and against others’.

However, if your organization makes decisions regularly on “political” grounds, leave. Culture is rarely changed from the ground up and many organizations are looking for good IT staff.

Ignoring the worse case scenario and IT professionals’ bias against self evangelism, the biggest problem with “corporate politics” is actually its name. The concepts behind “agile” and “technical debt” came into common usage once the correct metaphor was found. Corporate politics needs rebranding from something avoided to a tool that IT professionals use to advance themselves. It badly needs a dose of optimism and open mindedness.

Image credit: http://thebluediamondgallery.com/p/politics.html. Usage under CC BY-SA 3.0.

InfoSec: Not just for hackers

everybody-needs-a-hackerI recently read Troy Hunt’s blog post on careers in information security. Troy makes good points about information security as a potential career and the benefits of certifications like the Certified Ethical Hacker. Hackers are getting increasingly sophisticated, requiring specific knowledge to counter, and cryptography is hard. We need more information security specialists.

However, one criticism of the post, indeed the information security industry, is its implication hacking is the sole information security career path. This binary viewpoint – you are either a security person or not and there is only one “true” information security professional – does more harm than good.

Hacking is technology focused. However, security’s scope is not just technical. Information security needs people that can articulate security issue impact, potential solutions and their cost in terms non-security people can understand. This requires expertise and credibility in multiple disciplines from individual contributor level to management to boardrooms.

Security solutions are not just technical. We live in societies governed by laws. These can be standardized government security requirements as FedRAMP or IRAP. These can be contractual obligations like PCI-DSS, covering credit card transactions. These can hold organizations accountable, like mandatory breach disclosure legislation, or protect or privacy, like the European Union’s Data Protection laws. Effective legislation requires knowledge of both law and information security and the political nous to get it enacted.

We are also surrounded by financial systems. Financial systems to punish those with weak security and reward those with good security will only evolve if we (consumers and investors) value security more. Cyber insurance has potential. Cryptographic technologies like bitcoin and block chain algorithms are threatening to disrupt the financial sectors. Information security has and will continue to impact finance.

The list goes on. Law enforcement needs to identify, store and present cybercrime evidence to juries and prosecute under new and changing laws. Hospitals and doctors want to take advantage of electronic health records..

The security technology focus drives people away non-technology people. In a world crying out for diversity and collaboration, the last thing information security needs is people focusing solely inward on their own craft, reinforcing stereotypes of shady basement dwellers, and not on systems security enables.

Bringing this back to software, many organizations contract or hire in information security experts. Unfortunately, the OWASP Top 10 changed little from 2010 to 2013 and some say is unlikely to change in the 2016 call for data. According to the Microsoft Security Intelligence Report, around half of serious, industry wide problems are from applications. Developers make the same mistakes again and again.

Education is one solution – security literate developers will avoid or fix security issues themselves. A better solution is tools and libraries that are not vulnerable in the first place, moving security from being reactive to proactive. For example, using an Object-Relational Mapping library or parameterized queries instead of string substitution for writing SQL.

Unfortunately, security people often lack skills to contribute to development and design beyond security. While information security touches many areas, information security expertise is not development (or networking or architecture or DevOps) expertise.

Information security needs different perspectives to succeed. As Corey House, a Puralsight author like Troy Hunt says in his course Becoming an Outlier, one route to career success is specialization. Information security is a specialization for everyone to consider, not just hackers.

Image credit: https://www.flickr.com/photos/adulau/8442476626

Agile: It does not mean what you think it means

Agile Blog PostMany organizations adopt Agile development methodologies, or just Agile, for the right reasons. They want a software development methodology that welcomes change. They want something to give management better visibility on team progress and teams better visibility into the longer-term product plans. They want something to give motivated, competent individuals the opportunity to take more ownership and build value.

However, many organizations adopt agile for the wrong reasons. The bandwagon effect and general hype have made Agile a panacea. Unfortunately, Agile exacerbates many problems instead of fixing them.

The biggest problem Agile exacerbates is lack of trust and respect. Management needs to trust software developers to estimate accurately, cooperate with other team members and ensure non-functional requirements are met, such as automated testing and performance/scalability. Team members need to trust management to not use Agile to push more work onto the team, not to blame the team for poor or late management decisions, not to use the increased visibility for performance management and to promptly address hurdles the team encounters.

Agile only works if people are willing to change. For example, if software developers are unwilling to “waste” time on daily stand-ups, estimations, automated testing or code reviews then they are missing the point. While Agile, specifically Lean, allows decision making to be delayed to the last reasonable moment, decisions still must be made, communicated and supported.

Poor communication makes Agile harder. Technical team members often have difficulty translating technical details into something the business can understand. Product owners, usually senior decision makers, often have many demands on their time and a team of software developers can be culturally easy to deprioritize.

Remote team members, particularly in different time zones, make face-to-face communication harder. Technology can partially compensate, such as teleconferencing and Internet chat applications, but communication occurs as much in incidental conversations as it does in meetings.

Agile requires customer involvement. Scrum, for example, has the product owner role where a customer is actively involved in the process by identifying and prioritizing work and being available and accountable. Agile emphasizes regular delivery of working software to customers.

However, this works against the contractual nature of most outsourced software development. Some products, such as enterprise software, have six month or longer sales cycles and delivering software more frequently just burdens support. Some customers lack the expertise or desire to be actively involved. Successful agile requires an agile-capable customer.

Successful agile also requires an agile-capable team. Not all team members are proactive, Kaizen-embracing go-getters. Some people are happy to be led. Frequent iterations and regular delivery require deep and broad technical skills that some individuals or teams may lack. Team members need to be focused on value, not solving technical problems.

Without addressing these problems beforehand, Agile adoption causes a lot of pain and suffering. While the agile manifesto is easily read and understood, the underlying wisdom is less so. Agile is a tool that allows an organization hamstrung by poor process to excel.  Otherwise, Agile is often blamed when the real cause is the underlying culture or structure.

That said, Agile can be an effective organizational diagnostic tool. It shows problems that people often did not see or did not want to see. Therefore, it is important to clarify intentions and understanding before adopting Agile, as Inigo Montoya recommended.

Blog post image is a modified version of the image by Froztbyte – Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=1121735. 

Thanks to Adam LG Ring (@adamlgring) for the title.

The Future of Product Management in Enterprise Software

PM Challenges

In traditional enterprise software, product managers represent the customer to software development teams. They draw on industry experience to propose and prioritize features, usually as the scrum product owner. They liaise between development and the rest of the business, particularly marketing and sales.

However the product management role has three main challenges. The first challenge is new business models, particularly the move from on premise software to the cloud.

Cloud software presents increased non-functional challenges. For example, cloud systems are expected to scale up further than on premise software. The burden of service level agreements (SLAs), upgrades, patching and backup/restore is on the vendor and not the customer. Cloud systems are usually Internet accessible, missing the protection of an organization’s perimeter security measures like firewalls. Multi-tenant systems and resource pooling mean quality issues and downtime can affect many or all customers and not one.

Where product management previously dictated product direction, product management now share responsibility with architects (enterprise, solution, infrastructure and application) and business analysts (responsible for internal business processes). While IT commoditization has made IT cheaper and more accessible, capable technical people are still required, albeit with a different skill set. The increasing emphasis on integration, such as exposing web service APIs to third parties, and multiple platforms, such as mobile and tablet, exacerbates this.

However, this gives a product manager more opportunities. With the move from capital expenditure (initial purchase + support) to operational expenditure (monthly charge), many organizations are liberated from a fixed purchasing cycle. Unburdened by the three to six month customer upgrade period, releases can be as frequent as the development team can accommodate. Product managers can offload technical problems rather than shoulder the responsibility of the whole product.

The second challenge confronting product management is the increasing use of analytics and metrics. Many industries, such as media, have been using analytics and metrics for some time but many traditional enterprise market segments are only now getting access to accurate usage information. For example, few organizations previously consented to on premise products sending usage data back to the vendor.

Many product managers rely on experience or recent customer conversations to make decisions. However, the on-demand provisioning, self-service and customer customization (as opposed to vendor customization) aspects of cloud products reduce customer contact. Analytics and metrics can help fill this gap but it is a different type of customer relationship.

Moreover, the quality and quantity of decision-impacting data is increasing. Tools and expertise to extract useful information are becoming cheaper and more prevalent. Intuition and experience are always useful but will be more focused to what metrics to gather and interpreting them. Decisions will have a grounding (or at least a rationalization) in data. Making a decision solely on a “gut feeling” will be less acceptable.

Also note that good analytics and metrics can easily be applied to tactical problems like improving a user interface or prioritizing between two features based on actual use. It is harder to apply analytics and metrics to strategic questions like market direction or new product acceptance. This is where product management insight will be most valuable.

Metrics also help mitigate the “squeaky wheel” effect, where vocal customers monopolize product management time and the product backlog. For example, it is easier for a product manager to dissuade a customer demanding improvements to a certain feature with evidence the feature is rarely used.

The third challenge is the rapid change. Many product managers come from a business role. For example, an HR application product manager may previously been a manager in HR. Others have come from customer-facing technical roles like support, QA or sales engineering.

Unfortunately, in some industries previous industry experience is quickly outdated. While product management’s customer-focused perspective is vital, being removed from the industry can quickly atrophy understanding of customer processes and priorities. Exposure to their own software product(s) and current organization threatens to limit their thinking to what is probable, not what is ideal.

This is particularly important for product managers that come from customer-facing technical roles. They usually come to product management with specific product improvements in mind but, once these are in the product, may be at a loss.

Instead, a product manager needs to build ways to learn and predict industry changes – developing a new feature often takes months and the feature must be competitive when it is released, not just today. This could be key customers, industry contacts, thought leaders, peers at competitors or industry press. Building personal relationships, such as at conferences and industry meet-ups, is crucial.

Moreover, many information sources available to product management, like metrics or competitor analysis, are available to others in the software development team. It is these alternate information sources and relationships that will differentiate a good product manager.

Product management can no longer horde this information. For example, architects need accurate customer usage predictions for scalability planning and infrastructure provisioning. Management needs to allocate staff. Operations needs to ensure they are looking for likely threats or issues. By the time these are included in a release’s requirements or a sale occurs, it may be too late. Hording assumes product managers are the only ones making strategic decisions. Product managers are often poor conduits for technical requirements, too.

Meanwhile, product managers have less available time. They are dragged into sales opportunities to demonstrate or demand commitment, support calls to placate unhappy customers and marketing for feature commitments. Agile software development methodologies, like scrum, involve the product manager more frequently. This creates a “red queen” effect, where product managers spend a lot of their energy merely keeping pace with the industry, competition and own products.

Product management has always been a challenging role – often all responsibility and no authority. While many technical people incorrectly equate product knowledge with industry knowledge, prior experience and a customer perspective are no longer sufficient to be a good product manager. Successful product managers will adapt to the new business models (e.g. cloud) and leverage the new tools (e.g. analytics and metrics) to be more effective. In the future, those that rely on outdated experience and intuition are likely to fail while those that learn and adapt quickly will succeed.

Indeed, the end goal of product management is to impart customer perspective and industry knowledge. There will always be a place for a coordinating customer voice but it will lead by teaching, not a requirements document. Those involved in development should not need to consult product managers for every new feature or for a customer perspective – product management should have already taught them to think like a customer.

Treating Enterprise Software like Game Design

Mechanics, Dynamics and AestheticsIn 2005, Robin Hunicke, Marc LeBlanc and Robert Zubek wrote an academic paper titled “MDA: A Formal Approach to Game Design and Game Research“. It was and is an influential attempt at quantifying game design and theory.

The “MDA” acronym stands for “Mechanics, Dynamics and Aesthetics”. Mechanics refers to the algorithms and data structures that drive the game, such as how running characters animate or the arc of a character’s jump. Dynamics refers to the run-time interaction between the mechanics and the player, such as the pressing this button to jump or showing the character’s health as a bar at the top left of the screen. Aesthetics refers to how the player enjoys the game and what the player gets out of the game.

Aesthetics is often the hardest to describe to non-gamers. Some games may offer multiplayer where players enjoy the social and competitive aspects, like the an online game of “Call of Duty” or “Doom”. Other games offer an easy way to pass the time, like “Angry Birds” or “Candy Crush”. Others provide intense challenge, like Chess. Most games focus on a few core aesthetics and this is reflected by the difference audiences for each game.

As the paper points out, game designers and developers approach games from the mechanics side then dynamics, which hopefully impart the desired aesthetics. Game players, however, experience the aesthetics through the dynamics. Outside of statistic-heavy role-playing games and sports simulations, players rarely encounter the mechanics. Game designers should always keep aesthetics in mind, if possible.

Recognizing different layers and viewpoints gives game designers a nomenclature for understanding games’ inner workings and highlighting shortcomings. For example, a game aimed at a social aesthetic needs some form of multiplayer or social network integration. A game aimed at competition needs a visible score or ranking and consistent, well communicated rules.

How does this relate to enterprise software? The MDA framework layers have equivalents. Mechanics refers to the code and database queries software developers create along with business processes. Dynamics is unchanged, referring to user experience and interaction with the software. Aesthetics refers to the business value.

Also like game design, enterprise software customers and users approach the benefits the opposite way to software developers. Like game designers, software developers tend to start with the mechanics and work up to dynamics. Management aims for the aesthetics and, for those that directly use the software, the dynamics. While some software developers may enjoy the technical challenges of enterprise software, they must not lose focus of the business value.

As with any classification or taxonomy, the MDA framework provides a way of dissecting and comparing different applications. For example, two applications can aim for the same aesthetic (business benefit) but use different dynamics (user experiences). One might be a touch-heavy mobile application. One might be a web site storing its data in the cloud.

The MDA framework can point out where a business need (aesthetic) is not supported through user experience (dynamics) or a user experience does not relate to any of the defined business needs. Software developers and architects could also create a reusable mapping of dynamics to aesthetics or mechanics to aesthetics, like linking tactics to quality attributes.

Software developers have traditionally split systems into different layers or components. The aim was to improve maintainability by localizing the effects of changes. However, the MDA framework reminds us that changes in one layer can and do affect other layers. For example, a database query change (mechanics) may affect the results shown in the UI (dynamics) and the business value (aesthetics). Conversely, new or different aesthetics may require changes to both dynamics and mechanics.

The MDA framework also reminds us of requirement compartmentalization. For example, problems occur when management or business users specify dynamics (user experience) instead of aesthetics (business requirements). Management and business users should have opinions and input but user experience designers are experts.

With the increasing popularity of IT consumerization and gamification, game design has already encroached on enterprise software. The MDA framework goes deeper by identifying things important to the target audience (whether they be games or management) and a structured way of providing them. The fact that a closely related field has also produced something similar to existing software architecture and design best practices reinforces them.

Indeed, despite the fact that games are also created with limited time and resource constraints, enterprise software has a poor record of user experience design. There is probably a lot more game designers can teach software developers about improving enterprise software, considering games succeed or fail purely on their ability to satisfy users.

Unit Testing: The 20/70/0 Rule

20-70-0 Rule

Automated unit testing, one of the most important foundations of software quality, is still a struggle for many software development teams. Justifying the extra upfront time to business is difficult, particularly when the team is under deadline or resource pressure. Many teams give up when confronted by huge amounts of untested, untestable legacy code. However, avoiding or delaying unit testing hurts everyone.

Many misunderstand automated unit testing, making the message inconsistent or less convincing. For example, automated unit tests do not initially reduce the number or severity of code defects. Good developers should already manually test their code thoroughly, stepping through it in a debugger where possible. Good developers also manually check error conditions and corner cases.

Many concerns are also unfounded. For example, automated unit tests do not replace QA (testers). QA check software developers’ work and test at the functional level. Their different perspective can help write better automated unit tests, too.

Many complain about brittle unit tests only to find brittle “unit tests” are usually functional or integration tests, such as calling web services on external systems or accessing a shared database. Since these are not segregated, the unpredictable actions of others cause tests to fail.

Indeed, the biggest barrier to automated unit testing is software design. If a method or function cannot be unit tested, the design is incorrect. For systems outside the software development team’s control, see Michael Feather’s work on legacy code. Testable code tends to be better designed code, too.

The main benefit of automated unit tests is unit tests capture the expected behavior of a single unit of code, such as a method or function. These tests can be repeated quickly and regularly with little manual effort, identifying when code changes, refactoring or experiments break the expected behavior.

Software developers also forget important details as they move to other features or projects. Capturing expectations as automated unit tests retains this experience and knowledge.

Nothing mentioned above is novel. However, questions remain once agreement to add automated unit tests is reached. For example, how much unit testing do developers add? How much extra time is needed? How do you explain this to non-technical stakeholders? The 20/70/0 rule answers these questions:

First, spend 20% of development time writing automated unit tests. A day’s worth of testing each week is a good compromise. This is part of the development task, not extra effort. Otherwise non-technical stakeholders will demand skipping it when under pressure.

Second, aim for 70% code coverage. This excludes third-party or generated code, so make sure code coverage tools can exclude this. Interestingly, technical people tend to think this is high, especially if no automated unit tests exist. Less technical people ask why the remaining 30% cannot be covered.

Third, ensure 0 failing tests. Running automated unit tests after an automated build is a critical part of continuous integration. Fix failing tests immediately.

The first rule, 20% development time on tests, tells project managers and stakeholders the extra time to add initially. It also allows project managers to compare the up front costs with time savings later (ideally greater than 20%).

The second rule, 70% code coverage, tells developers what the team expects, particularly when code reviews highlight missing or poor unit tests. In an agile process, automated unit tests are part of “done” for development tasks.

Code coverage is an imperfect metric and heavily debated. Ideally, the team should target functional coverage. Behavior Driven Development (BDD) is one option. However, for a team without unit tests or a superior metric, code coverage is unambiguous, automatable and easy explained to less technical people.

The third rule, 0 failing tests, reinforces that quality is critical, especially to less technical people once again.

Software developers often get caught up in technical debate. Unit tests and quality are no different. However, projects can rarely wait for perfect understanding. The 20/70/0 rule is unambiguous and understandable, even to less technical people. Attaining it or, more specifically, the quality goal it represents is still a challenge but attaining it is now about metrics instead of gut feel and hand waving.

What makes a “good” software developer?

Good Software DeveloperSpend any with software developers and the question of whether a software developer is “good” has invariably arisen. Want to hire a new software developer? Want to promote someone to a lead developer position? Need someone to refactor or redevelop a piece of code? Need someone to work with a different team? You always want a “good” software developer. The challenge is “good” has many different meanings and few can be more specific. What does “good software developer” actually mean?

The first skill of a “good” software developer is delivery: writing code quickly with as few bugs as possible. Some software developers are faster than others, some create slightly more bugs than others but the balance should be there.

The second skill is product knowledge: knowing where to fix or improve something, the rationale for the current design and impacts of any change – often the hardest parts of software development. This includes the technical ecosystem (what products it integrates or interacts with and how).

This list excludes skills like communication and building a network of contacts. They are important but are not software development specific.

This list also excludes business knowledge because most software developer “business knowledge” is just product knowledge. Software developers with business knowledge exist and some product owners lack the business knowledge they should have. However, few software developers can step into a non-technical role and perform as well or better than someone hired specifically for the role.

Both delivery and product knowledge provide strong business value. However, does that mean the best software developers are those that have worked on the product the longest? Is “good” synonymous with “experience”?

Similarly, with little design up front, regular pivots, increased visibility and pressure to deliver, Agile practices tend to encourage finding the minimal effort to solve a given problem. The constant focus on delivery may dissuade developers from stepping out of their immediate task and learning how to improve themselves, their process and their team.

Consequently, there are other skills to consider. For example, optimizers examine a process or product, identify its deficiencies and make concrete improvements. Epitomized by Sam Saffron’s recent post, these developers excel at improving nonfunctional aspects (e.g. performance, scalability, security, usability) or automation (e.g. build process). These are the developers constantly pushing for new tools and libraries because they are better, not just because they are shiny and new.

Software developers may also focus on good design. For new products or features, designers focus on how the software works as much as what it does and usually head for the whiteboard first and not an IDE. For established products, designers seek technical debt, fix it and move to the next problem.

Software developers may also be quality-focused. Quality-focused developers consider code incomplete until it is both testable and has automated tests. They often espouse the benefits of Test Driven Development (TDD) and code coverage (an imperfect but useful metric) not necessarily for themselves but as easy ways to encourage (or require) others to focus on quality.

This list is also hardly conclusive. Some software developers may focus on security, others on user interface design and so on.

Optimizers, designers and quality-focused software developers are disruptive and may detract from delivery by focusing too much on improvement. For example, they may spend too much time and effort on switching tools, redesign and over testing. The disruption may extend outside the development team, too. For example, new processes may impact other areas of the business and purchasing new tools may put pressure on budgets.

However, these disruptive developers can drive positive change and be a source of innovation. Without them, software developers would still be writing code in assembler using waterfall processes on computers that filled buildings.

The first consideration is focusing disruptive software developers on important problems, often difficult for managers not used to software developers driving initiatives from the ground up. Changes must be driven to completion and implementation and not just the initial research and proof of concept. Managers also need to make sure all team members can follow and benefit from the improvements, not just team members driving changes.

The second consideration is managing expectations. A disruptive developer brought into the team may expect or be expected to drive change. Setting boundaries and goals beforehand can avoid problems before they occur.

However, having the skills mentioned above is not necessarily the best answer to “What makes a good software developer?”. Yes, “good” software developers deliver and amass product knowledge. Some software developers are disruptive and those that drive improvements while delivering may be “great” software developers. However, these are goals. How does a software developer become “good” or “great”?

Great software developers strive for continual improvement. If a software developer truly wants to improve, reads widely, experiments and keeps what works then even a “poor” software developer will become “good” or “great” eventually.

Continual improvement is often mentioned when discussing Agile development processes but it also applies down to the individual level, such as scripts, key bindings, macros or IDE add-ins. It also applies to the organization level, although this is harder for individual software developers to influence.

To look at it another way, great software developers focus on goals and motivations , not just rituals. A good developer may follow Test Driven Development or the SOLID principles but a great developer will know the benefits they bring and when to bend the rules. This is also why great developers learn new languages quickly and apply concepts from one in others.

That said, choosing a great developer is not always the best choice for every role or task. Great developers expect more from an organization. This is not just a higher salary and benefits. They expect to make more of an impact but not every organization can provide or allow that.

The key is to match the software developer to the task, not just find a “good” or even “great” software developer. This requires understanding what the team needs and how developers improve or lose their edge over time, which requires “good” management. However, that is a whole other blog post.

The Software Development Employment Jungle

When I first looked for software development jobs after leaving university, most employers were looking for candidates with the right attitude, right aptitude and good marks. However, as the years passed and my positions were increasingly senior, I found employers wanted the right person for the role, not just someone with the right skills. Having recently left Symantec (where I spent almost 13 years), waded through the employment jungle (as one of my peers called it) and started a new job, I wanted to capture my experiences and observations. Hopefully this applies beyond just software architect and software development roles in Sydney, Australia.

Recruiters

Recruiters

Many have negative opinions of recruiters and employment agencies. They try to provide candidates from the limited pool they can attract for clients whose requirements they sometimes do not fully understand. However, such an opinion is unconstructive and unfair. Many recruiters work long hours to fill roles whose remuneration sometimes dwarfs the recruiter’s in a rapidly changing technology and business landscape. Instead, consider the following:

Recruiters are paid by employers so candidates are the product, not the customer. Recruiters are very interested in candidates while applying and going through interviews and work hard to sell positions to candidates  and candidates  to employers. Rejected candidates  are rarely worth recruiters’ time (“no news” is generally “bad news”) while successful candidates are pursued, particularly for recurring contract roles.

Recruiters find candidates for positions not positions for candidates. Candidates are responsible for finding their next job, not recruiters. Candidates need to market themselves (such as joining job search sites, talking to hiring managers, attending conferences and user groups), generate as many good leads as possible (such as applying for jobs and distributing their resume), following up the leads, understand the employer needs (such as emphasizing particular skills on a customized resume) then sell themselves in interviews. This process changes over time, as the candidate learns and the market changes, and recruiters and employment agencies are one part of the process.

Preparation

Preparation

Start working toward the next job or promotion now, even for those happy with their current position. Why now? Because you cannot control when new job opportunities will arise or when your current position may end or change for the worse. Starting now means you will be prepared.

For example, identify activities that look good on a resume or will help for that next promotion, even if they are not immediately appealing. Plan conversations in advance, such as introducing yourself to peers at networking events, thinking of an insightful question for the boss’s boss and answering the inevitable “Why do you think you would be the right person for X?”

Start networking with people that could help you find a job in the future, such as peers and managers within the industry or related industries. Many are put off by the time commitments others recommend but simply reaching out, getting contact details and occasionally (once every few months) commenting on a post or tweet is usually more than sufficient. Aim for mutually beneficial relationships but realize peoples’ needs change overtime. Beyond getting the next role, it also gives context across the industry, identifies the skill level of peers, the challenges they solve and gaps in your own skills.

Create an online presence. It can be as simple as a LinkedIn profile that lists your employment history, important projects and key skills (and is a great way to keep your resume up-to-date). It can be as complex as a lengthy blog, strong social networking presence, a high Stack Overflow reputation and contributions to multiple open source projects. Start small, work up and do not be afraid to experiment.

Some fear others will infer the worst from making achievements visible – Dunning-Kruger is rife in software development – but being able to answer an interview question thoroughly because it was the topic of your recent blog post or pointing to your project on Git Hub when asked about a library or framework is invaluable. Unless you are aiming for a thought leadership position, most people simply do not care enough to discover other’s mistakes and having a demonstrable history is better than not, all other things being equal.

Some are deterred by the implicit pressure to maintain a blog or social networking activity. While thought leadership is may be best maintained by a steady stream of content, a different strategy is to post or blog fewer but better articles and content. A small number of insightful, relevant articles can be more useful than a regular stream of retweets, for example, because the articles show original work. This reduces the time commitment and interviewers will see better articles when they browse your content, too.

Be open minded. What you think you need may not be what you actually need and the people you think you should be talking to may not be who you should actually be talking to. Talking to those outside software development can be insightful and there are always things to learn and different perspectives to respect. Do not neglect soft skills, either.

Focus the resume. Customize it for the role to emphasize relevant skills. Write a cover letter but do not expect anyone to read it.

Interviews

Role

Employers and interviewers come with their own preconceptions. They have their own backgrounds and experiences, their immediate need for a new hire and their vision of a suitable candidate for the role. This leads many interviewers to grill the candidate about the candidate’s fit for the interviewer’s idea of the role, to see if the candidate “fits in the box”.

The “box” approach works well for software development roles where the programming languages and frameworks are known but interviewer skills vary. Bad technical interviews fixate on minutiae under the misplaced assumption good software developers will have touched those areas. Good technical interviews involve writing code, explaining technical concepts or defending decisions.

However, candidates with broad or unusual experiences may not easily fit the box, encouraging interviewers to label candidates (“Are you an X or Y? You cannot be both!”). Finding a candidate that fits a box is less successful the more senior the role gets, particularly for leadership roles like management and software architect positions, where strategic and broader thinking is required, because people the box approach prefers are less likely to have new ideas and different perspectives.

Understanding the interview strategy can help the candidate identify the sincerity of the role. For example, is the role being advertised as a technical leadership role, like a software architect position, but they are conducting a “box” style interview? This hints at an impressive title being used to attract people for an otherwise straight coding job. Alternatively, a software development role containing lots of open questions may hint at a more senior role or a higher expected standard of candidate.

Moreover, what many interviewers forget is interviews are as much about the interviewer as the candidate. Even without asking questions, the candidate learns about the types of problems the interviewer thinks are important, their priorities and the interviewer’s communication skills. Is the interviewer asking questions about challenges you are interested in or have experience with? Is it something you enjoy talking about? Are they happy with high level answers or do they want detail? How did they react to your last answer? Did they ask for clarification or move to the next question on the list?

The usual recommendations about interview preparation apply. For software development and particularly software architect roles, understand your last few projects, the important design decisions and why they were made. Have scenarios prepared for behavioral questions on leadership, dealing with difficult stakeholders or working under pressure. Focus on the interviewer’s business first and talk about the candidate’s benefits at the end. Good questions for the interviewer include questions about how to be successful in this role and what challenges do they expect to face but do not be afraid to ask questions throughout they interview if they are relevant at the time.

Final Thoughts

Much is written about things candidates can do to improve their chances of finding a job, like resume writing or interview practice. However, a key part is patience. All the hard work will help you if and only if the job you want is available – the Australian IT employment market hibernates over summer, for example. Some people often need a new job due to financial or other pressures and, by all means, adapt to the market’s needs  but getting a new role does not have to be a question of choosing what to sacrifice. Be good at what you do, have faith in yourself and the jungle will not seem so bad.

Intellectual Property Ownership

Whenever anyone involved in intellectual property starts a new job, particularly in software development, the employment contract usually includes a lengthy intellectual property agreement where the new hire assigns ownership of all intellectual property created over to the employer. Many new employees balk at this, concerned that the employer will claim any side projects they are working on and with recent court battles over software patents worth billions of dollars, for example, intellectual property ownership is a complex area worth exploring.

Employment contracts are usually “work for hire” contacts. Although the legal requirements and obligations vary from country to country, it usually means the employer owns any works created instead of the employee, in return for payment or salary. Few software developers dispute this. However, unlike authors and artists usually contracted for a specific work, developers are often contracted for all intellectual property created during their employment, irrespective of whether it is performed during working hours on work equipment.

Software developers are increasingly working on side projects like open source software or apps for mobile or tablet app stores. Many developers code in the spare time, whether it be tweaking the website for a friend’s business or experimenting with new libraries or languages. It is also increasingly common for students to bolster their resume this way or for software developers to teach themselves new languages or libraries.

This differs from traditional “working under the table” or “moonlighting” in that much of the work is not paid. The software developer may want to capitalize on the work in the future but usually just does not want others intruding or demanding ownership of the work, akin to a lawyer doing “pro bono” work. Software developers are also producing assets other than code. For example, software patents produced during software development can be more valuable than the code.

The barrier of entry for software development has never been lower. Thirty years ago, software development was difficult, usually performed on expensive, centralized computers and proprietary software. This is a far cry from today where one can create complex websites using free tools running on free operating systems hosted on commodity priced servers. Compare that to chemists, physicists or engineers that may require thousands or millions of dollars of equipment and dedicated teams of support staff to perform their research and yet more to monetize it.

Indeed, the problem with software patents and intellectual property ownership in software development is development only part of the cost. There is the marketing and sales required to turn products into revenue, for example, and the IT infrastructure, HR and accounting structures required to support all of this.

Patents are similar. Beyond software development, legal expertise to file patents is required along with the time and resources needed to find and deal with infringements. Patents are also often cross licensed, either earning additional revenue or allowing access to other organizations’ patents. Patents may also become more valuable over time as products they are used in become more widespread.

The word “ownership” also carries many mis- and preconceptions. If an employed software developer (“inventor”) wants to own all or part of his or her inventions, what does this mean? Does the inventor want royalties, like an actor may receive for a movie? Does the inventor want the option to use it in their own work, possibly for a job at a competitor? What about an open source project the inventor contributes to in his or her spare time? Does the inventor want the option to stop others using it, like competitors or those the inventor disagrees with? Will the developer help fund the sales, marketing and legal infrastructures required?

Even if those in software development can make a case for increased “ownership” of their products, it is not in employers’ interest to allow this. The creative process cannot be constrained to occur within work hours or on work equipment – many have  inspiration when asleep, exercising or in the shower – and increasingly flexible working arrangements further blur the distinction. Work for hire contracts are also well understood and widespread, making them lower risk.

Some would argue software development is like a painting selling and reselling for increasing amounts but the original painter seeing none of the profit.  However, a better analogy is performers selling music. Do they go through a record label where they get greater exposure and marketing but sacrifice income or do they produce the songs themselves and sell it through iTunes, where they can make a greater cut but a much reduced sales volume?

Many employers are also quite reasonable. If the idea is unrelated to current or likely projects and the employee is not going to make much money, pursuing it is not worth the expense. If the organization files patents, most reward developers with bonuses for doing so. Other employers take the opposite position and talking to the employer first before producing anything important is prudent.

It will be interesting to see what the future holds. Younger generations, those that have grown up with social networking, are used to sharing their lives on social media and regularly blur the lines between professional and social. With software development tools more accessible than ever and collaborative source code repositories like github gaining in popularity, will developers from younger generations look at coding the same way? While they may have different politics from their GNU and GPL espousing forefathers, will they see “social coding” or “social software development” as an obvious direction? If so, what compromises will be made?

Software Development: An Overtime Culture

Software development is rife with overtime. Driven by passion and perfectionism, many developers throw everything into it, reveling in a culture of coding in darkened rooms late at night. Coupled with customers pushing for more or larger deliverables with fewer staff and a legal exemption from overtime pay in many circumstances, developers often shoulder the burden. Unfortunately, consistently working late nights and weekends is not sustainable but many seem unable to break free of the rut.

To put this in perspective, overtime is not unique to software development or even IT – just ask accountants around the end of financial year or salespeople in danger of missing their quota. There is also nothing wrong with flexible working hours and arrangements. Nevertheless, a work/life balance is important particularly for those with families or other commitments outside work.

It is easy to blame overtime on poor management. While true, saying “manage better” does not solve the problem. Indeed, there are many causes of software development overtime, including: (1) poor estimation or underestimation, (2) too few or under-skilled developers, (3) perfectionism, (4) redoing work due to past mistakes or (5) changing requirements/poor communication with other parts of the business. Specifically:

Poor estimation or underestimation: Few cling to the Newtonian beliefs that estimation is merely substituting initial measurements into well understood formulas to predict the delivery date.  The issue is one of negotiation – experienced software developers often know how much features will really cost in time and resources but software developers are often outclassed by customers or management used to arguing their case and, once agreed, deadlines are traditionally immutable, like a salesperson’s quota.

Too few staff or under-skilled developers: No development team is has “enough” developers and with the current trend of outsourcing and offshoring, developers working on project may be under-skilled or unfamiliar with the business problem. However, “you have to go to war with the army you have” as they say but the Dunning-Kruger effect and poor planning rarely take this (and Murphy’s Law) into account.

Perfectionism: Unfortunately, software developers can be their own worst enemies. Well architected, readable, tested software is the goal but “production quality code” is really a tautology – real developers ship. Include time for refactoring but time box it. Include time for design but realize it will change. Include time for testing but test important features first.

Correcting development mistakes: Unfortunately, mistakes happen but software developers must take responsibility for their own mistakes. To put this in context, at least development has the option of working late to fix their mistakes, unlike sales or management.

Changing requirements or poor communication: Many developers think of this as an “us versus them” situation – the “villainous” business “demands” changes and the developers are “helpless victims.” Instead, work on negotiation and trade-offs. After all, if the project is not delivered, everyone suffers, not just development. Make it a business problem or a business risk.

Note the steady addition of new requirements is a smell, possibly indicating the release cycle is too long. If possible, consider multiple smaller releases rather than a single large one.

There is no silver bullet to prevent overtime. Software development is a profession and there will always be work to do. However, it is possible to reduce it. Apart from focusing on soft skills, developers need to:

  1. Improve negotiation skills. For example, treat estimates as immutable. Trade features rather than agree to reduce estimates. Otherwise the estimates lose credibility. Treat everything else (deadlines, delivery mechanisms, support, documentation, maintenance releases and so on)  as negotiable.
  2. Improve estimation. Each development team faces its own challenges in this area but a few suggestions include:
    1. Improve prioritization. Many customers want everything but estimating the revenue or customers for each feature (or loss if not implemented) is usually the best prioritization strategy. Deprioritize features that have no customer impact (the lean software development definition of waste), too.
    2. Estimate with 10-20% fewer resources and less time. Customers often spot padded estimates (reducing their credibility leading them to press for reductions) but reducing the resources and time instead is usually seen as prudent.
    3. Present estimates using the least accurate unit of measure. Instead of saying “172.3 hours”, say “about 1 month”. Defer more detailed estimates until the problem is better understood later in the project.
  3. Track the previous work estimates against actual time and resources. Use that to support current estimates and improve future estimation and negotiation.
  4. Focus on delivering. Many developers love learning new things and striving for the ultimate solution. There is nothing wrong with this but a critical project is usually not the place for it. Temper or time box this experimentation and perfectionism.

Agile development methodologies might also be useful but suggest them tentatively. Many have presented ill-defined “agile development” practices as a panacea, thus some consider the term “agile” a less than credible buzzword. Agile development methodologies require a wider organizational changelike the appointment and recognition of single product owners in scrum, and may require more trust before allowing this.

The challenge with much of this is not sacrificing status of development within the organization or being seen as an impediment. The business may expect overtime and changing this perception can be hard. Metrics that demonstrate how things get worse as overtime increases, like as lines of code produced and bugs introduced, might help particularly against those that espouse the “pressure makes diamonds” fallacy.

Reducing it requires developers to recognize their part in the problem. There is nothing wrong with developers striving to create the best solution and putting in the extra effort when they want to but required and expected overtime is problematic. Ignoring extreme cases like those described by EA_Spouse, the best solution is usually a mutually constructive one, particularly with developers improving their negotiation skills.

%d bloggers like this: