Processes Differentiate World-Class Software Engineering

By Ed Carroll, executive, Agilis Solutions

Software companies – with their continuous product release cycles, large IT programs and ever-changing business demands – are dependent upon continuous and repetitive project success. However, we continue to hold steady with an industry statistic showing that upwards of 70% of large software projects fail to meet their objectives by more than 35% (35% over budget, 35% off schedule, etc.). This article will compare the implementation of project management processes, engineering standards and continuous process improvement as ways to plan for success (repetitively). Click here to read more about these statistics.

When is a process-driven approach better?
When is a process-driven approach to developing consistently high-quality software better than a chaotic, ad hoc, hero-driven approach? When there is a need for a software application with a predictable delivery date, at a predictable cost, and of consistently high quality, repetitively. The executive in a large IT shop who allows development projects to continue with a chaotic, ad hoc, hero-driven approach is saying it is okay to have frequent failures. That would be the 70% of projects that overrun cost, schedule or other key objectives by more than 35%. By the same measure, the executive at a software company who allows development projects to continue with a chaotic, ad hoc, hero-driven approach is saying it is okay to allow 70% of his product releases to ship with significant defects, or to overrun key objectives by 35%.

How to repeat success
There are well-proven steps toward achieving repeatedly successful projects – through the use of well-published software engineering methodologies. It is not as important which specific method (Agile, RUP, or other) is used, as it is to establish processes in a way that enables feedback and encourages continuous improvement and success.

Begin by adopting a project management method, train all project managers and start using the approach (on all new projects – retrofitting only causes more chaos). The objective is to build repetition with the method before tailoring the method too much. A key to success will be to build into the chosen methodology some measurable objectives that can be compared from one project to the next, in order to define success.

Then build into the chosen methodology a feedback loop which will trigger analysis and process improvements where needed. This feedback -> analysis -> process improvement loop can be the mechanism that drives tailoring of the overall methodology. However, I recommend that tailoring of the methodology be disallowed by individual project managers until the overall methodology is well established across the development teams. Be prepared for the establishment step to take a while to happen, perhaps even a couple of years.

Communication processes
The number one reason for projects to fail is due to poor communication. And communication only becomes more difficult and more complex as projects get larger or become multiple projects, and use geographically dispersed teams. For this reason, improvement in communication is a primary objective of the chosen project management methodology, throughout the software development lifecycle (SDLC). Ironically, very few software methodology publications outline even the most basic communication practices. The assumption in most methods is that everyone on the development team is sitting around the same table, when in reality the larger the company the more likely there are team members geographically dispersed, sometimes anywhere in the world.

Several human communication studies have identified that as much as 55% of our communicated cues are picked up through watching body language, 38% through eye contact, and only 7% aurally. Therefore, if a team relies on only a telephone conference call to communicate requirements to an offshore engineering team in India as their primary communication mode, how much of the message is being missed?

Allow me to add this process to whatever software methodology has been chosen: “Communicate everything in triplicate.” In a face-to-face meeting one has already experienced visual and aural communication modes, just add written action items. When sending instructions via email, follow-up with a phone call (aural), and then maybe answer any clarifying questions with an IM chat. If a telephone conference is the standard tool (this includes Skype or other VoIP tools), add follow-up email and IM chat until it is clear or obvious to everyone involved that the topic of interest is well understood and all of the action items are being handled. Good communication on a software development project requires aggressive participation by all members. For these reasons, a better tool for long-distance communication is videoconferencing. Even if the video is bad or cumbersome, there will be far more effective communication happening than with a telephone conference or email exchange. And of course, stay with the “triplicate” theme by following up the videoconference with a confirmation telephone call and/or email, or IM chat.

There is no such thing as too much communication on a large software development project. Left to their own devices, most people will assume the best and take the easy way out. Setting out a process like this helps to ensure that every team member remembers that miscommunication is easy, and easy is not necessarily the objective.

Defined engineering standards
Once project management processes enable repeatable project success, then the time is right to standardize engineering. Standardizing engineering is a little different in that how one codes or even designs an application is often constrained somewhat by the software development tool kit (SDK) used. Defining engineering standards, then, is not so much about what steps – or processes – one should take in a project (as it is in repeatable project management methodologies) as it is about defining commonalities between releases. For example, standardizing the user interface, product architecture and code objects from one release to the next or between multiple products can have a big impact on the success of a software company or IT operation.

The objective in project management methods is to clarify communication, while the objective in defined engineering standards is to gain efficiencies across operations – building on a foundation of clear communication and repeatable project processes. For example, a common user interface should ease user acceptance, installation and training – improving sales and decreasing customer support cost. A standard technology architecture and reusable code library should ease both application development and technical support – constraining design variations and the probabilities that increase defects. A standard architecture can be as broad as requiring that all applications be developed in the same programming language, but the more detailed the standards can be toward true reusable code objects, the better should be the quality of the code base.

Reusable code from a trusted library can significantly reduce the defect rate. Fewer defects mean less time (and cost) serving customer complaints. Fewer customer complaints mean lower customer support (cost) requirements, which in turn means more resources for new development. There are programming and engineering teams that spend upwards of 80% of their available development time simply doing maintenance activity on their existing applications. It should be no surprise that those applications are not moving the company ahead of the competition.

Standard application architecture and reusable code across applications should enable more flexible use of resources across those applications, both on maintenance and new development projects. In theory (or as an objective), a coding problem should be solvable by any engineer on the team, given an understanding of the problem domain and the engineering standards of the organization. This is possible because all code should be in the same language – reusing proven objects, following the same design techniques, integration model and code-commenting standards, etc. (An exception is granted for the differences between such things as Java vs. database development.)

Managing what is measurable
There is an old axiom that goes, “You cannot manage what you cannot measure.” In software development, that means metrics. The first premise of metrics is that team members are following procedures and abiding by standards. Many managers new to metrics start and stop with metrics that simply identify compliance to rules.

To me, measuring whether an engineering team is following the rules is the meanest level of metrics management and provides the least value for the team's efforts to gather the needed data (did you ever see a manager gather metrics data?). Measuring metrics of project management process compliance and engineering standards use is of most value to highly trained and typically highly motivated employees as a tool for efficiency – to provide the data necessary in decisions about tailoring the SDLC methodology. I stated previously in this article to allow considerable time to pass (even a couple of years) before allowing project teams to deviate from established SDLC processes. My thinking is that until the processes are well ingrained and the standards truly standard throughout the organization’s processes and implemented applications, the teams do not have the foundation data necessary from which to base a tailoring decision. Without this data, decisions on what processes or standards to tailor are little more than gut-feel decisions. Unfortunately, gut-feel decisions are the basis for most of the original estimates for those 70% failed projects. Gut-feel decisions are not representative of systematic engineering.

To gain a significant return on investment in gathering metrics, gather the kind of data that can provide meaningful comparisons between processes, teams, products, and methods. For example, as trade-offs are made and processes tailored, measure the effect of the tailoring by comparing how the project succeeded against other non-tailored projects. Comparisons can be made between and across projects in defect rates, team performance, individual processes (affected and not affected), etc. Then ask the tough questions and ferret out what process or lack of process makes some teams, products, projects, etc. better than others. This is the start of process improvement.

Metrics can also provide a significant competitive advantage. For example, a large historical project database can be used as the effective basis for very accurate future project estimating. As described in my previous article (Estimating Software) project metrics have been gathered for over seven years from what is a growing database from now hundreds of projects per year. This model provides accurate project estimates 95% of the time across these hundreds of projects.

Data gathered in this way provides incredible value back to the business. Can you imagine having accurate project estimates before the annual budget process begins? Can you imagine having rock-solid product release dates, every time? Dates that not only you, but the entire company (marketing, sales, operations, the CEO) can depend upon?

Optimizing what is managed
There is another old axiom that I really like, “Attack the process, not the person.” If metrics are used to judge personnel deficiency, a real risk is created that will drive employees either away from the organization, or away from the metrics program – either way renders the metrics program moot. Real value in any metrics program is missed if the results of the data gathering and analyses are not fed back into the methodology for improvements. Notice that I wrote ‘methodology’, not ‘person.’ Remember, “Attack the process, not the person.”

A rule of thumb that I like to use in order to gather truly effective metrics is to have those who will benefit most from the metrics identify what metrics to gather. For example, have the business and systems analysts define the data to be gathered in order to make decisions on how to improve the requirements-definition processes. Have the design engineers define the data to be gathered in order to make decisions on how to improve the Test First processes within each sprint. The people who do the work are much closer to what is right or wrong with a process. Having them define the data to be gathered not only collects the right data, but it also gives them a sense of ownership in the improvement efforts.

In order to make the correct decision about improving a process, it is essential to empower those who work the process to define what data to gather. This empowerment is a fundamental idea upon which Continuous Process Improvement (CPI) is based. An engineering team that continuously improves will very quickly become a world-class engineering organization, and will continuously outperform the competition.

Conclusion
There are many talented engineers, some even destined to become heroes. Would you bet your business on the proclamation from a hero that he will never quit, never get hit by a bus, or will always come through in a pinch? Or would you rather build or partner with a true world-class engineering organization? Software engineering at the world-class level starts with established and repeatable project management processes. These repeatable processes are the foundation for engineering standards that provide the organization with the efficiencies necessary to overcome any competition. World-class organizations measure their processes, progress and productivity through copious metrics at all levels of the organization. And what actually differentiates these true world-class engineering teams is their continuous push to improve their processes.

About the author
Ed Carroll has been building software products for more than 20 years, with particular expertise in automating economic analyses, decision support and supply-chain management processes. He has provided strategic technology leadership in roles such as the vice president of engineering for Egghead.com, director of technology at Nike and director of software engineering at Boeing. He can be reached at EdCarroll@AgilisSolutions.com.