I saw this tweet from Scott Ambler earlier today:

The link in the tweet leads to the July 24, 2006 edition of Dr. Dobb’s Agile Journal, in which Scott writes:

In astronomy there is the concept of red-shifting and blue-shifting: red shifting occurs when a body is moving away from you (the wavelength of the light increases) and blue-shifting occurs when a body is moving towards you (the wavelength of the light decreases). This is useful input for determining the speed and direction of something. In software development we have green shifting which occurs when people rework the information contained in status reports to make them more politically palatable to their manager.

A few years ago I was involved with a seriously challenged project. In my weekly status report to the project manager I would very bluntly list all of the problems we were experiencing. Strangely, even though myself and several others were clearly documenting the problems, all of which were out of our control, nothing ever seemed to happen. After a few weeks of this I ran into the CIO and she congratulated me on how well the project was going. I was surprised, and told her that the project was in serious trouble and summarized the critical issues for her. It was news to her.

After a bit of investigation we discovered that although myself and team members had been reporting the serious political challenges that the project team faced, when our project manager summarized our reports he decided to put a better spin on it and reported that the team found the project challenging. His manager in turn reworked it in his report that the team enjoyed the challenges it faced. This was the report which the CIO received.

The problem is that the project status “green shifted” as it rose through the various levels of management. The people on the ground were very clearly reporting a red project status, our project manager wanted to play it safe so reported yellow status, and his manager was more political yet and reported green status.

This “green shift” isn’t just a problem in the software world. Any time somebody has to report upward to a more powerful person in the organization, there’s going to be some tendency to try and make any bad news less unpalatable. Filtered through multiple reporting levels, it’s all too easy for the bad news to be completely lost by the time the top person gets the report.

There are, fortunately, a number of ways for leaders to overcome this problem. Scott describes one simple one: the agile practice of a short daily standup meeting, where each team member provides a quick verbal status report. Anyone else interested in the project status is welcome to attend the meeting as an observer.

I saw a couple of other techniques while I was serving on a nuclear-powered submarine in the US Navy. Unsurprisingly enough, there’s a very clear hierarchy on a warship:

• the junior enlisted report to their division’s Chief Petty Officer
• the Chief reports to the division officer (usually an ensign or very junior lieutenant)
• the division officer reports to the department head (a senior lieutenant)
• the department head reports to the Executive Officer
• and the Executive Officer reports to the Captain.

Obviously, there’s a lot of room for the sort of “green-shifting” Scott mentioned to occur. One way the Navy counteracts this is with what is effectively a second, parallel hierarchy within the command. The division Chief Petty Officers report directly to their division officers, but they also communicate amongst themselves, and to the Chief of the Boat (the senior enlisted man on the submarine, typically a Master Chief Petty Officer). The Chief of the Boat has the direct ear of the Executive Officer and the Captain; this gives the skipper a second communication channel to cross-check the reports bubbling up from the junior officers. In addition, the Captain would spend part of every day at sea walking throughout the ship, seeing exactly what the conditions were without any filtering (deliberate or not) from the people reporting to him. That latter technique has crossed over into consulting lore with the label “Management by Wandering Around.”

The above techniques are clearly not specific to software development. One technique that is very software-specific is the agile practice of running short iterations, each of which delivers working code. Frequent releases are pretty binary; either you release working software every X days or you don’t. Suppose the person who’s signing the checks on a one-year project insists on seeing working software every two weeks. If the project hasn’t delivered anything new for six weeks, the person signing the checks knows something’s wrong, regardless of any green-shifting that might be occurring in the status reports.

Alas, all of these techniques depend on the leader devoting time and energy to supplementing status reports with hard data. If that leader chooses to restrict himself to status reports, he runs the serious risk of green-shifting hiding bad news for months or even years. The consequences can be calamitous; just ask anybody who’s been involved in a multi-year project that gets cancelled after burning through millions of dollars and thousands of hours of people’s lives.