What software product managers can learn from the current 737-MAX crisis

On October 29, 2018, a Lion Air flight with a brand new 737-MAX took off from Soekarno-Hatta airport in Jakarta. The plane followed a very erratic flight path before crashing into the Java Sea just 12 minutes after take-off. On March 10, 2019, an Ethiopian flight with a just as new 737-MAX crashed only 6 minutes after take-off. In the weeks following, more and more countries grounded all planes of this type. 

Almost all major plane crashes are investigated thoroughly. Although there is usually at least some politics involved, investigative engineers generally do a very thorough analysis of the chain of events leading to the crash. There seems to be hardly any industry that tries to learn so systematically from technical errors and human mistakes. One of the major contributing factors to crashes is the – erroneous – interaction between human (pilot) and machine (plane).

Although the investigations into the Lion Air and Ethiopian crashes are still ongoing, there seems to be crucial role for the MCAS system, one of the many systems in a modern airliner supporting the pilots in flying the plane. In both crashes, the MCAS system seems to have turned against the crew. It is hardly a secret that more regular computer software – although usually meant to support a human task – also frequently turns against its users.

Ever been annoyed by Word’s autocorrect replacing properly spelled words or abbreviations with what Word felt was ‘right’? Or by a form refusing to be submitted because you failed to enter a value in a certain field, without telling you where that field was between the thirty other ones? Ever questioned the outcome of some assessment without being able to find the logic behind it? 'Computer says no' is an internet meme for good reasons, obviously.

The analysis of the human – machine interaction of airplanes is so interesting because of the potentially catastrophic impact of errors, combined with the systematic evaluation of crashes and the way in which improvements are often implemented industry-wide. Air crashes can therefore be a major source of inspiration (be it a slightly morbid one) for people working in software product management.

Below, I elaborate on three core rules of software product management: Make the application logic transparent to the user, Cutting corners will catch up with you, and Make your software the perfect dance partner. All three of them relate directly to the mistakes Boeing seems to have made in implementing the MCAS system.

Make the application logic transparent to the user

The MCAS system in the 737-MAX compensated for a new position of the larger engines compared to older 737 models. It uses the input of a single 'angle of attack' sensor to push the nose of the plane down if the plane is approaching stall. It is therefore a safety system, designed to prevent the plane to crash because of a lack of lift caused by a too steep angle of attack. But when the Lion Air plane crashed, the pilots did not even know about the existence of that system.

The pilots of the Lion Air flight simply did not have the possibility to troubleshoot the issue at hand, because there was no way for them to know about the MCAS system, to what it responded, how to switch it off or how to deal with it in a different way. Boeing responded by claiming that most pilots would never experience the system to function at all, because it would only kick in in extreme circumstances. But those are exactly the kinds of circumstances in which pilots should know all about it.

What was more, the MCAS system responded to a single angle of attack sensor. When that sensor was broken (as was the case in both crashes) the system 'assumed' that there was an extreme situation and responds to that. Which was exactly what brought the two planes down. Although the pilots of the Ethiopian flight were aware of the MCAS system and troubleshooted the problem correctly, they did not have the time and force to correct it manually.

Takeaway: Never create blackboxes in software if they affect core functionality. Users should always be able to follow the logic of what your software does in order to detect the consequences of faulty input.

Cutting corners will catch up with you

According to this The Verge longread, the 737 MAX crashes were accidents waiting to happen, because of wanting too much in too little time with insufficient control, in a competitive battle with Airbus, which introduced its fuel efficient A320neo. In software development, reducing spend on development, resulting in corner-cutting, spaghetti-type code is usually referred to as 'technical debt'.

One could argue that the MCAS system was a quick 'bug fix' for a huge change in the very design of the plane, namely the aerodynamic change due to larger engines needing a different position on the wings. Quick bug fixes usually do not address a root cause, but only 'hide' the effect of a more fundamental flaw in normal use. Clearly, this 'bug fix' costs Boeing a multitude of the costs it has saved by corner cutting.

There is more to this cutting corners, again addressing the transparency issue. An important safety system such as MCAS needs to be clearly visible in the user interface as well. Being 'hidden' completely, there is no way for pilots to find out about its functioning, except by its very effects when they kick in. Although I wouldn't suggest that another warning signal would be a viable solution in an already dense cockpit dashboard setting, obviously a pilot has to be made aware when a safety system becomes active.

This is a case I would call 'UX debt', a situation in which either an interface becomes a Christmas tree so that core functionality gets obscured by bells and whistles, or where there is no way of interfacing with a particular function at all. UX debt can only be avoided by continuous evaluation of the interface and taking appropriate action for simplification and properly and logically adding new functions.

Agile software development too often seems to be an excuse for delivering immature software; don't cut corners and keep technical and UX debt to a minimum.

Make your software the perfect dance partner

All the stories about robots taking over our work and autonomous cars would make some people believe that soon, we will be apathetic creatures staring into nowhere while computers do everything for us. A well-known joke says that in the future, an airplane cockpit will accommodate two living souls: one is a pilot, and the other is a dog, preventing the pilot from touching anything. Why is this so far from the truth?

Because for the foreseeable future, we need the human experience and creativity to solve complex incidents. And in order for pilots be effective on those rare occassions, they need to be a 'team' with their equipment. They need to feel in control of the airplane, to know how it responds to their actions, like team mates or dance partners. The same is valid (to an extent, let's not exaggerate) for software.

For software to be the perfect dance partner of the user, it needs to express itself properly, not keeping inconvenient stuff to itself, and it needs to be predictable and consistent in its actions and responses. Software also needs to guide users in focusing on the right subject at the right time, by separating major from minor issues. The mixed signals and obscure and inexplicable actions by the MCAS system are a prime example of application and interface design to avoid.

Support for focusing on the right issues under relevant circumstances, and reliable and consistent behaviour, are crucial for good software

Laurens Mommers

Interested in more posts on software product management?

How to improve software products by making them simpler - why less is still more

Why you need a healthy dose of marketing in product management

Why software engineering should be rocket science

How to handle cognitive biases in product management

Bob Ross was the real father of the agile movement - and why that is bad news

What software product managers can learn from the current 737-MAX crisis

Seven golden rules to avoid software becoming a Christmas tree

Seven secrets of building beautiful software

Building compliance software: five major pitfalls and how to deal with them