Responding to Change over Following a Plan
Last year, I started writing about the agile software development values. This is the last post of the series. You can find links to the other posts at the end of this one.
As is the case with the other agile values, "responding to change over following a plan" seems almost like an obvious piece of advice. Honestly speaking, are we able to find professionals who don't aspire to act this way? Is any of us willing to publicly argue this value? And if this value is both obvious and undisputed, why do we need to list it as one of the agile values? Why would a software development team choose to follow a plan over responding to change?
One reason for ignoring additional information or a change in the world around us is that we might not like this new perception of reality. It might question the image of us as strong decision makers. We often choose to save face instead of admitting our mistakes. The more responsible for a decision or a plan we are, the more likely we are to justify its excellence even when negative events start to pile up.
In their research paper, Viktoria Stray, Nils Moe, and Tore Dybå study this phenomenon with an agile software development team over the span of four years. In the beginning of this period, the team made a decision whether to base the technology of their product on an existing framework already used inside the company or build their own framework from scratch.
The existing framework seemed to be too immature and complex for the project and therefore, the team decided to build a completely new framework instead. However, three years later, a new plan was made to reverse this decision. All the code for the new framework would be trashed and the team would move current features to the existing framework.
Development with the independent framework had been painfully slow. Almost all the new features required changes to the underlying framework code. These changes would take either hours or days but it was extremely difficult to estimate how big of a change was actually required. New developers entering the team would constantly complain about the code quality and architectural decisions of the framework.
Leading up to the decision to switch frameworks, the daily meetings had become a mess. Instead of coordinating work, developers would report on their finished work in detail in order to convince others of their technical skills. The product owner and new developers would judge the current framework as flawed while the original team members would defend their initial plan as best they could.
This is what escalation of commitment looks like. It's a situation where we refuse to alter course despite the increasingly negative outcomes we face. In group decision-making, escalation of commitment as a behavioral pattern is even more prevalent compared to individual decision-making. We don't like to admit to ourselves that we're wrong. But we like to admit it to others even less. We want to hold on to our social status. We want to be seen as competent and rational. Even after the decision to switch frameworks was made, it took five more months before the original team members accepted this new direction.
Stray, Moe, and Dybå caution us to stay aware of the escalation of commitment at our daily meetings: When we hear our teammates defending their decisions, we should take action. We should work on improving trust and communication. Teams that have open communication and high psychological safety are much more capable of a project turnaround than a team of silence and fear.
Too often our egos and insecurities get in the way. It's easy to delude ourselves into continuing with a bad decision or investment. It's obvious that we should alter course when needed, but the task of responding to change is harder than we think.
Additional notes
To read my other posts about the agile software development values, follow the links below: