Computer systems are like political systems, people tend to see the problems instead of what’s working well.  There is a constant tension between moving forward with what exists by trying to change it slowly over time to make it better and tearing down the entire thing and starting over.

My experience is that it’s almost always better to change it over time in a measured manner. When a complete overhaul happens all at once, it only works when the new system is already built, running and proven reliable. Unfortunately, given that all systems have components that aren’t well understood, it is almost impossible to have a system completely built, fully running and completely reliable. Reliability, in particular, is only achieved by having real people use the system in real circumstances. No amount of testing fully replaces experiencing the real world. In short, even successful revolutions always fail to achieve the ideal state the revolutionaries envision.

A universal tendency of computer systems, and businesses, is that they grow over time. People using systems always want more features. Build a system that counts steps and as soon as people start using it, they’ll start asking ”Can it also give me the best route to go to get 10,000 steps?” or “Can it track my heart rate, too?”. It’s a natural progression.

The same thing happens with businesses. Take Starbucks, they started selling coffee, then they expanded into more designer drinks. Now in any Starbucks you’ll also find all sorts of food items available. Their services have expanded.

Much of this has to do with economies of scope; where if you are selling one thing you can sell related things at a lower cost than someone who is only selling that thing. If you are already selling coffee, then you already have most of the infrastructure set up to also sell designer drinks. If your app counts steps, then you have a lot of what’s needed to also track heart rate while walking.

Over time, the focus of the system veers away from its original intentions. Instead of putting the company’s major efforts into finding the best coffee, they go into supporting the new products. This is somewhat offset by the additional resources and revenue from the new products, but over time the management focus moves towards what is making them the most money, which is usually the more complex products.

The Innovator’s Dilemma, by Clayton M. Christensen, talks about how this opens up an opportunity for new entrants. In the example above, when a company stops being solely focused on the best possible coffee, or on accurately counting those steps, and starts focusing on the other things, competitors who are more focused can step in. The old company is usually happy to give up market share of their original product, which is now one of their least profitable.

Notice, however, that what doesn’t happen in a market economy is a company trying to do it all at once. No company would buy 10,000 locations and open them all at once in an attempt to compete with Starbucks. Rather, they would open a store or two, test out the product, the supply chain, and the overall concept before expanding. They will make sure all of the pieces work before scaling. It’s inevitable that some of the pieces won’t work. No matter how hard you try, even experienced people will encounter issues. Moreover, opening 10,000 stores at once would mean that anything that went wrong could affect all 10,000 stores instead of just a few if a more measured approach had been taken.

And yet, with computer systems I see repeated attempts to rebuild systems and switch everyone over all at once. I see this more on very large and complex systems. On small systems, I see where people use two systems for awhile as they test the new system to see if it will work for them. For instance, if you are trying a new shopping site, you will tend to try the new site without giving up the old one. Only if the new site does everything you need will you switch to only using the new site.

With larger systems, where the move is much more difficult, it is much more common for companies to make large all-at-once switchovers. That is because these systems are heavily used, and trying to use two systems effectively doubles the work of everyone using the system. I have been through dozens of these major transitions, and they are always difficult. It’s common for the users to be quite unhappy as the kinks are being worked out of the new system. In short, it’s difficult. Major failures are common.

Despite these common failures, companies still want to do these ‘big bang’ projects because of a misguided understanding of the complexities of modern computer systems and businesses. People often think about moving systems like moving to a new home: you pack everything, the movers come and move it to the new place. Sure, you have to unpack and there can be a little confusion and cleanup to get everything in the new place, but within a few weeks you have everything put away and figured out.

Moving a major system is more like trying to move an entire city. A city is not just a bunch of buildings; it is a complex network of infrastructure, public services, and communities. Everything must move, not just the furniture. Unlike a house move, where the impact is contained within one family or entity, relocating a city affects thousands or even millions of individuals, each with unique needs and dependencies. The logistics of coordinating such a vast array of elements without causing major disruptions is exponentially more challenging.

Cities are upgraded piece by piece in a never-ending cycle of renewal. For most cities, there are always parts just finished and others that have desperately needed updating for decades. It is a slow cycle of renewal and decay. Rebuilding an entire city at one time just isn’t feasible. I was living in Doha Qatar as they underwent a massive building campaign and even there, with major construction literally on every block, it was being done in pieces so traffic and commerce could continue to flow.

Companies resist moving systems piece-by-piece because of the double work and also due to a general sense of wanting things to be “finished” so the company can move on to other things. Companies would be better served to think of their systems as cities, to be constantly updated piece-by-piece, instead of as a new house where the move is done and the old house left behind.

Lately, I’ve even heard a lot of talk about how various political and cultural systems are broken and should be utterly dismantled. There is even talk of revolution in some quarters. While much of the criticism of current politics and culture is accurate and in dire need of reform, dismantling the current system and trying to create a new one is unlikely to work. While in America we think of the American Revolution and its beneficial results, most revolutions, from the French Revolution to the Russian Revolution, and even recent ones such as the Arab Spring, have brought more destruction than benefits. Far from realizing the idealistic dream of the revolutionaries, most revolutions end badly for most of the population. As bad as the current system is, tearing down the existing system and rebuilding it is more likely to make the situation worse and take us even further from the ideal.

For the good of humankind, we should be building up, not tearing down.