Archive for February 28, 2010

Agile can feel ungainly when migrating complex systems

February 28, 2010 2 comments

At some point, almost everyone in the software business either dealt or will have to deal with a complex system migration.

System migrations carry a much higher risk of being over budget and requiring more time to complete.   Can Agile development process help mitigate these risks?   By definition, system migrations require a significant amount of planning (I will talk about these elements shortly).   In some parts of the plan, Agile development process will help.  In others, it will feel very ungainly and may create more risks than benefits.

First – even before discussing planning elements, let’s talk about factors which will influence how the system migration will be planned.

– Time:  some systems have to be migrated before a busy shopping season or a critical financial milestone.  Time compression will almost always lead to a decision to split the project into parallel work streams, requiring more coordination and advance planning.

– Budget:   self-explanatory.

– Technology platform shift:   migrating a system within the technology family or between different technology families (for example, from .NET to Java or vice versa) will introduce separate considerations and additional risks.

– Technology paradigm shift:  migrating a system from a custom-built order management system to an enterprise ERP suite will introduce – again – a set of new considerations and risks.   In 1998, FoxMeyer Drug – a drug distributor – failed as a company after an implementation of a well-known ERP software also failed.    Quick Google search will produce many articles about this case.

– Data migration:  incremental, one time?  Data migration planning requires more rigor than any other stream of work in a complex system migration project.

– Operational performance:   will the new system be able to accept a 300% increase in orders?  In 5 new countries?  These – and other metrics – have to be clearly identified because the architecture required to support a certain level of business activity may be very different.

It’s important to recognize that the support from business owners (marketing, order processing, financial operations) is essential for success of any complex system migration project.   I won’t discuss business transformation elements in detail.  However, your plan should have a separate work stream with very clear milestones:  training instance ready, training program for Merchandisers developed, training scheduled, lessons learned from training processed, training delivered to 5 locations.  Anything less – and the project will fail even if all technology goals have been met.

It’s also important to recognize that the above factors make the planning phase by far the most important first step when considering how to migrate a system.  “Measure seven times, cut once”.

Applying Agile development principles can be very helpful in several areas of a complex system migration effort:

– Developing data extraction and conversion processes:  iterative data discovery approach can work very well when dealing with systems that are poorly documented

– Developing interfaces:  only after all architectural considerations have been addressed (especially performance and scalability), developing interfaces in an iterative manner can be also very effective.   I found that ‘design by contract’ approach also works very well.   Complex interfaces usually have stages of evolution, i.e. first – develop an ability to obtain real-time pricing, then – develop an ability to obtain real-time availability, etc.  Each stage of interface evolution can be designed and built in parallel with other efforts.  Again – the caveat is the same.  The design should have been already completed, including understanding of integration milestones with old and new systems.

– Developing functionality which – as a result of system migration – is new or has significant changes, for example new order approval workflow.

Can Agile feel very ungainly during complex system migration efforts?  Yes, it can.

– Any effort which has a high degree of architectural uncertainty, for example a critical interface that has significant performance implications on the rest of the system.    Once designed, the development approach to build the interface can be structured as a series of multiple iterations.

– Where there is a need to integrate with legacy systems that are not well understood or documented.

– Where there is a need to integrate with 3rd party external systems:  complying with their interfaces and requirements becomes the most important goal.

“The more complex the problem, the greater the need to consider a multitude of solutions techniques”.

Complex system migrations are a perfect example where the above holds true.

Categories: Software Engineering