Revamping systems – The Agile way

Irena Shaigorodsky posted July 26, 2019

When revamping a system using a waterfall methodology, legacy systems are modified while new systems are built. At feature parity, they need to run in parallel for a reasonable amount of time to ensure that the new system can genuinely replace the existing one. Using an agile rewrite, we can start seeing wins sooner — kind of like rebuilding an airplane piece-by-piece while still in flight.

The decision to rewrite an existing system does not come lightly. From a business perspective, in most cases, there is no obvious business value in such an undertaking. Engineering pushes for a rewrite because: 

  • they can no longer maintain the system
  • it is impossible to scale it for the new requirements
  •  the technology is outdated and no longer supported
  •  it is impossible to hire people that know the outdated tech stack
  • licensing cost changes from the vendor

It is easier to get buy-in for user-facing changes, like replacing an outdated user experience.

In either approach, the rewrite comes with a significant cost attached to it. When done in the waterfall style, the old system will continue to change and grow as the new one continues to be developed. Creating end-to-end specifications for a new system will require a lot of upfront work. Performing acceptance tests on the new system will take time. Even when all tests have passed, switching to a new system will have associated risks (especially when money or health is involved). Because of these risks, the systems will have to run in parallel for some time pending the decision to shut down the old one. As a result, the waterfall rewrite initiative will take a significant amount of time and resources. Such an effort has the potential of running overtime or over budget. The benefits of the rewrite will be delayed until the completion of the whole project. End users will have to work in 2 systems for some time. 

As with the development process, there is an alternative to the waterfall rewrite: iterative system rewrite. You can think about the process as of iteratively rebuilding the plane you are flying. The approach minimizes the risks, simplifies requirements gathering and acceptance testing, and brings benefits at each iteration seamlessly to the end-user. There is no date on which new system has to replace the old one. Instead of this, at each iteration, additional features are switched other until none are left whenever that time comes. Additionally, the new features that the business needs are added to the new system only. 

Making the two systems work as one requires some throw-away code creation on both sides. Besides, each migrated feature needs to be carefully suppressed in the old system, and periodic synchronization of the changes for the functionality between the systems put in place. The sequence of the incremental changes is also essential, and first iterations will be spent on the groundwork of enabling the rewrite.

The iterative rewrite was the approach that Experiences Supply team started in autumn of 2017. The rewrite has been ongoing until 2019. New and returning suppliers were able to use modern-day user experience (powered by React) on mobile and desktop, while new features were being developed. Product management was able to improve on their OKR (objective and key results) metrics. Engineering got a lot done on revamped scalable microservices tech stack with continuous releases. And the list of benefits can go on and on.

The first couple of milestones were used to lay the groundwork and enabled the changes. 

The first milestone introduced proxying functionality from the new system to the old one. It was just a shell that served the legacy system UI. The traffic was hitting the new system at that point.

With the second milestone, the login page with the new look and feel was introduced. The rest of the UI still was proxied from the old system, and some cross-system handoff was needed for logging in, and password reset.

With the third milestone, the first net new feature was added: user management. In the old system, the user did not have an option for self-servicing of their accounts. Now primary users became able to add/delete additional users associated with their organization, grant and revoke permission. The amount of customer service calls went down as a result of the addition. The effort started bringing returns on investments by the end of 2017.  To power the new feature, microservices were introduced. Logging into the system began to be handled in the new system.

The fourth milestone moved the navigation from the old system to the new one by suppressing the navigation. Minor CSS changes were applied to make the UI align. The iframe solution that worked seamlessly with the existing system required an entire milestone to work out. Having it in place allowed for additional feature migration to begin in full speed.

At the end of the fourth milestone, the team had the foundation in place that allowed the team to migrate one feature at the time from the old system to the new one. The migration continued for a duration of over a year until nearly all the features were migrated to the new system or product management decided that from ROI perspective it is not feasible to support the feature(s). Having similar iteration planned for your rewrite will put you in the pace of delivering the improvement and keeping business and customers happy while getting closer to the end goal and replacing the legacy system with the new one.

References & Bibliography

  1. “Rewrite (Programming).” Wikipedia, Wikimedia Foundation, 29 June 2018, en.wikipedia.org/wiki/Rewrite_(programming).
  2. “Agile Software Development.” Wikipedia, Wikimedia Foundation, 1 Mar. 2019, en.wikipedia.org/wiki/Agile_software_development.
  3. “Waterfall Model.” Wikipedia, Wikimedia Foundation, 4 Feb. 2019, en.wikipedia.org/wiki/Waterfall_model.

Author’s Biography

Irena is a Principal Software Engineer/Tech Lead on TripAdvisor’s Experiences Demand Services Platform Team. She is a seasoned full-stack professional with over 20 years of experience.  Irena was fortunate to be part of the incremental system rewrite from the early implementation steps. She enjoys public speaking at company-wide Tech Talks and outside conferences. Irena is the founder and first president of CA Technologies Toastmasters club in Framingham, MA.