A dialogue with the legacy system: AI-powered code analysis for migration projects

Executive Summary: The effort involved in a migration is usually estimated before the details of how a legacy system actually works are known. AI provides a solution here: when organised as a knowledge graph, even a codebase that has grown over decades can be queried by a language model. Assumptions about logic and special cases can thus be systematically verified, and estimates become more robust. At the same time, time-consuming and costly corrections in later project phases are avoided.

Nobody knows the whole system

At some point, the time comes when a legacy system must be replaced. The causes are well known: regulatory pressure, increasing maintenance costs, a successor system already waiting in the wings. The decision to undertake a migration project has been made, and it is precisely at this point that the question often arises: what exactly does this system do?

Everyone in the organisation knows part of the answer. But the knowledge of how the system actually works as a whole is usually either incomplete or difficult to piece together. Too many customisations have been added over the years.

This is where the real risk of a migration begins: the effort involved is estimated before it is fully known what needs to be migrated. Every estimate therefore contains assumptions about the logic and special cases in the legacy system that have not yet been systematically verified.

The limits of readability have shifted: language models make code accessible

This systematic review is possible, but until now it has simply not been feasible within the timeframe available for a migration project. A legacy system that has grown over decades often comprises several million lines of code. Anyone who works through it ends up with notes, but no structure that makes the knowledge easily accessible.

The limits of what can be read within the project timeframe have shifted in recent years. Language models can analyse code on a scale that was previously impractical, provided that it is not submitted as text but in a form that both reflects its structure and subsequently allows for token-preserving processing.

Whether a special treatment in the code was a deliberate product decision or a workaround from an old release that nobody needs anymore is not stated there. The code exhibits every behaviour, but does not reveal which of these was intended. This distinction is the cause why, at the end of the analysis, classification by subject matter experts is still required. AI helps to organise topics and makes things visible. The final decision remains with humans.

From source code to a queryable graph

The starting point is the source code, exactly as it stands. There is no preliminary stage involving tidying up or restructuring; otherwise, the process would only work under conditions that are rarely found in migration projects.

We process the code and construct a graph from the syntax trees, in which modules, functions and their dependencies are represented as nodes and edges. What was previously text spread across thousands of files is now queryable: a knowledge graph of the codebase.

Queries are run on this graph. A language model can search through it and thereby find the entry point that previously required extensive system knowledge: where a rule is defined and which modules it involves. What remains for a human to read afterwards are only individual sections, rather than the total codebase. Product data and configuration are stored in separate files alongside the graph and are also taken into account in the AI analysis. The information contained therein is thus incorporated into the response.

However, the graph alone does not solve any problems. It simply makes the code and its interrelationships accessible in a different way. The added value only arises in dialogue with the legacy system: by using the graph to ask the right questions in a targeted manner.

The legacy system itself is new to us, but the typical problem areas are not. From decades of migration experience, we know where special cases are likely to arise and which questions need to be answered for a robust effort estimate.

A look behind the scenes

What does the use of such a graph look like in practice? In an ongoing project, a migration rule would have incorrectly classified part of the data. The cause: a field that controls the handling of payments contains plain-text values in the configuration that read like a tariff type. Anyone deriving the rule from this would be migrating based on a criterion that does not exist in the professional context. The source code reveals what the field actually controls, and the rule was corrected before it was implemented in the target system.

The discrepancy was found because both sources were side by side in the same analysis and someone asked about inconsistencies. Such issues do not stand out when read by a human; the codebase is simply too large for that. The analysis, including references to the relevant code, reveals what actually applies in the legacy system. The professional decision as to which criterion should apply in the target system remains open. The business department can now make this decision at an early stage based on a verified finding, thereby avoiding time-consuming corrections in a later phase of the project.

The system at the centre of this project:

• Code base: around 32,500 files, over twelve million lines of code, built up over decades
• Largest single file: just under 378,000 lines of code
• Time taken to reach a queryable state: just over three hours, a one-off process, on a virtual machine with two processor cores

Human-in-the-loop: verification and validation

However, human verification must not be omitted. In one instance, for example, the analysis delivered the correct answer for the wrong cause: the tool reliably identifies where something is defined and which segments are used together, but does not always recognise who calls whom. This proximity in the graph was interpreted as a dependency. It is precisely such errors that are dangerous. They do not stand out as obviously incorrect answers, but are hidden behind plausible results. The solution lies in a dialogue about the graph; the answers provide the basis for approval by the client.

A look at the workshop shows that the use of this new technology is a work in progress. Not everything has been fully explored yet, but the learning curve is steep. For example, customer-specific customisations have not yet been incorporated, and the graphs that can currently be queried are based on C++ and Java code bases. However, a technical workaround is already available for older programming languages such as COBOL, so the methodology promises to be applicable there too.

Experience remains the key

There are now quite a few code analysis tools available. They can reveal structures, dependencies and anomalies. Whether anything useful comes of this, however, depends on the questions one asks of the system. The most heavily connected node in our graph is a memory management function. In terms of the number of its connections, it ranks ahead of every professional module. Nevertheless, it is irrelevant for the migration.

Knowledge regarding the quality, reliability and efficiency of the AI methods used is not, on its own, sufficient to successfully implement AI-based migration processes. Equally important is knowing where the costly areas are hidden within a life insurance portfolio. We have gained this knowledge through more than 25 years of in-house migration experience. Combining both aspects within a single team makes the difference between a mere code analysis and a robust preliminary migration study. A cost estimate does not become reliable simply because more is known, but because such questions are asked before anyone quotes a figure.

No one who reads code asks about inconsistencies between configuration and source code. It is asked by someone who knows how costly a misclassified portfolio group becomes once it is only noticed in the target system. The language model answered this question. A human asked it.