Reference Data Management: gather, govern, provision

Having a centralized process of managing classifications and hierarchies across systems and business lines, it facilitates unified reporting and ensures integrity.

Important Topics

• Industry Classifications
• Custom Classifications
• Governance
• Automation
• Provisioning
• Versioning

In a complex organization the number of systems can be significant and each of these systems is working on a predefined List of Values. Imagine coding a simple "client's state" in various ways, like Active = A = ACT = 1 = true. Reference Data Management gathers and unifies these lists of values matching them and provides one single place to manage them all.

Each industry has their own standard list of values, which comes from regulatory or governmental platforms. A centralized RDM gathers these classifications or hierarchies once under one platform and makes them available for usage, avoiding individual loads for each system in the organization. Working with AbInitio it is easy to deliver graphs to access public/private web services or adapt to any technology necessary to load these data.

Having in one single place the reference data it is important to have a governance flow, especially on top of the custom ones. Any enhancement or change in the available values over a classification can be easily handled in one single place, but is mandatory to have approval and acceptance. This is why consumers of that data must agree and be able to react upon changes with appropriate consent.

In case we are dealing with Industry standards every list of values should support automation to ensure immediate update for all consumers, but not only in gathering the reference data, as Life Cycle Management requires propagation of both these Industry and Custom classifications to be delivered across the other environments (TEST, DEV, ...).

AbInitio delivers a standard out-of-the-box API to access metadata, and it supports an easy consumption of the reference data being performant on both reporting as operational systems.

Challenges

Performance - One of the greatest challenges is the performance in accessing the reference data. A clear architecture ensures the timeline for consumption, where operational systems would decide and reconcile upon the time spent to daily usage against on-update synchronization.

Results

Defining the Data Quality Framework from the beginning based on the principles mentioned above we were able to introduce to clients information about their current quality, trend information about quality improvements over time, decision-making information to stop or proceed with processing or reporting, and even automation for fixing or improving the quality. Nevertheless, in our experience we realized that this kind of solution must be supported by other streams of a complex organization, like Modelers, Architects, and it always succeeded by being effective if we have a good Metadata Management next to it.