Complete Metadata Management Implementation: Business Terms and DQI Governance, Lineage

In any organization, having a good Governance implemented helps it to easily react to market changes as well as to regulatory requirements. The term Governance it specifically expresses "to govern" and it is to be used in organizations on one hand to monitor the health and status of it, but also to facilitate initiatives impacting multiple stakeholders.

Important Topics

• Business Catalog
• Data Catalog
• Data Quality
• Governance
• Relationships
• Multi Tenant

Our involvement in the project started with Metadata Management responsibilities through gathering all requirements from the client and conduct a "vendor selection" for the "best bread on the market" approach. Reaching the final steps and organizing the POC, we presented all markets options with viable solutions that would cover all mandatory requirements.

All complex requirements involving multi-tenant architecture and one single place of control and access were solved and proved possible with the help of AbInition.

Having multi-tenant architecture requirement, it was mandatory to have a Business Glossary governed on multiple levels: Group Level where business departments would create and manage definitions, as well as Country Specific Level where each country would have the own governance policies. The tool's capabilities allowed us to customize the approval workflow, where states would have single or majority approvals over multiple owners and stewards.

Part of the Life Cycle of a Glossary, we implemented as well cross-life-cycle synchronizations in order to ensure lower environments with the appropriate list of business terms as well to take into account Release Management handling, where future version (not approved ones) are to be delivered on various environments to support development or testing activities.

Getting to know a business term, it is mandatory to understand also where does it come from and where it is reported. In the end, a glossary must keep up with the organization evolution and impacts caused by a definition change to be easily estimated. This is why, also a strong argument of a Metadata Management implementation is imperious to define relationships or trace business definitions in the physical data.

As beneficiaries of the tool and especially of the content of information helping decide on their levels, involved both business and technical specialists, it was mandatory to track definitions on multiple levels of understanding and this is why we helped decide and implement in AbInitio 3 levels of lineage with individual business definitions relationships:

Business Lineage Visual representation of various connection points from architecture perspective, like identification of a business term into specific operational systems, aggregation and calculation engines, until the reporting of it to various consumers.

Mapping Specification Lineage As part of any Data Warehouse implementation any requirement starts with the analysis phase where specialists help identify the business definition in operational systems, continues with specifications where to be stored and aggregated for reporting, until where to report the information further. This entire process can be documented and visually represented by identifying the key points as well as targeting the correct technology and processes. In the end, the beneficiary would not be represented only by the Business Analysist, but can also support automation tasks in development tasks.

Technical Lineage ETL processes are composed by programs that transfer the organization's data from operational systems through various models of historization and aggregation in order to support "helicopter-views" or regulatory reports from semantic models. All these technical programs or executables construct a Data Flow throughout organization layers where the data is matched, enhanced, aggregated or consumed. In the end, the time-to-market of technical initiatives becomes easy to estimate, track and support.

Challenges

Versioning - one of the requirements from the client was that the Glossary should be able to support current and future version of the same term. Business definitions were supporting future or unreleased developments, so, we had to define Release Management Cycles and have it linked to development statuses in order to ensure at once 2 versions active for the same term.

Many-To-Many Relations - a business definition can be stored in multiple columns or fields, as well as a column/field could contain multiple terms. Starting from Lineage perspective to avoid circularity and still visually represent this type of relationship, we received a major support from AbInitio where the model behind received an intermediate layer to document each relationship individually.

Results

All initial requirements were implemented and along the way new ones surfaced to meet the criteria defined by the organization's old and new processes and market's best-practices. It wouldn't have been possible without the support of internal organization by embracing new technologies and facilitate digitalization and trust over content, but also the huge help received from AbInitio where client's challenges were covered and fixed by product releases and patches along experience sharing.