1. 2. 3. 4. 5. 6. 7.
Informatica MDM On-Premises walk through
Definition and introduction Landing and stage process Trust & Validation Process Delta detection and audit trial Cleanse functions Load process Match and Merge
HTTPS://INVENTMODEL.COM 81810
+91 98703
[email protected]
MDM basic introduction Definition:Informatica Master Data Management (MDM) is a software solution that helps businesses manage and govern their master data. It creates a single, reliable record for customers, products, and suppliers.
How it works: Consolidate data from multiple sources Resolve conflicts and inconsistencies Establish data governance policies Create a centralized hub for master data management.
HTTPS://INVENTMODEL.COM 81810
+91 98703
[email protected]
MDM Architecture
HTTPS://INVENTMODEL.COM 81810
+91 98703
[email protected]
Landing and Stage Process Landing Process: •
Entry point for source data into the MDM Hub.
•
Data is loaded into landing tables via ETL processes , data loading tools , SQL inserts etc from heterogeneous sources.
Stage Process: •
Cleanses and moves raw data from landing tables to staging tables.
•
Mappings are used to move data from stage to landing table.
•
Cleanse function gets applied to cleanup bad data.
HTTPS://INVENTMODEL.COM 81810
+91 98703
[email protected]
Delta Detection and Audit Trail
Delta Detection: •
Identifies new/modified records in landing table via PRL table.
•
Ensures only new/modified data is added in staging tables.
Audit Trail: I.
Maintains a history of changes in corresponding RAW tables.
II.
Keeps history based on defined retention policy.
III.
Tracks duplicates and rejected records.
HTTPS://INVENTMODEL.COM 81810
+91 98703
[email protected]
Trust & Validation Process Trust Settings: These determine the reliability of data from different source systems. Each source system is assigned a trust score, which reflects how much system's data is trusted. When multiple sources provide conflicting data for the same attribute, the trust score helps decide which value to prioritize.
Validation Rules: These are rules applied to ensure data quality and consistency. Helps to reserve minimum trust if defined. For example, a validation rule might check if a field contains valid values or meets specific criteria. If the data fails validation, it may be flagged or excluded from further processing.
Together, these processes ensure that the most accurate and reliable data is used in the MDM Hub.
HTTPS://INVENTMODEL.COM 81810
+91 98703
[email protected]
Cleanse List and Cleanse Functions
Cleanse Functions: •
Standardizes and verifies data to cleanup or transform bad data.
•
pre-built and custom functions are used.
•
Provide function to lookup DB table to derive another values.
Cleanse List: •
Predefined lists for noise filtering and data standardization.
HTTPS://INVENTMODEL.COM 81810
+91 98703
[email protected]
Cleanse Functions
HTTPS://INVENTMODEL.COM 81810
+91 98703
[email protected]
Load Process Overview: •
Moves data from staging tables to base objects in the Hub Store.
•
Applies trust and validation rules to ensure data reliability.
•
Staging table is truncate table i.e it deletes the data before loading new dataset.
Steps: •
Updates existing records based on trust scores.
•
Inserts new records into base objects.
HTTPS://INVENTMODEL.COM 81810
+91 98703
[email protected]
Match Process Match Process: • Identify similar or identical records in the base object. • Determine candidate records for automatic consolidation. • Determine candidate records for review by a Data Steward prior to consolidation. Match Strategy: • Exact match - looks for exact match based on defined rule. • Fuzzy Match - makes probabilistic determination of match between records based on variations in data patterns such as misspellings, omissions, truncation, and phonetic variations. Allow to define Exact match within fuzzy match.
HTTPS://INVENTMODEL.COM 81810
+91 98703
[email protected]
Merge Process Merge Process: Based on the Matching strategy, the consolidation will begin. Match table (_MTCH) is populated with matched records that are identical or duplicate with ROWID_OBJECT and consolidation indicator in the base object of these records changes from 4 (New Record) to 3 (Matched Record). These records are then queued with the consolidation indicator as 2 indicating that they are ready to go for the merge process. The merged data (records) are flagged with the Consolidation Indicator 1 indicating that the records can be considered as “golden” (Unique or BVT – Best Version of Truth).
• • •
Auto merge – Achieved by auto merge job which are flagged as auto merge during matching process. Manual merge – manually merged by data stewards for records flagged as manual merge during matching process. Tokenization – generates match tokens that are used subsequently by the match process to identify candidate base object records for matching.
HTTPS://INVENTMODEL.COM 81810
+91 98703
[email protected]
Match & Merge
HTTPS://INVENTMODEL.COM 81810
+91 98703
[email protected]
