In this article we will deal with Master Data Management (MDM) as a specific area in which IT and business work together. We will describe what makes Master Data special, what are typical areas for improvement and how to approach them.
Master Data are sometimes defined as data about customers, vendors, business partners, assets, human resources, products, materials, projects and other entities. These are however only examples. In order to understand what are the Master data, it is much more important to realize that these data are shared across various systems. If we imagine an organization that will have all the data in a single system (application), the master data topic would not be a concern for it. However, the reality of each and every bigger organization si different. Various business processes use various applications that have been deployed gradually as the company was growing and was realizing need to support various areas. It is true that the very information about business partners, employees, products, projects etc. are clear examples of master data. We work with them both in front office applications (sales people, people at the front office counters and desks...) and back office applications (accounting and finance, logistics etc.).
Master Data are relatively statics (they do not change frequently) and non-transactional. Orders, invoices, ledger entries do not belong to master data.
Reference data are a special topic of master data. They represent valid values in attributes of another data. E.g. order status, customer type, product category etc. Therefore, when transfering data across systems we are not only working with the primary entites like customers, products etc but also a number of other entities that define their properties.
If the data are spread over a more places, question of consistency has to come to our mind necessarily. What if I create a customer record in one system but forget to do that in another one? Is it possible that we could have a one price list in one system and another price list in another system? Do we have everywhere same statuses, types, categories etc. of customers, products, materials…?
If above questions resonate in your minds, you probably recognize importance of Master Data Management. If you are creating today data in various systems, with variable quality and inconsistent attributes, then Master Data Management can help to unify the data and ensure their propagation across the organization.
Solution design
If we start with master data, we are going to cover these steps:
- Identification of Master data sources
- Sources classification
- Mapping data flows
- Definition of responsibilities
- Development of data model for Master data
- Master data architecture, rules and roadmap to the architecture, process for managing Master data in daily operations
- Selection of technologies
- Actual move to the target Master data architecture and implementation the day-to day process
Lets look at these steps in more detail.
1. Identification of Master data sources
We will start with information collection. We will put down a list of applications that work with certain master data. E.g. customer data may be in an ERP system, CRM, marketing application, data warehouse etc. This information can be already available somewhere. Use list of applications as a starting point, speak to their administrators and start organizing information about master data.
2. Sources classification
We will distinguish producer applications (data originate in them) and consumers (they are receiving data from outside and just use them) for particular master data entities. E.g. customer master data are created in the new CRM system and ERP and Data warehouse are just using them. Each application can be a producer of some master data, consumer of other master data and other master data are not a concern for them.
Figure 1: Classification of systems to producers and consumers (always from perspective of a particular data entity)
3. Mapping data flows
In this step are ging to map the data transfers. We will be finfing out from which source system to which target system we are transferring which data how frequently, by which technology in which format. E.g. data about customers are transferred from CRM to ERP every 10 minutes by means of a XML file on a shared disk. Once we have mapped the current status, we will be able to define improvement opportunities and the way how to move to the target situation.
4. Definition of responsibilities
It is necessary to define who is responsible for which data. It is good to distinguish between the data owner and the data steward. Data owner is responsible for data quality. It should be somebody from the organization management. Data steward maintains data, their structure and rules according to the data owner decision. Various data can have data owners and data stewards in various teams. Processes for master data management, roles and responsibilities should be described in procedures that are available to everyone and all the concerned people should be familiar with them.
5. Development of data model for Master data
Master data have mutual dependencies. E.g. customer data have a relationship to types of customers, products, that the customer orders, and employees, that care about the customer. Product have their own hierarchy of product categories and types, products use certain materials etc. Data model in form of Entity Relationship diagram will capture these relationships in a transparent way.
Figure 2: Example of Entity Relationship diagram
6. Master data architecture, rules and roadmap to the architecture, process for managing Master data in daily operations
Current flows of master data, as we mapped them in point 3, do not have to be optimal and actually they may be a root cause for current issues. If you maintain list of products and their hierarchy in more systems, you will probably face issues caused by the fact that you forgot to update the data in one system or did that only partially. There are several concepts of working with master data. One of them is is data creation and master data management in one dedicated system and data distribution to all other systems. This concept of a single place of truth can solve the previous issue in a nice way. However, due to pragmatic reasons other various approaches are also taken. Moving to this system always requires modification of applications that will not manage the data anymore but will take them from the central system. Some applications will not be able to do that and their modification would be very costly. Therefore, we will have to accept that e.g. some data will be also in future managed in ERP and imported to the MDM system only in a second step. We should describe particular approaches, set rules for their application and plan an overall transition to the new system.
7. Selection of technologies
According to the selected architecture and strategy we will select particular technologies that will facilitate our work with master data.
8. Actual move to the target Master data architecture and implementation of the day-to day process
Transition to the target architecture depends on the organization starting conditions. Organizations that recognize importance and benefits of master data at the beginning of building the systems` architecture have another starting position than a company that experienced a precipitous information system development and that keeps certain internal debt in this area. If we are starting with Master Data Management and use many applications, transition to the target architecture can be realistically several years when we will be applying our approach to the master data within other projects working on the application architecture. By a gradual application of rules we begin improving data quality and solving current issues. The fact that this will be a long track run in case of a developed information system should be accepted as a matter of fact and we should not balk at it. We will start seeing benefits soon since we start setting good system grounds and this effort will pay back in future.
What will be the benefits of the above solution?
- Correct, reliable and accurate data will enable the right informed decision.
- Data consistent across systems will have the necessary credibility and users will be willing to work with them further.
- If you avoid issues with data, your productivity and process speed will increase. User will not have to wait anymore for IT correcting the data.
- Costs of other IT projects and day-to-day IT operations will go down, because data corrections will not be needed any more, same as complex connecting to unconsolidated data.
Initial situation when starting with mater data management
We may need to launch the MDM project at the time of another company acquisition. Consolidation of information systems with the parent company may be a too expensive and difficult project. Sometimes we do not even think about consolidation, e.g. due to legislation reasons or because the daughter company may be sold again in the long term and therefore it has to keep its source information systems. However, data shall be shared to support business processes. Another example is an operation of legacy systems, that do not share data with other applications. Maintaining data that are entered in parallel to several systems is not sustainable due to constant issues with data consistency.
If we speak about CRM or ERP, we always associate such a topic with a particular information system that the company operates. However, with MDM we care mainly about the selected architecture, approach to maintenance and distribution of data as we described it in point 6 above. Despite that a particular technology can save a lot of work with master data.
Company that is starting with Master Data Management often faces a question in which system Master Data should be maintained. Sometimes people decide to create new master data in an existing system that already keeps some master data. This may sound as a logical step but it can also be a very bad decision if such a system is e.g. ERP. ERP system is typically not easy to reconfigure to take-over data about customers from a central master data repository and therefore we may want to maintain the data primarily in ERP and transfer them to the central master data management system as a secondary system only for customer data. However, it should not be an argument to maintain all other future master data in ERP. Why? There are several reasons for that.
- ERP systems support complex business processes and therefore any change requests are often reviewed in a special change management process. Deployment of a new master data table can be very difficult. User interface of ERP systems is often also less user friendly.
- Typically only some users work with ERP. Therefore, we would have to provide data stewards with licences without them actually having to work with ERP itself.
- ERP also does not support specific Master data functionality.
What should be the Master Data system functionality?
We have said that Master Data are nontransactional and relatively static, i.e. they do not change frequently. Despite that or maybe rather due to that the change is a critical master data feature. If we record transactional data (e.g. invoice), we do not change this record anymore. If we realize later on that the invoice should have been issued to another customer or by another company in the holding, we will either cancel the invoice or we will create a new transaction (credit note) that will effectivelly cancel the invoice.
We have a different situation with master data. If we change e.g. the product composition, we will plan its validity by means of properties Valid from and Valid to. A similar principle shall be used for exchange currencies when we will set from when till when certain currency has a certain value for us. If we give our customers several colors to choose from and internally we are changing a vendor resulting in a small change in the color shade we should define in the data that e.g. a red color was from a vendor A with shade X till 20. 4. 2019 and from 21. 4. 2019 it was vendor B and shade Y. Master Data system should ensure that there will not be inconsistencies caused e.g. by overlapping of these intervals or that we have periods for which the values are not known.
Figure 3: Time validity of the red colour – change of vendors in the course of time.
This functionality is not available in ERP systems and therefore ensuring consistency will be on data stewards with a high risk of error given by human factor. Master data structures, data rules, screens for people entering the data, interface for applications sending or subscribing can be developed by data stewards, because the system supports these functions and is ready for that. In case of ERP system we will have to involve external or internal developers for these activities which will make everything more costly and it will also take much longer to impelement these changes. Therefore, acquiring speciliazed software supporting master data may be a decision that will pay off a lot. It will facilitate work with master data and will have a solid place in our enterprise architecture. On top of that such a tool does not have to be expensive.