In this piece, I will cover the power of prototyping a BI product and how this creates opportunities for change such as implementing Agile Master Data Management (MDM) into your product development cycle. One of my responsibilities as a test analyst is ensuring the sanctity of the data warehouse and to perform quality checks on the BI products being developed. For certain projects, I’m asked to organize an acceptance test or perform a demo in which I present the BI product to the business and explain in brief the tests I have performed. In most cases, this demo occurs at the rear end of the development cycle of the BI product; it is nearly finished and the business is eagerly waiting for the BI product to go live. If this method is not swift enough and/or the projects seeks for an alternative to scrum, prototyping can come in hand.
A prototype can be a data mart which contains most of the relevant source data or a dashboard. This method enables the business to get acquainted with the prototype (read data) and determine which data can be used to derive meaningful information and attain actionable insights in their business operations. But there are other benefits as well:
- Getting business involved at an early stage will help the project team to iteratively develop the prototype into a beta version.
- Discussing the content of the prototype will help the business to become crystal clear about the why (do we need this information? What’s the main purpose of the BI product?), and the how ( for example which information do we need and which ICT information standards are going to be used)
This method gives the business plenty of time to figure out how it will support the business process and fine tune it accordingly. Furthermore, this process might also uncover future business needs in the early stages of the development process.
Once the prototype has iteratively been evolved to a beta version, the project team can seize the opportunity to kick-start parallel activities in the domain of agile MDM:
- identify master data,
- set up and/or maintain a business glossary
- document meta data
discuss data quality issues and how to deal with these.
Concerning the latter, in case the data quality is really poor, it is best practise to come up with a pragmatic strategy where you get the business involved at an early stage. Here are two examples on how to manage data quality issues:
Categorizing data into categories via a Risk/Impact analysis. Asking key questions will get the process started: what's the impact if this data/information isn't correct?
For example: customer data: you don't want to send a health insurance bill to a client who's just deceased.
Product information: having the correct sales figures (which type of cars of have been sold) is far more important than which colours are most popular.
Categorize data into usage categories; which data/information will be mostly used?
Prioritizing your data Risk or usage categories makes it possible to prioritize data quality issues as well and will make it easier to manage data quality issues. Bearing in mind that data quality issues in the data warehouse chain do not have the same level of urgency for the source owners, you might want to start discussing the impact of poor data quality/information on various levels in the organization. This will help them to get a greater understanding and level of support (read time and money) to fix the data quality issues at the source.
To conclude; whether your organisation needs to abide to compliancy regulations, pass internal audits, or start working towards integrating Hadoop technology with your data warehouse, all of these can be used as a positive drive to incorporate agile MDM practices into the product development cycle. This can either be done by the project team itself or by adding MDM professionals to your development team. And yes, it’s easier said than done. Being open minded and willing to grow, learn and change, both as an individual and as a group, will eventually lead to success. It’s also more important than doing the tasks at hand. How about you? Do you see any opportunities for change?