Not all combinations can be tested
It is clear that it is not possible to test all these combinations separately. For this reason, it is wise to determine beforehand what combinations must be tested. Creating a thorough test plan that includes a sophisticated test strategy, different testing scenarios and concrete test cases, is important in order to actually round off Business Intelligence projects. It enables us to perform qualitative tests, yet in a pragmatic manner.
Large amounts of data
While testing it is not sensible to start using large amounts of data, simply because it is time consuming: loading a month’s data into the data warehouse, for example, can take several hours. If we discover crucial errors (during testing), all data will needs to be reloaded – after the error has been resolved –, which again takes time.
Start with a small test set
All in all, it is better to start with a small ‘restricted’ test set to save time on the loading and reloading of data. Generally, the major errors will surface and they can be solved quickly. Once these major errors are resolved, we can load larger test sets in order to run performance tests.
Figures might differ from the general ledger
It is quite possible that the acceptance test shows that there are differences between the figures in the Business Intelligence system and those in the general ledger, for example. It often happens that the revenue, for instance, does not exactly match.
Adjust the data warehouse and rerun the tests
While analyzing the loading processes in the data warehouse, we may conclude that these processes have not been developed in accordance with the specifications in both the functional and the technical design. If this is the case, we ‘simply’ adjust the data warehouse and repeat the test.
Open up the interface to the general ledger
However, it is also possible that the load processes are fine and do match the specifications. In that case, we must ‘open up’ the interface between the source system and the general ledger, which may be very interesting! We may discover that the interface – for some reason – does not include certain transactions, or that revenues are manually booked or corrected in the general ledger, outside the operational source system. In short: testing the data warehouse and the Business Intelligence system is difficult.
Additional budget sometimes needed
Therefore reserving additional budget in the planning is to be recommended in case the Business Intelligence system has to be (re-)aligned with other systems such as the general ledger.
When the BI system is rejected
At the end of the testing phase, the users either reject or accept the Business Intelligence system giving reasons either way. The BI system may be rejected due to crucial errors such as incorrect figures, poor performance, poor data quality, poorly designed reports or the fact that the system does not meet the criteria for maintainability and scalability. If the system is rejected, we will need to return to an earlier stage of the project. Which stage that is depends on the reason for rejection.
Go back to the drawing board
In some cases, it is even necessary to go back to the drawing board in order to revise the blueprint. In any case, we will need to carry out a thorough evaluation. Lastly, if only a few –less crucial – key performance indicators are not yet correct, it is wise to persuade the users into at least accepting parts of the Business Intelligence system. This prevents us from remaining in the test phase for weeks on end without the BI system being put into production. However, if we do this, we must be a hundred percent sure that the indicators we do accept are in fact correct and that there are no fundamental errors in the Business Intelligence system.
Daan van Beek MSc, speaker, trainer and author of the BI & AI book 'Data Science for Decision-Makers and Data Professionals'. He is the CEO of the Passionned Group and you can follow Daan on Twitter.
