What is Process Mining?
Process Mining is the umbrella term for a collection of techniques, tools, and methods that, using event logs, allow you to uncover, visualize, analyze, monitor, and improve the actual course of business processes. This way you can screen your processes for bottlenecks, wastage, deviations, and inconsistencies in the prescribed process flow.
Process Mining
Process Mining works roughly as follows: people who use computer systems within an organization leave traces. These large amounts of stored data can then be analyzed to reveal how processes run, with the ultimate goal of making them more efficient.
Figure 1: Coherence of operational processes and process mining: a simple purchasing process.
Process Mining is a field that has been under the radar for a long time. Before Process Mining took off, people spoke of Workflow Mining, for example. Workflow Mining can be seen as a predecessor of Process Mining. How should organizations approach the great potential of Process Mining?
Process Mining in 13 logical steps
Process Mining is an analytical process that ideally follows a three-phase plan:
Each phase has a number of steps that are logically connected. In total, you go through 13 steps.
Phase I: Applicability Analysis
The entire process of Process Mining starts with a clear problem analysis. However, there are a few preconditions. There has to be a problem: an undesirable situation which can be improved. The problem also has consequences for the organization’s performance. There is also a problem owner who recognizes the problem and is willing to do something about it. They have the authorization to do so. In this phase, you go through the following steps:
- Drafting a problem analysis
- Identifying the process context
- Determining process data
- Identifying system context
- Determining applicability
Phase II: Data analysis
Data analysis is a phase where the data is inspected, cleaned up, transformed, and modeled to then extract the most valuable information from the data. You will go through the following steps in this phase:
- Creating an event log model
- Collecting raw process data
- Cleaning up raw process data
- Determining the validity of the process data
Phase III: Mining analysis
In this last phase, you work towards the end goal:
Concrete recommendations for process improvements. You will also evaluate the entire process of Process Mining.
This is necessary because Process Mining doesn’t stop here. Process Mining is in fact a continuous process. You’ll go through the following steps in this phase:
- Applying Process Mining
- Interpreting results of mining analysis
- Recommending process improvements
- Evaluating the Process Mining process
Process Mining is NOT Data Mining
The concepts of Data Mining and Process Mining are very similar at first glance. Both techniques have a direct relationship with business processes. Both concepts fall under the umbrella of Business Intelligence, where users strive to gain valuable insights for the organization based on Big Data. Yet there are most certainly differences to be found between the two concepts.
Looking for hidden patterns
The aim of both Data Mining and Process Mining is to provide key insights into processes, enabling users to make better decisions. For both disciplines, the role of artificial intelligence and algorithms is becoming increasingly prominent. The discovery of causal relationships and hidden patterns is central to this. Hidden patterns are patterns that would otherwise be invisible to the human brain.
The Artificial Intelligence handbook
Process mining, data mining, and text mining are just some of the techniques covered in the AI book 'The intelligent, data-driven organization'. Learn how to become an AI-first organization, how to make the business case, and how to stay ahead of the competition. With this book you get a complete recipe for data-driven working in 10 steps.
Time stamps make all the difference
Data Mining is a computer technique to detect complex relationships and non-obvious patterns in (big) data. With Data Mining you analyze data to discover or predict patterns. For example: which ad campaign leads to the highest conversion rate or which product group performs best in the supermarket? There is another important difference. The input of Data Mining consists of tables of data. The input of Process Mining consists of so-called event logs, audit trails, data, and events from the IT system which are also time-stamped.
The three basic forms of Process Mining
The goal of Process Mining is to optimize process efficiency, reduce process costs, and reduce process complexity. In Process Mining, the event log is always the central starting point. These log files are already kept and stored somewhere by default in most information systems but are usually poorly accessible. Process Mining can be used to improve the efficiency of processes and to get a better understanding of them.
Process Mining has three basic forms in which the event logs play a key role:
Figure 2: Basic forms of Process Mining (source: Process Mining Manifesto, Van der Aalst, W. et al.)
- Discovery: this basic technique uses an event log as the basis for a model. No a priori information is used in this process. This form of Process Mining is most commonly used in practice. The outcome can be a model that you will use in basic form 2.
- Conformity checking: here, an event log is used to confirm whether or not the reality as logged matches the model, and vice versa. Applications are mainly in the field of procedural and organizational models, declaration processes, business rules and policies, and legislation and regulations.
- Enhancement: the idea behind this form of Process Mining is to extend or improve an existing process model by using information about the actual process captured in event logs. Time stamps from the event logs, for example, expose bottlenecks and provide insight into service levels, lead times, and frequencies.
Sample Questions on Discovery
Process Mining is not an abstract, theoretical exercise but, if applied in the right context, actually answers very practical questions within different domains.
Possible analysis questions in the Customer Journey:
- What are the most common payment methods?
- Which factors influence the customer journey positively or negatively?
- What is the most common first point of contact between customer and company?
Possible analysis questions in Customer Support:
- How long does it take to process support requests?
- How long does the customer have to wait?
- What possible variations does the support process have for customers?
Possible analysis questions in Claims Handling:
- How long does it take to replace defective goods?
- Which common product problems lead to more than average complaints?
- Which product versions lead to the most amounts of refund claims?
Five examples of process deviations
In the context of Process Mining, conformity checking is an important tool to achieve a clear process diagnosis. The tool helps to investigate why certain processes are stalling, or why they deviated from the regular path. In concrete terms, you can distinguish five points of leverage with which you can make improvements:
- Some activities should not have been done in the first place.
- Other activities were carried out by the wrong person.
- Still other activities were carried out too late.
- One or more planned activities were not carried out at all.
- Two activities were mixed up with each other either intentionally or by mistake.
Process Mining software: how to distinguish the wheat from the chaff?
Process Mining is gaining more and more popularity. There are now several dozen vendors active worldwide who claim to have developed Process Mining software and platforms that, according to them, perfectly meet customer needs. These vendors particularly emphasize the power of the algorithm underlying the process discovery tools. For customers, however, it is difficult to judge such claims on their merits. The differences in quality of process mining tools, perceived or not, can be significant in practice. Functionality is an important selection criterion when looking for the right software vendor. Aspects such as data visualization and usability also deserve more attention in general.
Tool selection: make a longlist based on 7 weighting factors
When evaluating the functionality of process mining software, pay attention to things like:
- User friendliness of the software: is there a clear, intuitive navigation?
- The number of relevant features: can users still see the forest for the trees, or are important features missing?
- The visualization possibilities: flow charts, decision, and connector symbols in all colors of the rainbow quickly lead to selection stress.
- Integration with other analysis tools: interoperability and compatibility are the key concepts here.
- Clear data representation: can you present the data in the right form that is understandable for everyone?
- The price: the license price per user is still an important factor when choosing a supplier.
- Collaboration possibilities/information sharing: Process Mining is not a solo project, but a project that is characterized by chain collaboration.
In addition, take into account the specific infrastructure in which you will deploy the process mining tool. Is access possible via any computer? Is there a mobile application? Will data be stored in the cloud? What is the data processing time? What about personal data protection and security? How fast (or how slow) is the Process Mining algorithm with large amounts of data? and so on.
5 obvious advantages of Process Mining
Although Process Mining is a relatively new field with a limited number of reference cases, a number of obvious advantages have already emerged which can be summarized as follows:
- Process Mining fulfills an important bridging function. This is between the traditional Business Process Management (BPM) and Workflow Management (WfM) systems, which took little account of so-called event data, and the more modern Data Mining (DM), Business Intelligence (BI), and Machine Learning (ML) systems, which take data as their primary starting point and are less concerned with end-to-end process models.
- Process Mining enables the prediction of processes. Among other things, Process Mining software allows you to “reenact” processes and detect problems in the business process, using Gaming techniques or not. You can also use Process Mining software to replay (simulate) processes with this same goal in mind. The possibility of using Process Mining to simulate processes and using Artificial Intelligence (AI) to predict the future course of processes as accurately as possible is promising. Time stamps play a crucial role in this type of analysis, as they can precisely date the chronological order of events.
- Process Mining makes intelligent use of Big Data. Process Mining’s solutions focus on both improving organizational performance and the compliance that your employees are expected to meet. Process Mining is at the same time also the hinge between the analysis of process models (simulation, verification, optimization, gaming, and so on) and the data-oriented analysis (Data Mining, Machine Learning and Business Intelligence).
- Process Mining eliminates bottlenecks. The challenge of Process Mining is to turn the vast amounts of Big Data into valuable insights that say something about process performance and compliance. With Process Mining, you get a better understanding of the bottlenecks, inefficiencies, anomalies, and degradation risks within your critical processes.
- Process Mining improves operational processes. Obtaining non-trivial, process-related insights through the analysis of event data is a relatively new development with which PhD students at universities (including R.P. Jagadeesh Chandra Bose), consultants, and commercial software vendors are increasingly gaining traction. Using the insights obtained, for example, in the form of process models, you can monitor and improve operational processes. According to scientists, the application of Process Mining to event data from information systems has already led to amazing insights and successes in practice. The application of Process Mining to event data that is not directly derived from information systems, but from all kinds of devices, such as X-ray machines and CT scanners, copiers, printers et cetera, is also booming.
Where can I deploy Process Mining?
Process Mining is proving its usefulness in numerous industries. Process Mining is a relatively new application that can be used to analyze Big Data in a variety of sectors: from credit card companies, industrial companies and real estate, to government agencies and hospitals. Think, for example, of the analysis of all the process steps that are necessary for the treatment of a hospital patient. Or the municipal handling of notices of objection for the Valuation of Immovable Property. Or all the actions that a housing corporation must perform before a house is delivered to a new tenant in accordance with the regulations and procedures. Process Mining is also proving its worth in the testing of so-called wafer steppers in the semiconductor industry.
5 pitfalls when applying Process Mining
As soon as you start applying a new technique, all kinds of pitfalls or unwanted side effects suddenly appear. Process Mining is no different. The pitfalls in Process Mining are on the one hand related to the people and on the other hand related to the data. Below, we list the 5 biggest pitfalls of Process Mining.
- Pitfall 1: Consultants, auditors, quality managers, and process owners are still relatively unfamiliar with the phenomenon and have little idea of the possibilities that the current Process Mining tools offer. You can get your Six Sigma Black Belt or be registered as a certified internal auditor without ever having delved into Process Mining. That says something about the status of the field.
- Pitfall 2: Due to the current emphasis on machine learning and artificial intelligence (AI), people don’t realize that Process Mining is something completely different.
- Pitfall 3: Middle managers in particular are fearful of the results and insights that come from applying Process Mining. The increased transparency into the flow of processes can reveal mismanagement, inefficiencies, and compliance issues. This comes across as threatening.
- Pitfall 4: Poor data quality and access to data can also be a barrier. The 80-20 rule holds true: the preliminary process (locating, selecting, extracting, and transforming the data) gobbles up 80% of the time. For Process Mining itself, usually no more than 20% remains.
- Pitfall 5: Process Mining usually brings up data quality issues that need to be addressed immediately. This is distracting.
How does Process Mining fit within data-driven work?
Most information systems continuously store exactly what happens in the form of so-called event logs. The difficulty is that this data is often hidden somewhere in the system or the computer. In addition, the volume of that data stored by traditional workflow systems, as well as by other systems, is growing explosively. One of the biggest challenges for organizations to innovate, grow, and strengthen their competitive position is to make intelligent use of all that available data. Within this context, Process Mining fits perfectly with the existing concept of the intelligent, data-driven organization and the growing interest in data-driven work and information-driven work based on the data and actual events.
3 tips for maximum return on Process Mining
In addition to avoiding the five pitfalls mentioned above that can crop up with Process Mining, you can also benefit greatly from the practical tips below.
- Elephant paths are effective but undermine the process. The deviant routes that people take in their pursuit of goal realization are sometimes compared to elephant paths when Process Mining is used as a lens (Van der Aalst, 2013). An elephant path, sometimes also referred to as a shortcut, is an unofficial cycling or walking path that is intentionally or unintentionally created over time by users of the regular cycling and walking paths. Also in organizations, people sometimes, consciously or unconsciously skip certain process links because they themselves are convinced that this leads to results faster. In practice, it can happen, for example, that a Central Purchasing Department is bypassed because people are not in the mood for all sorts of bureaucratic procedures, such as tenders, bid competitions, and so on.
- Walk the Happy Path and stay in the Happy Flow. It will be clear that the aforementioned elephant paths can cost the organization a lot of money, for example, because they miss out on purchasing discounts that the Central Purchasing Department has agreed with some preferred suppliers. Ideally, therefore, employees in the organizations do not deviate from the beaten path and follow the mapped-out route. Within the field of Process Mining, we then speak of a so-called “Happy Path” or “Happy Flow.”
- Realize that business processes are the foundation of your business strategy. An intelligent organization monitors its performance on a continuous basis from a translation of strategy to KPIs and to so-called Process Performance Indicators. Process Mining software is an indispensable link in the pursuit of a data-driven, intelligent organization.
Process Mining: a good mix of science and practice
Process Mining arose in part as a result of scientific research by TU Eindhoven. The inventor and one of the founders and promoters of this new field in the Netherlands is Eindhoven professor Wil van der Aalst who is building Process Mining software from within the university community together with PhD students. Meanwhile, there are also several dozen commercial software vendors that support the concept of Process Mining with their software solutions. With the rise of streaming social media, sensor data within the Internet of Things, and techniques such as RFID, much more event data will become available that lends itself ideally to analysis using Process Mining software.
Video explaining Process Mining
Process Mining is a specialized field
In summary, Process Mining is a perfect way to visualize and analyze the actual execution of business processes by using logged data about these processes in operational/transactional systems, such as for example an ERP system. Process Mining on the one hand requires an understanding of business processes, process analysis techniques, and process compliance. On the other hand, it also requires specific knowledge and skills in the generation and validation of reliable process data and the application of Process Mining software. Process Mining is therefore a specialized field. Our Process Mining consultants are happy to help you further.
About Passionned Group
Passionned Group is a leading specialist in Process Mining issues and solutions. Our seasoned process experts help larger and smaller organizations transform into intelligent organizations. Every other year we organize the Dutch BI & Data Science Award™, the election of the Smartest Organization of the Netherlands. 
