CollabDays Hungary 2024 - Discovering Process Mining and Task Mining with Power Automate

Editor's Notes

• #5: Professor Wil van der Aalst has published a total of over 400 books, journal articles, book chapters, conference papers and reports. He refers to process mining as “the bridge between data science [which includes algorithms, machine learning, data mining and predictive analytics] and process science [which covers operations management and research, business process improvement and management, process automation, workflow management and optimization].”
• #9: 2013 – Rasto Hlavac, founder of Minit with the team of process mining specialists, top software architects and programmers started to work on the Minit v1 2015 – Minit v1 released – main product focus was from the beginning on how the business executives can get the value out of process mining from the day 1. 2017 – Minit corp. founded, followed by the seed investment from Earlybird VC 2018-2019 – major customers in Europe, high product and customer focus 2019-2021 – product innovators, first on the market with Hierarchical process mining (patent pending), AI powered Route cause analysis and AI powered simulations 2020-2021 – major growth in North America 2021 – Minit named top vendor in proceed mining research by Everest Group, Nelson Hall and ISG 2022 - Microsoft acquired Minit
• #11: Process mining is a research area and technology that helps businesses understand their real processes, how they're operated, and identify opportunities for improvement, automation, and digitization. The process mining capability in Power Automate works with the existing Microsoft platform and ecosystem to provide end-to-end solutions and enable faster business decisions. By extracting event data from your system of records, it visualizes the processes happening in your organization, allowing you to customize your process mining reports, compare processes, determine the root cause of inefficiencies, and monitor KPIs. Overall, the process mining capability is a valuable tool for businesses seeking to improve their operational efficiency and make informed decisions. https://learn.microsoft.com/en-us/power-automate/process-mining-overview
• #12: The business value of the process mining capability lies in its ability to help organizations gain insights into their operational processes and identify areas for improvement. By analyzing event data from various sources, the process mining capability provides a clear view of how processes are actually being executed in practice. This helps businesses to: Improve operational efficiency: Process mining can help identify bottlenecks and inefficiencies in processes, allowing organizations to streamline their operations and improve productivity. Enhance customer experience: By identifying process pain points and eliminating them, organizations can improve the customer experience and increase satisfaction. Optimize resources: Process mining enables businesses to identify opportunities for automation, reducing the need for manual intervention and allowing resources to be allocated more effectively. Ensure compliance: Process mining can help organizations to identify non-compliant processes and take corrective action to avoid legal and financial risks. Improve supply chain management processes: Create processes that help warehouse and operations managers gain insights into the material flow in the warehouse. This can help improve the performance of the warehouse.  https://learn.microsoft.com/en-us/power-automate/process-mining-overview#benefits-of-the-process-mining-capability
• #13: Business examples Overall, process mining provides valuable insights that can help organizations improve their operations, enhance customer satisfaction, and stay competitive in a rapidly changing business landscape. <cover Slide content> These areas represent key challenges and opportunities for businesses to improve their operations, stay competitive, and meet the needs of their customers. https://learn.microsoft.com/en-us/power-automate/process-mining-overview#business-examples
• #14: These are the main components for the process mining capability https://learn.microsoft.com/en-us/power-automate/process-mining-overview#components
• #15: Data requirements Event logs and activity logs are tables stored in your system of record that document when an event or activity occurs. For example, activities you perform in your customer relationship management (CRM) app are saved as an event log in your CRM app. For process mining to analyze the event log, the following fields are necessary: Case ID (caseId when mapping) Case ID should represent an instance of your process and is often the object that the process acts on. It can be a "patient ID" for an inpatient check-in process, an "order ID" for an order submission process, or a "request ID" for an approval process. This ID must be present for all activities in the log. Activity Name (activityName when mapping) Activities are the steps of your process, and activity names describe each step. In a typical approval process, the activity names may be "submit request," "request approved," "request rejected," and "revise request." Start Timestamp (startTimestamp) and End Timestamp (endTimestamp) Timestamps indicate the exact time that an event or activity took place. Event logs have only one timestamp. This indicates the time that an event or activity occurred in the system. Activity logs have two timestamps: a start timestamp and an end timestamp. These indicate the start and end of each event or activity. Where to get log data from your application The process mining capability needs event log data to perform process mining. While many tables that exist in your application’s database contain the current state of the data, they might not contain a historical record of the events that happened, which is the required event log format. Fortunately, in many larger applications, this historical record or log is often stored in a specific table. For example, many Dynamics applications keep this record in the Activities table. Other applications, like SAP or Salesforce, have similar concepts, but the name may be different. In these tables that log historical records, the data structure can be complex. You might need to join the log table with other tables in the application database to get specific IDs or names. Also, not all events that you're interested in are logged. You may need to determine what events should be kept or filtered out. If you need help, you should contact the IT team that manages this application to understand more. Connect to a data source The benefit of connecting to a database directly is keeping the process report up to date with the latest data from the data source. Power Query supports a large variety of connectors that provide a way for the process mining capability to connect and import data from the corresponding data source. Common connectors include Text/CSV, Microsoft Dataverse, and SQL Server database. If you're using an application like SAP or Salesforce, you might be able to connect to those data sources directly via their connectors.  https://learn.microsoft.com/en-us/power-automate/process-mining-processes-and-data
• #16: After you select the data source you want to use, you're taken to the Power Query Editor. The Query Editor is a powerful tool to transform your data. Here are some reasons why you might want to transform the data: You might not be interested in all the activity names that are logged, and so you want to filter for specific activity names that are important to the process you're trying to mine. You might want to rename some of the activity names to be more descriptive and understandable. This is often not the case with names in a database. You might be interested in only a specific date range, and not the entire history of data. You might want to combine multiple ID columns to form the case ID. Filter activity names Select the caret next to the activity name column to bring up the sort and filter menu. If there's a message that says List may be incomplete, select Load more. Select only the activity names that you want to analyze. Uncheck any name you want to exclude. Rename activities Ensure that the Activity Name column is selected. Above the toolbar, select the Transform tab. On the toolbar, select Replace values. Reduce the number of total records One strategy for reducing the total number of records is to use only the latest records. To do this, you need to first sort the data by time. Combine multiple IDs You can use Ctrl + click to select multiple columns that you want to combine. On the Transform tab toolbar, select Merge columns. Map data Mapping tells the process mining capability what columns represent case ID, activity name, or timestamp. In the toolbar, select Map to entity. https://learn.microsoft.com/en-us/power-automate/process-mining-transform
• #17: Process map The process map empowers you to visualize and gain insights from processes. By looking at a graphical representation of how your business processes are performed, you can glean insights about where opportunities exist. Activities describe tasks or actions, the sequence of which makes up a business process. Activities can be performed by humans, or, in the case of automation—by machines. In the process map, different activities appear as nodes, and transitions between activities appear as edges. Each process sequence will have a start and an end. Different activity combinations and variants are shown separately on the process map. A process variant is a unique sequence from start to finish of the process. Each variant differs from the others by at least one activity. You can easily switch between different layers and associated metrics on the process map using the controls on the top left of the process map visual. Use KPIs and visualizations for analytics You'll get several prebuilt KPIs and visualizations to help you to understand your process. You can filter by selectors, such as Activity and Case Id (Premium), and custom filters (Premium) if you added the custom attributes (data columns) when you uploaded your data for analysis. The screenshot here is an example of visualizations and analytics you'll see in the premium version. If you didn't purchase the premium version, you'll have access to the default version. The top of the default version shows only the four KPIs listed in the KPIs section in this article and not the two KPIs with (Premium) in the title. Also, you won't see the Average duration of cases over time chart. KPIs These KPIs are the same metrics that you see at the top of your report. Filters pane To drill down into the process, use the filters in the filters dialog. To see the filters, select Filters in the upper-right side of the Summary tab. Time Analysis (Premium) The Time Analysis (Premium) doesn't have an alternative default view. It allows you to drill down into time bottlenecks by cases, variants, and activity. The Time Analysis (Premium) also shows you average time spent per case and per variants ordered by the time spent. Variant DNA The Variant DNA view is available in both the default and premium views. To show the order of all the activities in each variant sorted based on the variants that happen the most often, select the Variant DNA tab. The activities are color coded and abbreviated to quickly show a high-level view of the order of the activities that occur. This also helps identify non-compliant processes, self-loops, and loops quickly. https://learn.microsoft.com/en-us/power-automate/process-mining-visualize
• #18: The process mining capability can help you gain valuable insights and optimize your cloud flows. You can visualize your flow's performance, identify bottlenecks and opportunities for improvement, and monitor for performance drifts. By utilizing the process mining capability to analyze your flow's run history with process mining techniques, you can generate these insights directly from the flow details page. Prerequisites To generate process insights, you must meet the following prerequisites: Flow runs must have been generated within the last 28 days. Only the owner of a flow has the ability to generate process insights. You must have access to Microsoft Dataverse (i.e., a low-code data platform that allows you to store and manage data for your applications) You must have the environment maker role. https://learn.microsoft.com/en-us/power-automate/process-mining-cloud-flow-process-insights
• #19: Process mining is a technique used to extract insights from event logs of a process. During the process, it is common to make edits to the mapping or query of an existing process. The following are some of the reasons why you might want to edit your process: Your data might have multiple ID columns that can be used as case ID, and changing what you use as a case ID might help you discover more insights. Some attributes may be mapped to something more specific, which can provide additional options during analysis, like financial or resource attribute types. If you have transformed your data in Power Query, you may want to adjust the transformation if the resulting process is not what you need. Edit mapping and query To edit your mapping or query, follow these steps: In the process details page, select Setup to return to the mapping page. To change the case ID or other mappings, select the values in the dropdown menu that correspond to the column name from the original data. Refresh data If you have a transactional data source, like Dataverse or SQL, select Refresh data to update your process with the latest data from your data source. The refresh process may take some time, and you will be prompted to confirm the refresh by selecting Refresh again. Schedule refresh To keep your data updated, you can use the schedule refresh feature. Select Schedule refresh on the right side of the Data Source card to display the schedule refresh panel.  Disconnect data If you want to change the data source for your process, you can easily do this. For example, you might have been using a CSV file before, but now you want to connect to a transactional data source instead https://learn.microsoft.com/en-us/power-automate/process-mining-data-source
• #20: When you create a process in the process mining capability, only you can see it. But you might want to share the analytics and insights with others. Share your process mining processes with others in your organization so they can also glean insights to make decisions. There are various ways you can share your process. Share from process lists page Share from process details page Share from analytics report page Share panel Share action Share a process with a co-owner Share a process with a viewer
• #22: Easily onboard your process mining data with process mining templates. Templates help you start mining your processes for insights with just a few selections. You'll avoid the data identification and manipulation that's required with traditional process mining tools.  https://learn.microsoft.com/en-us/power-automate/process-mining-azure-templates
• #23: Take advantage of additional enterprise process mining capabilities with the Power Automate Process Mining desktop app. For example, there are many ways to break down your process cycles into smaller pieces for analysis. With the Power Automate Process Mining desktop app, you can easily delve into the details of your processes. To fully use the Process Mining desktop app, you need an add-on to unblock the enterprise process mining capabilities. Business example Power Automate Process Mining analyzes data from processes that you create using the process mining capability in Power Automate. For example, you can get deep insights into how your processes run, uncover the root cause of problems, and build useful outputs for the rest of your organization. https://learn.microsoft.com/en-us/power-automate/minit/minit-desktop-application-overview
• #24: https://learn.microsoft.com/en-us/power-automate/minit/minit-desktop-application-overview#components
• #25: A process map provides a view of the activities performed in a process and their sessions. It represents the behavior of the process captured in data by means of activities and directed edges. The process map corresponds with the mining result of the process. Process map activities (also known as nodes) represent a unique activity, or unique values of different selected mining attributes, performed, executed, or passed through in the process. Chart edges represent transitions between individual events and describe the sequence of activities in the process. All nodes/activities starting process instances are connected to the start node. All nodes/activities ending process instances are connected to the end node.  Chart nodes starting and ending process instances contain colored indicators in the right part of the node. The green indicator represents the initial activity (upper right), the red indicator represents the final activity (bottom right) of the process.  When you open a project view for the first time, a process map with the default view parameters displays. The structure of the process map and the information vary depending on your view settings. Map clustering allows you to visually encapsulate activities in the process map view into clusters. You can also do this in the social chart view with a resource attribute selected.  Hierarchical process maps use clustering technique to enable you to visually encapsulate activities in clusters. They group them into further clusters in the process map view or resource in the social chart view.  The social chart shows the important parts and parameters of your process through links and dependencies between resources involved in the process. The Variant panel contains a list of variants created during the process reconstruction. Variants can be arranged according to the number of process instances they group or by the number of events in case of variants.  https://learn.microsoft.com/en-us/power-automate/minit/process-map
• #26: With process animation, you can visualize process development over time. You can watch the animation play over your process map or social graph on the Animation screen. Control animation speed, advanced settings, and more on the Animation screen. The animation playback control is located in the bottom part of the screen. Use it to start the animation, pause it, or skip to any position in the process. The information about the actual time in the process appears to the right of Play animation. https://learn.microsoft.com/en-us/power-automate/minit/process-animation
• #27: Statistics are shown in the top panel. The listings of specific records displayed in the statistics appear in the bottom panel. The statistics always reflect the data after filters, if any, have been applied. The following types of statistics are available: Case overview: Statistics per single cases with case level metrics. To learn more, go to Case overview statistics. Activity statistics: Activities that display their various event level properties. To learn more, go to Activities (event-level) statistics. Attributes imported with the process marked as Others and Source have the same statistical summary as the statistics for the activity. Edge statistics: Process paths (edges) metrics. To learn more, go to Edge statistics. Cases duration influence: Analytical insight that evaluates the influence of case-level attributes (including financial) on the mean case duration. To learn more, go to Case duration influence in this article. Business rules: Overview of defined business rules. To learn more, go to Business rules overview. You can export your statistics chart in .png format, or your table of records in .csv format. Stastistics samples Statistics allow you to quickly identify issues in the process. The following sections provide examples. <Ist image> Shows the average activity time of resources. As in the following screenshot, it's apparent that some resources need much longer time to complete their activities than others. You can narrow your list of resources to get more data. <2nd image> Shows the number of cases with varying duration intervals. If the chart shows that there are rare cases in the data that distort the average values, such cases might require attention, or possibly should be excluded from process analysis as non-standard. https://learn.microsoft.com/en-us/power-automate/minit/statistics
• #28: Root cause analysis (RCA) allows you to find hidden connections in your data. For example, it helps you understand why some cases take longer to complete than others, or why some cases get stuck in reworks while others run smoothly. RCA will show you the key differences between such cases. RCA can use all your case level attributes, metrics and custom metrics to find connections among them, and a metric of your choosing. How RCA works The RCA algorithm will compute a tree structure where each node will split the dataset into two smaller parts. This is based on one variable where it finds the best correlation between the variable split and the target metric. From this, you can see the hidden connections in the data. This is where it will tell you which combination of attributes will influence the case in which way. RCA example In this example, we want to see the root cause behind the case duration. In the data, we have case level attributes supplier country, supplier city, material, total amount, and cost center. The average case duration is 46 hours. By looking at each value of each attribute separately, we can see that the highest influencer of case duration is when supplier city is Graz, which on average increases the duration of the case by additional 15 hours. From this initial analysis, we can see that the other values of attributes influence the target metric far less. However, when we compute the tree model, we can see that the computation above is misleading (as in the screenshot). The tree structure looks like this: The first split is the data along the material variable. The data with aluminium is on one side and all other materials is on the other side. The aluminium branch is split further by supplier country into Germany and Austria. The Austria branch continues with a split by supplier city, with Graz on one side and Vienna on the other. In the node Graz, the average case was 36 hours slower than the overall average duration of 46 hours. https://learn.microsoft.com/en-us/power-automate/minit/root-cause-analysis-overview
• #29: You can use filters to adjust which cases will be included in the process analysis. For example, you can use filters to define only a certain time period, only specific resources or scenarios in the process, or modify cases by excluding activities that aren't important for the current analytical problem. Filtering can also be helpful in identifying and focusing on the problematic parts of the process. A filter is a rule or a set of rules determining which cases and/or events from the process will be later included in process mining. There are two groups of filters available in the Power Automate Process Mining desktop app: event level and case level.  Event level filters Event level filters modify cases by removing events based on the applied filter criteria. Event level filter types: Event attributes Subprocess Event metrics Case level filters Case level filters are always applied to complete cases in the process. Case level filter types: Timeframe Case attributes Attributes (conditional) Edge (conditional) Variants Case metrics Sequence End events Conflict of interests Case and event level filters can be combined to form a single combined filtering criteria in a view. There are filters that have the same syntax, but different semantics based on the type. For example, the Event attributes filter can be applied on the event and case level. https://learn.microsoft.com/en-us/power-automate/minit/filtering
• #31: Task mining is a technology that enables organizations to capture detailed steps for tasks performed on users' desktops, either independently or collaboratively with colleagues. By analyzing recorded user actions, the task mining capability in Power Automate allows organizations to gain insights into how they perform tasks, identify common mistakes made during task performance, and pinpoint tasks that can be automated, all of which can help optimize their business processes. Benefits of the task mining capability The task mining capability helps busunesses to: Streamline workflows. Identify inefficiencies and bottlenecks in workflows. Reduce costs. Foster a culture of continuous improvement where employees are encouraged to optimize their work processes.
• #32: Before you can use the task mining capability to visualize and analyze your processes, you need to: Create your process in the process mining capability in Power Automate. Record the activities that make up the process using the Power Automate recorder. Prepare the recording for analysis. (Optional) Create recommended activity names to make it easier for you and others to prepare the recording for analysis. https://learn.microsoft.com/en-us/power-automate/process-advisor-processes
• #33: View your process map and related analytics To begin analyzing your process, start by checking the status of your recordings and using the Analyze button in the command bar. This button will add the selected recording into the process map and related analytics. Please note that the analytics reports will only include insights from recordings that have the "Analyzed" status. After the task is analyzed, the process map and related analytics are available when you select Analytics in the menu. On the Details card, you can check the status of the analysis for your process and whether it has been previously analyzed. Analysis happens at the process level for all recordings that are ready to be analyzed. Pay attention to the recording status, which you can find under the Status column of the Recordings pane. Only recordings with the "Ready to analyze" status will be considered for analysis. View your recording status The status of each recording can be found under the Status column of the Recordings pane. Recordings can have one of the following statuses: -In progress: This means that the recording is currently being made, or it may not have started yet. The web portal cannot determine the status of the recording until it has been saved because the recording happens in the desktop client. Even after a recording has been saved, it's possible to see the "In progress" status because some processing is required to make the recording ready to view. Failed: An error occurred while processing the recording. You'll need to create a new recording. Not analyzed: The recording has been processed and is available to view and edit, but it hasn't been marked as ready for analysis. Ready to analyze: The recording has been marked as ready for analysis and will be included in the report and analytics the next time you analyze the process. Analyzed: The recording has been analyzed and is included in the analytics output that can be viewed through the Analytics screen. An analyzed recording might also have one of the following icons next to it:  This icon indicates that the recording has been analyzed but has been marked not ready to analyze, and it won't be included if the process is analyzed again.  This icon indicates that the recording has been modified and may be out of sync with the previous analysis. If you analyze the process again, the recording will be synchronized with the analysis output and the process map. https://docs.microsoft.com/en-us/power-automate/process-advisor-analyze https://learn.microsoft.com/en-us/power-automate/process-advisor-analyze
• #34: The automation recommendation feature in the process mining capability helps you identify automation opportunities and guides you through automating your processes using Microsoft Power Automate.  The blue recommendation icons on the process map activities indicate automation opportunities. To automate an activity, select the +Automate activities option above the process map. This takes you to the Power Automate form designer, where you can see the connector recommendations for the activities in your process map. From there, you can select and add the connectors to your flow to automate your process.  https://learn.microsoft.com/en-us/power-automate/process-advisor-automation
• #36: The process mining capability relies on environment security and Microsoft Dataverse security roles and privileges to grant access to its features in Power Automate.  Security Some privileges are set by default in Dataverse. This allows built-in security roles to use the process mining capability without further actions from system administrators. Specifically: A user in the process mining capability can create, share, and contribute to processes. This role has a restricted set of privileges that are only meant to be used by the process mining capability. If you need to provide access to other Power Platform features, make sure to assign less restrictive security roles on top of the user role for the process mining capability. Environment makers can use the process mining capability to create, share, and contribute to processes. Administrators and system customizers can access all processes created in the environment. Important The process advisor Application is an internal security role that process mining uses. (Process mining was formerly named process advisor). Don't assign this security role to users. Don't modify the set of privileges in the process advisor User or process advisor Application security roles. https://learn.microsoft.com/en-us/power-automate/process-advisor-security
• #37: Privacy Sharing processes and their recordings is essential to create rich analysis and insights in the process mining capability. Users can add recordings to a process. They can then use Power Automate to record processes and then import the processes into the process mining capability. Owners and contributors can see some data from the process and its recordings. While most of the information process recordings capture can be essential to the understanding of the process activities, some steps might contain sensitive information. You can modify and delete sensitive information such as personally identifiable information (PII) from your recordings. In Power Automate, users can: Delete sensitive steps or modify input data information. Pause and resume recording to avoid recording sensitive information and PII. After you import the recording into the process mining capability, you can remove sensitive information by doing the following: Rename step names or descriptions. Delete screenshots. https://learn.microsoft.com/en-us/power-automate/process-advisor-security
• #38: To ensure that sensitive data is not exposed, the process mining capability allows you to remove screenshots and text entries stored during the recording session in the task mining capability. Before preparing your recordings, you can also remove any confidential information in the text or images to further protect your data. You can delete screenshots. Remove text In the process mining capability, you can protect sensitive data by removing it from any text entries recorded during a session. To do this, select the text entry that contains the sensitive information and modify the step description to remove it. For example, you could replace the sensitive data with a generic term or placeholder. Be sure to save your changes before sharing or analyzing the recording. https://learn.microsoft.com/en-us/power-automate/process-advisor-protect
• #39: You can copy or move process mining processes from one environment to another using Microsoft Power Platform solutions. You need to be an owner or co-owner of a process to export it. Export a process Sign in to Power Automate. Make sure you've selected the environment where the process you want to export has been created. Select Solutions in the navigation pane on the left. In the list, select the solution you want to use to add your process, or create a solution and select it. Select Add existing > PM Inferred Task. Search for and select the process you want to export, and then select Add. You can now export your solution and import it in another environment. Import a process To import a process into another environment: Make sure you've exported your process with a solution. Go to the environment where you want to import your process. Follow the instructions in Import a solution. https://learn.microsoft.com/en-us/power-automate/process-advisor-export
• #40: Made acquisition and now we have deeper insights, customizability and templates
• #42: AI powered process discovery Process Mining is the “blueprint” that informs businesses of what processes in their organization need optimization and where generative AI can maximize ROI on process excellence. Accelerate your process analysis Once you complete the process discovery, process map creation and root-cause-analysis, get a written summary in natural language without the need to analyze reports. Democratize process optimization Leverage co-pilot to democratize process optimization and expedite time to action by getting suggestions for deploying solutions including automation and apps AI embed from start to finish Revolutionize process mining with AI embedded through the entire process mining customer journey. Democratize process optimization Leverage co-pilot to democratize process optimization and expedite time to action both within product, as well as meeting users where they are via Business Chat in Teams.
• #43: AI-centric – Root cause analysis, Simulations, Predictions Most complete low-code platform What is the most important message we would like to share with Bernard via demo? The most complete digital transformation platform – not only Process mining, but also task mining, extensive ETL and storage capabilities – Dataverse and Data Flows, Power Automate - full stack automation including low code and no code, PowerBI – leading BI platform, AI builder all running on Azure Cloud first - Natively integrated with seamless extensibility to enable central governance, security and collaboration for enterprise trust, bring your own lake in roadmap User centric – known experiences of standard products, citizen friendly, easy to use platform with out of the box results AI centric – Improve processes smarter by leveraging AI, AI powered RCA, Simulations, Vision demo