Dynamics 365 Finance Insights
Hello the Community !
If you were like me part of the MBAS 2020 in May, you must maybe have seen a presentation / session on Dynamics 365 Finance Insights for our loved ERP system : Dynamics 365 Finance and Operations. This session was done by Shivam Pandey - PPM Microsoft. So first, after this session I was really enthusiast and ask them to be part of the Preview with them. So first, thank you again for let me test as early adopter and hope that you enjoyed my feedbacks so far :) . Thank you also on behalf of the whole community to let me write this public article to let them well understand Finance Insights !
So this is the aim and main goal of this article that you will see. Giving my global review on all aspects with this new AI Apps.
Hope it will be digest for you but I will also do - at the end of this article - 1 main global video on Youtube in English.
Also please forgive me for the name CDS everywhere, it’s now Dataverse but let’s wait a little bit before that everyone know this name !
Introduction :
So what is this new AI App : Dynamics 365 Finance Insights ?
As you can notice on Microsoft strategy, it’s to clearly add some Insights on Data combined with the global Business Apps eco-system. This is the real force compare to other solutions : when you include all together : this will help you gain real value on your ERP : aka PowerPlatform (AI Builder) and Azure Services.
Microsoft added a lot of Insights Apps, especially in Dynamics 365 CE / CRM system ; like Sales Insights - Customer Service Insights and even also the CDP Platform : Customer Insights. On the ERP side, we didn’t get a lot of Insights / ML / predictions - while yet we’ve got a rich & global DATA system with large volume and cross-global module business process.
For me it was a real challenge to ask & provide some insights on Finance Data, right ? As an AI Expert and ML/Prediction model experiences, I always seen Sales or Marketing Insights for Customer experiences but not really on backend system for voucher/transactions stuff. But as you will see, Microsoft added this new AI Apps to provide some real Insights for people like Credit Risk Officer or Accounting team / CFO. But the main global part, like AI Builder of PowerPlatform, is to help you to configure & deploy a quick model without to be a real data scientist. And this is the main global change here : in a couple of hours, you can configure and even do some extension if you want & need to (especially if you want to improve the accuracy of the model) but you will not go on a 6 months project just to respond 1 question : is this customer will pay me late or not … :) ?
Finance Insights can be summarize like this :
“Plan more accurately and efficiently using customer payment predictions, budget proposals, and cash flow forecasting with Finance Insights”
Besides, being part of COVID19 situation, you can easily understand that many business is currently suffering from problems of cash flow, budget, restructuring and it is complicated to review the entire trend to come at the financial level. Of course, these AI Model will give you insights on your internal DATA and not really (well at first) external Data ; so ever a global situation like COVID19 that was not expecting is really a challenge : after all it’s just prediction right ?
These are the main features & goals of Dynamics 365 Finance Insights for Business side :
Run global financial operations more effectively
Improve your cash flow with built-in AI tools that enable proactive collections through payment predictions, easier budget creation using intelligent budget proposals, and more accurate cash flow management using intelligent cash flow forecasting.
Increase accuracy of projected cash flow
Monitor cash flow in real time, predict future trends, and make data-driven decisions with an intelligent and customizable cash flow forecasting solution.
Adapt quickly with intelligent budget proposals
Reduce the time and effort of budgeting by letting the system perform the tasks for you. Quickly consolidate and analyze years of historical data to forecast your budget.
Reliably predict customer payments
Use AI to predict when customers will pay their invoices. Proactively reduce write-offs and improve margin with immediate visibility into the probability of late or at-risk payments.
While I write this article, Finance Insights is still in Private Preview . To be part and test on your side (like me) you can sent directly an email to the Microsoft Team :
Public preview will come soon, but no ETA on this.
Of course, some parts can change before the GA that seems to be for the middle of 2021.
So please be aware, every part on this article can change somehow, but I will keep updated as soon as I have more information / confirmation.
Also, at this time, we don’t know yet the real price behind the scene. I assume that Microsoft will ask to pay for AI Builder Service in the PowerPlatform system (cost around 500$/month) or even integrate this as a Cost Addon on your D365 F&O Subscription. For me the extra charge as AI Builder will be clearly the minimum knowing that you can use it for other needs or processes (like the MRP, Supply Chain Insights Apps sometimes ? the expense report part on which I also made an article).
Before going on the Technical part on how-to, you will need to configure your Finance Insights App, here are the main pre-requisites. But of course by being part of the Private Preview, Microsoft will help you to figure out what you really need.
Finance and Operations instance - Sandbox TIER2 needed in version 10.0.12
Copy/refresh your production database on it
A CDS Environment - somehow also with an AI Builder Service Trial activated. For that I really don’t know how it works - seems that being part of Private Preview will not be asked to have a real subscription on AI Builder.
An Azure Data Lake with the LCS Addin activated on this same Sandbox instance
A DataLake with your Entity Store / Aggregate Measurements on it.
Activate the preview Features in the Workspace of your F&O instance. AS I remember, Microsoft will provide you some SQL request to add features if it doesn’t appear on your instance.
As you can see again, DataLake is everywhere :) and here it’s mandatory so…. time again to jump in and read my past articles about this.
Technical Side :
Let’s start first by explain the global architecture before going on each steps of the configuration.
Here is my unsexy schema :)
First stage, it’s the main one and standard one for months now. It’s to have your Entities Store of your Dynamics 365 F&O (aka AxDTWH) on your own Azure Subscription with your Data Lake. Every aggregate measurements will be calculated from your financial transactional data here (especially those coming from 10.0.12 version that Microsoft released) will need to by sync to an ADLS before going further. Of course I will go deeper on the configuration just after.
The second part is here to setup the integration between a CDS environment in a custom Entity provided by Microsoft with your DataLake in order to sync the global DATA. At first, you will have a global warmup process to sync.
AI Builder can now read your DATA into your CDS environment. AI Model can now calculate prediction and process the model.
On a custom entity in your CDS, the results will be stored by AI Builder on every transactions with a scoring.
A Data Integration Project in the PowerPlatform (very similar as Dataflow) can read the results
And Sync the results on Dynamics 365 F&O instance
Here is now the global sneak peek technical configuration. First, please be aware this is also well explained by Microsoft in a Word Document that they will provide to you when you will be part on the Preview Program. I will explain the “manual” configuration, meaning that they also provide a Powershell script in order to setup almost 75% of this manual process. Of course, I don’t know if they will provide in their backend a global shadow process to init everything for you with just a button, but let’s hope to be more easier in the future when it will be in GA State. Since I worked on Azure for the begining I know quite well this part, but I assume that every D365 F&O Architect are not on the same page as me… (or even for the CDS / AI Builder configuration part)
For the first part, you will need to sync your Entity Store to Azure Data Lake.
For that, you can go directly on my global article on ADLS in All is Glory :) - Chapter 3
After it’s well synced and configured, you can have a look on your Entity Store.
Double check that the 3 are well activated on Automatic Refresh Enabled so that the batch Full Reset can sync it after in your ADLS.
Customer Payment Insights
Cash flow time series aggregate measure
Budget time series measure
In Azure, you can also check that everything is here. Like PayPredMeasure CSV files ; like this :
After you will need a CDS Environment. If you have already try or activate the DualWrite Framework or CDS Virtual Entities for F&O ; maybe you have already one. Altough, I suggest to create a specific one for this Finance Insights Preview.
Create a new Common data services environment in the same Active Directory Tenant.
Go to https://admin.powerplatform.microsoft.com/
Click New environment
Create in the same region as the LCS environment (note that you won’t be able to pick a specific data center - but of course don’t create your CDS in APAC if your LCS / D365 F&O is in EMEA).
Select Sandbox for the environment Type. (toggle yes to create database, select the language & currency - doesn’t really matter) and leave default values for the rest of it
After you will have to change just one setting for your CDS instance to increase Email Attachment to 50 MB
You will need to add some AAD Apps in your Azure Portal. Add Applications to the Azure Active Directory Tenant like this :
Go to Azure Active Directory (be aware that you will need to be a Global Azure Admin - especially on Azure AD part to configure this part)
Careful also, you will need to be on the Azure Tenant as your CDS tenant that you created before !
On the left-side menu bar select Manage > Enterprise Applications and search for the following applications
Microsoft Dynamics ERP Microservices
ID : 0cdb527f-a8d1-4bf8-9436-b352c68682b2
Microsoft Dynamics ERP Microservices CDS
ID : 703e2651-d3fc-48f5-942c-74274233dba8
AIBuilder [ADLS] Finance Insights Client Application
ID : 23363606-4627-4a7f-b8e5-2e6f935c4d28
If you can’t see it, Microsoft can provide you a simple Powershell Script.
Create after a specific Azure Active Directory Application to have a client ID and client secret ; and also add API Permissions for Azure Key Vault
You will need now also to create a new Azure Key Vault ; this Key Vault will be used after in LCS and it’s the secure way to store all connections informations (like the Appid & secret of your previous as well as the connection string for your ADLS)
You will need to add Access Policies to your key vault ; of course the 3 Services Apps that you have registered and also the Application created before with Get + List operations.
Finally on the Azure part, you just need to go to your Azure Data Lake Storage (ADLS) and add some permissions.
AI Builder Authorization Service (or AIBuilder ADLS Finance Insights Client Application like Above) and also your Azure Active Directory App created before - me (DatalakeDyn365)
Now you can go to your LCS portal on your Sandbox instance.
You will need to install the 2 follow Add-ins : Export to Data Lake and Get Insights (aka Power AI before)
For the first one (ADLS), you will need to provide previous information noted during the setup ; including the name of your secrets in your Azure Key Vault and once more time the AppId / Client Secret on your AAD created before.
For the second one, just 2 things to setup ; it’s here to have the link between F&O instance and your CDS / AI Builder instance that you created at the beginning. This is the URL of your CRM / CDS environment (like https://orgxxxxxxxx.crm4.dynamics.com) and the ID can be found directly in the Settings Page of your instance like this :
Now you can go back to your F&O Instance in the Feature Management Workspace in order to activate the AI Models Features :
(If it doesn’t appear, Microsoft will provide you an SQL Request in order to add it directly in your TIER2 Database)
The last part of setup before jumping to the warmup process, is to check something that I didn’t notice at first but seems to be installed under the hood… :) it’s the solution in your CDS instance / environment. You will see that during the Preview Process you will need to provide information to Microsoft so don’t know which tasks are executed in that part, like if they have installed right away the solution. But it’s of course very important because it’s here where you will find all the Entities needed for the AI Builder & to store all your DATA for prediction. Assuming again that this part will be automatically done, ever by installing like the LCS Add-in feature that we did before.
First init / warmup :
The technical side is now done - meaning the global first schema / architecture that I’ve presented earlier.
The first init will be done directly in your F&O Sandbox instance. Go to the setup like this :
First, you will see of course the 3 AI Models at left. For the Cash flow forecasts and Budget Proposal, select just “Enabled feature” that’s it :)
For the last one, Customer Payment Predictions (my favourite), you will to be careful on 1 global parameter : “Very Late Period” before clicking on Enable feature.
On my side, I just selected 60 days ; but clearly it’s up to you and MAINLY to your business needs ! Depending on what you can consider and past historical behaviors of your customers on payments. You will see that the results of this AI Model are on 3 parts / categories : On time, Late and Very Late
When it’s done, you will need to wait a little for the first init process (1 hour depending of course of the volume of financial transactionnal data you have)
But as I will describe after, the more there are, the better it will be !
To sync back DATA (prediction results) you will need to setup Data Integration Project on PowerPlatform Admin Center. Sync CDS Entities => to F&O Data Entities.
1. Go to the CDS instance and follow the steps here, https://docs.microsoft.com/en-us/power-platform/admin/data-integrator
Important: read through the following steps before starting and integrate the steps into the setup process.
Setup first a connection set. Of course the Sandbox instance with an D365 F&O Admin Account as well as the ADMIN CDS Account that you have use to create your CDS environment. On the Organization part you can pick whatever Legal Entity you want.
You will need after to create 3 Data Integration Projects of each AI Model.
Pick the connection set created before. Pick the right template coming from the CDS Solutions
After creating, review the mapping. You can refresh entities if you have seen some errors on mapping, but normally we don’t have to change anything.
You can schedule after the refresh of the Results Entities in the CDS to F&O (like every day) ; it depends on how much you refresh your DataLake with your Entity Store. Every new DATA that will come after to CDS will be scored with your current & latest AI Model deployed daily ; this score can be after synced to F&O with this Data Integration Project.
You can be notify every time by email of the resultats of this Data Integration Project, as well as the Execution History panel (very similar as the Dual Write framework)
Understand the results & adapt / Lifecycle :
Finally, all is good now. But of course, this is standard way. Of course each AI Model (especially the Customer Payment Predictions) can be improved. On my side, I was at first around 67% of global scoring. After few changes, I was around 80% of accuracy on the AI Model deployed.
So let’s take an example with this same one. How to change & improve that. For that go back to your F&O instance.
At this time where I write this article, you can see the Data fields that it’s currently used to calculate the prediction. We can’t extend it right now, like adding some key values, especially maybe custom fields that you have added that you know that may be relevant to this AI Model. This was part of my feedback actually. Well, it’s still in preview, assuming that we can create new & extend already Cubes in the EntityStore (PayPredMeasure here). You can’t add external data also. For that maybe the good way can be to extend also the CDS entities but not really sure about the results :) I am currently trying this part.
You can also check the DATA back to your CDS entities (good to check results and analyze it)
To check and publish new AI Model you can go directly to your F&O instance again. At this time, we can only do that for Customer Payment Prediction Model - for Cash flow forecasts & Budget Proposal it’s only the Microsoft standard model.
Click on Improve model Accuracy.
You are now on the AI Builder part :)
First part to analyze is the “Most influential data”
Edit the Model to publish new one after and uncheck all unnecessary fields. (below <1% or even 0%)
Believe me, you will be like a Data Scientist by uncheck/check fields to try new version of your Model and analyze the metrics. By doing this, I managed to improve by 5%.
With this Sandbox instance, I had only less than 1 year of DATA volume. So not so huge, but even the scoring is really great.
If you are more aware of ML/Predictions model, you can still download the metrics in a Matrix way on AI Builder Model.
If you are now more confident after changing your AI model, you can publish the new one. Before publising, it will train the model and you will directly the scoring model.
One more thing, don’t forget to setup in your Model the Run Settings to “Daily”
And also the Retrain part ! For me I assume that weekly can be enough.
Functional / Business side :
And voila ! Here is the most important part ! Of course, this is the real interest of this solution ; especially for the Business side / Key Users in ERP system.
I will be here like a Credit Risk Officer that use every day D365 F&O Instance in the Credit Collection Module.
I will have now 2 Menu Items to analyze the Customer Payment Predictions.
First, on a Customer Level : I have got directly insights with of course the 3 categories of predictions :
On the right panel, on each Customer I can have more related information :
As a Credit Risk Officer, I can have other Insights, like how Invoices are Open, Late, average of day late with the amount associated to the late payment.
And again new other insights.
Also I can have a look directly on every transaction level side. Very helpful for me to adjust the Credit Limit for my Customers part based on this result.
Now I am part of the global Accouting Team and want to analyze the Cash flow forecast by Legal Entity and Bank Account. Here we go !
Here I can have a global Cash flow forecast. I can have the global forecast by each legal entities.
I can save any snapshot at any time.
I can add also external DATA to this cashflow. And also filter by other Main Account.
I can have also now a Budget Proposal by Main Account : This feature uses historical budget register entries and general ledger historical balances to create a model that's used to predict future budget amounts
Conclusion :
Hope you have now a global complete overview of what is Finance Insights and what you can achieve with that for your Business with Dynamics 365 Finance and Operations.
I still continue to test more the Cash flow forecast & Budget proposal right now so I will update this article as soon as I discover new things for you :)
Thanks ! See you soon !
Here is my global video in English for you below :