Alternative to data obfuscation to TEST your D365FO integrations

 When doing integrations one rely completely on the data that is generated by the LOB applications to test integrations. Moreover, using this data for testing purposes can lead to issues related with privacy and security. For this reasons there are sample data that is available but the scope is very narrow and it is only applicable for process testing. 

It is desirable to have ways to produce this data in an ad-hoc manner as when it comes to integrations the scenarios can be quite diverse.

In this article I'll explain a set of scripts that can produce huge amount of valid randomized data considering the master data available in D365FO. 

Consider the following scenario: 

  • A LOB that generates huge amount of sales orders (100.000+ lines per day) and send them to D365FO for creation, and further invoicing. 
  • 5.000+ customers
  • 30.000+ items
  • You cannot re-use production data. 
Although obfuscation of the data is a possible solution does not guarantee that all the relevant scenarios will be generated, plus the effort - without a proper tool - to do it could be considerable in order to maintain all the environments in sync for the testing.

Another solution can be as follow: 

  • Grab all available customers in your instance
  • Grab all available items in your instance
  • Generate randomized amount of sales order per Customer with a randomized numbers of sales order lines per item. 
  • Randomize unit prices and quantities
With simple python and odata calls I was able to generate as many data I wanted. Later, it is possible to generate a valid list of SQL sentences to create tables and records - if SQL fits your scenario - to be used by your integrations. 


Conclusion

Using this tool I generated more than 15.000+ valid sales order lines in less than a minute. This technique fit is purpose and provides a flexible tool that can be adapted to any scenario. While testing the integration it is likely to have errors with D365FO validations - for instance decimal quantities but the item only accepts whole numbers - but it is exactly the purpose of testing, identify an error and decide upon it. 


Disclaimer: All the data used for this example was taken out of a regular CONTOSO dataset. 

Comments

Post a Comment

Popular posts from this blog

Telemetry and Monitory with Applications Insights and Dynamics 365 F&O

Image
Telemetry and Monitory with Applications Insights and Dynamics 365 F&O With version 10.0.39 Applications Insights integrations is GA. Hence, here is a very simple example how to start using this great feature and start monitoring the performance of your application.  For more about D365FO and Application Insights integration:  Monitoring and telemetry using Application Insights - Finance & Operations | Dynamics 365 | Microsoft Learn Dynamics 365 Finance and Operations - Monitoring and Telemetry using Application Insights -TechTalk (youtube.com) In this example we will monitor the execution time of a batch job. In particular, we will use a metric to analyze the execution over time.  // Create a class file and paste this code class SlowBatchJobClass extends RunBaseBatch implements BatchRetryable { [SysEntryPointAttribute] public void run() { int maxWaitInSeconds = 20; // Maximum wait time in seconds           SysApplicationInsightsTelemetryLogger logger =
Read more
Image
Distributed Printing Architecture with Power Platform  Going back to posting after a long period. Now I have started a new blog where I'll be posting content related to my work with Microsoft technology, and how you can combine Dynamics 365 + Power Platform + Azure to solve virtually any problem in a reliable and robust architecture. In this first post I'll share how to build an architecture for unattended distributed printing from Power Apps. Although there are other options (look at PCF controls ) I found this architecture simple and reliable. This solution will serve to customers looking to: Use network shared printers across their organization. Serve multiple users allowing them to distribute their printing task according to specific business rules (ex. some group will only print on 'Printer 01') Rely on the SO to print as you can use the print command Access your local resources (printers) in a secure way To connect from the canvas app to the A
Read more

Creating a DMF data package from files stored in an Azure blob storage

Image
Dynamics 365 for Finance and Operations support different scenarios for data integration.  Each scenario is listed on the following documentation: https://docs.microsoft.com/en-us/dynamics365/fin-ops-core/dev-itpro/data-entities/integration-overview One of the most common scenarios is when a large number of records need to be imported in D365FO. These record normally are coming from a third-party application as batches. If we follow the documentation guidelines for this scenario, the recommended pattern to be applied is Batch Data API For this kind of pattern we have the following API available:  Recurring Integration API  Data Management API  Many people uses the Recurring Integration API, however, I found it lacks of features (like asking for the task status) and also I haven't seen any further investment to improve it. For these reasons I choose to use the Data Management API.   Data Management API is a robust API and their specification can be found here: Data management pac
Read more