Accelerator between Microsoft Dynamics CRM 2011 and SAP ERP for BizTalk Server 2010 / 2013 White Paper Published on: September 2013
Inhaltsverzeichnis: 1. Introduction... 3 2. Components of the Accelerator... 4 2.3 Component Architecture Diagram... 4 3. Roedl Accelerator Components... 5 3.1 Rödl Microsoft Dynamics CRM 2011 Event Pipeline Solution... 6 3.2 Roedl Microsoft Dynamics CRM Adapter for BizTalk Server... 7 3.2.1 Mapping with the Standard WCF adapter... 7 3.2.2 Mapping with Roedl Adapter... 8 3.3 Predefined interfaces of most used entities... 9 3.3.1 Customer... 9 3.3.2 Sales Order... 10 3.4 Ticket system for controlled calls of SAP and CRM... 12 3.4.1 System Demand Control through tickets... 13 3.5 BAM Templates for interface monitoring... 14 3.6 ESB Toolkit Templates for exception handling... 15 3.6.1 Other... 16 3.7 Rödl BizTalk Server Runtime Monitor Service... 17 3.8 Roedl Accelerator Function Library... 18 3.8.1 HelperLibrary for Microsoft Dynamics CRM... 19 3.8.2 HelperLibrary for SAP R/3... 22 3.8.3 Helper Library for Logging... 23
1. Introduction Why Accelerator was developed? BizTalk Server is the tool to connect Microsoft Dynamics CRM 2011 and SAP ERP. Most of the projects tend to have the same basic processes; therefor while working on individual projects it is highly possible to be repeating the same process pattern, which leads to the exhaustion of resources and time consumption. Therefor it is crucial to equip a developer with a toolset to create interfaces in a nearly default way. Such a toolset delivers fast results in a fully modular manner. Taking into consideration the above approach, a developer s skills and experience in both Microsoft Dynamics CRM and SAP are a beneficial factor, however the high level of expertise in the both applications dose not play a significant role in the development process. The out- of- the- box Accelerator installation delivers more than satisfactory outcomes. In fact all most common and relevant fields are transferred from one business object to another in an automated way. The advantages are straight forward: Performance Customers receive the running interface including standard entities in a matter hours! Extension Due to the modular concept of the Accelerator, the solution allows the continuous extensions to its standard functionality. Simplicity The Microsoft Dynamics CRM and SAP experience and knowledge are the secondary factor. Optimization The Accelerator creates the basic interface between Microsoft Dynamics CRM and SAP, which can be enhanced and extended upon the individual customer demand.
2. Components of the Accelerator Microsoft Dynamics CRM - SAP Accelerator for BizTalk Server 2.3 Component Architecture Diagram SAP R3 4.6c 7.x IDOC IDOC RFC BizTalk Adapter v3.0 for mysap Business Suite BizTalk Server 2010 / 2013 Microsoft Dynamics CRM 2011 / 2013 Roedl Dynamics CRM 2011 Event Pipeline WCF XML WCF Azure ESB Roedl Microsoft Dynamics CRM WCF Adapter Roedl Accelerator ü BAM ü ESB ü Tickets ü Functions ü Functoids ü Exceptions ü Tracing ü Interfaces Templates ü Monitoring ü Libraries SAP Microsoft R&P Accelerator complies with the all SAP Systems, starting with SAP/R3 4.6C to the current version SAP ERP Suite 7.X. Roedl Accelerator is available for most of the currently and still used Microsoft Dynamics CRM releases (3.0-2011). Moreover it is the only and unique solution compatible with Microsoft Dynamics CRM 2011 Online and provides an extraordinary interfaces integration capabilities in cloud, based on the Azure technology.
3. Roedl Accelerator Components The inevitable advantage of Accelerator is that fact the solution is not a one and constant solution. It is a set of flexible but generic based tools for interfaces creation and modification.
3.1 Rödl Microsoft Dynamics CRM 2011 Event Pipeline Solution Microsoft Dynamics CRM - SAP Accelerator for BizTalk Server By default, Microsoft Dynamics CRM 2011 is not able to publish events to external systems, except for the Service Bus. However, in on-premise scenarios this approach is not the preferred way. Furthermore the complete configuration has to be done by using an administrative tool. In contrast, the Event Pipeline allows you to configure all endpoints and their associated events declaratively. More information regarding Roedl Microsoft Dynamics CRM 2011 Event Pipeline is available at here For each event, following properties can be defined. 1. Entity, who fire the event 2. Internal event (e.g. create, update, delete, won, lose, merge) on this entity 3. Event stage (pre or post, synchronous or asynchronous) 4. Filter attributes by update (by changing of which attributes the event will be fired) 5. Ignore Users (mostly the BizTalk Server User in CRM) that can t send an event 6. Trigger is a defined option set who must be set on a defined value to send the event.
3.2 Roedl Microsoft Dynamics CRM Adapter for BizTalk Server The Roedl Adapter creates interfaces between Microsoft Dynamics CRM 2011 / 2013 and BizTalk Server 2010 / 2013. Roedl Adapter supports not only Microsoft Dynamics CRM standard entities, it is also compatible with the individual custom entities in regards to all the methods (create, update, delete, fetch and retrieve). Roedl Adapter also supports all Request/ Response Methods both from the Microsoft Dynamics CRM SDK and XRM SDK (e.g lead conversion). The installation of the adapter is easy like the installation of any custom WCF Adapter. 3.2.1 Mapping with the Standard WCF adapter With Standard WCF adapter the integration can be only realized via an original WCF service. The original WCF adapter does not deliver the functionality of the attribute mappings and visual control over the process.
Moreover, the original WPC functionality provides only a general schema for all entities, therefore attributes can be only addressed via key value pairs with the names of specific attributes defined in advance. That means concrete: One generic schema for all entities Attributes only accessible through key-value pairs Attribute names have to be known Named attribute mapping not possible No visible mapping 3.2.2 Mapping with Roedl Adapter The Rödl Adapter for Microsoft BizTalk Server seamlessly integrates BizTalk Server with Microsoft Dynamics CRM 2011 with no installation effort on the CRM system. The Roedl Adapter delivers a handy tool for attributes mapping in a simple manner allowing visual control of the process. The solution defines all entities in the noambiguous way on the basis of one, entirely generic request. Meaning: Well-known and strong named entities Strong typed attributes Simple directly and real visible attribute mapping Completely generic fetches Security Token for AD, Live-ID, Office 365 More information regarding Roedl Adapter is available at here.
3.3 Predefined interfaces of most used entities Roedl Accelerator delivers by default the interfaces for the most common CRM/ SAP function entities (e.g. Product, Account, Contact, Quote, Sales Order, and Invoice). Moreover, Roedl Accelerator complies with the SAP system using the standard IDOC and BAPI functions. Therefore, no individual SAP development is required. Due to the modular Roedl Accelerator schema, the integration of additional and/or individual custom interfaces is possible upon the minimum development resources effort and time consumption. Examples of Interfaces: 3.3.1 Customer A customer is displayed in Microsoft Dynamics CRM by the entity account. A customer is displayed in SAP by the business object debtor. 3.3.1.1 From SAP R/3 to Microsoft Dynamics CRM This interface sends debtors from SAP to CRM and there either creates of changes the entity account. Process: The customer data set newly created or changed by SAP is sent directly to the BizTalk-Server. There is a checkup if the data set already exists in Microsoft Dynamics CRM. If it exists, it is updated with the data from SAP. If it doesn t exist, a new data set is created. If the data set exists more than once in MSCRM, an error message appears. Technical Background: For transferring purposes, the SAP standard IDOC (Intermediate Document) DEBMAS06 is used. To clearly identify the data sets in MSCRM, the applicable primary keys are synchronized. That means that in the SAP system, the distinct GUID (Global Unique Identifier) of the CRM system is listed. If this identifier is contained in the SAP data set, the lookup in MSCRM is executed via this parameter. If the CRM-GUID is not yet contained in the SAP data set, the applicable CRM data set is looked up by the debtor number from SAP. SAP Component The IDOC component DEBMAS06 is used for creating and changing. As soon as a debtor is created or changed in SAP, the systems sends the data to the IDOC port via the change event. Involved entities in CRM In Microsoft Dynamics CRM, debtors are filed in the entity account Customizing in Microsoft Dynamics CRM
The new attribute new_roedl_sapid is filed in the entity account. The debtor number from SAP is saved in this field. Mapping The following fields are transferred in the standard: o Debtor number o CRM-GUID (if available) o Name o Street o ZIP Code o City o Telephone 1+2 o Facsimile o Language tag 3.3.1.2 From Microsoft Dynamics CRM to SAP R/3 This interface sends accounts from CRM to SAP, where debtors are created. Process: MSCRM sends out a trigger message that an account has been newly created or modified. There is a checkup if the last modification was caused by the interface itself in order to avoid unnecessary update loops. If the last modification was caused by a CRM user, a DEBMAS06-IDOC is created and sent to SAP. Technical Background: SAP Component The IDOC component DEBMAS06 is again used for creation and changing in SAP. If the initial generation occurred in Microsoft Dynamics CRM, no debtor number is given to the account because SAP is the leading system. In order to ensure synchronization of the data sets, the MSCRM-GUID is saved in the SAP data set. For this, the field NAME4 in the SAP system is utilized. Alternatively, the WebSite field (KNURL) can be utilized as long as it s not occupied by customer data. Involved entities in SAP Accounts are created as debtors in SAP. Mapping The following fields are transferred in the standard: o Debtor number o CRM-GUID o Name o Street o ZIP Code o City o Telephone 1+2 o Facsimile 3.3.2 Sales Order A sales order is displayed in Microsoft Dynamics CRM by the entity salesorder. A sales order is displayed in SAP by the business object salesorder. 3.3.2.1 From SAP R/3 to Microsoft Dynamics CRM This interface sends salesorders from SAP to CRM, where salesorders are created or changed. The items of the quote are created in the entity salesorderdetails. Process: A new sales order is created or modified in SAP. The altered data set is sent directly to the BizTalk-Server. There is a checkup if the data set already exists in MSCRM. In case it doesn t exist, a new sales order headline is created for MSCRM. Among other information, the customer is looked up and attached. After creating the headline in MSCRM, the individual sales order details are created in a loop. For this, the applied products and standard units (uom) are looked up in MSCRM and attached.
If the sales order already exists in MSCRM, the headline is being updated. Furthermore, the sales order items are first deleted and then newly created. Technical Background: SAP Component Sales orders are being transferred by the SAP-Standard-IDoc ORDERS05. Involved Entities in CRM The headline of a sales order is being represented by the CRM entity salesorder. The sales order items are listed in the entity salesorderdetail. There are lookups for account, product and basic unit. Mapping The following fields are transferred in the standard headline: o Order number o Customer ID o Requested date of delivery The following fields are transferred in the standard quote items: o Order number o Line number o Basic price o Product number o Amount o Quantity unit 3.3.2.2 From Microsoft Dynamics CRM to SAP R/3 This interface sends salesorders and salesorderdetails from CRM to SAP and there creates or updates salesorder. Process: MSCRM sends out a trigger message that a sales order has been newly created or modified. First, the account connected to the sales order is looked up in MSCRM in order to extract the SAP debtor number. Afterwards, all sales order items are being evaluated and the SAP number of the connected products is extracted. Then, the information is forwarded to a BAPI request that creates the SAP sales order. Technical Background: SAP Component Sales orders are created and modified in SAP via the BAPI component BAPI_SALESORDER_CREATEFROMDAT2. Mapping The following fields are transferred in the standard headline: o Sales Organization o Sales Channel o Industry o Date The following fields are transferred in the standard sales order items: o Customer number o Line number o Product number o Amount o Quantity Unit o Request date
3.4 Ticket system for controlled calls of SAP and CRM Microsoft Dynamics CRM - SAP Accelerator for BizTalk Server It is commonly known that the Microsoft Biztalk Server is capable of transmitting the large amount of data in a continuous time, which might cause queues and data loss in other systems. Roedl Ricket system facilitates and controls the data transition between the CRM and BizTalk or SAP and BizTalk System (and/ or any other external solution). For each outgoing message in the Biztalk Server a unique ticket can be created. The maximum ticket count of any ticket type can be defined and modified dynamically in the live environment of the Accelerator depending on the requirement (e.g. when the number of outgoing messages changes periodically). The number of ticket types and tickets is unlimited.
3.4.1 System Demand Control through tickets The ticketing system of our Helper Library is generally feasible for all orchestrations that need quantity-limited design. Within our Accelerator, the tickets are included in the orchestrations of the Roedl.HelperLibrary.CRM to better coordinate the workload of the CRM and SAP systems. The settings of the ticket.xml can be adjusted in the runtime. This enables a fine-tuning of the interface in real time. The configuration file ConfigurationTicket.xml This file contains all information necessary for the ticket class. <config> <TimeOut value ="2000" /> <LoopDelay value ="1" /> <ThrowErrorOnTimeOut value="false" /> <Debug value="false" /> <Entity key="account" value="20" /> <Entity key="salesorder" value="20" /> <Entity key="contact" value="20" /> <Entity key="fetch" value="20" /> </config> Description: Nodename Type Suggestion Meaning Timeout value Integer 10000 100000 Ends opening the tickt after this time in ms LoopDelay value Integer 2000 Busy if tickets occupied, waiting x ms for next request Throw ErrorOnTmeOut value Boolean FALSE If true exception if Timeout reached Debug value Boolean FALSE If true, then Debuginfos for Debugview Entity key String Name of the entity Entity value Integer 10-50 Maximum amount of parallel tickets for this entity
3.5 BAM Templates for interface monitoring Microsoft Dynamics CRM - SAP Accelerator for BizTalk Server Microsoft BizTalk Server delivers standard interface monitoring by default (BAM- Business Activity Monitoring). Additionally, Roedl Accelerator allows monitoring of the implemented interfaces between Microsoft Dynamics CRM 2011 and SAP. Furthermore, the BAM is possible to be integrated with the independently created custom interfaces.
3.6 ESB Toolkit Templates for exception handling Enterprise Service Bus (ESB) toolkit is a free add- on for BizTalk Server, which delivers, among the others, the exception handling functionality. Accelerator and the ESB are the compatible tools. Once an exception occurs in Accelerator it is going to have a graphical and fully descriptive representation in the ESB. Therefore, all the incidents can be tracked leading to the well- organized and detailed process control.
Example of Exception Handling Microsoft Dynamics CRM - SAP Accelerator for BizTalk Server This orchestration is usually requested in case of a catch. It is given the error causing message. The message is edited so it can be read and sent out via a send-port group. 3.6.1 Other 3.6.1.1 System Demand Control through tickets The ticketing system of our Helper Library is generally feasible for all orchestrations that need quantity-limited design. Within our Accelerator, the tickets are included in the orchestrations of the Roedl.HelperLibrary.CRM to better coordinate the workload of the CRM system. The settings of the ticket.xml can be adjusted in the runtime. This enables a fine-tuning of the interface in real time. The configuration file ConfigurationTicket.xml This file contains all information necessary for the ticket class. <config> <TimeOut value ="2000" /> <LoopDelay value ="1" /> <ThrowErrorOnTimeOut value="false" /> <Debug value="false" /> <Entity key="account" value="20" /> <Entity key="salesorder" value="20" /> <Entity key="contact" value="20" /> <Entity key="fetch" value="20" /> </config> Description: Nodename Type Suggest ion Meaning Timeout value LoopDelay value ThrowErrorOnTi meout value Debug value Entity key Entity value Integ er Integ er Bool ean Bool ean Strin g Integ er 10000 Ends opening the tickt after this 100000 time in ms 2000 Busy if tickets occupied, waiting x ms for next request FALSE If true exception if Timeout reached FALSE If true, then Debuginfos for Debugview Name of the entity 10-50 Maximum amount of parallel tickets for this entity 3.6.1.2 Logging The orchestrations are created in a way that many debug outputs are written by default. In addition to that, the logging framework Log4NET is being used, which enables very simple configuration in the runtime. Several log targets can be defined, such as: the error console of the system a log file There are further settings auch as the size of the maximum amount of log files or whether a file may be overwriten. 3.6.1.3 The configuration We developed our own small library to configure various aspects of the interface. It is able to read multiple configurations from XML-files in the runtime.
3.7 Rödl BizTalk Server Runtime Monitor Service Microsoft BizTalk Server has by design the problem that Receive Locations stopped working, when the source for the messages is not available! For that, Roedl writes the BizTalk Server Runtime Monitor! The functionality of the BizTalk Server Monitor is: ü Automatic recognize of deactivate Receive Locations ü Automatic recognize of deactivate Send Ports ü Automatic recognize of deactivate Host Instances ü Automatic starting of each Receive Location, Send Port or Host Instance is possible ü Sending of Emails to the Administrator, when Receive Locations, Send Ports or Host Instances are down ü Working in background as a service We create also a Client, who can call the service as a WCF Server. With this client, the Admin can see from his local machine, if all is fine or not with the BizTalk Server. The configuration of the service is also really simple. All information are stored in a XML config file: <BiztalkMonitor> <ConnectionString>server=.;database=BizTalkMgmtDb;Integrated Security=SSPI</ConnectionString> <EMailSender>biztalk@roedl.com <EMailSender> <SMTPPassword></SMTPPassword> <SMTPServer>200.204.26.10</SMTPServer> <EMailTo>jBizTalkAdministrator@roedl.com</EMailTo> <EMailSubject>Error in BizTalk customization</emailsubject> <Applications> <Application name="esbtest"> <ReceiveLocations> <ReceiveLocation portname="fileinrp" name="fileinrl" autostart="true"> </ReceiveLocation> <ReceiveLocation portname="rp1" name="receive Location1"> </ReceiveLocation> </ReceiveLocations> </Application> <Application name="biztalk Application 1"> <ReceiveLocations> <ReceiveLocation portname="rpba1" name="receive Location2"> </ReceiveLocation> </ReceiveLocations> </Application> </Applications> <HostInstances> <HostInstance name="biztalkserverapplication" autostart="true" /> </HostInstances> </BiztalkMonitor>
3.8 Roedl Accelerator Function Library The Roedl Accelerator Helper Library is a toolset of functionalities that facilitates a developer s effort. The Library contains the functionalities that are not normally defined in the Microsoft BizTalk Server scope. Therefore, a developer is equipped with code templates and additional functions (e.g. a complete Debug Logger is available in the system).
Examples of the Helper Library for Microsoft Dynamics CRM and SAP: In the introduction it was mentioned that large parts of the SAP and CRM know-how has been encapsulated into an easy to use API. This API will be explained in further detail now. The Accelerator utilizes several utility classes to simplify the communication with both SAP and MSCRM significantly. In addition, logging and error handling functions are included. Since no interface works without configuration, a module has been integrated that enables convenient configuration of the interface while the systems are running. These utility modules will also be explained in further detail in the following. 3.8.1 HelperLibrary for Microsoft Dynamics CRM The Helper Library for Microsoft Dynamics CRM consists of several orchestrations that simplify the communication significantly. 3.8.1.1 CreateEntity The orchestration has a parameter (ref) of the type System.Xml.XmlDocument that transfers the message to an entity. The same message is given back with the GUID of the newly created entity added to it. Name Type Direction Description msgentity System.Xml.XmlDocum ref Entity that is supposed to be created (in) ent Entity (with GUID), that has been created (out)
3.8.1.2 UpdateEntity The orchestration has a parameter (ref) of the type System.Xml.XmlDocument that transfers the message to an entity. The same message is given back. The GUID must be part of the transferred message. If there is no entity in Microsoft Dynamics CRM containing the GUID, the orchestration creates an exception. Name Type Direction Description msgentity System.Xml.XmlDocum ref Entity that is supposed to be changed (in) ent Entity (with GUID), that has been changed (out) 3.8.1.3 DeleteEntity The orchestration has a parameter (ref) of the type System.Xml.XmlDocument that transfers the message to an entity. The transferred message can contain the GUID only. All other transferred attributes are being ignored. If there is no entity in Microsoft Dynamics CRM containing the GUID, the orchestration creates an exception. Name Type Direction Description msgentity System.Xml.XmlDocum ref Entity (GUID) that is supposed to be deleted (in) ent Entity (with GUID), that has been deleted (out)
3.8.1.4 GetEntityByFetch The orchestration is given a fetch message. The first entity that matches the fetch request is given back as well as the number of entities found. Name Type Direction Description msgfetch System.Xml.XmlDocument In Fetch Request msgreturnentity System.Xml.XmlDocument Out Entity Message intcount System.Int32 Out Number of entities found
3.8.2 HelperLibrary for SAP R/3 Microsoft Dynamics CRM - SAP Accelerator for BizTalk Server This library encapsulates all SAP requests. BAPI and RFC components as well as IDOCS are being requested via this library. In case trfc components are being requested, the library takes care of commit and rollback of the components. The library cooperates with the latest Microsoft WCF-Custom-SAP-Adapter 3.0. 3.8.2.1 RfcCall The RfcCall orchestration serves to request RFC, trfc und BAPI components. Name Type Direction Description msgrequest System.Xml.XmlDocument In Request from SAP msgresponse System.Xml.XmlDocument Out Response from SAP 3.8.2.2 SendIDoc This orchestration serves the purpose of sending arbitrary IDOCS to the SAP system. Name Type Direction Description msgrequest System.Xml.XmlDocument ref Request from SAP
3.8.3 Helper Library for Logging The orchestrations are created in a way that many debug outputs are written by default. In addition to that, the logging framework Log4NET is being used, which enables very simple configuration in the runtime. Several log targets can be defined, such as: the error console of the system a log file There are further settings as the size of the maximum amount of log files or whether a file may be overwritten.