BizTalk Mapping Patterns and Best Practices at BizTalk User Group Sweden in Gothenburg

Editor's Notes

• #4: My name is Sandro Pereira and I’m working as a BizTalk Consultant at DevScope in Portugal an amazing company and I’m a Microsoft Integration MVP since 2011 Writer of numerous articles for Portuguese eMagazine “Programar” Author “Sandro Pereira BizTalk Blog” http://sandroaspbiztalkblog.wordpress.com Member of “BizTalkAdminsblogging.com” and “BizTalk Brasil” community Member NetPonto community MSDN BizTalk Forums Moderator TechNet Wiki author (Wiki Ninja) TechNet Gallery, Code Gallery and CodePlex contributor Public speaker Technical Reviewer PACKT Publishing BizTalk Server 2010 Cookbook (April 2012)
• #5: Last year I told here that I was writing a book about Mapping Patterns… It is a pleasure to say that the book is already available for free… almost 400 pages about mapping so I hope if you have not done so go there download it and hope you enjoy
• #9: Syntax Transformations: This type of transformations occurs in the receive or send pipelines and aim to transform a document into another representation, e.g. CSV to XML. Here the document maintains the same data (semantics), but changes the syntax that is represented. I.e. we translate the document, but typically we don't modify the structure. Normally, this type of transformation is bidirectional, since we still have the same semantic content, we can apply the same transformation logic and obtain the document in its original format. Data Translation Change the format of data between messages Example: translate between a flat file and an XML file Semantic Transformations: This type of transformation usually occurs only in BizTalk maps. Here the document maintains the same syntax that is represented (XML), but changes its semantics (data content). This type of transformation are typically one-way, since that when we added and aggregate small parts of the information, that compose the document into another differently document, we may miss important details for its reconstruction. Data Transformation Perform computational and other data operations Copy the data from one message to another
• #10: BizTalk map. A file that defines the correspondence between the records and fields in one schema and the records and fields in another schema. BizTalk maps are implemented in XML Extensible Stylesheet Language Transformations (XSLT). Extensible Stylesheet Language Transformations (XSLT). An industry-standard specification defined by the World Wide Web Consortium (WC3) for expressing transformations between two documents. The XSLT generated by BizTalk is fully W3C compliant.
• #11: The biggest difference between using maps in ports or in orchestrations is that, when we use maps inside orchestrations we can have multiple messages inputs (transformations of many documents into one final document – transformations N1) and ports only allows a single input message (transformations 11).
• #15: Basically on this mapping problem there are two similar schemes, which we intend to map the source elements in their proper destination and for which we implemented the following challenges: Concatenate the first and last name in order to give the full name (Concatenation of values); Map the Address in its destination element (Direct copy) Transform the date of birth in age (Custom scripts); The Zip Code should only be mapped if it has a valid string, otherwise it will not be mapped (Conditional selection); The element Civil Status is optional and as we have no evidence in the source to map it, it should be ignored (Add new values). Additionally, we perform a more advanced mapping logic to calculate the total of national and international calls using cycles, conditional selections and mathematical operations. The first element found is "Address"; Since there is link associated to this element, the link is translated by one XPath expression (”Address/text()”) that defines the element to be extracted from the source
• #16: The second element found is “ZipCode” also with a link associated; It is a conditional selection that will be translated into a XSLT condition (xsl:if):
• #17: The third element is "CivilStatus", since there are no links associated with this element, it is ignored in the mapping process
• #18: The fourth element to be processed is “FullName” and once again there is a link associated, which corresponds to the concatenation of two elements of the origin; If you check the generated code, is assigned to the element "FullName" the value of variable “$var:v3” that is generated from the execution of the function userCSharp:StringConcat that visually represents the String Concatenate Functoid:
• #19: The fifth element is “Age”, a link is found which means that custom script found inside the CSharp Inline Scripting Functoid will be carried out:
• #20: Finally we found the record “PhoneBilling” that contain two elements: “TotalInternational” and “TotalNational” both with links associated. Note that although we have not defined any cycle through loop functoid, the compiler is smart enough to realize that we are dealing with a cycle and translate it correctly. However, unfortunately, it will create an own cycle for each element. For a better optimization, it will require us to use a custom script. Both elements are calculated using conditions and mathematical calculations as you will see in the generated code
• #22: In fact the order with which we associate links (Drag&Drop) from the source to different destination elements is irrelevant, since the compiler, as previously explained, will process them in the correct order… Except if we have to associate several links to the same functoid. In this cases the order in which the link association takes place is extremely important and can lead to unexpected results. If we change the order in which the links are associated to the functoid, it will lead to mapping errors or unexpected results, according to the functoid used.
• #23: Except if we have to associate several links to the same element or record. In this cases the order in which the link association takes place is extremely important and can lead to unexpected results. If we change the order in which we associate the links on the same element in the destination schema we can also have an impact on the desired final result. Unfortunately, when we associated different links in the same element, there is no way or place in the graphical editor where you can check the order of the association, for example, similarly to what happens with the functoids. The only way to verify the order in these cases is to inspect the XSLT generated code or testing the map.
• #24: However there is one important exception to this rule of Link Sequence, especially when using custom scripts in recursive records or elements. Once again, a good example of this scenario is the use of custom scripts to increment counters. We can illustrate this scenario by adding two Scripting Functoids to the map We would expect that in the first cycle the result of the second script was the value "1", in the second cycle we obtained the value "2" and so on. However, if we test the map, we will see that the reality is different:
• #27: Readability and Maintainability: For new and even for expert developers, or even when working with developers from other teams, using multiple grid pages will make the map easier to read and maintain if necessary make changes Level of effort: by being easier to read and maintain you will reduce the development time. Documentation: Using this technique will also help you to make a better map documentation, and sometimes this could be enough to self-documenting the map
• #29: Label and Comments tab Label – Enter the new name. The maximum number of characters allowed is 256. If a string with more than 256 characters is specified, the first 256 characters are accepted and the rest are discarded. Comments – Enter the comments for the functoid. The maximum number of characters allowed is 1024. If a string with more than 1024 characters is specified, the first 1024 characters are accepted and the rest are discarded.
• #30: Label and Comments tab Label – Enter the new name. The maximum number of characters allowed is 256. If a string with more than 256 characters is specified, the first 256 characters are accepted and the rest are discarded. Comments – Enter the comments for the functoid. The maximum number of characters allowed is 1024. If a string with more than 1024 characters is specified, the first 1024 characters are accepted and the rest are discarded.
• #31: We can make an analogy with C# code, if we are reading a function or a piece of code that doesn’t have any comment or inadequate nomenclature can become difficult and time consuming to understand, but if it’s well structured and commented the developer's life will be facilitated, same principals applies here
• #33: Validating a Map Before a map is deployed, it should be tested to ensure that the resulting message contains the desired results. The BizTalk Mapper provides the tools for validating a map and testing a map with sample data, much as can be done with schemas Testing a Map Before you can test a map, you need to specify the type and location of an instance message to be used for testing. Several properties need to be configured, which are set in the .NET properties for the map. The .NET properties, as distinguished from the BizTalk properties, are accessed by right-clicking the map in Solution Explorer, and then clicking Properties. Validate TestMap Input. Specifies that the input message should be validated. Validate TestMap Output. Specifies that the output message should be validated. TestMap Input Instance. This is the message that you will use as the source message instance for the map. This instance should be validated using the schema validation steps described in Module 2, “Creating Schemas.” TestMap Input. This specifies the format of the test message (XML, Native, or Generated Instance). Native specifies that Visual Studio must convert the input file from flat file format to XML before executing the map. TestMap Output. This specifies the format of the test message (XML or Native). TestMap Output Instance. Specifies the file location for the message to be written to. If the file exists in the same location, it will be overwritten. After configuring the test properties, right-click the map in Solution Explorer, and then click Test Map. A link to the input and output message will be shown.
• #34: The advantages of using the built-in functoids: Overview: Using the functoids can give a developer a better overview of what functionality a map provides, and sometimes also provides a good overview for not developers also. Whereas Scripting Functoids with custom XSLT needs to be read by a developer to be able to understand what is happening. Maintainability: A BizTalk developer knows about maps and functoids without necessarily having to know or be an expert in XSLT (although I find extremely important). By staying with the functoids you give a BizTalk developer, new, expert or even to external team members, the best possible opportunity to understand a new map and for them to maintain. This reason is valid if you don’t use very complex functoid chain and use best practices in how to organize and document your maps. Level of effort: by being easier to read and maintain you will reduce the development time. Disadvantages or precautions to take when using built-in functoids: Limit of operations scope: Although BizTalk Server provides many functoids to support a range of diverse operations, we are limited to the number of existing operations. The advantages of using Scripting Functoid: Performance. It seems rather obvious that automatically generated XSLT can’t perform as well as custom XSLT. So if you really need high performance, use your own XSLT. Maintainability. If you remember I used this argument also in the built-in functoids as well with conditions and the same appends here if you use Scripting Functoids (C# or XSLT) to simplify the complex functoid chain or to be more reusable, then you are improving the Readability of the map and therefore enhance the maintaining process. Everything is possible: Basically you can do what you want. Use different script types or combine them to solve your transformation problems. Disadvantages or precautions to take when using Scripting Functoids: Reuse: We need to rewrite over and over again the code! Compiler limitations: When you create a script, for example a C# function, remember to give it a correct name because all scripts with the same name are treated as one by the compiler regardless of whether the code is equal or not. Support: If you are thinking in use XSLT language: it supports only XSLT 1.0 (XSLT 2.0 not supported).
• #35: An alternative to wrap repeated transformation rule logic in custom functoids is to put this rules in an external assembly and call this assembly through a Scripting functoid. The advantages of using External Assemblies: Maintenance: If you have an external assembly with your transformation rule logic, then you can easily maintain it (making fixes or updates) without having to update and redeploy the maps. This means that you can change the way all those maps work by modifying and replacing the assembly in the global assembly cache (GAC). Level of effort: They are reusable. Because the .dll containing the transformation rule logics is deployed into the Global Assembly Cache (GAC), you can reference them in your project you can reference and use this functionality in your Scripting Functoids as often you want and again considerably simplifying the development task at hand. Development: You can have several operations inside a single .dll and you can easily implement version control and have different versions running side-by-side. Disadvantages or precautions to take when using External Assemblies: Overview: Because they are encapsulated inside Scripting Functoids, is more difficult to read visually on the map grid compared with built-in functoids or custom functoids. Compiler limitations: By register the assembly in the GAC will make your function available to BizTalk, but the function will not be available to the map until you add the .dll as a reference to your mapping project. So you need to reference your assembly in every single project that you want to use. Development: it requires experience with .NET Programming and since there may be multiple instances of a map running at the same time, you have to ensure that the code used in an assembly called from a Scripting functoid must be thread safe.
• #36: Second rule: A good example for using scripting functoids is to solve grouping problems, complex transformations rules with multiple elements or if/else decisions inside cycles.
• #37: The advantages of using External Custom XSLT file: Development: XSLT file can be developed separately and hosted in a BizTalk map. Disadvantages when using External Custom XSLT file: Level of effort: Not quite as intuitive, Functoids are more easy to read visually on the map grid and therefore Requires “geeky” coding skills Overview: You loss of visual map representation. Some people also mention that using External Custom XSLT file can also improve: Performance and Compiler limitations: Direct XSLT is more powerful, fewer limitations than the BizTalk Mapper and will definite will improved performance in complex transformation rules. I don’t agree because you can also implement use custom XSLT and use this approach.
• #44: 02- JoinMultipleMessages 03- HowToSendOrchestrationVariablesIntoMaps 04- WorkingWithConstantValues 05- MappingWorkingWithMultipleOutputMessages 06 – WorkingWithConditions 07- BizTalkMapperIFinLoppingRecord 08- MuenchianGroupingSortingWithoutLosingMapFunctionalitie 10- SandroPereira.MappingToNameValueRecord 11- SandroPereira.NameValueToHierarchical CommonMappingLogicConstructs