White Papers: EDIDocument Crystal Universe Software Document Version: 1.2 Overview EDIDocument is a powerful state-of-the-art component that provides a fast and easy way to create EDI files. EDIDocument follows an object-oriented hierarchy schema that makes it easy to use and understand. EDIDocument enables developers to create any EDI file (X12 and EDIFACT) fast and easily. It s as simple as creating a new EDIDocument object and adding loops, segments and data elements to it. EDIDocument contain many built in smart features that prevents malformed EDI files without compromising speed. Developers will experience how powerful EDIDocument is with their first use. General EDI Segment Structure NM1*40*1*ABC CORP*A:B:C*SS~ Text Description NM1 Segment 40,SS Data Element A,B,C Composite Data Element * Element Separator Character : Composite Separator Character
~ Segment Separator Character General EDI File Structure The ANSI X12 standards dictates the structure of X12 EDI files. The general EDI file structure contains loops, segments, elements and composite elements. Loops contain segments, a segment contains elements and elements may or may not contain other elements (composite elements). An example EDI file format is: Loop: Interchange Header (Start of EDI File) Segment: ISA Loop: Functional Header Segment: GS Loop: Transaction Header Segment: ST Segment: BHT Segment: SE Segment: GE Segment: IEA (End of EDI document) EDIDocument and RDPCrystal EDI Library mirrors this parent-child model when creating and/or loading EDI data. EDIDocument contain DataLoop objects which in turn can contain DataSegment objects which in turn can contain DataElement objects and so on. When generating EDI files RDPCrystal navigates this parent-child hierarchy and visits each node in order to retrieve their data. Creating a new EDI file is easy with the EDIDocument. Simply create an instance of the EDIDocument, add DataLoops and DataSegments, then call the file generation method of EDIDocument. Delimiters EDI files contains a list of segments. Each segment is delimited by a special character called a Segment Terminater Character. Segments also contain elements. Each element is delimited by a special character called an Element Terminator Character. Some elements may contain other elements as well. These sub-elements are called composite elements. Composite elements are also delimited by a sepcial character called a Composite Terminator Character. Every EDI file contains these special characters that are used to separate data into logical units. EDIDocument contains a Delimiters object where these special characters are assigned. By knowing what these special characters are EDIDocument and EDIValidator can correctly create, validate and load EDI files
File Buffer Size and Performance EDIDocument uses an internal buffer to enable it to quickly create EDI files. EDI Document does not put a limit on the size of EDI files that can be created. A limit is imposed by the environment. This limit is a file with up to approximately 2 billion characters. This length is more space than is required for any EDI file. Because EDI files can be very large, generating EDI files can consume a lot of memory and effect performance. Speed improvement is attained by specifying an approximate length of the EDI file that you are in the process of creating in terms of the amount of characters. The internal buffers will not need to recreate itself with more space if this limit has not been reached. By default the file buffer size is set to 2000 characters. This limit should be change while generating an EDI file larger than 2000 characters (which is most likely the case).. Auto Placement of Number of Segments One of the bodies that govern some of the EDI specifications is ANSI X12. There is a category of EDI files that relate directly to healthcare. These EDI files contain a segment, SE, used to specify the end of a header. According to the rules, this segment must contain the total amount of segments used within that specific header. EDIDocument will automatically keep a count of the number of segments used in the header and goes as far as to actually enter that data in the SE segment. This feature can also be turned off in cases where developers want to put other data in the SE segment. The property used to toggle this setting is AutoPlaceCorrectNumOfSegments. Segment Separator String EDI data can become cumbersome to look at visually. Some parsers may place a return character or a carriage return character so data can be more readable and appear on different lines. This is perfectly acceptable in the industry. Aside from placing a Segment Terminator Character at the end of each segment, an extra piece of formatting text can also be placed. This extra piece of formatting text is called the Segment Separater String. Developers who use this property can add their own formatting to their EDI files. Truncation of Empty Elements For Smaller Files and Faster Processing While developing EDI files certain data elements may be required. In some cases the data in the elements are known to exist and correctly entered. However, in other cases, there is no data to be entered in those elements. Because each element must be delimited by the Element Terminator Character, if there is no data in one or more elements at the end of a segment what we have is a series of Element Terminator Characters followed by a Segment Terminator Character. For example, if the Element
Terminator Character is * and the Segment Terminator Character is ~ then we will have the following segment: Figure 1. NM1*41*2*ABC Corp********~ EDIDocument has the ability to locate and remove these empty data elements. One immediate benefit of this is smaller EDI files. Another benefit of this is that parsers need not load empty data into memory for loading and validation, wasting valuable processor time. The property that governs this is TruncateEmptyElements property. After removal the data segment will look like the following: Figure 2. NM1*41*2*ABC Corp~ Truncation of Empty Composite Elements for Smaller Files and Faster Processing While creating EDI files certain data elements can contain other data elements. These are called composite data elements. In some cases the data in the composite data elements are known to exist and correctly entered. However, in other cases, there is no data to be entered in those elements. Because each composite data element must be delimited by the Composite Element Terminator Character, if there is no data in one or more composite data element what we have is a series of Composite Element Terminator Characters followed by an Element Terminator Character. For example, if the Element Terminator Character is * and the Segment Terminator Character is ~ and the Composite Element Terminator is : then we will have the following segment: Figure 3. NM1*41*2*ABC Corp*A:B:::::*34~ EDIDocument has the ability to locate and remove these empty composite data elements. One immediate benefit of this is smaller EDI files. Another benefit of this is that parsers need not load empty data into memory for loading and validation, wasting valuable processor time. The property that governs this is TruncateEmptyCompositeElements property. After removal the data segment will look like the following: Figure 4. NM1*41*2*ABC Corp*A:B*34~ Padding of Data Values to Reach Length Contraint Requirements One of the bodies that govern some of the EDI specifications is ANSI X12. EDI specifications require that both data elements and composite data elements stay between a mininum and maximum length. Failure to adhere to this length constraint may result in an EDI file failing validation and being rejected. If this is time sensitive data, then the
sendor may have to regenerate the EDI file and resend it to the receiver wasting considerable time. EDIDocument has to ability to pad or trim element data to reach mininum and maximum element length requirements. While generating EDI files, as EDIDocument visits each node (both data element and composite data elements), it checks the data to verify that they are within the minimum and maximum lengths. If the data element adheres to these length constraints they are left alone. On the other hand, if they violate the length constaints, EDIDocument will either pad or trim the element in order the satisfy the constraints. Automatic Trimming of Data Elements Incorrect or extra data in EDI files may cause file rejections. Extra spaces in either data elements and/or composite data elements both cause file bloat and file rejections. In extreme cases where a developer does not have control of what data is placed in data elements, there may be an enormous amount of empty characters ( ). EDIDocument has the ability to trim blank spaces from the beginning and ending of element data. While generating EDI files, EDIDocument visits each node (both data element and composite data elements) and checks them for extra blank spaces ( ). EDIDocument then removes them. Graphical Generation of EDI Tree Structure With Data After generating a new EDI file, EDIDocument has the ability to generate a graphical tree representation of the entire EDI file structure. The graphical tree contains the entire EDI file schema as well as element data. The data type of this tree is a System.Windows.Forms.TreeNode object and can be added to a TreeView control for viewing. Users can click to the treenodes to view its contents. An example of this treenode is below: Figure 5.
Generating EDI Files in Test Mode EDI data can look confusing to new EDI developers. Large EDI files can even be intimidating to seasoned EDI developers. EDIDocument has the ability to work in Test mode. When in test mode and an EDI file is generated, extra information is included in the EDI file. This enables the newly created EDI file to be more readable. Developers will also be able to see their EDI data and determine if it is correct. After developers determine that their EDI files are being generated correctly, they can switch back to production mode and continue generating EDI files the same way. In production mode the EDI file will not contain the extra data used to increase readability. For example, while in test mode, an EDI file will look like the following: Figure 6. ------------------------------------------------------------------------------------------------------------ Functional Header Loop GS*HC*ApplicationSenderCode*ApplicationReceiverCode*2005*132334*1*X*000 10X098A1~ ST Transaction Header ST*837*1~ BHT*0019*00*1*20070515*094553*CH~ SE End Of Transaction SE*3*1~ END SE End Of Transaction
END ST Transaction Header GE End Functional Group GE*1*1~ END GE End Functional Group END Functional Header Loop IEA End Interchange IEA*1*1~ END IEA End Interchange END Interchange Header ----------------------------------------------------------------------- EDI File Generation Directly To Memory The EDI file generation process often ends with an EDI file being generated on the file system of the developer or program that initiated the process. The developer or program can then retrieve the EDI file from the file system and send it to an EDI receiver. This process, straight forward as it may seem, requires data to first be written to the file system only to be read in again. This IO (input/output) task wastes a lot of processing time and power. This simple delay can add up considerably when generating hundreds or even thousands of EDI files. EDI Document has to ability to generate an EDI file directly into memory. This is a one step process that avoids writing and reading to and from the file system. The newly created EDI data in memory can then be used immediately. EDI File Generation with Batch Writing The EDI file generation process often ends with an EDI file being generated on the file system of the developer or program that initiated the process. By default EDIDocument will write the EDI data to to stream while creating an EDI file. This can lead to many IO operations. EDI Document has to ability to generate an EDI file directly into memory and then make one batch write to the file system efficiently. This results in just one IO call and increases speed of the EDI file creation operation. XML File Generation XML is currently the leading standard for sharing data in a loose and flexible way. XML data is not attached to any system or programming language. By using XML, applications from disparate systems can communicate with each other.
EDIDocument taps into this paradigm by having the ability to generate EDI data in XML format. Applications can be written to parse the XML EDI data in any fashion imaginable. An example of the XML generated from EDIDocument is displayed below: Figure 7. <?xml version="1.0" encoding="utf-8"?> <Loop Name='Interchange Header'> <Segment Name='ISA'> <Element>00</Element> <Element> </Element> <Element>00</Element> <Element> </Element> <Element>ZZ</Element> <Element>InterchangeSenderID</Element> <Element>ZZ</Element> <Element>InterchangeReceiverID</Element> <Element>070303</Element> <Element>18:04</Element> <Element>U</Element> <Element>00401</Element> <Element>T</Element> <Element>:</Element> <Loop Name='Functional Header Loop'> <Segment Name='GS'> <Element>HC</Element> <Element>ApplicationSenderCode</Element> <Element>ApplicationReceiverCode</Element> <Element>2005</Element> <Element>132334</Element> <Element>X</Element> <Element>004010X098A1</Element> <Loop Name='Transaction Header'> <Segment Name='ST'> <Element>837</Element> <Segment Name='BHT'> <Element>0019</Element> <Element>00</Element> <Element>20070515</Element> <Element>094553</Element> <Element>CH</Element> <Loop Name='End Of Transaction'> <Segment Name='SE'> <Element>3</Element> </Loop> </Loop> <Loop Name='End Functional Group'> <Segment Name='GE'>
</Loop> </Loop> <Loop Name='End Interchange'> <Segment Name='IEA'> </Loop> EDI Document Solutions Issue #1: When creating EDI documents I sometimes have trailing empty data elements. How does EDIDocument solve this? Solution #1: EDI Document has an option to automatically remove empty trailing data elements and empty trailing composite data elements. You can be assured that your segment structure will be correct. Issue #2: Sometimes there are trailing empty spaces in my data element values. This can cause my EDI files to be rejected by my organization. How does EDI Document solve this? Solution #2: EDI Document has an option to automatically remove trailing spaces from data elements. Issue #3: It is often difficult and time consuming to check all the minimum and maximum lengths of data element values when creating EDI documents. How does EDI Document solve this? Solution #3: EDI Document has an option to automatically pad or trim all data element values to make sure that they meet minimum and maximum required lengths. This way your data elements will always contain the correct data length. Issue #4: It is often difficult to keep track and enter the correct number of segments in a transaction. How does EDI Document solve this? Solution #4:
EDI Document has an option to automatically count and enter the correct number of segments in a transaction in the SE (X12) or UNT (EDIFACT) segments. You don't need to keep track of this number. Issue #5: I want to see the EDI document I just created graphically. How does EDI Document solve this? Solution #5: EDI Document creates a TreeView node that contains a hierarchical view of the EDI document just created. You can add the node to a TreeView control. You will be able to see all the loops, segments, elements and values graphically. Issue #6: I want to see the EDI document I just created in XML format. How does EDI Document solve this? Solution #6: EDI Document creates allows you to save the EDI document in XML format. You can then use the XML file however you desire