TIBCO Engage Integration Guide Release 1.0.0 May 2014 Two-Second Advantage
Important Information SOME TIBCO SOFTWARE EMBEDS OR BUNDLES OTHER TIBCO SOFTWARE. USE OF SUCH EMBEDDED OR BUNDLED TIBCO SOFTWARE IS SOLELY TO ENABLE THE FUNCTIONALITY (OR PROVIDE LIMITED ADD-ON FUNCTIONALITY) OF THE LICENSED TIBCO SOFTWARE. THE EMBEDDED OR BUNDLED SOFTWARE IS NOT LICENSED TO BE USED OR ACCESSED BY ANY OTHER TIBCO SOFTWARE OR FOR ANY OTHER PURPOSE. USE OF TIBCO SOFTWARE AND THIS DOCUMENT IS SUBJECT TO THE TERMS AND CONDITIONS OF A LICENSE AGREEMENT FOUND IN EITHER A SEPARATELY EXECUTED SOFTWARE LICENSE AGREEMENT, OR, IF THERE IS NO SUCH SEPARATE AGREEMENT, THE CLICKWRAP END USER LICENSE AGREEMENT WHICH IS DISPLAYED DURING DOWNLOAD OR INSTALLATION OF THE SOFTWARE (AND WHICH IS DUPLICATED IN THE LICENSE FILE) OR IF THERE IS NO SUCH SOFTWARE LICENSE AGREEMENT OR CLICKWRAP END USER LICENSE AGREEMENT, THE LICENSE(S) LOCATED IN THE LICENSE FILE(S) OF THE SOFTWARE. USE OF THIS DOCUMENT IS SUBJECT TO THOSE TERMS AND CONDITIONS, AND YOUR USE HEREOF SHALL CONSTITUTE ACCEPTANCE OF AND AN AGREEMENT TO BE BOUND BY THE SAME. This document contains confidential information that is subject to U.S. and international copyright laws and treaties. No part of this document may be reproduced in any form without the written authorization of TIBCO Software Inc. TIBCO, Two-Second Advantage, and TIBCO Engage are either registered trademarks or trademarks of TIBCO Software Inc. in the United States and/or other countries. All other product and company names and marks mentioned in this document are the property of their respective owners and are mentioned for identification purposes only. THIS DOCUMENT IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. THIS DOCUMENT COULD INCLUDE TECHNICAL INACCURACIES OR TYPOGRAPHICAL ERRORS. CHANGES ARE PERIODICALLY ADDED TO THE INFORMATION HEREIN; THESE CHANGES WILL BE INCORPORATED IN NEW EDITIONS OF THIS DOCUMENT. TIBCO SOFTWARE INC. MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE PRODUCT(S) AND/OR THE PROGRAM(S) DESCRIBED IN THIS DOCUMENT AT ANY TIME. THE CONTENTS OF THIS DOCUMENT MAY BE MODIFIED AND/OR QUALIFIED, DIRECTLY OR INDIRECTLY, BY OTHER DOCUMENTATION, INCLUDING BUT NOT LIMITED TO ANY RELEASE NOTES AND "READ ME" FILES. Copyright 1999-2014 TIBCO Software Inc. ALL RIGHTS RESERVED. TIBCO Software Inc. Confidential Information
Contents iii Contents Preface........................................................................ vi Related Documentation..................................................................... vi Typographical Conventions.................................................................. vii Connecting with TIBCO Resources............................................................ ix How to Join TIBCOmmunity.............................................................. ix How to Access TIBCO Documentation...................................................... ix How to Contact TIBCO Support........................................................... ix Chapter 1 Data Integration Introduction.............................................1 Implementation and Data Integration............................................................ 1 TIBCO Engage as an Integration Hub.......................................................... 2 Chapter 2 Data Integration Process.................................................3 Technical Integration Process................................................................. 3 Data Integration Prerequisites............................................................. 3 Data Integration Project Plan and Tasks......................................................... 4 Project Task Descriptions.................................................................... 5 Chapter 3 Data Exchange Overview.................................................9 Data Content Types......................................................................... 9 Transaction........................................................................... 10 Customer............................................................................ 10 Product Catalog....................................................................... 11 Store................................................................................ 11 Location............................................................................. 11 Rewards............................................................................. 11 List................................................................................. 12 Profile............................................................................... 12 Generic Event......................................................................... 12
Contents iv Email................................................................................ 12 Data Exchange Requirements................................................................ 13 Data Exchange Methods.................................................................... 15 Batch-file Exchange.................................................................... 15 Real-time SOAP-based APIs............................................................. 16 Data Exchange: Batch and API Differences.................................................. 17 Chapter 4 Batch Imports.........................................................19 About Batch Files.......................................................................... 19 File Processing Constraints.............................................................. 19 Batch File Summary Descriptions............................................................. 20 Data Content Types........................................................................ 22 Transaction........................................................................... 22 Customer............................................................................ 22 Product Catalog....................................................................... 23 Location............................................................................. 23 Rewards............................................................................. 23 List................................................................................. 23 Profile............................................................................... 24 Generic Event......................................................................... 24 Batch File Guidelines and Specifications........................................................ 25 Upload Products Using Batch Files............................................................ 26 How to Upload Product Data via Batch Files................................................. 28 Upload Transactions Using Batch Files......................................................... 28 What s In The Transaction File I - Transaction Details.......................................... 28 What s In The Transaction File II - Returns................................................... 30 What s in the Transaction File III - Customer Data............................................. 32 Chapter 5 On-Demand File Exchange..............................................34 About On-Demand File Exchange............................................................. 34 List Import............................................................................... 36 File Format........................................................................... 37 Sample File........................................................................... 37
Contents v List Export............................................................................... 38 Profile Export............................................................................. 41 Email Statistics Export...................................................................... 41 Reward Catalog Import..................................................................... 43 File Format........................................................................... 44 Custom Questions Import................................................................... 44 Custom Question File Details and Requirements................................................. 45 Chapter 6 Flat File Formats and Examples..........................................48 Import Files.............................................................................. 48 Sample of Product File from Clients........................................................... 52 Format and Sample of Store File.............................................................. 56 Chapter 7 File Import Reports....................................................57 About the File Import Report................................................................. 57 Creating a File Report...................................................................... 58 Chapter 8 Email Update Import File................................................60 General Description.................................................................... 60 Email Update Import File Use Cases........................................................... 61 Standard XML Sample File and XSD Schema................................................ 61 File Usage Guidelines, Data Elements and Formats............................................... 62 Samples and Formats of Email Update Files from Clients....................................... 62 Email Update File and Record Failures......................................................... 63 Chapter 9 SSO Integration.......................................................66 SSO Overview............................................................................ 66 Secure Data Transfer................................................................... 67 How SSO Works.......................................................................... 67 Implementation Details..................................................................... 70 Query String Authentication Parameters.................................................... 72 Configuration of Client SSO Handler....................................................... 72 Return Codes for SSO Authentication.......................................................... 72 Index..........................................................................74
vi Preface This book explains the technical requirements for enabling specific TIBCO Engage features and benefits. It includes How-To information for managers, project managers, technical leads, data managers, programmers, and anyone responsible for implementing a TIBCO Engage solution. Topics Related Documentation, page vi Typographical Conventions, page vii Connecting with TIBCO Resources, page ix Related Documentation The following documents form the TIBCO Engage documentation set: This book serves as a guide for exchanging customer, product, and transaction information with TIBCO Engage. TIBCO Engage API Reference This document explains how to use TIBCO Engage API calls, including authentication, and provides useful information about the entities involved. TIBCO Engage Release Notes The release notes provide a list of new and changed features. These documents are located on the TIBCO Product Documentation site: https://docs.tibco.com/
Preface vii Typographical Conventions The following typographical conventions are used in this manual. Table 1 General Typographical Conventions Convention ENV_NAME TIBCO_HOME <ProductAcron ym>_home Use TIBCO products are installed into an installation environment. A product installed into an installation environment does not access components in other installation environments. Incompatible products and multiple instances of the same product must be installed into different installation environments. An installation environment consists of the following properties: Name Identifies the installation environment. This name is referenced in documentation as ENV_NAME. On Microsoft Windows, the name is appended to the name of Windows services created by the installer and is a component of the path to the product shortcut in the Windows Start > All Programs menu. Path The folder into which the product is installed. This folder is referenced in documentation as TIBCO_HOME. TIBCO <ProductName> installs into a directory within a TIBCO_HOME. This directory is referenced in documentation as <ProductAcronym>_HOME. The default value of <ProductAcronym>_HOME depends on the operating system. For example on Windows systems, the default value is C:\tibco\<ProductAcronym>\<ReleaseNumber>. code font Code font identifies commands, code examples, filenames, pathnames, and output displayed in a command window. For example: Use MyCommand to start the foo process. bold code font Bold code font is used in the following ways: In procedures, to indicate what a user types. For example: Type admin. In large code samples, to indicate the parts of the sample that are of particular interest. In command syntax, to indicate the default parameter for a command. For example, if no parameter is specified, MyCommand is enabled: MyCommand [enable disable]
Preface viii Table 1 General Typographical Conventions (Cont d) Convention italic font Key combinations Use Italic font is used in the following ways: To indicate a document title. For example: See TIBCO ActiveMatrix BusinessWorks Concepts. To introduce new terms For example: A portal page may contain several portlets. Portlets are mini-applications that run in a portal. To indicate a variable in a command or code syntax that you must replace. For example: MyCommand PathName Key name separated by a plus sign indicate keys pressed simultaneously. For example: Ctrl+C. Key names separated by a comma and space indicate keys pressed one after the other. For example: Esc, Ctrl+Q. The note icon indicates information that is of special interest or importance, for example, an additional action required only in certain circumstances. The tip icon indicates an idea that could be useful, for example, a way to apply the information provided in the current section to achieve a specific result. The warning icon indicates the potential for a damaging situation, for example, data loss or corruption if certain steps are taken or not taken.
Preface ix Connecting with TIBCO Resources How to Join TIBCOmmunity TIBCOmmunity is an online destination for TIBCO customers, partners, and resident experts. It is a place to share and access the collective experience of the TIBCO community. TIBCOmmunity offers forums, blogs, and access to a variety of resources. To register, go to http://www.tibcommunity.com. How to Access TIBCO Documentation You can access TIBCO documentation here: http://docs.tibco.com How to Contact TIBCO Support For comments or problems with this manual or the software it addresses, please contact TIBCO Support as follows. For an overview of TIBCO Support, and information about getting started with TIBCO Support, visit this site: http://www.tibco.com/services/support If you already have a valid maintenance or support contract, visit this site: https://support.tibco.com Entry to this site requires a user name and password. If you do not have a user name, you can request one from your Customer Loyalty Management representative.
1 Chapter 1 Data Integration Introduction This section introduces key concepts about integrating data with TIBCO Engage and discusses the various files needed for information exchange. Topics Implementation and Data Integration, page 1 TIBCO Engage as an Integration Hub, page 2 Implementation and Data Integration Your company, as a TIBCO Client, uses our code base, infrastructure, and professional services to set up and maintain a experience management program. TIBCO imports your data into our normalized and highly scalable databases. This greatly reduces the impact of our technology on your IT department. TIBCO data integration specialists define any necessary data file format transformations necessary. Data is transferred via batch files and real-time API processes. Once the integration setup work is completed, your technical staff is responsible only for routine batch file data transfers (for example, automatically generating and forwarding report files). When the basic file transfer mechanisms are in place, your data is normalized, matched, transformed, and processed to help you with marketing segmentation, targeting, reporting, and analysis. All data in the system is regularly updated, and includes data volunteered by site visitors and the information tracked by the system across all channels. Profiles are updated daily and applied across the platform to revise offers, communications, messages, notifications, and results. The platform provides a wide set of flexible reports that enable you to evaluate performance, learn from the results, and modify activities accordingly.
TIBCO Engage as an Integration Hub 2 TIBCO Engage as an Integration Hub The TIBCO Engage integration hub is made up of application interfaces that allow for seamless data flow between TIBCO Engage and customer point of contact and analytical systems such as POS, ecommerce, Email, and CRM systems. The simplicity of our integration reduces the high cost and time requirements of a custom-built solution.
Data Integration Process 3 Chapter 2 Data Integration Process This section explains the TIBCO Engage data integration process. Topics Technical Integration Process, page 3 Data Integration Project Plan and Tasks, page 4 Project Task Descriptions, page 5 Technical Integration Process As part of the initial integration process, TIBCO Professional Services Group (PSG) devotes a day to review integration requirements and develop the project plan. Using our Onboarding Checklist, Pre-built Project Plan along with the technical documentation, we review the process in collaboration with clients. Required participants from your company might include the Program Manager, Marketing, the IT Manager, IT Developers, and Finance (to discuss customer crediting processes), as needed. Data Integration Prerequisites Once the integration process has started, participants typically want to make rapid progress, without stopping or slowing to collect fundamental data and resources. TIBCO has therefore identified the following information gathering and planning milestones, which should be completed before the technical data integration effort begins: Client receives and completes the Onboarding Questionnaire, or completes it to a reasonable degree. The TIBCO PSG representative for the individual client determines when there is sufficient information to proceed.
Data Integration Project Plan and Tasks 4 Client receives documentation, data-mapping guidelines, templates, and sample files to define the mapping between the TIBCO native representation of transaction data and the client s native data format. Client receives standard site flow diagrams and Client Review Documents (CRDs) from TIBCO for reference. (Working with these standard design starting points, the client can customize the design of its own site.) Completion of the final site flow design is not required at this point. Client receives a project plan and tentative timeline from TIBCO. These planning tools allow TIBCO and its clients to work efficiently together towards their shared implementation goals. TIBCO applies standard project management tools and its own practical experience to guarantee progress. Note that the integration progress can begin only after all four of these prerequisites are met. Data Integration Project Plan and Tasks TIBCO typically manages the data integration process with a project plan similar to the one represented in this table. This summary plan information includes both the tasks performed by TIBCO and the tasks that you perform as a client. Table 2 Data Integration Project Requirements and Project Plan Summary Task Send Technical Integration Documents and Sample Data Technical Integration kickoff Client sends sample data in accepted format Create client data mapping and requirements documents Set up a secure FTP connection Review sample data and give feedback Client revises and sends sample data (round 1) Review revised sample data and give feedback Owner Data Integration Data Integration Client Data Integration Client Data Integration Client Data Integration
Project Task Descriptions 5 Table 2 Data Integration Project Requirements and Project Plan Summary (Cont d) Task Client revises and sends sample data (round 2) TIBCO signoff on final client data formats Send Final Data Mapping document to client Client signs off on Data Mapping document Owner Client Data Integration Data Integration Client Note: The project description details listed above are only samples. Actual project plans are customized for each client, according to specific needs and circumstances. Project Task Descriptions The data integration tasks for which you, as a client, are primarily responsible are listed in Table 3. Table 3 Data Integration Task Descriptions Actions Task TIBCO Client Allocate a technical point of contact in the client organization to receive technical integration information. Provide the client with the contact information for the TIBCO Technical Integration Lead and a back- up contact. Provide TIBCO with the name, email address, phone number, and time of availability of the technical point of contact who will receive technical integration documents and sample data. Confirm that technical contact has access to the TIBCO Client Extranet." Provide links and passwords to client technical contact, as necessary. Confirm access. Resolve any access problems, as necessary.
Project Task Descriptions 6 Table 3 Data Integration Task Descriptions (Cont d) Actions Task TIBCO Client Send Data Mapping document templates and file samples. TIBCO Customer Loyalty Management representative sends data mapping documents and sample files (and related links) for the baseline data integration. The set of documents sent to the client corresponds to the client s needs, as documented by their responses to the Onboarding Questionnaire. Client receives all appropriate data mapping templates, sample files, and related documentation (including this Integration Guide). Client initiates data file mapping project. Set up a secure FTP connection. Client provides completed data mapping documents and sample files for transaction, product, and retail stores data. Note: This step is repeated as necessary (round 1, round 2, etc.). TIBCO s native file format is XML, but flat files are accepted. Sample files must not contain valid credit card numbers. Configure secure file transfer protocol to allow access to ToInStorecard and FromInStorecard directories for the client. Deliver SFTP or FTPS username and password to client in a secure manner: by secure email, phone, personal meeting. Verify that completed data mapping documents and sample files have arrived, are readable. Help the client make corrections. Download secure client software, as necessary. Complete SFTP or FTPS setup and log in. Attempt to upload a file to ToInStorecard directory. Attempt to download a file from FromInStorecard directory. Communicate attempts and results to TIBCO. Transfer completed data mapping documents and sample files to TIBCO over secure transfer (or email, if secure path is not yet set up). Provide a point of contact to answer questions and make corrections relating to the file formats.
Project Task Descriptions 7 Table 3 Data Integration Task Descriptions (Cont d) TIBCO provides client with sample reward and sample customer files. Actions Task TIBCO Client Test file transfer process. Provide files for export to client. Test file transfer process. Test ability to consume export data. Provide technical assistance. TIBCO provides client with sample reward and sample customer files. TIBCO reviews files, ensures that transaction, product and retail store file formats have required fields. Notes: Transaction data should include customer (customer) data. This step is repeated as necessary (round 1, round 2, etc.). Client signs off on final data mapping document. TIBCO creates a process to import the client s transaction, product and retail store data into the client s account. In some cases, set up for transfer of historical transaction data file. More rarely, set up for transfer of email address update file. Test file transfer process. Provide files for export to client. Check the documentation and sample files to ensure that all of the TIBCO required fields exist and that the file format is understood. The TIBCO QA department develops a test plan based on the approved data mapping document. Create import process for client-specific file formats if the data is not in TIBCO native format. Provide technical assistance. Test file transfer process. Test ability to consume export data. Provide technical assistance. Provide technical assistance. Ensure that TIBCO file format and transfer requirements are understood by technical associates at client. (For example: If the file format ever changes, the changes must be communicated to TIBCO. ) Client approves (via email) a final data mapping document that identifies the final formats to be used in production. Provide technical assistance. Set up plan to notify TIBCO and implement changes if a local (client-specific) file format changes. Provide technical assistance.
Project Task Descriptions 8 Table 3 Data Integration Task Descriptions (Cont d) Client creates a process to export transaction, product and retail store data to the ToInStorecard directory. Actions Task TIBCO Client Provide technical assistance. Ensure successful secure transfer capability. Automate uploading/downloading to/from sftp directories. Automate export of three files: transaction, product, and retail store data. Client creates a process to accept/consume the customer file and rewards file from the FromInStorecard directory. TIBCO tests and validates import process. Provide technical assistance. Test import process and validate data. Provide technical assistance. Provide technical assistance. Test regular end-to-end import process. This requires a period of live file transmissions from the client to ensure that systems on all sides are working correctly. Establish file monitoring process (to ensure timely file processing and catch errors). Turn on automated import process. Data Integration Team monitors the status of all incoming and outbound files. Turn on automated file delivery process. Provide technical assistance during test period. Establish an email contact (ideally an alias for several people in the client organization) for any data integration/monitoring issues. Turn on production import process. Turn on production processes. Turn on production processes.
Data Exchange Overview 9 Chapter 3 Data Exchange Overview This section introduces the types of data that TIBCO exchanges with its clients, and briefly describes the available methods Batch file transfer, Real-time API, and On Demand transfer for performing these data exchanges. Topics Data Content Types, page 9 Data Exchange Requirements, page 13 Data Exchange Methods, page 15 Data Content Types TIBCO and its clients exchange the following types of data files: Transaction Customer Product Catalog Store Location Rewards List Profile Generic Event Email Notes: Import text files must be in ASCII format. Other text formats may cause imports to fail.
Data Content Types 10 The particular set of batch files your company provides or consumes depends on your particular experience management program. Typically, only a subset of the batch files is required. These files are identified during your onboarding process (when you first define and implement your program with TIBCO). Transaction Transaction files include data concerning purchases or returns of products or services, and also customer data. The Transaction File describes a client s (retailer s) transactions, and usually contains the transactions for a 24-hour period. The Transaction file includes data about the client s customer, the item (product or SKU), and the store or location associated with the transaction, as well as the quantities and amounts. Transaction data allows TIBCO Engage to award points and give other rewards based on offers. In addition, it enables targeting customers based on their purchase behavior. Typical TIBCO Engage implementations involve the loading of a historical transaction file that provides two years worth of data and a daily transaction file. The daily transaction files transmitted to TIBCO includes new transaction information. During initial setup, TIBCO will work with you to determine the historical and daily volume of data transfers. Customer Sometimes called Shopper data or experience management Member data, the Customer file data refers to a client s customers. Clients submit customer information together with transactions to be able to track behavior, report, target, and communicate to customers. After the initial loading of the customer data, customer updates (changes only or the delta) can be provided to TIBCO on a daily, weekly, or monthly basis, using the Transaction File format.
Data Content Types 11 Product Catalog The Product Catalog includes information for each individual product SKU or service supported by a client. Use this file if you want to: Present SKU-specific and category-specific offers. Target on past purchases of specific SKUs or categories. Collect category preferences from Customers and target on them. The Product Catalog file should always be the master file. All Product data will be overwritten on each file load. Product files can be transmitted daily, weekly, or monthly depending on how frequently your product catalog changes. Store The Store file provides a list of client stores or locations, and may include variations such as web-based retail channels. This data is required for any features that include store-based offers. Similar to the Product file, clients should always transmit their master store list to TIBCO since the latest transmitted file completely replaces the previous. Location The Location file includes store or sales channel information for each store. The Location file allows clients to target on store information, including store locations. For example, the zip/postal code of stores can be compared to the zip/postal code for experience management program members to determine their proximity to a store for targeting. Similar to the Product Catalog data load, the Location file should always be the master file. All store information will be overwritten on each file load. Rewards The Rewards file provides the current list of rewards due to be delivered to customers for a given period. It includes specific data about the reward and the recipient customer. Rewards refer to goods, gift cards, credits to a reward account, etc. Point-based programs generate rewards, typically by converting points into rewards, such as gifts or gift cards. Rewards data (for example, Reward Export files) generally go to the client or a client-designated third party for fulfillment of the rewards.
Data Content Types 12 List Lists are fixed groups of customers as defined by their email addresses. Most marketing systems treat customer groups as lists; TIBCO Engage also provides native support of the concept. Exchanges of list information typically involve transferring lists from one system to another so that the recipient can perform an action based on the list. Clients can import customer lists to be used for sending email within TIBCO Engage or for reporting. Clients can also export customer lists to be used in external systems. List data import includes email addresses. Note: Because the email address serves as a unique identifier, it allows access any other existing customer data. Profile A profile is a set of business rules that defines a group of customers. A profile dynamically defines a customer segment by filtering a customer population with targeting parameters such as gender, location, prior purchases, etc. For example, a profile might comprise a set of demographic criteria that are used to filter one list of customers in order to define a new list. Clients can define profiles in TIBCO Engage and export profile information from TIBCO Engage (for example, lists of customers who match a profile). Generic Event Use generic events to capture data about customer activities and behavior beyond the TIBCO Engage system. Examples of generic events include customer actions such as writing a product review, abandoning a product in an online shopping cart, playing a game online, or adding an item to a list of favorite products. Email This is not a standard option. The Email Update Import file allows clients to update email addresses in the TIBCO Engage database. TIBCO Engage imports this batch file when the email address is the only available identifier for the customer and TIBCO is not the party responsible for maintaining the "system of record. System of record refers to when the customers new and current email addresses are maintained in another non-tibco application. In this case, the updated record is imported by TIBCO Engage to update its database. Refer to Email Update Import File for more information.
Data Exchange Requirements 13 Data Exchange Requirements TIBCO accepts data from clients in XML, flat delimited ASCII files, or flat fixed-format ASCII files. XML files are much less prone to error and require less maintenance over the medium and long term. Notes: Import text files must be in ASCII format. Other text formats may cause imports to fail. TIBCO also delivers data files to clients. There are two options for delivery: XML or a flat, delimited ASCII file. Data exchanges containing client data must be done via a secure file transfer system, never through an unsecured file transfer mechanism. However, the sample files used to set up data mapping standards, which have no live or sensitive data, can be sent via email. Notes: Make sure your file names are consistent with the naming convention and in XML format. Make sure the data in your file is consistent with your data mapping. Validate your files using the TIBCO Engage XML Validation tool at http://new.loyaltylab.com/xmlvalidator/validate.aspx Table 4 shows the methods(s) of data exchange that may be used for transferring each different type of data. As indicated in this table, there is not a one-to-one relationship between the method of transferring and the content-type of the data. To convey some types of data, two or more different methods may be used. For example, customer data may be updated by a Transaction file as a batch process or by a real-time API.
Data Exchange Requirements 14 Table 4 Data Exchange Methods used for Data Content Types Batch Transfer On Demand Transfer Real-Time API * Transaction Customer Product Catalog Location Transaction file (ISCLog.xsd) Transaction file (ISCLog.xsd) External Registration File For export only: Customer Registration File (Shopper.txt) Product File (ISCProduct.xsd) Store File Example*: GetTransactions (by Customer) Examples*: CreateShopper GetRegisteredCard GetShopperOffers GetShopperPointBalance RedeemReward Content Reward Reward File (RewardsSample.txt) List List.txt Import List (TIBCO Engage) Export List (TIBCO Engage) Example*: RedeemReward Profile Reward Catalog Points Export Profile (TIBCO Engage) Reward Catalog Import (TIBCO Engage) Point Expiration Forecast Report Export (TIBCO Engage) Example*: GetShopperPointBalance Generic Events ISCEventInstance.xsd Example*: CreateEventInstance Custom Questions CustomQuestions.csv Custom Question Import (TIBCO Engage)
Data Exchange Methods 15 * Note: For specific information about obtaining and updating data in real time, and API methods involved, see the TIBCO Engage API Reference. Each type of data mentioned in Table 4 is explained in the Data Content Types section. The TIBCO Customer Loyalty Management team can assist you with instructions, guidelines, and data mapping templates, and other tools to help you with data exchange processes. Data Exchange Methods The standard data exchange options include: Batch-file exchange Real-time SOAP API On-demand file exchange (TIBCO Engage) Each data exchange method supports different sets of data. During on-boarding, and depending on your requirements, TIBCO advises clients on the data exchange methods that are available and recommended. For example, you can import transactions as batch files but not on demand. During the process of defining and setting up the experience management program for a new client (the onboarding process) the emphasis is on Batch-file exchange and/or Simple Object Access Protocol (SOAP) APIs. Batch-file Exchange Batch file processing is ideal for high-volume, asynchronous processing. Most of the TIBCO Engage batch processes require files in XML format that adhere to common retail standards, such as ARTS and IXRetail. Note, however, that TIBCO can usually use files in your existing proprietary format, if you make arrangements with TIBCO during your onboarding process. TIBCO can write transformations to convert flat files in delimited text format (for example, commaor tab-separated value files) into usable XML files.
Data Exchange Methods 16 The TIBCO batch file integration works as follows: By default, you send data files using a secure transfer protocol (for example, SFTP) according to The TIBCO standard schedule, unless your company has negotiated a different batch file transmission schedule. TIBCO picks up these files and processes them, applying different transformation routines, as necessary, to convert the data into our required format. Inbound XML files are processed in their native format. Flat files that match The TIBCO standard file formats are handled using standard transformations. Flat files that do not match The TIBCO standard file formats are handled using custom transformations. Requests to change these file formats must be submitted through TIBCO's Customer Loyalty Management team and may incur a fee. (Consult your TIBCO Customer Loyalty Management representative for details.) In return, TIBCO posts the resulting files using a secure transfer protocol according to our standard schedule, unless your company has negotiated a different batch file transmission schedule. Refer to File Import Reports to see a file report that provides data about file imports. Batch File Exchange Benefits Flexible frequency based on your needs and capabilities. Rapid transfer of large quantities of data (up to 200K transactions or 200M per day). Flexible file requirements use common XML retail standards including ARTS and IXRetail, or use existing data feeds and TIBCO can write the transformations. Monitoring monitor your imports using the File Import Report. High security Visa CISP-certified, secure FTP. Real-time SOAP-based APIs The TIBCO Engage real-time SOAP-based APIs are used to add, retrieve, update, or delete one record at a time, with almost instantaneous response to the request for service.
Data Exchange Methods 17 Clients most often use real-time data transfer to get, add, or update customer data, or to get or update a customer s reward balance in operational systems. For data security, this data is sent via Hypertext Transfer Protocol over Secure Socket Layer (https). Real-time APIs are used for client companies who require real-time processing, and who can dedicate technical resources to the project. Typically, the real-time SOAP-based APIs provide for Point of Sale (PoS), ecommerce Shopping Cart sites, and/or Payment Terminals or Gateways. Implementing a real-time API requires additional effort, including programming, from the client s technical staff. This data transfer method is meant for experienced programmers already familiar with web services and programming via SOAP-based APIs. SOAP-based API benefits include: Near-instantaneous data exchange. Support for POS and/or Payment Terminals or Gateways. Different solutions for each channel. The ability to mix real-time with batch by location and/or channel. The TIBCO Engage SOAP-based APIs provide a simple language- and platform-independent interface that enables implementers to work in whatever language and platform they are most comfortable. The only requirements are that the implementation language and platform support https and XML. For more information about the TIBCO Engage SOAP-based APIs, please see: http://api.loyaltylab.com/loyaltyapi/help/index.html. Data Exchange: Batch and API Differences API and batch file processing are the two key technologies for data transfer between TIBCO and its clients. There are some basic differences between two methods of data integration: API and batch processing. The two processes are designed to serve different needs, and as such behave differently. Record Level Business Process Application Batch Process: The batch process enforces certain data requirements and business logic based on the setup decisions made during the TIBCO onboarding process. Loyalty API: The Loyalty API will not apply any business logic to your incoming data regardless of application settings. API use assumes that business logic is implemented and applied in the application that calls the API. The application set up completed during the onboarding process does not apply to API processing.
Data Exchange Methods 18 Dates and Times Batch Process: Batch processing will apply default date and time stamps to various data. Loyalty API: The Loyalty API will not automatically populate all relevant date-time fields. However, you can populate them yourself using external business logic. Shopper Matching Batch Process: Using the batch process you can configure multiple matching criteria (unique customer identifiers). This ensures that shoppers are matched correctly. For example, this ensures that shoppers are correctly matched to transactions when you are using a transaction batch file. Loyalty API: The Loyalty API will only enforce email address uniqueness when adding a record. If you use other customer identifiers, TIBCO suggests that you use available Loyalty API methods to check for uniqueness before adding the record. Bulk Data Processing Batch Process: The batch process is designed for inserting multiple records at once, thus much of the processing overhead is streamlined for this process. Loyalty API: The Loyalty API is meant for singleton manipulation of data. Although processing may occur almost instantaneously, only one record is processed at a time. Null Data Processing Batch Process: The batch process will not update a non-null field to a null. If null data is passed through by means of a batch file and there is existing data, the existing information in the database will be retained. Loyalty API: The Loyalty API will allow you to null out any field that is nullable.
Batch Imports 19 Chapter 4 Batch Imports This section provides information about batch file imports. Topics About Batch Files, page 19 Batch File Summary Descriptions, page 20 Data Content Types, page 22 Batch File Guidelines and Specifications, page 25 Upload Products Using Batch Files, page 26 Upload Transactions Using Batch Files, page 28 About Batch Files Use batch files to update the TIBCO Engage database and to provide updates with the latest information maintained by TIBCO. Most of the data exchanged between TIBCO and its clients is transmitted through batch files. Batch processing is a highly efficient method of transferring large volumes of data. File Processing Constraints Under normal circumstances, TIBCO Engage processes data at a rate of approximately 100,000 records per hour. Note that regularly scheduled file import processes are not intended to support the population or re-population of a client's entire collection of records, nor is normal file export processing set up to handle large "dumps," that is, downloads of the complete collection of a client's records. If you believe you may need to import or export batch files that far exceed the normal and customary processing limits, contact TIBCO Customer Loyalty Management.
Batch File Summary Descriptions 20 Batch File Summary Descriptions This section describes the batch files and their usage, and includes guidelines and specifications to help you define and format the batch files. Note: Import files must be ASCII text files. Other formats may cause imports to fail. This section includes detailed information about the following files: Transaction and Customer Import File Also called the Transaction file, describes a client s (retailer s) transactions, and usually contains the transactions for a 24-hour period. The Transaction file includes data about the client s customer, the item (product or SKU), and the store or location associated with the transaction, as well as the quantities and amounts. Product Catalog Import File - Sometimes called the Product file, this is a complete listing of all products offered by the client. To use TIBCO Engage features that include SKU-based programs or offers, you must supply product data. The Product and Store files are the only complete replacement files at this time. Each time the Product File is loaded, all previous product data is inactivated. Product files can be transmitted daily, weekly, or monthly depending on how frequently your product catalog changes. Store Import File - Provides a list of client stores or locations, and may include variations such as web-based retail channels. This data is required for any features that include store-based offers. Similar to the Product File, clients should always transmit their master store list to TIBCO since the latest transmitted file completely replaces the previous file. Reward Export File Provides the current list of rewards due to be delivered to shoppers for a given period. It includes specific data about the reward and the recipient customer.
Batch File Summary Descriptions 21 Customer Export File - TIBCO Engage extracts all customers whose profile information was updated in the system between the current time and the time at which the previous extract occurred. More specifically, a customer's record is included in the Customer Export File if there are changes in the following types of information in that customer's record: Customer Name Customer Address Email Address Opt-In/Out of Email Change in point balance Custom attribute data changes Tier status Loyalty membership status Customer Status Registered card/loyalty card. Generic Event Import File - This file allows clients to submit instances of events to be tracked with the Generic Event framework. The file includes new event data such as the event ID, subject, object, master value, and time. A wide variety of information can be passed using the generic event file; for example, instances of customers writing product reviews, instances of customers adding items to a wish list or favorites list, data about customer activities captured from social networking sites, and web analytics data about customer activities. Email Update Import File - This is not a standard option. The Email Update Import file allows clients to update email addresses in the TIBCO Engage database. TIBCO Engage imports this batch file when the email address is the only available identifier for the customer and TIBCO is not the party responsible for maintaining the system of record. System of record refers to when the customers new and current email addresses are maintained in another non-tibco application. In this case, the updated record is imported by TIBCO to update its database. Refer to Email Update Import File for more information. The particular set of batch files your company provides or consumes depends on your particular experience management program. Typically, only a subset of the batch files is required. These files are identified during your onboarding process (when you first define and implement your program with TIBCO). Based on your business needs, TIBCO Customer Loyalty Management designates the batch files that your company needs to send and receive.
Data Content Types 22 Data Content Types TIBCO and its clients exchange the following types of data files: Transaction Customer Product Catalog Location Rewards List Profile Generic Event Transaction Transaction files include data concerning purchases or returns of products or services, and also customer data. The Transaction File is the data feed that clients most commonly provide to TIBCO. Transaction data allows TIBCO Engage to award points and give other rewards based on offers. In addition, it enables targeting customers based on their purchase behavior. Typical TIBCO Engage implementations involve the loading of a historical transaction file that provides two years worth of data and a daily transaction file. The daily transaction files transmitted to TIBCO include new transaction information. During initial setup, TIBCO will work with you to determine the historical and daily volume of data transfers. Customer Sometimes called Shopper data or Loyalty Member data, this data refers to our client s customers. Clients provide TIBCO customer information together with transactions to be able to track behavior, report, target, and communicate to customers. After the initial loading of the customer data, customer updates (changes only) can be provided to TIBCO on a daily, weekly, or monthly basis, using the Transaction File format.
Data Content Types 23 TIBCO returns new registration and updated customers so that clients can maintain their own records. Product Catalog The Product Catalog includes information for each individual product SKU or service supported by a client. Use this file if you want to: Present SKU-specific and category-specific offers. Target on past purchases of specific SKUs or categories. Collect category preferences from Customers and target on them. This file should always be the master file. All Product data will be overwritten on each file load. Location The Location file ncludes store or sales channel information for each store. The Location file allows clients to target on store information, including store locations. For example, the zip/postal code of stores can be compared to the zip/postal code for experience management program members to determine their proximity to a store for targeting. Similar to the Product Catalog data load, the Location file should always be the master file. All store information will be overwritten on each file load. Rewards Rewards refer to goods, gift cards, credits to a reward account, etc. Point-based programs generate rewards, typically by converting points into rewards, such as gifts or gift cards. Rewards data (for example, Reward Export files) generally go to the client or a client-designated third party for fulfillment of the rewards. List Lists are fixed groups of customers as defined by their email addresses. Most CRM and marketing systems treat customer groups as lists, so TIBCO provides native support of the concept.
Data Content Types 24 Exchanges of list information typically involve handing lists from one system to another so that the recipient can perform an action based on the list. Clients can import customer lists to be used for sending email within TIBCO Engage or for reporting. Clients can also export customer lists to be used in external systems List data import includes email addresses. Because the email address serves as a unique identifier, it allows access any other existing customer data. Profile A profile is a set of business rules that defines a group of customers. A profile dynamically defines a customer segment by filtering a customer population with targeting parameters such as gender, location, prior purchases, etc. For example, a profile might comprise a set of demographic criteria that are used to filter one list of customers in order to define a new list. Clients can define profiles in TIBCO Engage and export profile information from TIBCO Engage (for example, lists of customers who match a profile). Generic Event Use generic events to capture data about customer activities and behavior beyond the TIBCO Engage system. Examples of generic events include customer actions such as writing a product review, abandoning a product in an online shopping cart, playing a game online, or adding an item to a list of favorite products. The generic event includes event-related data that a client provides to TIBCO. This event data enables TIBCO Engage (and by extension, the client) to: Identify specific events Collect data about customer actions performed during these events Target these customers (by creating profiles) based on the collected event data. TIBCO Engage uses generic event data to extend its customer-analysis capabilities so that clients can capture and use customer data from a wider variety of sources. Using generic events, you can capture records of any customer action that marketers consider significant, as long as the event data meet the basic requirements defined for a generic event record. In the context of TIBCO Engage's generic event processing, an event is defined as "an action taken by a customer (known as the subject) on an object (a data entity, such as an item or person) at a given time". As an option, an event can be defined to include a "master value" that adds greater specificity. For example, you might define a customer review of a product as an event. Referring to this event, the
Batch File Guidelines and Specifications 25 basic definition is: customer X (subject) reviews (action) product Y (object) on date (time). You could add a product-rating value (for example, a zero to five star rating) as the master value, to gather even more detail about this customer activity. Generic Event data includes such information as: Event ID Event Reference tag name Subject Object Event Date and Time Master Value The ability to target customers by event data adds a new dimension to your existing customer profiling tools and extends the reach of your marketing resources. Batch File Guidelines and Specifications The following information is provided for each data file: General description Use cases File data elements, formats and guidelines Samples and formats of files from clients File failures After your company s initial kickoff meeting with TIBCO, we will send you all of the templates and guidelines for each batch file you should use. Contact your Customer Loyalty Management representative if you do not have the information you need. Use this guide to obtain additional information about the integration process and the technology involved. The client data-mapping templates that are included in this section will help you map your company s data fields to the corresponding fields used as input by TIBCO Engage. These templates help you move your company s data to TIBCO Engage, that is, from your files to ours.
Upload Products Using Batch Files 26 TIBCO will establish fundamental guidelines regarding the transmission of these files during the integration. These guidelines will address information such as the following: Expected maximum size of a daily file Filename format for this file Schedule transmit time Expected number of files sent per day. TIBCO has identified the following recommended processing thresholds for import files listed in Table 5. Clients expecting to exceed these thresholds should notify their Client Service contact. Table 5 Processing thresholds for import files Import File File Size Number of Records Transaction File 500 MB 200,000 Product Catalog 100 MB 100,000 Store Import 100 MB 200,000 Generic Event 10 MB 50,000 Email Update 10 MB 500,000 Note that these thresholds are not file processing limitations but serve to assist The TIBCO operations teams with capacity planning. Upload Products Using Batch Files Use the Product file to send TIBCO your product information via batch files. As you can see from the graphic below, the Product file is an XML file that contains your product names, SKUs, department/category hierarchy, price and brand information. Once this data is in the system if can be used to target offers by product, category and brand. For example, with product data you can run offers like these: Buy Brand X, get 20% off Buy two or more Product Y, get double points on your purchase
Upload Products Using Batch Files 27 Get double points when you shop across two or more departments This is an example of a Product file. As shown in the example, data in the Product file includes the following: 1. <caption></caption> - The name of the product. 2. <department></department> - This places the product in a department/category hierarchy. Marketers often use the department/category hierarchy to build offers for specific departments or categories of products. TIBCO Engage supports up to five levels of product categories. At least one product department/category must be defined. 3. <description></description> - A more detailed description of the product. If no description is provided, the system will use the Caption optional field. 4. <product></product> - The SKU, list price and brand of the product. a. The SKU (or Stock Keeping Unit) is the product identifier within the TIBCO Engage system. As such, each product must have a unique SKU. b. The List Price is required, but it isn't currently used by the system. When the actual list price isn't known, we generally use a value of "0.00." c. The Brand is often used for brand-based offers like "Buy two pairs of ACME jeans "
Upload Transactions Using Batch Files 28 5. delta - A flag indicating whether this product file is a Full or a Partial import. a. If delta="0", or if the tag is missing, the TIBCO Engage system will treat the file as a full content import. b. If delta="1", the Engage system will treat the file as a partial content import. How to Upload Product Data via Batch Files Follow these steps to upload a Product file via batch file: 1. Prepare the Product file and name it using the following convention: <clientname>_products_yyyymmdd.xml 2. Connect to the TIBCO Engage SFTP server at sftp01.loyaltylab.com using the credentials we supply you. 3. Drag and drop the Product file into the ToInstoreCard folder. The TIBCO Engage system will automatically pick up and import the document. 4. Log in to the TIBCO Engage system. 5. Click Reports. 6. Click Specialized. 7. Click File Import Report. 8. Navigate to Reports>Specialized>File Import Report to verify that your Product file uploaded successfully. Upload Transactions Using Batch Files The Transaction File contains the details of your members', and in some cases non-members', transactions. It often includes Customer/Member information, too. Most TIBCO Engage clients automate the process by which this file is created and uploaded by sending a new file every day. What s In The Transaction File I - Transaction Details The Transaction file, in its basic form, contains the details of your transactions.
Upload Transactions Using Batch Files 29 This is an example of a Transaction file. As shown in the example, data in the Transaction file includes the following: 1. <TransactionID></TransactionID> Identifies a particular transaction. TIBCO Engage ignores transactions with duplicate TransactionIDs. 2. <RetailStoreID></RetailStoreID> Identifies the Store at which the transaction was made. 3. <RegisterID></RegisterID> Identifies the Register at which the transaction was made.
Upload Transactions Using Batch Files 30 4. <SalesAssociateID></SalesAssociateID> Identifies the sales associate who took the transaction. 5. <ReceiptDateTime></ReceiptDateTime> Identifies the date and time of the transaction. This value is evaluated to determine whether a transaction falls between the start and end dates of an offer. 6. <ShopperEMail></ShopperEMail> Identifies the person making the transaction. If TIBCO Engage can match the ShopperEMail to an existing account, it will link the transaction to that account. If TIBCO Engage can't match the ShopperEMail to an existing account, it will create a new, non-member customer account for the transaction. While ShopperEMail is often used to identify the person making the transaction, other options are available. 7. <LineItem></LineItem> LineItem tags surround the purchase, return and payment details. These roughly correspond to the line items on the receipt. a. <Sale></Sale> Indicates that the line item is a sale b. <LineNumber></LineNumber> Indicates the line number on the receipt c. <ItemSKU></ItemSKU> Identifies the product purchased. ItemSKU is used to link the product purchased to the product taxonomy d. <Quantity></Quantity> Identifies the number of that item purchased e. <ExtendedAmount></ExtendedAmount> Identifies the cost of the item x the number purchased 8. <Tender TenderType="CASH"><Amount></Amount></Tender> Identifies the method of payment and the amount paid. What s In The Transaction File II - Returns The Transaction file, in its basic form, contains the details of your transactions. In addition to purchases, the Transaction file may also include returns. Returns are represented in the Transaction file between <LineItem></LineItem> tags.
Upload Transactions Using Batch Files 31 This is an example of a Transaction file with Returns data. The Transaction file includes the following returns data (as located in the graphic below): 1. <ItemSKU></ItemSKU> The SKU of the item the customer returned. 2. <Quantity></Quantity> The number of items of that SKU the customer returned 3. <ExtendedAmount></ExtendedAmount> The value of the return. TIBCO Engage will use this number to calculate the number of points to deduct, if points were awarded for the initial purchase. 4. <TransactionLink></TransactionLink> The TransactionLink makes it possible to match the Return to the original purchase. a. <TransactionID></TransactionID> The TransactionID of the original purchase transaction. b. <RetailStoreID></RetailStoreID> The StoreID of the store at which the original purchase was made. c. <RecieptDateTime></RecieptDateTime> The date and time of the original purchase d. <LineNumber></LineNumber> The line number of the purchased item from the original receipt.
Upload Transactions Using Batch Files 32 What s in the Transaction File III - Customer Data In addition to transaction information, the Transaction file can convey Customer data. To do so, we include additional fields between the <shopper></shopper> tags. What TIBCO Engage does with the customer data it receives depends on whether the system can match the incoming data to an existing active member. Note the following about Transaction imports with customer data: If the data is for an active shopper and the system is able to match the incoming data to the existing active member, the system will update the member data. If the data is for a new member, the system will create the member in the system. The Transaction file includes the following customer data (as located in the graphic below): 1. <ShopperID></ShopperID> This is a unique identifier created by the TIBCO Engage system when the shopper is created 2. <ShopperName><ShopperName> This is where you would pass the customer's first, last and middle names 3. <ShopperAddress></ShopperAddress> This is the customer's street address 4. <ShopperPhoneNumber></ShopperPhoneNumber> The customer's phone number 5. <ShopperEMail></ShopperEMail> The customer's email address 6. <LoyaltyProgramID><LoyaltyProgramID> If the customer has an ID assigned by the Loyalty program, you can pass that here 7. <LoyaltyMember></LoyaltyMember> Is the customer a member of the Loyalty program. 0 = is not a Loyalty Member. 1 = is a Loyalty Member 8. <RegistrationDate></RegistrationDate> The date/time the customer became a member of the Loyalty program 9. <EMailOptIn></EMailOptIn> Has the customer agreed to receive email. 1 = Opt in. 0 = Opt out.
Upload Transactions Using Batch Files 33 10. <ShopperStatus></ShopperStatus> Is the customer Active or Inactive. A = Active, which means TIBCO Engage will track activity. I = Inactive, which means TIBCO Engage will not track activity.
On-Demand File Exchange 34 Chapter 5 On-Demand File Exchange This section provides information and specifications required to perform different types of on-demand data exchanges. Topics About On-Demand File Exchange, page 34 List Import, page 36 List Export, page 38 Profile Export, page 41 Email Statistics Export, page 41 Reward Catalog Import, page 43 Custom Questions Import, page 44 Custom Question File Details and Requirements, page 45 About On-Demand File Exchange TIBCO and its clients typically use On-Demand file transfers to import or export certain types of data. Clients can import or export certain files using TIBCO Engage. While these files are formatted as batch files, they are not imported or exported according to a regular schedule. These file transfers are available to clients "on demand." A client identifies and/or defines a data collection using TIBCO Engage, and sets up the transfer of a file, which contains that data. For example, a client may decide to import a list of shoppers who have opted in to receive email into TIBCO Engage. The client prepares this list of data in an appropriate file format and imports the file using list import. Thereafter, the list data is available in TIBCO Engage, where the client can use and manipulate the data in many ways.
About On-Demand File Exchange 35 From TIBCO Engage, you can import or export the following files: List Import The List import file enables clients to import lists of customers in a self-service fashion from their desktop via TIBCO Engage. Once imported, lists can be used for additional segmentation and targeting. Clients can create an import list by uploading a file that includes email addresses. List Export List export allows clients to export all customers in a list at any time from TIBCO Engage. Profile Export Profile export enables clients to export a defined profile of customers at any time from TIBCO Engage. Email Statistics Export Clients can export up-to-date email data for any email communication using TIBCO Engage. Data includes all customers who were sent the email with email status. Reward Catalog Import Clients can upload a file with reward item data that can be used to create or update the reward catalog. Custom Questions Import Clients can import Custom Question records (which include answers) that are added to the pool of custom questions available in TIBCO Engage. While these files are formatted as batch files, they are not imported or exported according to a regular schedule. These file transfers are available to clients on demand. A client identifies and/or defines the data collection in one of these files, and uses TIBCO Engage to set up the transfer of the file. For example, a client may decide to import a list of shoppers, who have opted in to receive email, into TIBCO Engage. The client prepares this list of data in an appropriate file format and imports the file into TIBCO Engage using List import. Thereafter, the list data is available in TIBCO Engage, where the client can use and manipulate the data in many ways. This List Import file is an example of an On Demand file exchange. Refer to the Data Exchange Methods section for a basic introduction to the subject and its context (specifically the subheading about on-demand file transfers). All on-demand imports and exports allow you to work with files from your desktop. That is, you can specify files to be uploaded from your computer, and you can access the files that are downloaded, assuming that your computer has appropriate access to the network resources. Note also that many reports are available through TIBCO Engage. Though not classified as exports, these reports include significant data.
List Import 36 File exchange constraints and processing limitations Import File File Size Number of Records List Import File 25 MB List Export File 200,000 Profile Export File 200,000 Email Statistics Export Reward Catalog Import Custom Questions NA NA NA Notes: On-demand downloads are not intended to handle "dumps", (that is, downloads comprising a client's complete collection of customer records). If you believe you may have processing requirements exceeding these limits, contact your TIBCO Customer Loyalty manager ahead of time. List Import The List Import file is a fixed list of customer email addresses. Because TIBCO Engage uses the email address field as a unique identifier for each customer, an email address matching an existing customer email address in the database effectively provides access to the rest of that existing customer s data. Note: Import Lists must be ASCII text files. Other formats may cause imports to fail.
List Import 37 To upload a new list, perform the following steps in TIBCO Engage: 1. Find the file to be imported, use a text editor to open it, and verify that the file follows the proper format. If you do not need to reformat the file, close the file without changing it. 2. Log into TIBCO Engage and navigate to the list page: Target>Lists. 3. Click Create New List from File Import. 4. Enter the list name and browse to locate the file. 5. Read and agree to terms of use by checking the I agree check box. 6. Review Import Settings. 7. Click Next step. 8. When the next step (Request Status) appears, verify that the information is correct, then click Done. The List is uploaded. Once the import is complete, verify the success of the import. File Format The content of the list to be imported consists solely of one data element: email addresses (no headers). The list can be in comma-separated or tab-separated format, although the simplest format, one email address per line, is recommended. Sample File Sample File Content: karen@example.com mars@example.com josh@example.com amy@example.com inderpreet@example.com If you need to import more complete customer profiles, including names, addresses and phone numbers, contact your TIBCO Customer Loyalty Management Representative to verify your current data integration prior to importing the list.
List Export 38 List Export The List Export file consists of Customer Export list data (also known as the Customer Export list). Be aware that data in the List Export file format also appears in the Customer Export file, and in the Profile Export file, and that the fields common to these files have the same meaning and format. These files do not include column headings (field labels). To identify the data in List Export files, refer to the File Format information provided in the File Formats section. To create this export file, perform the following steps in TIBCO Engage: 1. Log into TIBCO Engage and navigate to the list page: Target>Lists. TIBCO Engage displays export links for all lists. 2. Locate the list to export, and click export. TIBCO Engage displays a pop-up window, which states that the data is being exported and provides the name of the file that will be created. TIBCO Engage moves the file to the FromInStorecard directory. 3. Log into the SFTP server using a pre-arranged username/password and download the file.
List Export 39 File Formats Table 6 List Export File Formats Field# Field Description Type Notes 1 FileId Integer TIBCO Engage internal file identifier. 2 RegistrationDate DateTime If the customer has signed up on the client s microsite, this value will be the date the customer completed signup. Otherwise, this value will be equivalent to the CreationDate. This is also known as Member Since Date. 3 CreationDate DateTime If the client provided a historical transaction feed, this value will be the date of the first transaction appearing for the customer. 4 ShopperStatus String A Active; I Inactive; When a customer opts out of the experience management program, this value will be I. Otherwise, the value will be A. 5 LoyaltyMember Bit 1 Member; 0 Non-member. 6 SourceOfChange [Note: Deprecated field] String Retailer, Shopper, CSR, or Other Note: This field is deprecated. Although the field may still exist, the data values are not used. 7 ShopperId String Client s identifier for this customer. 8 InStorecardShopperID Integer The TIBCO Engage unique identifier for a customer. 9 FirstName String First name of the customer. 10 MiddleName String Middle name of the customer. 11 LastName String Last name of the customer. 12 AddressLine1 String Address line 1 of the customer. 13 AddressLine2 String Address line 2 of the customer.
List Export 40 Table 6 List Export File Formats (Cont d) Field# Field Description Type Notes 14 City String City where the customer resides. 15 State/Province String State/province where the customer resides. 16 PostalCode String Postal code where the customer resides. 17 Email String Email address. 18 EmailOptIn/Bulk email Bit If the customer has opted out of receiving marketing emails, this value will be 0. 19 PhoneNumber String Phone number of the customer 20 LoyaltyCardNumber String If a client assigns a experience management program card ID/number, it can be recorded here. 21 TierName String If a client uses tiers, then tier data appears, otherwise no data appears. 22 TierJoinDate DateTime If client uses tiers, then tier data appears, otherwise no data appears. 23 StatusUpdateDateTime DateTime Customer record update time stamp. 24 PointBalance Integer Customer's current point balance, as of time of report. Here is a sample record file: 1229,2006-12-29 12:45:41.693,2006-12-29 12:45:41.693,"A",0,"Other","452566743 ",49602261, "John","","Brown","200 Main Street",,"San Francisco","CA","94114","john@example.com",1,"415-555-5555","11134 56","Gold Member",2013-12-29 12:45:41.693, 2013-12-29 12:45:41.693,1300
Profile Export 41 Profile Export The Profile Export file corresponds to a profile defined in TIBCO Engage. This file consists of the Customer list generated by the application of that profile. That is, the file format of the profile export file and the Customer (shopper) list file are the same. For a description of the Profile Export file format, see the example included in the List Export section. To create this export file, perform the following steps in TIBCO Engage: 1. Log into TIBCO Engage and navigate to the target>customer profiles page. TIBCO Engage displays export email links for all customer profiles. 2. Locate the profile to export, and click export email. TIBCO Engage displays a pop-up window, which states that the data is being exported and provides the name of the file that will be created. TIBCO Engage moves the file to the FromInStorecard directory. 3. Log into the SFTP server using a pre-arranged username/password and download the file. Email Statistics Export TIBCO Engage users generate this export file (as needed, on a self-service basis) to obtain communication statistics. For each email communication, the system generates a file containing information such as deliveries, opens, clicks, and bounces. To create this export file, perform the following steps in TIBCO Engage: 1. Log into TIBCO Engage and navigate to the email>communications tab. TIBCO Engage displays extract stats links for all communications that were ever live at any point in the past, including emails that are currently live. 2. Locate the email for which you wish to receive statistics. Use the sorting options provided to reduce the number of communications that are listed. 3. Click extract stats.
Email Statistics Export 42 TIBCO Engage displays a pop-up window, which states that the data is being exported and provides the name of the file that will be created. TIBCO Engage moves the file to the FromInStorecard directory. 4. Log into the SFTP server using a pre-arranged username/password and download the file. File Format The exported file includes the data described in this table. Table 7 Email Statistics File Data Descriptions Field Name Type Length Description CommunicationInter nalname nvarchar 1000 Communication Name CommunicationId int 4 Unique Identifier CustomerEmailAddr ess nvarchar 200 ShopperKey.EmailAddress CustomerNumber nvarchar 200 ShopperKey.RetailerShopperId CommunicationSent Date datetime 16 Email Sent Date Delivered int 4 1=Yes/0=No Opened int 4 1=Yes/0=No OpenDateTime datetime 16 Open Time. Only provided if the email was opened Clicked int 4 1=Yes/0=No ClickedDateTime datetime 16 Click Time. Only provided if a link in the email was clicked Bounced int 4 1=Yes/0=No BounceDateTime datetime 16 Bounce Time. Only provided if email bounced
Reward Catalog Import 43 Here is an excerpt from an Email Statistics Export File: PRS: Send email 1 (exclude not sent email 3) 1723 joe_moore@example.com 2013-01-02 12:50:40.477 1 0 0 0 PRS: Send email 1 (exclude not sent email 3) 1723 john_doe@example.com 2013-01-02 12:50:40.490 1 0 0 0 PRS: Send email 1 (exclude not sent email 3) 1723 mike@example.com 2013-01-02 12:49:38.350 1 1 2013-01-02 20:55:05.390 0 0 Reward Catalog Import TIBCO Engage users import their current reward catalog in TIBCO Engage. Using this feature, clients can upload their items and then edit and manipulate the catalog directly into TIBCO Engage. Follow these steps to upload the reward catalog file using TIBCO Engage: 1. Log into TIBCO Engage and navigate to rewards>reward items. 2. Click Action>Upload Rewards File. 3. Browse to locate and select the file to import. 4. Verify that the file format is acceptable. Notes: Links to sources of help on file and data format are provided on the screen. If the files are not correct, make changes as necessary. Use the appropriate editor application to change and save the file. You may need to click Browse again. After selecting the file, click Upload. 5. Once the upload is complete, navigate to the rewards>reward items tab to see the items listed.
Custom Questions Import 44 File Format Reward Catalogs must be imported as ASCII text files. Other formats may cause imports to fail. Your Rewards Items file must include the following columns in this table, in this exact order. Field Name Required Description Reward ID Yes Reward ID for internal use Reward Name Yes Reward Display Name Reward Description No Reward Description used for display Redeem Value Yes Points required to redeem the item Image URL Yes URL of the image for the item Custom Questions Import TIBCO Engage users can import a file containing custom question records. The unique contents of the file are bulk uploaded and added to the list of available custom questions in TIBCO Engage. Then TIBCO Engage users can use these custom questions and answers in questionnaires. Once imported, questions (and their answers) are added to the pool of all Custom Questions. TIBCO Engage users can edit and work with uploaded questions exactly as they do with questions created within TIBCO Engage, using the Create a New Custom Question page. You can also use the file import method to make certain type of changes to existing custom question records. Note: Imported custom question records are not automatically added to a TIBCO Engage questionnaire, except in the case of an imported question that exactly duplicates the text characters of an existing question. In this case only, the associated elements of the record replace those same elements that exist in the questionnaire.
Custom Question File Details and Requirements 45 Custom Question File Details and Requirements Custom Questions must be imported as ASCII text files. Column headers should not be specified in the file. Every line in the file represents a question and its associated answers, including formatting parameters. Every line must include all the following columns (field values) in this exact order: 1. Question text (required). 2. Question category (optional - leave blank if unspecified). 3. Answer format (required), as follows: - Specify "single" to set up the format so that the user can select only one answer by clicking a radio button. - Specify "multiple" to set up the format so that the users can select more than one answer by selecting multiple checkboxes. - If format is unspecified, "multiple" will be selected by default. 4. Answer layout (required), as follows: - Specify "list" to present answers in a list layout. - Select "column" to present answers in a multi-column layout (better for one-word and very short answers). - If unspecified, "list" will be selected by default. 5. One or more columns containing the answer choices for this question. Refer to Sample CSV File (displayed as text) for examples or answer records that apply these requirements. For example, each column-value must be followed by a comma. Two commas with no text between them indicate that a value has been omitted. File formatting technical note: If question or answer text contains a comma, you must enclose that text in quotation marks to prevent the comma from being read as a value separator.
Custom Question File Details and Requirements 46 Sample CSV File (displayed as text) What are your dietary preferences/restrictions?,food habits,multiple,list,no restrictions,ovolactarian,dairy-free,gluten/wheat-free,low carbohydrate,low calorie,low sugar,low fat,diabetic, Vegetarian,Vegan,Organic,Adjusted for allergies not specified here What strongly influences your buying decisions of electronic products?,,multiple,column,advertising,word of mouth,reviews,brand recognition,brand Reputation,Conscience factors/political Correctness, immediate need How do you prefer to be contacted?,,single,list,email,instant Message,Home phone,cell phone,mail,carrier pigeon How did you learn about our product?,advertising,multiple,list,magazine ad,newspaper ad,internet advertisement,internet search,acquaintance or associate - word of mouth,radio,television,telephone book,multiple print media,don't know To prepare a new custom question file for the bulk file uploading process: Refer to the sample CSV file and instructions above, or to the equivalent information in the the Target>Import Custom Questions section of TIBCO Engage Help system. Use Excel or a generic text editor (such as Notepad) to open the sample file. Use this as a reference when you create your own file. Create a CSV file that contains your Question and Answer text and formatting instructions, and store the CSV file in a location on your computer. Locate and upload the CSV file by using the Browse and Upload options in TIBCO Engage Target>Import Custom Questions page. To make changes in existing custom questions using the bulk file upload process: Be aware that you cannot change existing answer-responses with an uploaded file. However, you can upload additional answers. The question text is the key data in each record: it must be an exact duplicate of the existing question text in the record to be changed. You can specify different Category, Answer Format, and Answer Layout data in a record, and when these are uploaded, the changes will be implemented. If you leave blanks where existing Category, Answer Format, and Answer Layout data already exists, the blanks will be treated as errors and the upload will fail.
Custom Question File Details and Requirements 47 After you have created a CSV file that contains your modified Question records, store the CSV file in a location on your computer. You can then use TIBCO Engage to locate and upload the CSV file. Note: If you want to make corrections to an existing custom question record by using the file upload method, the best practice is to start with an exact copy of the existing record and then make changes and additions to it. File Failures Note the following about file import failures: Reward Catalogs must be imported as ASCII text files. Other formats may cause uploads to fail. The Custom Question file upload will fail if the file size exceeds 1 MB. If one answer record fails, the entire file upload (import) fails. After making repairs to the file, repeat the upload process for the entire file. If the upload fails, an error report appears. The report is designed to help locate and identify any errors at the individual record level. The report identifies a faulty record by number (where the first record in the file is number 1, the second is 2, etc.). In many cases, the error reporting will specify the problem to be solved in the record.
Flat File Formats and Examples 48 Chapter 6 Flat File Formats and Examples This section provides information about flat file formats. Topics Import Files, page 48 Sample of Product File from Clients, page 52 Format and Sample of Store File, page 56 If your Company is not using the standard XML format files to transmit data to TIBCO, but is instead using flat files (non-xml files in which the values are separated by commas, tabs, spaces, or bars, or in which values are positionally defined), refer to the flat file examples in this section. Fundamental information about the content of files is provided in Batch File Guidelines and Specifications. Also note that much information about the data elements and data hierarchy prescribed for each batch file can be obtained from the XML schema documentation. Refer to the TIBCO Engage XML namespace: http://www.instorecard.com/schema/v1/retailintegration/ Import Files Transaction and Customer Sample Files Clients agree to send Transaction files to TIBCO in a format that is strictly defined. This format is the result of a careful needs analysis and a corresponding data mapping effort that identifies which of the client s available data fields will be included in the Transaction file supplied to TIBCO.
Import Files 49 The preferred format is an XML file corresponding to the requirements of the XSD schema (ISCLog.xsd).This file format is based on ARTS/IXRetail data model, and this is the preferred format for the Transaction files received from clients. TIBCO can also transform Transaction files sent in other formats so that they can be used to supply the necessary transaction data. TIBCO has extensive experience in performing such transformations. Samples of some transaction files appear below. Each sample is followed by a corresponding table that shows the sample file format and its data formatting parameters. Typically, a client company provides a sample file and a formatting table to indicate the data and formatting it will provide in its transactions files. The file format samples and tables convey information about the fields in their native file format. The client and TIBCO must formally agree on the information provided in these tables, and TIBCO must be able to use the sample files (after any agreed-upon transformations). When a live Transaction file is imported by TIBCO, any required transformations are performed automatically, based on the understanding that each transaction file will always follow the formatting guidelines established and documented in the tables. Each live file is transformed (as necessary) and ultimately processed as part of a TIBCO ISCLog file. Note: A client must notify TIBCO any time there is a change in the format of this file. XML File Format - Sample See the sample of a transaction file in the TIBCO Engage XML format at http://www.instorecard.com/schema/v1/samples/isclogsample.xml. Fixed Width File Format - Sample 123456789112345678921234567893123456789412345678951234567896123456 78971 23456789812345678991234567890 260300348700520060412701161201287653095 0000003487000JOHNDOE CA946091305200810152005091500A 370400323700520060412701211473530958765 1003003487000VERONALMARTINEZ
Import Files 50 CA926092505200810152006111500A 2705203487005200604120410711360395308795 0081003487795PAULAMDECRISTOFARO CA946096405200810152005091600A 430300348700520060412701161404387653095 0520063487345ROBERTGRAVEST MA021381305200810152005092800A 260300348700520060412802351201254284023 6530903452006LILLIANBOSWAINE MA021381305200810152005092800A Table 8 Fixed Width File Formats Field # Field Description Fixed Width File Format Table - Sample Table 8 provides information about the fields to be included in a Transaction data file in their native file format. A table such as this enables TIBCO engineers to set up an automated process for transforming the file every time it is sent. Type Length Decimal Location Notes 1 Transaction ID A 9 1 2 Store number S 4 0 10 3 Date S 8 0 14 CCYYMMDD 4 Time S 6 0 22 HHMMSS 5 Line item S 6 0 28 1 6 Line item amount S 9 2 34 Money (for example, 2000) 7 SKU A 10 43 for example, 8765309 8 Line item quantity S 6 0 53
Import Files 51 Table 8 Fixed Width File Formats (Cont d) Field # Field Description 9 Account number Type Length Decimal Location Notes S 13 0 59 10 Name, first A 16 72 11 Name, middle initial A 1 88 12 Name, last A 20 89 13 State/Province A 2 109 14 Zip/Postal Code A 9 111 15 Expiration date S 8 0 120 CCYYMMDD 16 Contract date S 8 0 128 CCYYMMDD 17 Reward amount allowed S 9 2 136 Money - 10% of dues amount paid 18 Shopper status A 1 145 A =Active, I =Inactive CSV (Comma Separated Value) File Sample 0009200618053,00012,2012-09-06 00:14:21,JANE@DOE.COM,002, 001469,000001, 000004.40,1111222233334444 0009200618053,00006,2012-09-06 01:36:43,JOHN@DOE.COM,001,462031,000001, 000012.65,2111222233334444 0009200618053,00070,2012-09-06 01:44:29,ZEBEDIAH@DOE.COM,001,896953,000001, 000037.40,3111222233334444 0009200618053,00024,2012-09-06 03:14:51,ATHENA@DOE.COM,001,876464,000001, 000034.64,4111222233334444 CSV (Comma Separated Value) Format Table - Sample
Sample of Product File from Clients 52 Field# Field Description This table provides information about the fields to be included in a Transaction data file in their native file format. A table such as this enables TIBCO engineers to set up an automated process for transforming the file every time it is sent. Type Length Sample Data Notes 1 TransactionID String 13 0009200618053 2 RetailLocID String 5 00072 3 ReceiptDateTime DateTi me 19 2013-09-06 00:14:21 4 CustomerEMail String Varies JANE@DOE.CO M Optional 5 LineNumber Nume ric 6 ItemSKU Nume ric 7 Quantity Nume ric 8 ExtendedAmount Nume ric 9 CardNumber Nume ric 3 002 6 001469 6 000001 9 000004.40 Includes decimal 16 111122223333444 4 Sample of Product File from Clients This format table has column headings set up for a CSV file format. A column of sample data has been added to provide clarification. Field Name Field Description Data Type Size Sample Data Notes product_c ontainer Product Information
Sample of Product File from Clients 53 Field Name caption* departme nt* Field Description Name or heading for a product. Client defined department (category) hierarchy. Data Type Size Sample Data Notes String 255 Great Product! This is the value that appears throughout TIBCO Engage, wherever a user sees product information. TIBCO Engage supports five department levels. These categories are used for targeting, reporting, and offer qualification. departme nt_l evel_1* Top-level product category String 100 Books DVDs CDs Software Toys The range of values for this field across all products are the categories available in the Shopper Application for the What I Like functionality. These same values appear in the Preferred Category Breakdown report. departme nt_l evel_2 Second-level product category String 100 Fiction, Books, DVDs, CDs, Periodicals Fantasy Educational If Department_Level_2 is specified, Department_Level_1 is required. departme nt_l evel_3 Third-level product category String 100 Mysteries, Fantasy Financial Cooking Jazz automotive If Department_Level_3 is specified, Department_Level_2 is required. descriptio n* Human Readable Description image_url URL to an image of the product. String 100 http://example.c om/image.gif This image is available when creating new offers and when adding rewards to the Reward Catalog.
Sample of Product File from Clients 54 Field Name sku* Field Description Required, must be unique. SKU, stock keeping unit, or product number. Data Type Size Sample Data Notes String 100 S100000123 Catalog management (additions, removal) is done on SKU, which must be unique. This appears in most of the application where product name (caption) does. This SKU does not correspond to the SKU used in the Rewards Catalog. list_price* List price of product. i.e. The manufacturer's suggested retail price. Deci mal 15.23, 34.78 sale_price Sale price of product. This is the price the customer should pay for the product. Deci mal 15.23, 34.78 brand brand String 100 Maytag Brand used for targeting * Required
Sample of Product File from Clients 55 The example of a Product Catalog file below is in tab separated value format (tabs spacing adjusted for easier reading below). Caption DeptLevel1 DeptLevel2 DeptLevel3 Description ImageURL SKU List_Price Sale_Price Brand Yan Can Cook! DVD Educational Cooking Great Chefs Teach http://ourcompany.com/image100.gif S100000123 34.99 30.99 EduData Wynton Marsalis Live CD Music Jazz Jazz legend at Monterey http://ourcompany.com/image200.gif S100003223 14.99 10.99 Vibe TortureTax! Software Educational Financial TaxProcessingSoftw are http://otherweb.com/image31231230.gif S100456123 49.99 44.99 Dodobucks Fix Your Ride Software Educational automotive CarRepair http://ottoweb.com/image42231420.gif S100654123 49.99 44.99 GreaseBooks
Format and Sample of Store File 56 Format and Sample of Store File Field Name Sample Format Table The format table below is set up with column headings for a Store file in CSV file format. A column of sample data has been added to provide clarification. Field Description Data Type 1 StoreID String 5 00010 Length Sample Data Notes 2 StoreName String Varies Zirconia Hill 3 StoreAddress1 String Varies 234 Zirconia Mall 4 StoreCity String Varies Woonsocket 5 StoreState String 2 RI 6 StorePostalCode String Varies 02895 7 StorePhoneNumber String Varies 401-555-3439 Sample Data file StoreID Storename StoreAddress StoreCity StoreState StorePostal Code StorePhoneNumber 00010 Pa Store #1 Westcenter Mall #344 King Of Prussia Pa 19406 610-555-3439 00011 Md Store #1 Southdale Mall #45 Columbia Md 21044 410-555-2344 00013 Ct Store #1 Town Center #6 Stamford Ct 06901 203-555-9089
File Import Reports 57 Chapter 7 File Import Reports This section details file import reports. Topics About the File Import Report, page 57 Creating a File Report, page 58 The TIBCO Engage File Import Report provides TIBCO staff and clients with information about the processing status of individual files. The File Import report tells you which files were successfully imported by TIBCO Engage, which report imports failed, and where applicable which reports were partially successful. (Note that some files cannot be "partially successful": either the entire file is imported or the entire file fails.) The File Report also provides details and failure reasons for the records in a file that failed processing. About the File Import Report To set up a File Import report, use the controls at the top of the report. Among other things, these controls allow you to specify what period of time the report should cover. For each file that was imported within the time frame that you specify, the File Import Summary report lists the following fields: File Import ID This is a file ID assigned by the system in accession order. Each new file is assigned the next number, so higher numbers indicate a more recent file. Files are listed from highest ID to lowest (within the specified time frame for the report) File Name Name composed of standard filename elements, including date. Start Date The start date of the file generation. End Date The end date of the file generation.
Creating a File Report 58 File Type The type of file that was imported, for example: TransactionImport, StoreImport, ProductImport, etc. Status Successful or Failed. Details link Click this link in the File Import Summary report to see the File Import Details report for the associated file. The File Import Details report has two variations: Failure: If the file import failed, the File Import Details report lists information in fields titled "Failure Reason and Suggested Action." Success: If the file import succeeded, then the File Import Details report lists file data and statistics, including the record type, the number of records in the file, and the number of records that were successfully transmitted. Note: In the case of certain files, a file import may partially succeed (some records are transmitted but others are not) and the file import is considered a success. In this case this report will specify the number of records successfully transmitted, and the number of records that failed to be transmitted. Creating a File Report The File Import report provides status updates and other useful information about the results of batch imports or list file uploads. It also provides access to details about each import file. Use TIBCO Engage to set up and generate this report with these steps: 1. Navigate to Reports>Specialized>File Import Report. 2. Click File Import Report. The File Import Report opens. 3. Use the controls at the top of the report to set up this report. If the time period specified by default is not the time period that you want the report to cover, select a new date range or period immediately. Do not wait for the default report to compile if it is not what you want to see. To select a time frame for the report, choose from the drop-down list of standard options, or customize the time frame by specifying a range of dates. If you change the time frame, click View Report to be sure that the report is regenerated.
Creating a File Report 59 4. Navigate through the report by using the arrow icons. You can also find data by searching for strings that occur in the report: use the data entry box in front of the Find control. To create another copy of the report with a different format, select a formatting option from the Select a Format drop-down box, then click Export. Click the Printer icon to print the current report.
60 Chapter 8 Email Update Import File This section provides information about the email update import file. Topics Email Update Import File Use Cases, page 61 Standard XML Sample File and XSD Schema, page 61 File Usage Guidelines, Data Elements and Formats, page 62 Samples and Formats of Email Update Files from Clients, page 62 Email Update File and Record Failures, page 63 For special circumstances, TIBCO provides an Email Update Import file. This is not a standard file import option. The Email Update Import file allows clients to update email addresses in the TIBCO Engage database. This batch process is used when TIBCO Engage does not maintain the system of record, that is, when the master version of each customer s email address is maintained in another, non-tibco application. In this case, the updated record is imported to update the TIBCO Engage database. This import is included because the email address is the TIBCO Engage primary means of identifying customers, and there is no way to update email addresses using the Transaction and Customer Import File. Contact TIBCO for information about using the Email Update Import file. General Description The email update file is simply a file that includes the old email address and the updated email address. If TIBCO Engage can successfully identify a record based on the email address, it will update the email to the new address provided in the file.
Email Update Import File Use Cases 61 Initial Integration Notes This integration is optional. The Email Update Import file is used only when TIBCO Engage is not the system of record (that is, the definitive, master version of the customer s records are maintained by the client) and when the email address on file with TIBCO Engage must be identical to the address maintained by the client. Email Update Import File Use Cases The Email Update Import file is typically used in the following cases: The client maintains its customer records and wants to synchronize that data with the TIBCO Engage corresponding data on a periodic basis. Customers update their information on the client s website or system, including their email addresses. These scenarios require data integration to enable the creation and transfer of the Email Update Import File. Standard XML Sample File and XSD Schema Note: At present, because of its simplicity, the Email Update file is processed as a flat file. There is no XML schema for this file. Refer to sections File Usage Guidelines, Data Elements and Formats and Samples and Formats of Email Update Files from Clients for details.
File Usage Guidelines, Data Elements and Formats 62 File Usage Guidelines, Data Elements and Formats If the client maintains the master list of customer email addresses, then it can send TIBCO a list of email addresses to be changed in the TIBCO Engage database. In this file, each record consists of a pair of email addresses: the first email is the old previous email address; the second address is the new replacement email address. Each old email address is used to identify a customer record in the TIBCO Engage system. If a record with a matching old email address is found, the record is updated by replacing the old address with the new address. Table 9 Email Update Data Table TIBCO Engage Field Name Description DataType Size Sample Data OldEmailAddress NewEmailAddress Previous Email Address: Use this field to populate the source email address Current Email Address: Use this field to populate the updated email address String 100 mstone@example.com String 100 michaelstone@example.com Samples and Formats of Email Update Files from Clients Sample Email update file in CSV format: old_email_address@wahoo.com, new_email_address@sool.com Test1@wahoo.com, Test2@wahoo.com
Email Update File and Record Failures 63 Email Update File and Record Failures The following table includes the error codes that apply to Email Update file imports. Table 10 Email Update File Errors and Error Notes Error Code Error Type Error Reason Suggested Action 1 Invalid E-mail address. 2 Record failed to update 3 Record failed to update 4 Record failed to update 5 Record failed to update 6 Record failed to update Email address format is invalid. Old Email address duplicates in dropped file New Email address duplicates in dropped file New Email address exists as old email address for another record New Email Address already exists in the system Old Email Address does not exist in the system Fix the email address and resend the file. An Email address cannot include spaces and has to include the following elements:<alphanumeric> @<alphanum eric>.<alpha> For additional, more specific details, see the "Email Address Validation Criteria " section. No action necessary. Records with errors are skipped. No action necessary. Records with errors are skipped. No action necessary. Records with errors are skipped. No action necessary. Records with errors are skipped. No action necessary. Records with errors are skipped.
Email Update File and Record Failures 64 Table 10 Email Update File Errors and Error Notes (Cont d) Error Code Error Type Error Reason Suggested Action 11 File failed This file is a duplicate of a previously sent file. 12 File failed The file failed to import because it does not include any records. 13 File failed The file failed to import because of a general error. 14 File failed The file failed to import of a data format error. 15 File failed The file failed to import because the file type is incorrect. 17 File failed The file failed to import because it failed a validation rule. Check the file sending process to ensure it is working properly and sending the correct file. If this file is expected to be a duplicate, ignore this notice. Check the file sending process to ensure it is working properly and sending the correct file. If this file is expected to be empty, ignore this notice. Contact TIBCO for diagnosis. You can correct the file by fixing data format errors. Common issues include having an alphanumeric value in a date field. This file was not recognized by our process. Common reasons for this issue include misnaming the file or accidentally sending a different file. You can correct this issue by sending the correct file. Contact TIBCO for diagnosis. Some of the above error types are described below with more specific information and examples.
Email Update File and Record Failures 65 Email Update File Error Descriptions and Related Notes New Email address duplicates in dropped file A file failure will occur if the client s file includes multiple entries with the same new email address, for example: oldemailaddress@xxx.com,newemailaddress@xxx.com Ye_oldemailaddress@xxx.com,newemailaddress@xxx.com abc@abc.com,def@def.com efg@efg.com,def@def.com The records are not processed and the file is rejected with an error: New Email address duplicates in dropped file Old Email address duplicates in dropped file A file failure will occur if the client s file includes multiple entries with the same old email address, for example: oldemailaddress@xxx.com,newemailaddress@xxx.com oldemailaddress@xxx.com,even-neweremailaddress@xxx.com abc@abc.com,def@def.com abc@abc.com,xyz@xyz.com The records are not processed and the file is rejected with an error: Old Email address duplicates in dropped file New Email address exists as old email address for another record A file failure will occur if the client s file includes an updated email address that is used as an old email address in another record, for example: oldemailaddress@xxx.com,newemailaddress@xxx.com newemailaddress@xxx.com,even-neweremailaddress@xxx.com abc@abc.com,def@def.com def@def.com,xyz@xyz.com The records are not processed and the file is rejected with an error: New Email Address exists as old email address for another record.
SSO Integration 66 Chapter 9 SSO Integration This chapter provides an information about Single Sign On (SSO) integration. Topics SSO Overview, page 66 How SSO Works, page 67 Implementation Details, page 70 Return Codes for SSO Authentication, page 72 SSO Overview The TIBCO Engage Single Sign On (SSO) feature provides shoppers with access to loyalty- building presentations and offers, such as your ecommerce sites and our integrated loyalty sites, without requiring them to log in twice. To implement SSO, your development staff must provide some coding to the TIBCO integration team to perform integration testing. The resulting SSO technology provides you with a secure means of giving your shoppers convenient and secure access to experience management program information. To use SSO, you must implement the following functions: When a customer logs in, use a TIBCO Engage real time API call to obtain an Authentication Ticket for that customer. When a customer opts into the experience management program, use a TIBCO Engage real time API to create a new customer record, if one does not already exist. When a customer clicks a link to the experience management program pages hosted by TIBCO Engage, retrieve the Authentication Ticket from the TIBCO Engage API Call and pass it with the URL as a query string. Provide a logout and error handling page.
How SSO Works 67 When shoppers need information other than their password, use the TIBCO Engage Real Time API to update the customer information. The TIBCO Engage Real Time API can also be used to mark a customer as Inactive. Secure Data Transfer TIBCO Engage is Visa CISP (Cardholder Information Security Program) compliant. This means that all data with sensitive information transferred between TIBCO Engage and the retailer must be transmitted using an encrypted protocol. Single Sign On uses 128-bit SSL for all connections and encrypts and encodes authentication tokens to ensure they are reasonably impervious to attack. This includes both the API call and the link to the customer application. How SSO Works TIBCO Engage provides a secure mechanism to pass customer credentials to the TIBCO Engage system and authenticate the customer on the TIBCO Engage customer-facing web site without requiring the customer to perform a second login. Single Sign On must handle the following five basic scenarios: 1. Customer opts into the experience management program When your customer is on your web site and agrees to join (opt in), you must create a customer record in the TIBCO Engage database using the Real Time API that we provide. The interface provides for exception handling in the case where the customer already has a record in the database. 2. Customer requests access to his or her experience management program information When the customer clicks on a link on your site to review personal experience management program information, your web site must retrieve the customer s email address and pass it to the TIBCO Engage API. The TIBCO Engage API will respond with an Authentication ticket for that customer, which is good for 30 minutes. Encode this ticket as a query string parameter named auth_t and add it to the URL of the TIBCO Engage - hosted experience management program pages. TIBCO Engage parses and decrypts the query string information, and then validates the authentication of the customer before redirecting the customer to the experience management program information.
How SSO Works 68 3. Customer s credentials are not accepted When a customer s credentials are not accepted by TIBCO Engage for any reason, TIBCO Engage will return an HTTP redirect response to a web page that you host. This response contains the error type so that you can provide appropriate messaging to your customer about the problem, or take other programmatic action to resolve the problem (like retrieve a new public key to replace one that has been rotated). For more information about error codes, refer to Return Codes for SSO Authentication, page 72. 4. Customer logs out of the experience management program pages When the customer clicks the logout link on our site, we will return an HTTP redirect response to the same URL that you host with a return code of 2 on the query string that implies that the customer logged out of our system. For more information about error codes, refer to Return Codes for SSO Authentication, page 72. 5. Customer makes changes to their information When the customer makes changes to the same information that we store about your customer (email address, mailing address, etc.), you must use the Real Time API we provide to update the information in the TIBCO Engage database. It is particularly important to keep the customer email up to date so that the customer can always access their loyalty information.
How SSO Works 69 Figure 1 SSO Process Flow Diagram
Implementation Details 70 Implementation Details This section provides information on implementing SSO via API. Engage API Call This is a SOAP call Basic authentication using your API User is required. (Note: Refer to XXX for how the API User is created using the TIBCO Engage application.) This call must be done over SSL 1. GetShopperAuthenticationToken This API returns Authentication Ticket for a given customer. Email address of customer is required. Sample Request <?xml version="1.0" encoding="utf-8"?> <soap:envelope xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xmlns:xsd="http://www.w3.org/2001/xmlschema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:header> <AuthenticationResult xmlns="http://www.loyaltylab.com/loyaltyapi/"> <RetailerGuid>string</RetailerGuid> <Authenticated>boolean</Authenticated> <Token>string</Token> <ICSUserID>int</ICSUserID> </AuthenticationResult> </soap:header> <soap:body> <GetShopperAuthenticationToken xmlns="http://www.loyaltylab.com/loyaltyapi/"> <shopperemail>testshopper@test.com</shopperemail> </GetShopperAuthenticationToken> </soap:body> </soap:envelope> Sample Response <?xml version="1.0" encoding="utf-8"?> <soap:envelope xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xmlns:xsd="http://www.w3.org/2001/xmlschema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:body>
Implementation Details 71 <GetShopperAuthenticationTokenResponse xmlns="http://www.loyaltylab.com/loyaltyapi/"> <GetShopperAuthenticationTokenResult>31-2F-43-44-32-55-6E-37-78-39-64-47-30-69-73-79-51-7F-53-65-2F-6E-38-66-61-4A-71-35-55-4C-52-36</GetShopperAuthenticatio ntokenresult> </GetShopperAuthenticationTokenResponse> </soap:body> </soap:envelope> 2. GetShopperAuthenticationTokenByRetailerID This API returns Authentication Ticket for a given Customer. RetailerShopperID of the customer is required. Sample Request <?xml version="1.0" encoding="utf-8"?> <soap:envelope xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xmlns:xsd="http://www.w3.org/2001/xmlschema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:header> <AuthenticationResult xmlns="http://www.loyaltylab.com/loyaltyapi/"> <RetailerGuid>string</RetailerGuid> <Authenticated>boolean</Authenticated> <Token>string</Token> <ICSUserID>int</ICSUserID> </AuthenticationResult> </soap:header> <soap:body> <GetShopperAuthenticationTokenByRetailerID xmlns="http://www.loyaltylab.com/loyaltyapi/"> <retailershopperid> testshopper@test.com </retailershopperid> </GetShopperAuthenticationTokenByRetailerID> </soap:body> </soap:envelope> Sample Response <?xml version="1.0" encoding="utf-8"?> <soap:envelope xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xmlns:xsd="http://www.w3.org/2001/xmlschema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:body> <GetShopperAuthenticationTokenByRetailerIDResponse xmlns="http://www.loyaltylab.com/loyaltyapi/">
Return Codes for SSO Authentication 72 <GetShopperAuthenticationTokenByRetailerIDResult>31-2F-43-44-32-55-6E-37-78-39-64-47-30-69-73-79-51-7F-53-65-2F-6E-38-66-61-4A-71-35-55-4C-52-36</GetShopper AuthenticationTokenByRetailerIDResult> </GetShopperAuthenticationTokenByRetailerIDResponse> </soap:body> </soap:envelope> Query String Authentication Parameters The value of the ShopperLoginToken should then be passed in as the auth_t parameter to the requested Loyalty Pages. Example /../..?auth_t=-64-35-37-79-69-69-30-71-72-2f-57-66-4e-70-35-59-30-73-39-4d-44-73-2f-70-70-67-6b-34-30-56-4f-35-46-72-56-61-58-50-53-55-48-73-3d Configuration of Client SSO Handler As mentioned above, you can configure one URL for handling SSO logout and error scenarios. To configure this URL you will have to work with TIBCO and provide the URL so it can be set in our system. This URL can also be changed at any time from its initial setup. Again, the change can only be handled through TIBCO. Return Codes for SSO Authentication The SSO functions can fail to authenticate a customer s credentials for several reasons. When that happens, the software directs the user s browser to a URL that you code and host with an rc= query string parameter containing one of the following values: (-1) Decryption of Customer Credentials failed (the most probable cause is it is an invalid ticket) (-4) Client GUID is missing from the query string and the cookies sent to the server (-5) Email address is invalid (-6) Customer record was not found (-7) Customer account is disabled
Return Codes for SSO Authentication 73 (-8) Customer account is deleted (-9) Authentication failed Ticket expired (1) Redirect to the signup page. The signup page was requested with a missing authentication cookie this implies that the customer needs to be directed to the initial signup page on your site to complete the first couple of steps of signup and then be transferred to TIBCO Engage customer pages using SSO. (2) Redirect to logout page. Customer logged out of TIBCO Engage pages. (3) Redirect to Customer Account page. Customer chose to edit account preferences, which must be done on your site. When the rc parameter is contains a negative value, the URL redirect will also contain a ru= query string parameter that will contain the URL that the customer was trying to reach when the SSO process failed, so that you have an opportunity to correct the problem and retry the request.
Index 74 Index A ASCII files 13 B Batch File Summary Descriptions 9 batch process 17 Batch-file Exchange 15 Bulk Data Processing 18, 18 C Connecting with TIBCO Resources ix Creating a File Report 58, 58 Custom Question File Details and Requirements 45 Custom Questions Import 44, 44 Customer 10, 22 customer support ix D Data Content Types 15, 15 Data Exchange Methods 15, 15 Data Exchange Methods used for Data Content Types 14 Data Exchange Overview 9 Data Files 13 Data Integration Project Requirements and Project Plan Summary 4 Data Integration Task Descriptions 5 E Email Statistics Export 41, 41 Email Statistics File Data Descriptions 42 Email Update Data Table 62 Email Update File and Record Failures 63, 63 Email Update File Errors and Error Notes 63 Email Update Import File Use Cases 61, 61 Event ID 25 F File Failures 47 File Format 37, 39, 42, 44 File Processing Constraints 19 File Usage Guidelines, Data Elements and Formats 62, 62 Format and Sample of Store File 56 G General Description 60, 60 Generic Event 12, 24 generic event 21, 24, 26 H How to Access TIBCO Documentation ix How to Contact TIBCO Support ix How to Join TIBCOmmunity ix I Implementation 1 Implementation and Data Integration 1 Import Files 48, 48 Integration Hub 2
Index 75 L List 12, 23 List Export 38, 38 List Export File Formats 39 List Import 36, 36 Location 11, 23 N Null Data Processing 18 T technical support ix The Data Integration Process 3 The Data Integration Project Plan and Project Tasks 4 The TIBCO Loyalty Lab Technical Integration Process 3, 3 TIBCO_HOME vii Transaction 10, 22 Transaction and Customer Sample Files 48 transaction file 26 Typographical Conventions vii O On-Demand File Exchange 34 P Prerequisites to Data Integration 3 Product Catalog 11, 23 Profile 12, 24 Profile Export 41, 41 Project Task Descriptions 5, 5 R Real-time SOAP-based APIs 16 Related Documentation vi Reward Catalog Import 43, 43 Rewards 11, 23 S Sample of Product File from Clients 52 Samples and Formats of Email Update Files from Clients 62, 62 Secure Data Transfer 2 Shopper Matching 18 Standard XML Sample File and XSD Schema 61, 61 support, contacting ix