Advanced XML Seminar. Objective. Personal Introductions Jeff Rosler. Agenda. XML Application Architecture Core Documentum/XML Features:

Transcription

1 1 Objective Advanced XML Seminar Jeff Rosler Flatirons Solutions Describe tools and methods to handle common problems found in consulting projects implementing XML Solutions with Documentum DevCon 2004 San Ramon, CA Agenda Personal Introductions Presentation Overview Problems/Solutions and Tools and Tricks Questions Personal Introductions Jeff Rosler Flatirons Solutions Full lifecycle consulting and implementation experience Strategic and business process consulting Systems Architecture and Information Architecture Systems design, integration and implementation Support and evolution Content-based solutions expertise Collaboration, document management, WCM, and imaging Integrations with ERP, CRM and financial systems Experts in information analysis and process re-engineering Leaders in XML-based solutions 20 years of software engineering experience in consulting and product development Over 3 years Documentum experience specializing in XML Solutions 5 years Content Management experience Jeff.Rosler@FlatironsSolutions.com 3 4 Presentation Overview Migrating XML content to Documentum Integrating DFC code in XSL Transforms Core Documentum/XML Features: XML Applications Rules for processing XML content in and out of Documentum XML Application config files XML Editor Integration Allow reuse of content Searching XDQL Verity zone searches Transforming XML Content XDQL/DFC integration 5 6

2 7 Simple block diagram: Dctm Desktop WDK/ Webtop Custom Code DFC DFC DFC DMCL DMCL DMCL Docbroker Docbase Server Database/ Filesystem XML Application Processing : Import, check in, check out, export Implemented in DFC com.documentum.operations.* (IDfImportOperation, IDfCheckinOperation, IDfCheckoutOperation, IDfExportOperation, etc.) DFC may be located on user s desktop (in case of Documentum Desktop) Operations may be heavyweight operations if large amounts of chunking are involved 8 Operation Logic: DFC Disclaimer The following descriptions are very general bullet points on DFC operations and are mentioned to allow users to better understand operations for debugging purposes. Import/Check in (DFC logic) Retrieve XML Application config file Parse XML Application config file May download Application Support files Parse XML content file and perform dctm updates May include SAX parse (using Xerces) Process based on config file rules Create content chunks, fix-up virtual document, set properties, etc. Operation Logic: Export/Check out (DFC logic) Retrieve XML Application config file Parse XML Application config file Download content file Download support files Fixup of content files May include adding DOCTYPE declaration Including SAX parse of content files (using Xerces) 9 10 Sample Operations Code: DFC java docs/samples Default location on windows install - C:\Program Files\Documentum\Help\DFC\api\samples Documentum Developers site ( tml ) Look under DFC and XML sections Flatirons Solutions site ( ) Important Points from Architecture to remember XML Config file is parsed XML Content file is parsed DTD and files included in DTD are parsed Most run time errors occur because of problems in the above content Use custom code or Flatirons Docbase Shell Tool to debug operations Be careful using Desktop Client to test, it can hide the real error messages (i.e. Unknown error) 11 12

3 13 Flatirons Solutions Docbase Shell Tool The following commands are accepted by the shell: cd <full or relative path> - Change current Docbase directory ci <relative path> - Check-in specified object co <relative path> - Check-out specified object cc <relative path> - Cancel check-out export <full or relative path> - Export specified object get <full or relative path> - Use getfile for file content h/help - Print shell commands import <full file system path> - Import specified document into Docbase ls - List all files in the current directory q/quit/exit - Exit the application shell trace dfc [level] <full file system path> - Toggle DFC tracing trace dmcl [level] <full file system path> - Toggle DMCL tracking XML Application Config File: Common Problems Incorrect element/attribute names No defaults specified (or empty </default> tag) Incorrect context for variables Incorrect context for content rule Can t find java class file (for java customizations) Make sure that jar or class is in CLASSPATH If using desktop, remember to restart desktop after CLASSPATH changes so that JVM is restarted 14 XML Content File: Common Problems Bad date format Remember to use <date_format> and make sure mask is correct (based on java SimpleDateFormat class) Missing content with no default Cannot find content for a <variable> (incorrect <element_selection_pattern> Missing content for context rule Have <context_rule><child_of> in <element_selection_pattern> on content rule and there is no parent in content Content not well formed DTD/Support Files: Common Problems Invalid Content (i.e. carriage return after <?xml?> decl) Missing files (i.e. entity files) Incorrect references (i.e. includes) DTD/Support Files: How to determine which is which Check content well formedness XML Editor (i.e. Epic, XMetal, XMLSpy, etc.) Use Xerces to parse content Run java sax.counter sample to parse and validate If errors are found, check content and then DTDs If no errors are found, re-check config file against content DTD/Support Files: Hardcore debugging If all else fails, use the DFC trace log DFC logs are huge, try and turn trace on just for the operation to debug (Flatirons docbase shell tool is good for this) Search from the bottom up for Exception see if there are any other exceptions before the first exception that might give more information (sometimes early errors mask final error Unknown error ) Based on methods called and output of methods, try and determine where problem occurred (i.e. loading config file, setting a variable, etc.) 17 18

4 19 Migrating XML content to Documentum Situation Client engagement to move from unstructured authoring to structured authoring using XML and Documentum DTDs and screen stylesheets were created early in the engagement before Documentum was available and certified Client wished to have authors start authoring content immediately Problem to solve Determine best way for authors to author content on filesystem Create methods to migrate the authored content into Documentum retaining the reuse of content Basics of adding custom java code to XML Applications Three custom applications from customer engagements Validate content during import/check in Setting object names on link rules Integrating XML into external database tables 20 The basics: XML Application Declare the java code in XML Application Call java code in <expression> Java Code Use static main() to create test scenarios Use logging to debug problems Installation Make sure that code is in CLASSPATH wherever DFC is installed After deploying code and updating CLASSPATH if using Desktop Client, will need to restart JVM (log off windows and back on or kill explorer) Validate Content during import/check in: Overview Requirements Client wants additional metadata validation before XML content is imported or checked in Solution Create custom java code in XML Application to run business rules for content validation If content not valid, stop the operation and report an error Validate Content during import/check in : XML Application Declare the Java code via <java_classes>: <java_classes> <java_class prefix= Validation" classname = com.flatironssolutions.documentum.validation"/> </java_classes> Create variables to gather data for validation <variables> <variable> <name>title</name> <content_of_element> <element_selection_pattern> <element>title</element> </element_selection_pattern> </content_of_element> </variable> Validate Content during import/check in : XML Application Use <expression> to set value and validate <metadata> <dctmattr> <name>title</name> <template><expression>validation:validatetitle('<var name="title"/>','<var name="language"/>')</expression></template> </dctmattr> </metadata> 23 24

5 25 Validate Content during import/check in : Java Function package com.flatironssolutions.documentum; import org.apache.log4j.logger; public class validation static Logger logger = Logger.getLogger(validation.class); public static String validatetitle(string title, String language) throws Exception String returnvalue = null; logger.debug(" validation:validatetitle (" + title + "," + language + ")"); // perform logic and set returnvalue // if validate logic fails run // throw new Exception( Validate failed for title + title + and language + language +. Please fix problem in file and re-run check in or import ); return returnvalue; } } Setting object names on link rules : Overview Requirements Client wants linked graphics to have name based on file name from fileref and language <graphic href= /graphics/foo.gif language= EN /> should be imported into the docbase as an object named foo EN.gif Solution Create custom java code in XML Application to parse fileref and construct object name 26 Setting object names on link rules : XML Application Declare the Java code via <java_classes>: <java_classes> <java_class prefix="util" classname="xmlapputil"/> </java_classes> Create variables to gather data for validation <variables> <variable> <name>language</name> <attribute_value_of_element> <this_element/> <attr_name>language</attr_name> </attribute_value_of_element> <default>en</default> </variable> <variable> <name>href</name> <attribute_value_of_element> <this_element/> <attr_name>href</attr_name> </attribute_value_of_element> <default>unknown href.jpg</default> </variable> Setting object names on link rules : XML Application Set the object name <object_name><expression>util:getobjectnamefromhref ('<var name="df_session"/>','<var name="href"/>','<var name="language"/>')</expression></object_name> Pass in href and language (and DF_SESSION?) to getobjectnamefromhref method Setting object names on link rules : Java Function public static String getobjectnamefromhref(string stringsession, String filepath, String language) throws Exception logger.debug("xmlapputil:getobjectnamefromhref(" + stringsession + "," + filepath + "," + language + ")"); try String normalizedfilepath = normalize(stringsession,filepath); File file = new File(normalizedFilePath); String objectname = ""; String extension = ""; String filename = file.getname(); int nfilenameend = filename.lastindexof('.'); if (nfilenameend == -1) nfilenameend = filename.length(); } else extension = filename.substring(nfilenameend); } String filenamewithoutextension = filename.substring(0,nfilenameend); Setting object names on link rules : Java Function What is normlize() and why do we need it? The fileref values fit into various types Fileref for new graphic (import scenario) <graphic href= /graphics/foo.gif language= EN /> Fileref for pasted graphic (Copy and paste reference) <graphic href= c:\documentum\export\foo.gif language= EN /> Fileref for checked out or exported content (check out without descendants) <graphic href= dctm://dfcdev_solora/0500ae198000f8f9?dms_object_spec= RELATION_ID language= EN /> objectname = filenamewithoutextension + " " + language + extension; return objectname; } catch (Throwable te) 29 30

6 31 Setting object names on link rules : Java Function private static String normalize(string stringsession, String filepath) throws Exception logger.debug("xmlapputil:normalize(" + stringsession + "," + filepath + ")"); String newfilepath = filepath; try IDfClientX clientx = new DfClientX(); IDfClient client = DfClient.getLocalClient(); IDfSession session = client.findsession(stringsession); // check to see if filepath is a DRL if (filepath!= null && filepath.length() >= 5 && filepath.substring(0,5).comparetoignorecase("dctm:")== 0) IDfXMLUtils xmlutils = clientx.getxmlutils(); xmlutils.setsession(session); IDfId id = xmlutils.getobjectidfromdrl(newfilepath); IDfSysObject sysobj = (IDfSysObject)session.getObject(id); // use the object name as a basis newfilepath = fixupobjectnamesuffix(sysobj.getobjectname()); } // else // if it isn't a DRL, let's see if it's listed in the Registry Setting object names on link rules : Java Function else // if it isn't a DRL, let's see if it's listed in the Registry IDfClientRegistry clientregistry = clientx.getclientregistry(); IDfViewedObject viewedobject = clientregistry.getviewedobjectbypath(filepath); IDfViewedObject viewedobjectexportfullpath = clientregistry.getviewedobjectbypath(clientregistry.getexportdirectory() + File.separator + filepath); IDfViewedObject viewedobjectcheckoutfullpath = clientregistry.getviewedobjectbypath(clientregistry.getcheckoutdirectory() + File.separator + filepath); IDfCheckedOutObject checkedoutobject = clientregistry.getcheckedoutobjectbypath(filepath); IDfCheckedOutObject checkedoutobjectexportfullpath = clientregistry.getcheckedoutobjectbypath(clientregistry.getexportdirectory() + File.separator + filepath); IDfCheckedOutObject checkedoutobjectcheckoutfullpath = clientregistry.getcheckedoutobjectbypath(clientregistry.getcheckoutdirectory() + File.separator + filepath); if (viewedobject!= null) // was it exported - is it already a full path? IDfId id = new DfId (viewedobject.getobjectid()); IDfSysObject sysobj = (IDfSysObject)session.getObject(id); // use the object name as a basis newfilepath = fixupobjectnamesuffix(sysobj.getobjectname()); } else... other conditions 32 Setting object names on link rules : Summary Test all of your use cases in your XML Applications to find issues for all chunked content Import (with and without reuse) Checkout (with and without descendants) Checkin (with and without reuse) Export Etc. All clients and custom code should be checked Whitepaper with sample code and XML Application can be downloaded from Flatirons Solutions Developer Zone - ava_ref_article.zip Integrating XML with Database Tables: Overview Requirements Client needs to export XML metadata to Oracle during an import or check in operation Multiple tables need to be updated based on content Automated mechanism desired to insert XML values into database rows Solution Through java extensions of XML applications and linking and registering the target tables, extracted values can be passed and written Integrating XML with Database Tables: The XML Application Declare the Java code via <java_classes>: <java_classes> <java_class prefix="staticdatatableutilities" classname ="dss.integ.dataobjects.staticdatatableutilities"/> </java_classes> For the XML mapped to metadata, do not make an object <xml_content_rule export="inline" editable_virtual_doc="false" make_object="false"> <element_selection_pattern> <element>admin</element> </element_selection_pattern> <variables> Integrating XML with Database Tables: The XML Application After the variables are mapped, pass them to your Java code: <metadata> <dctmattr> <name>title</name> <template> <expression> StaticDataTableUtilities:insertAdminElement ("<var name = "DF_SESSION"/>", "<var name= "admin_org_name"/>", "<var name="admin_prog_name"/>", "<var name = "admin_prog_type"/>", "<var name= "DSSSchemaName"/>", "<var name="docbaseschemaname"/>", "<var name="createuserid"/>") </expression> </template> Note the session passed from the XML application 35 36

7 37 Integrating XML with Database Tables: Java function All arguments are strings public static void insertadminelement(string ssessionid, The query is built from the arguments as an IDfQuery string and executed IDfQuery oinsertquery; oinsertquery = new DfQuery(); oinsertquery.setdql( INSERT INTO IDfCollection col = oinsertquery.execute(osession, 4) col.close(); Integrating XML with Database Tables: The Import Code (optional) To pass the schemas as variable, the import needs to be called from code oimportoperation = (IDfImportOperation) clientx.getoperation("import"); oimportoperation.setsession(osession); oimportoperation.setexternalvariable("docbaseschemaname", sdocbaseschemaname); oimportoperation.setexternalvariable("dssschemaname", sdssschemaname); Specify the XML file and XML application and execute oimportnode = (IDfImportNode) oimportoperation.add(oxmlfile); oimportnode.setxmlapplicationname(sxmlappname); oimportnode.setkeeplocalfile(true); if (oimportoperation.execute()) // log results 38 Aside External variables: Pre-defined External Variables that can be used DF_USER_NAME DF_HOME_CABINET DF_DOCBASE DF_SESSION_ID XML Application config file <metadata> <dctmattr> <name>title</name> <template> <expression>myutils:useallvars ("<var name = "DF_SESSION"/>", "<var name= DF_USER_NAME"/>", "<var name= DF_HOME_CABINET"/>", "<var name = DF_DOCBASE"/>") </expression> </template> Further Aside other ways to get params: Query other parameters from Documentum If based on a user, consider specifying alias on user Create custom configuration object Other parameters outside of Documentum Resource Bundle (properties file) although this would need to be accessible The most pervasive complaint about authoring XML content in Document is the performance. When decisions are made to chunk many objects assembly and disassembly can take a long time how can this be addressed? Out of the box Chunking and authoring decisions Webtop versus Desktop Desktop with Citrix Draft Objects (in editing environment) Draft Objects in workflows Out of the box: Chunking and authoring decisions You should chunk - Core components that will be shared between documents Major components that are redundant between documents Components that form a logical unit of work for authors When you want to allow authors to edit smaller pieces of the overall document without locking other authors out When you want separate approval workflows for pieces of the document, or to be able to control access to chunks You should not chunk XML components at too low a level When you don t have a clear reason to do so Just because some of the information in a component is redundant 41 42

8 43 Out of the box: Chunking and authoring decisions Checking out content Check out at the appropriate virtual document level When updating structure don t download descendants Out of the box: XML Processing location Even using good chunking and authoring decisions many solutions may take too long for authors to checkout and check in content Consider using an Application Server Architecture to move assembly/disassembly to a server Webtop Desktop with Citrix For XML Editor integration consider Arbortext Epic 5.1 with WDK/XML adapter Arbortext Contributor 5.1 Blast Radius XMetal ActiveX control 44 Draft Objects: Overview If all else fails and you still need further performance improvements, consider these custom solutions Draft objects are objects which contain preassembled content Draft objects are created by assembling and locking content on server and copying to a new draft object Authors edit/check in/check out as much as desired When authoring is complete, draft object is disassembled and individual chunks are checked in Draft Objects (In Editing Environment): Overview Current Data Flow: Queries and Assembly/ Disassembly of objects Author Future Data Flow: Author Each requested object of Virtual Document, thousands of objects Multiple Requests Network Traffic Single File Single Request Documentum Server Queries and Assembly/ Disassembly of objects Documentum Server Draft Objects (In Editing Environment): Architecture Server Documentum method to handle all Draft Object assembly/disassembly Server side methods can be pushed to more then one java method server to balance load Client Epic Documentum Adapter customizations Additional menus and dialogs to work on draft objects Call server code for all heavy lifting Draft Objects (In Editing Environment): User Interface Epic Editor Menu customizations 47 48

9 49 Draft Objects (In Editing Environment): User Interface Browse and assemble content into a Draft Object Draft Objects (In Editing Environment): User Interface Operate on draft objects (My Draft objects) What happens if some content is locked? 50 Draft Objects (In Editing Environment): Summary Testing of the solution showed an average 50% reduction in the amount of time it took to view/edit and check in content Exact numbers were greater and lesser depending on the number of objects in a virtual document Additional gains were seen based on the number of times draft objects were checked in and out (i.e. checking a draft object in and out was very fast since no XML processing was done) Reduced network load (useful for remote authors) Reduced need for upgrading underpowered authoring PCs Even with other remoting mechanisms (Citrix, Web application), still saw performance improvements Draft Objects (In Workflows): Overview If authoring tasks are determined through a more formal process, consider using draft objects in workflows. Authoring workflow is started Method is launched to create draft object for author and attach to workflow Task is put into author s inbox with attached draft object package Author edits draft object and makes appropriate changes Author completes task Workflow can continue and create renditions for review and pass/ fail content Workflow method runs to disassemble draft object and check changes back in to individual chunks Integrating DFC code in XSL Transforms How to call DFC within an XSL Transform How to call your own DFC classes within an XSL Transform Allows you to implement more complex logic Conditional / Fallback logic Execute side-effects Error handling Potentially faster execution Integrating DFC code in XSL Transforms Call DFC within an XSL Transform: Declare java in your xsl:stylesheet <xsl:stylesheet version="1.0" xmlns:xsl=" xmlns:xalan-nodeset=" xmlns:java=" xmlns:xalan=" exclude-resultprefixes="xalan" xmlns:jsp="jsp"> Establish a session <xsl:param name="dms_session_id" select="'default value'"/> <xsl:param name="dms_input_object_id" select="'default value'"/> <xsl:variable name="clientx" select="java:com.documentum.com.dfclientx.new()"/> <xsl:variable name="client" select="java:getlocalclient($clientx)"/> <xsl:variable name="session" select="java:findsession($client, $DMS_SESSION_ID)"/> 53 54

10 55 Integrating DFC code in XSL Transforms Call DFC within an XSL Transform: Set object to XSL variable and make DFC calls <!-- Get DFC Object --> <xsl:variable name="persistentobject" select= "java:getobjectbyqualification($session, $qualificationquery)"/> <! Get Subject value --> <xsl:variable name= ssubject" select="java:getstring($persistentobject, subject')"/> DFC method definitions: public com.documentum.fc.client.idfpersistentobject getobjectbyqualification(string qualification) throws DfException public String getstring(string attrname) throws DfException Integrating DFC code in XSL Transforms Call your own DFC classes within an XSL Transform: Set an xsl:variable to a new object instance <xsl:variable name= oxslthelper" select="java:com.company.common.xslt.helper.new()"/> Call Methods with oxslthelper, the DFC Session, and required arguments <xsl:variable name="osysobject" select="java:getsysobjectfromobjectid($oxslthelper,$session, $sobjecttype,$sobjectid)"/> Method Definition public IDfSysObject getsysobjectfromobjectid(idfsession osession, String sobjecttype, String sobjectid) throws DfException 56 Advanced XML Seminar Summary Remember where XML developer resources can be found Documentum Developers ml Flatirons Solutions Developer Zone Documentum Support Site Groups (XML) Yahoogroups Documentum XML newsgroup (Documentum-xml) Questions? 57 58