XTM Web 2.0 Enterprise Architecture Hardware Implementation Guidelines. A.Zydroń 18 April Page 1 of 12

Transcription

1 XTM Web 2.0 Enterprise Architecture Hardware Implementation Guidelines A.Zydroń 18 April 2009 Page 1 of 12

2 1. Introduction XTM Database JVM and Tomcat considerations XTM Engine Translator Workbench Spell Checker TM Manager Terminology Manager Workflow Engine Recommended Configurations Small Medium Large Very Large Enterprise...11 Page 2 of 12

3 1. Introduction XTM is built on an extremely scalable modular architecture. How to maximize this architecture is the subject of this paper. The design of XTM is based on SOA (Service Oriented Architecture) principles, Open Standards and an Open Architecture with the following distinct components: 1. XTM Database 2. XTM Engine 3. XTM Translator Workbench 4. XTM QA Manager/Spell Checker 5. XTM TM Manager 6. XTM Terminology Manager 7. XTM Workflow Engine Page 3 of 12

4 All of these components can be located on the same server, or distributed across multiple servers. Written in Java, XTM is platform independent, which means that individual components can be resident on different operating systems if required. It also means that XTM is 64 bit ready from the outset; you can run XTM on 32 bit or 64 bit environments completely transparently. The benefits of a 64 bit architecture are greater scalability enabling the handling of much larger volumes of data on a single server with enough RAM. The following diagram represents the logical flow of data within XTM: The XTM Workflow component is only relevant if use is made of the XTM Project Manager to create and monitor projects. If XTM is being integrated with an existing workflow then this component is not typically used. XTM Statistical Machine Translation (SMT) is an optional component that will utilize the services of an external SMT engine. 2. XTM Database The Database is the core component of XTM. Most other services rely on the Database. The Database represents the current state of the XTM system regarding most XTM data apart from the actual XTM Workbench XLIFF translation files and the XTM QA Manager Spelling Dictionaries. XTM uses the Hibernate v3.3.1orm (Object to Relational Mapping) methodology to create a layer of abstraction between itself and the underlying database engine. XTM supports any relational database that is supported by Hibernate. These include, among others: 1. Oracle 8i, 9i, 10g 2. IBM DB2 7.1, 7.2, 8.1, 9.1 Page 4 of 12

5 3. Microsoft SQL Server Sybase 12.5 (JConnect 5.5) 5. MySQL 4.1, PostgreSQL 8.1, SAP DB InterSystems Cache' A lot of time and effort has gone in to guaranteeing enterprise scale database performance in XTM. The design of the table structures is optimized for maximum efficiency and performance. XTM provides the user with a wide and rich choice of commercial or Open Source database engines depending on expected throughput and performance. In addition, XTM uses the Lucene engine for text indexing. This provides a very efficient and underlying database independent method of indexing text for fuzzy matching and concordance. It is important to note that the database is the core component that all other parts of XTM refer to. The design of XTM allows placement of all other services to be completely arbitrary, thus providing exceptional flexibility. Suffice it to say that for very large installations a dedicated 64 bit clustered database engine will provide the very best performance possible. Additional advantage can be taken from any clustering facilities that the database may offer. For small to medium sized installations PostgreSQL provides a very effective database platform. 3. JVM and Tomcat considerations All XTM components must run within a Java Virtual Machine (JVM). Most XTM components also run within Apache Tomcat, a Java Server Pages and Web Services container. All XTM components can be offloaded onto separate servers with each component running in its own Tomcat container. Alternatively all XTM components can be run on one Tomcat container on one server. 4. XTM Engine The XTM Engine is responsible for the initial conversion into XML (if required), analysis and matching of documents in order to present the resultant XLIFF file via the XTM Translator Workbench to the translator. An Open API is provided, that allows for the transparent integration of XTM with XML CMS systems. The XTM Engine is usually run as part of the Project Manager 'New Project' facility or as part of the Workflow in Page 5 of 12

6 an XML based CMS system. XTM Engine operations are very CPU and memory intensive and as such are best off loaded onto a separate server. In a LSP environment the XTM Engine operations are run as part of a single threaded Java Messaging System (JMS) queue, so that multiple concurrent jobs are queued up to minimize the effect on other parts of the system. In a large scale enterprise installation the XTM Engine should reside on one or more dedicated servers. The XTM engine will store the XLIFF files to be translated in a special directory structure. This structure can be accessed transparently by mounting the appropriate file system from other servers that are running the Translator Workbench. 5. Translator Workbench The Translator Workbench is at the crux of XTM operations. It is imperative that translators are given the best performance possible. To this end it is best to run the Translator Workbench on one or more dedicated servers. As a general rule of thumb, XTM can support 20 concurrent translators per 1.5Gb of RAM per 1 Intel Xeon processor. It is strongly encouraged to run the Translator Workbench on its own server/s. The Translator Workbench is also an ideal application to run in a load balancing configuration when translators are automatically routed to the server with the least load. The only limitation is that for documents that will be worked on by multiple concurrent translators and reviewers, then all of the concurrent participants must be on the same server. 6. Spell Checker The XTM Spell Checker module runs as a self contained Web Service. It is also quite memory and CPU intensive as an application. As such, it is advisable in busier installations to offload the Spell Checker away from the Translator Workbench. Ideally it should run on a dedicated server, albeit a relatively small one: a 32 bit server with 2Gb of RAM will suffice. 7. TM Manager The TM Manager runs as a Web Services component and is therefore very easy to offload/relocate. Due to the nature of its operations it does not present a heavy load on even an enterprise scale installation. 8. Terminology Manager The Terminology Manager also runs as a Web Service and is similar in nature to the TM Manager. 9. Workflow Engine The workflow engine is also a relatively low overhead component, but nevertheless can be easily relocated as required. Page 6 of 12

7 10. Recommended Configurations The following is an example of possible XTM configurations depending on number of concurrent users. These are presented here as potential configurations, although the exact set up will be dependent upon actual throughput and mix of file types being processed. These should therefore be regarding as guidelines and not definitive rules: 10.1 Small This configuration will be sufficient for up to 20 concurrent translators and one project manager on a dedicated server: 32 bit OS 1 Xeon 1.86Ghz CPU 2Gb RAM 250Gb disk Page 7 of 12

8 10.2 Medium This configuration will be sufficient for medium sized operations up to 40 concurrent translators: 1 Quad Xeon 1.86Ghz CPU 4Gb RAM 500Gb disk Page 8 of 12

9 10.3 Large This configuration will be sufficient for large scaled operations up to 80 concurrent translators: Database and Main Services Server 2 quad Xeon 1.86Ghz CPU 16Gb RAM 1Tb disk Translator Workbench Server 12Gb RAM 250Gb disk Page 9 of 12

10 10.4 Very Large This configuration will be sufficient for between 80 and 500 concurrent translators: Database and Main Services Server 32Gb RAM 1Tb disk NAS Server 1Tb disk Translator Workbench: 2x 6x, the more Workbench servers, the more concurrent translators that can be supported. 12Gb RAM 250Gb disk Page 10 of 12

11 10.5 Enterprise This configuration will be sufficient for medium sized operations up to 1000 concurrent translators: Database server 32Gb RAM 1Tb disk XTM Engine 32Gb RAM 500Gb disk NAS Server 1Tb disk Web Services Server for Spell Checker, TM Manager, Terminology Manager, Workflow 16Gb RAM 500Gb disk Translator Workbench: 12x 12Gb RAM 250Gb disk Page 11 of 12

12 Page 12 of 12