EUROPEAN COMMISSION DIRECTORATE-GENERAL FOR MARITIME AFFAIRS AND FISHERIES MEDITERRANEAN AND BLACK SEA Integrated Fisheries Data Management Version 0.11 INTEGRATED FISHERIES DATA MANAGEMENT PROGRAMME PHASE 1: FISHERIES CONTROL AND MONITORING FLUX Subject: FLUX outline 1. INTRODUCTION This document describes FLUX (Fisheries Language for Universal exchange) in general terms. The intended readership is anyone who needs a general overview of what FLUX is about. 2. BACKGROUND Fisheries control and management is largely based upon the collection, storage, exchange of large sets of data between the parties involved. Data sets are very diverse, ranging from tiny reports on the whereabouts of individual fishing vessels to aggregated reports of monthly (yearly) catches of the complete fleet of a country. This data is collected for different purposes. Sometimes it is used to closely monitor the behaviour of a single vessel, in other cases it serves scientific purposes in preparation of scientific advice for establishing TAC for a future fishing season. The requirements for data availability have historically grown and changed. The consequence is that for each business need individual data sets have been defined, and specific technical solutions have been developed. Today, a large patchwork of (partial) data management solutions is in place. This diversity hinders data exchange, and often delivers questionable quality at high operating cost. As part of the solution for this problem, the FLUX (Fisheries Language for Universal exchange) project aims at defining a universal and efficient data Commission européenne/europese Commissie, 1049 Bruxelles/Brussel, BELGIQUE/BELGIË - Tel. +32 22991111 Office: J-79 - Tel. direct line +32 229-+ 32 296 40 13 francky.callewaert@ec.europa.eu
exchange "language" compatible with (but not limited by) regulations and international requirements. This language will be promoted for use in the EU, as well as for RFMO and FPA. 3. FLUX CONCEPT FLUX is based on web technology. Applicable technical terms 1 are XML (extended Markup Language), XSD (XML Schema Definition), WSDL (Web Services Description Language), SOA (Services Oriented Architecture) FLUX contains two distinct but related parts: The FLUX business layer The FLUX transportation layer FLUX is supported by software developments and implemented on a networked infrastructure called the Data Exchange highway (DEH). Summarised, FLUX can be easily compared with a paper based mail system: The FLUX business layer brings a series of standardised electronic forms that can be filled in with concrete business data that has to be sent to another party. The FLUX transportation layer delivers an envelope that can contain any of those forms. The DEH, built with FLUX enabled software, serves as a postal infrastructure sending the envelopes (and their content) from sender to destination. 4. FLUX CONTENT 4.1. The FLUX business layer 4.1.1. Standardisation of data elements The core of the FLUX business layer is the detailed and standardised description of each and any data element needed the standardised grouping of those data elements in messages required by the business for exchanging data between parties Standardisation of the data elements and formats is based upon the UN/CEFACT 2 approach. This immediately allows to also describing the typical business processes. 1 The explanation of these terms is beyond the scope of this document 2 http://www1.unece.org/cefact/platform/display/cnp/electronic+interchange+of+fisheries+catch+data 2
Technically speaking, UN/CEFACT standardization provides a standardized schema for business process (XSD) and a standardized content (Core Components). The practical outcome of a UN/CEFACT standardisation project is a technical file called XSD (XML Schema Definition) for the business processes and requirements subject to the project. This XSD can be used for all data exchanges and processes described by the project. The data exchanged are also harmonized and published in standardized library (UN/CEFACT Core Component Library). An extra advantage is that UN/CEFACT ensures compatibility with similar standardisation exercises taking place in other business area's which may influence the data requested from the fisheries sector like customs, food traceability and trade. 4.1.2. Modular data domain approach Not every party is involved in all aspects of the fisheries business. As one example, a country not involved in Bluefin Tuna fishing has no interest in implementing the data elements, messages and processes for that particular fishery. The FLUX business layer is based on individual stand-alone business modules that allow parties to implement only the modules they need. The UN/CEFACT standardisation approach guarantees that modules are compatible. Once a party has completed a FLUX data exchange installation for a single module, it should be fairly easy to plug in extra modules for other data exchanges. 4.1.3. Selection of code lists In many cases the potential content of a specific data element is limited to a number of values; this is e.g. the case for gear types or species. Often, these lists are already standardised but two issues arise: Standardised code lists are sometimes outdated (e.g. list of gear types) For the same data elements, there are several "standards" available (e.g. several lists of country codes) The FLUX approach is to base the code lists needed to the extent possible on existing standards, but to deviate from these if unavoidable. In such cases the standardisation body responsible for the code list may be contacted with a request for updating the code list (e.g. UN/LOCODE port list). Note: To avoid any uncertainty about the current versions of code lists, data elements, formats, etc all this information will be published on a master data register. This web site starts of as a simple HTML based site but will be replaced by a dedicated web service. 4.2. The FLUX transportation layer The messages, formulated according to the above UN/CEFACT standards, are sent around between the computer systems of the parties during the business processes. 3
These "data streams" have been implemented in many ways which has been leading to a range of specific technical implementations for specific data streams and between specific parties. This scattered technical infrastructure leads to high cost for development and maintenance, and a constant need for new developments for new data streams. The FLUX approach is to create a business agnostic transportation layer facilitating the exchange of data between parties. In fact, this transportation layer could be used for any business (e.g. exchange data on airplanes). The FLUX transportation layer provides description for: The FLUX Envelope, one single yet universal message format that can encapsulate any business-specific message or structured data in a predictable way whatever the business system and associated data types and formats, using industry standard data representation techniques The FLUX Protocol, a mechanism describing how to reliably deliver the FLUX Envelopes to their destination reliably and without human intervention, leveraging state-of-the-art existing technologies (SOAP Web Services) in a sensible manner so as to as much as possible avoid interoperability issues between FLUX implementations based on different vendors' solutions. 5. FLUX SOFTWARE A party's system that needs to interact with a system of another party using FLUX obviously needs to adhere to FLUX conventions and behaviours. In other words it must implement the FLUX protocol. One solution is to have such a system incorporate a tailor-made module that implements FLUX messaging built according to the technical FLUX specifications. In many cases however, the cost of adding a custom-made FLUX module in an application can be very expensive, even impossible in case of legacy systems built a long time ago and for which the detailed technical expertise or knowledge of its internals is no longer available Therefore DG MARE may provide a reference implementation of a FLUX Gateway that can on one hand communicate with legacy systems, and on the other hand connect to the DEH. 6. FLUX IMPLEMENTATION ON THE DATA EXCHANGE HIGHWAY 6.1. Data Exchange Highway (DEH) In the EU alone, 22 countries are involved in fisheries and are exchanging data with each other, with DG Mare, with Eurostat, the EFCA, ICES and other organisations. There are about 13 RFMO with a large number of contracting parties and a large number of bilateral fisheries partnership agreements. As an example: A typical EU country exchanges data with +/- 15 parties. This means that, working bilaterally, 22*15 connections, or 330, have to be made to 4
cover for existing practices. Any changes in business could lead to a growing need for more connections. The cost of establishing (and maintaining) all these bilateral technical "links" between each parties' computer systems needing to exchange data with another one grows exponentially with the number of parties involved and prohibits setting up an efficient system. To avoid this exponential growth an architecture is set up grouping parties around "central nodes". These central nodes are themselves connected to other central nodes. This means that if a party is connected to one central node, it can securely communicate with all parties connected to any central node. This approach is called the "data exchange highway". The FLUX software is supporting this approach. Note: The provider of a central node does not "own" the data passing through it and does not have any right to store, change, or read the business content in the messages. Providers can however look at the envelope content to investigate and solve communication errors. 6.2. Central services Parties may develop central services and make these available on the DEH to other parties. The logic behind this is that it can be counterproductive (from an efficiency and data quality point of view) and expensive for each party to develop, maintain, and operate IT systems of common use. Typical examples are: A Master Data Register with all standard formats, code lists A spatial service as unique reference for geographical fishing zones The methods to request and receive data from such services can be incorporated in the FLUX business and transportation layers and are supported by the FLUX software. 7. VERSION AND HISTORY Version Author Date 0.11 Francky Callewaert 31/08/2012 5