Conversion of Measurement Data from MDF to ATFX

Size: px

Start display at page:

Download "Conversion of Measurement Data from MDF to ATFX"

Drusilla Boone
8 years ago
Views:

1 Conversion of Measurement Data from MDF to ATFX Dr. Olaf Flebbe science + computing ag IT-Dienstleistungen und Software für anspruchsvolle Rechnernetze Tübingen München Berlin Düsseldorf

2 Agenda science + computing ag at a glance Motivation Conversion of MDF to ATFX Data structures Data types Conversions Units Software available ODS 5.3 Future? Page science + computing ag

3 Locations and employees Office Duesseldorf Headquarters Tuebingen Office Berlin Office Ingolstadt Office Munich Number of employees: > 300 Locations in Asia: Peking Singapore page science + computing ag

Automotive industy Chemistry and pharmaceutics Aerospace

4 Customers by sectors Public service Plant and mechanical engineering 2 semiconductor manufacturers Microelectronics Automotive industy Chemistry and pharmaceutics Aerospace Hardware and software manufacturers page science + computing ag

5 Mapping of Measurement Data Structures MDF ATFX <AoEnvironment><AoTest><AoSubTest> ID HD. DG (Data Group) <AoMeasurement> CG (Channel Group) <AoSubMatrix> CN (Channel) <AoLocalColumn> DG <AoMeasurement> CG <AoSubMatrix> CN <AoLocalColumn> Seite 5

6 MDF Konvertierung Context: Importing Existing Measurement Systems creating MDF3.x and MDF4.0 format into an ASAM ODS 5.2 storage. No exotic features in MDF Files MDF3 -> ATFX ATFX -> MDF3 MDF4 -> ATFX Seite 6

7 Mapping of Measurement Data Structures Converter only writes ATFX entities below <AoMeasurement> Second step is to enrich the missing parts with Meta- Information which is not available at conversion time. Seite 7

step is to enrich the missing parts with Meta-

8 Mapping of Data Type MDF Integers can be arbitrary, from 1bit up to 64 bit ODS: The integer quantities available are: 8bit, 16bit, 32bit and 64bit (DT_BYTE, DT_SHORT, DT_LONG, DT_LONGLONG) ODS: DT_Boolean (1 Bit) Implementation have problems with packed DT_BOOLEAN in AoExternalComponent (i.e. DS_BOOLEAN) ODS: No unsigned 64Bit integer (java limitation) Strings: Encoding! -> Conversion needs to write External Component Seite 8

Bit) Implementation have problems with packed DT_BOOLEAN in AoExternalComponent (i.e. DS_BOOLEAN) ODS: No unsigned 64Bit integer (java limitation) Strings: Encoding!

9 Mapping of Conversions Conversion is the mapping of raw (measurement) data to physical data MDF has a wealth of conversion methods between raw data and physical dimensions: CCBlock attribute cc_type 0 = 1:1 conversion 1 = linear conversion 2 = rational conversion 3 = algebraic conversion (MCD-2 MC text formula) 4 = value to value tabular look-up with interpolation 5 = value to value tabular look-up without interpolation 6 = value range to value tabular look-up 7 = value to text/scale conversion tabular look-up 8 = value range to text/scale conversion tabular look-up 9 = text to value tabular look-up 10 = text to text tabular look-up (translation) Seite 9

value tabular look-up with interpolation 5 = value to value tabular look-up without interpolation 6 = value range to value tabular look-up 7 = value to text/scale

10 Mapping of Conversion Conversion is the mapping of raw (measurement) data to physical data ODS supports only 3 conversion AoLocalColumn sequence_representation external_component (=7) raw_linear_external (=8) raw_polynomial_external (=9) raw_linear_calibrated_external (=11) => Fortunately only trivial rational conversions found, until now. Seite 10

external_component (=7) raw_linear_external (=8) raw_polynomial_external (=9)

11 Mapping of Units In MDF Units are optional Strings, no relations between units. In ODS Units have relations between each other and are mandatory MDF Unit -> N:1 -> AoPhysicalDimension (Unit) (exponents 0) (empty) -> N:1 -> AoPhysicalDimension ( - ) AoPhysicalDimension 1:1 AoUnit Name Unit AoUnit 1:1 AoQuantity Problem: No relation of AoUnits (mm, m, km) Need for Unit Database What is the unit of %? Encoding of Unit (MDF4 should be part of a XML Document, but it is not all the time. C encoded in ISO8859 found. Seite 11

(empty) -> N:1 -> AoPhysicalDimension ( - ) AoPhysicalDimension 1:1 AoUnit Name Unit AoUnit 1:1 AoQuantity Problem: No relation of

12 Software Available The Software is (should be soon) available in the openmdm community. Pure Java License openmdm Uses javolution (BSD License) for mapping of MDF datastructures to java. Uses openatfx for reading in ATFX Files. Contains own simple ATFX Writer with an easy API Optionally contains CSV Writer for dumping raw values for integrity checks. Seite 12

to java. Uses openatfx for reading in ATFX Files.

13 Performance Issues Reading of unsorted files and indexing on the fly Indices are RAM intensive. Recommended to use 64Bit Java VM Uses memory mapped I/O, performance improvements on new (1.7 java VM) Seite 13

Recommended to use 64Bit Java VM Uses memory mapped

14 TODO The software mdf3atfx and mdf4atfx implements only a small subset of MDF3 and MDF4: Supports a many Real Data Files. No virtual Channels No support for MDF 4.1 Extensions (i.e. compression etc) Seite 14

15 ODS 5.3 ODS 5.3 contains additions for better supporting MDF3 and MDF4 used directly as external component AoExternalComponent Attributes ao_bit_count ao_bit_offset Chapter of ODS 5.3 Standard describes what MDF 4.x file features can be used for AoExternalComponents or as-is The software presented here can be to do the functionality Possible Postprocessing steps to achive ODS Compliant MDF Files on many MDF4 files described in the ODS Standard. Seite 15

Attributes ao_bit_count ao_bit_offset Chapter 2.11.3 of ODS 5.3 Standard describes what MDF 4.

16 ODS Wishlist Standard Unit Database System MDF have comments almost everywhere, ODS almost none. Seite 16

17 Vielen Dank für Ihre Aufmerksamkeit. Dr. Olaf Flebbe science + computing ag o.flebbe@science-computing.de

ASAM Transport Format in XML (ATFX) Overview and Best Practices

ASAM Transport Format in XML (ATFX) Overview and Best Practices Dr. Peter Stoll science + computing ag IT-Service and Software Tübingen München Berlin Düsseldorf Content CAT data management @ s+c ASAM