XML::DT - a Perl down translation module

Size: px

Start display at page:

Download "XML::DT - a Perl down translation module"

Miranda Conley
8 years ago
Views:

1 SGMLDocuments:WhereDoesQualityGo? 1 XML::DT - a Perl down translation module José Carlos Ramalho jcr@di.uminho.pt José João Dias de Almeida jj@di.uminho.pt Table of Contents XML::DT, what? Context of use Feature description Behind the scene Examples Future developments XML Europe jcr 2

uminho.pt José João Dias de Almeida jj@di.uminho.pt Table of Contents XML::DT, what?

2 SGMLDocuments:WhereDoesQualityGo? 2 What was the motivation? project GEIRA 30 museums; several libraries; 1 national archive;... XML documents are produced everyday: Electronic news system Catalog construction Source documents we have to generate several different formats for the same document Control over... XML Europe jcr 3 XML::DT, what? a simple tool easy to learn with unlimited power!? to process down translations... going towards a transformation tool XML Europe jcr 4

several different formats for the same document Control over... XML Europe 1999-29.04 - jcr 3 XML::DT, what?

3 SGMLDocuments:WhereDoesQualityGo? 3 XML::DT architecture expat XML::DT... XML::Parser... Perl XML Europe jcr 5 Processing with XML::DT Document Head Body Paragraph List Process(Document) = f( Process(Head), Process( g( Process(Paragraph), Process(List) ))) With XML::DT: f, g = concatenation Expected result of Process is a string XML Europe jcr 6

04 - jcr 5 Processing with XML::DT Document Head Body Paragraph List Process(Document) = f(

4 SGMLDocuments:WhereDoesQualityGo? 4 Example 1: XML file <?xml version="1.0" encoding="iso "> <article> <title>the XML Down Translator</title> <author>j. João Almeida</author> <author>josé Carlos Ramalho</author> <keyword>xml</keyword> <keyword>language processing</keyword> <keyword>perl</keyword> <abstract> Once upon a time... </abstract> </article> XML Europe jcr 7 Example 1: DT file 1 #!/usr/bin/perl 2 use XML::DT ; 3 my $filename = shift; 4 %handler=( 5 '-outputenc' => 'ISO ', 6 '-default' => sub{""}, 7 'title' => sub{"<b>$c</b>"}, 8 'author' => sub{" <i>$c</i>"}, 9 'article' => sub{"$c<br>"} 10 ); 11 print dt($filename,%handler); XML Europe jcr 8

.. </abstract> </article> XML Europe 1999-29.04 - jcr 7 Example 1: DT file 1 #!

5 SGMLDocuments:WhereDoesQualityGo? 5 Example 1: DT file perl ex1.pl art.xml 1 #!/usr/bin/perl $c - processed content 2 use XML::DT ; <b>the XML Down Translator </b> $q - element gi 3 my $filename = shift; <i> J.J. Almeida </i> $v{attrname} - value of attribute <i> J.C. Ramalho </i> dt( arg1, arg2 ) - main down <br> 4 %handler=( translation function 5 '-outputenc' => 'ISO ', 6 '-default' => sub{""}, 7 'title' => sub{"<b>$c</b>"}, 8 'author' => sub{" <i>$c</i>"}, 9 'article' => sub{"$c<br>"} 10 ); 11 print dt($filename,%handler); XML Europe jcr 9 dt( arg1, arg2 ) function arg1is the filename of the XML file or a string with XML content arg2is a structure of the form: pattern => action patterns available '-default' - matches every element not matched in any other pattern '-outputenc' - by default is UTF8 'element-gi' - triggers action for elements with that gi '-pcdata' - selects #PCDATA in mixed contents '-end' - processing to be applied to the transformed tree XML Europe jcr 10

Ramalho </i> dt( arg1, arg2 ) - main down <br> 4 %handler=( translation function 5 '-outputenc' => 'ISO-8859-1', 6 '-default' => sub{""}, 7 'title' => sub{"<b>$c</b>"}, 8 'author' => sub{"

6 SGMLDocuments:WhereDoesQualityGo? 6 Is it hard to write?...skelgen.pl Ex: perl skelgen.pl art.xml #!/usr/bin/perl use XML::DT ; my $filename = shift; %handler=( # '-outputenc' => 'ISO ', # '-default' => sub{"<$q>$c</$q>"}, 'title' => sub{"$q:$c"}, 'author' => sub{"$q:$c"}, 'article' => sub{"$q:$c"}, 'abstract' => sub{"$q:$c"}, 'keyword' => sub{"$q:$c"}, ); print dt($filename,%handler); XML Europe jcr 11 skelgen.pl Printing: print <<'END'; #!/usr/bin/perl use XML::DT ; my $filename = shift; Generates XML::DT processors %handler=( It is programmed with XML::DT It shows a different use END of XML::DT Analysis: two parts: analysis and printing # '-outputenc' => 'ISO ', # '-default' => sub{"<$q>$c</$q>"}, for $name (keys %element){ #!/usr/bin/perl print " '$name' => sub{\"\$q:\$c\"},"; use XML::DT ; print '# remember $v{', join('},$v{',keys %{$att{$name}}), my $filename = shift; '}' if $att{$name}; %xml=( '-default' => sub{$element{$q}=1; print "\n"; for (keys } %v){$att{$q}{$_}=1 }; ""}); print <<'END'; ); dt($filename,%xml); print dt($filename,%handler); END XML Europe jcr 12

sub{"$q:$c"}, 'abstract' => sub{"$q:$c"}, 'keyword' => sub{"$q:$c"}, ); print dt($filename,%handler); XML Europe 1999-29.04 - jcr 11 skelgen.pl Printing: print <<'END'; #!

7 SGMLDocuments:WhereDoesQualityGo? 7 skelgen.pl: future developments script will become a module function It will have a behavior switch to enable incremental processing: if you add something to the structure of your XML file you do not need to hand code the new elements XML Europe jcr 13 Ex2: Proceedings List of papers <?xml version="1.0" encoding="iso "?> <proceedings> <title>xml Europe 99</title> <chair>pam</chair> <abstract> Once upon a time in Granada... </abstract> <article file="art1.xml"/> <article file="art2.xml"/> <article file="art3.xml"/> </proceedings> XML Europe jcr 14

structure of your XML file you do not need to hand code the new elements XML Europe 1999-29.04 - jcr 13 Ex2: Proceedings List of papers <?xml version="1.

8 SGMLDocuments:WhereDoesQualityGo? 8 Proceedings.pl #!/usr/bin/perl use XML::DT ; my $filename = shift; <H1>Proceedings: XML Europe 99</H1> 'title' <H2>Chair: Pam</H2> <b>the XML Down Translator</b><BR> <I>J. JoÃ o Almeida</I><BR> <I>JosÃ Carlos Ramalho</I><BR> ); <P> %p_art=( <b>the XML Parser</b><BR> <I>Clark Cooper</I><BR> 'title' <I>Larry Wall</I><BR> <P> ); <b>the expat library</b><br> <I>James Clark</I><BR> <P> %p_proc=( '-outputenc' => 'ISO ', '-default' => sub{"$c"}, 'proceedings' => sub{"<h1>proceedings: $c"}, => sub{if(inctxt('proceedings')) {"$c</h1>"} else {""}}, 'article' => sub{ dt($v{file}, %p_art) }, 'abstract' => sub{""}, 'chair' => sub{"<h2>chair: $c</h2>"}, '-default' => sub{""}, => sub{" <b>$c</b><br>"}, 'author' => sub{" <I>$c</I><BR>"}, 'article' => sub{"<p>$c<p>"}, print dt($filename,%p_proc); XML Europe jcr 15 Proceedings.pl #!/usr/bin/perl use XML::DT ; my $filename = shift; dt can be used in a functional way enabling subdocument processing nested processing processor combination %p_proc=( '-outputenc' => 'ISO ', '-default' => sub{"$c"}, 'proceedings' => sub{"<h1>proceedings: $c"}, 'title' => sub{if(inctxt('proceedings')) {"$c</h1>"} else {""}}, 'article' => sub{ dt($v{file}, %p_art) }, 'abstract' => sub{""}, 'chair' => sub{"<h2>chair: $c</h2>"}, ); %p_art=( '-default' => sub{""}, 'title' => sub{" <b>$c</b><br>"}, 'author' => sub{" <I>$c</I><BR>"}, 'article' => sub{"<p>$c<p>"}, ); print dt($filename,%p_proc); XML Europe jcr 16

$Clark</I><BR> <P> %p_proc=( '-outputenc' => 'ISO-8859-1', '-default' => sub{"$c"}, 'proceedings' => sub{"<h1>proceedings: $c"}, => sub{if(inctxt('proceedings')) {"$c</h1>"} else {""}}, 'article' =>$

9 SGMLDocuments:WhereDoesQualityGo? 9 Context dependent processing ctxt(number) Suppose we have: CONTEXT = "article/header/title" ctxt(0)matches "title" ($q) ctxt(1)matches "header" ctxt(2)matches "article" inctxt(pattern) inctxt('art.*') true if current element is under article and article is the top element inctxt('proceedings article') true if element is son of proceedings or article match LIFO structure article/header/title XML Europe jcr 17 What do we want? Proceedings: XML Europe 99 Chair: Pam Ex3: Adding a keyword index The XML Down Translator What do we have? A set of J. JoÃ o files like: Almeida <?xml version="1.0" encoding="iso "?> JosÃ Carlos Ramalho <article> The XML Parser <title>the XML Down Translator</title> Clark Cooper <author>j. João Almeida</author> Larry Wall <author>josé Carlos Ramalho</author> The expat library <keyword>xml</keyword> James Clark <keyword>language processing</keyword> Keyword index <keyword>perl</keyword> <abstract> C++ The expat library Once upon a time... </abstract> XML </article> The XML Down Translator The XML Parser The expat library... XML Europe jcr 18

*') true if current element is under article and article is the top element inctxt('proceedings article') true if element is son of proceedings or article match LIFO structure article/header/title

10 SGMLDocuments:WhereDoesQualityGo? 10 Ex3: How? sub mkkeyind{ my $r = "<H2>Keyword index</h2>"; for $term (sort keys %ind) { $r.= "<P> <B>$term</B> $ind{$term}"; } $r } %p_proc=(... term1 term2 term3... 'proceedings' => sub{"<h1>proceedings: $c". mkkeyind()},... ); %p_art=(... 'title' => sub{ $tit = $c; " <b>$c</b><br>"; }, 'keyword' => sub{ $ind{$c}.= "<BR> $tit"; ""; },... ); key1 key2 key3 XML Europe jcr 19 The main algorithm dt( document-handler, processor) = let tree = Parse(document-handler) in process( tree, processor) processor{'-end'} (process( tree, processor)) process(pcdata(p), processor) = p processor{'-pcdata'}(p) process( element( e, sons ), processor ) = let args = concatenate( [ process( x, processor ) x sons ] ) in if( e in domain(processor) then processor[e](args) else processor['-default'](args) XML Europe jcr 20

.. ); key1 key2 key3 XML Europe 1999-29.

11 SGMLDocuments:WhereDoesQualityGo? 1 Last example: gcapaper2tex.pl %handler=(... What '-pcdata' is the => sub{ problem? if(inctxt('(section SUBSEC1)')) {$c =~ s/[\s\n]+/ /g; $c } reduce spaces inside #PCDATA mixed content in SECTION and $c }, SUBSEC1... elements. convert 'TITLE' list => of sub{ authors into: \author{a1 if(inctxt('section')){"\\section{$c}"} \and a2 \and a3} elsif(inctxt('subsec1')){"\\subsection{$c}"} else {"\\title{$c}"} }, 'AUTHOR' => sub{ $c ; ""}, 'ABSTRACT' => sub{ sprintf('\author{%s}\maketitle\begin{abstract}%s\end{abstract}', join $c) }, 'XREF' => sub{"\\cite{$v{refloc}}"},... ); XML Europe jcr 21 New developments To Enable dt to create a data structure instead of a string: a mapping: elementgi content for a certain domain of actions dt will generate a DSSSL/XSL specification To create the $utc un-transformed-content XML Europe jcr 22

$convert 'TITLE' list => of sub{ authors into: \author{a1 if(inctxt('section')){"\\section{$c}"} \and a2 \and a3} elsif(inctxt('subsec1')){"\\subsection{$c}"} else {"\\title{$c}"} }, 'AUTHOR' => sub{$

12 SGMLDocuments:WhereDoesQualityGo? 12 Resources Web page last version available for download examples documentation XML Europe jcr 23

José Carlos Ramalho jcr@keep.pt jcr@di.uminho.pt

José Carlos Ramalho jcr@keep.pt jcr@di.uminho.pt 1 Database migration CLI José Carlos Ramalho jcr@keep.pt jcr@di.uminho.pt 1 Intermediate Representation DBML; Hardware and Software independent; It can