ehg New Trends in e Humanities Amsterdam

Transcription

1 ehg New Trends in e Humanities Amsterdam

2 Overview 1) Dialect geography 2) A unified structure for Dutch dialect dictionary data 3) Dialectgebieden in Brabant. Geografische clustering op basis van de ruwe lexicale gegevens van het Woordenboek van de Brabantse Dialecten 4) Visualization as a Research Tool for Dialect Geography Using a Geo browser

3 1 Dialect geography

4 Dialect geography What dialect differences are there between villages or towns? What dialect areas are there? What relations are there between dialect areas and other geographic data?

5 Dialect geography Dictionary of the Brabantic Dialects (WBD)

6

7

8 2 A unified structure for Dutch dialect dictionary data Folkert de Vriend, Lou Boves, Henk van den Heuvel, Roeland van Hout, Joep Kruijsen, Jos Swanenberg (2006). In: Proceedings of The fifth international conference on Language Resources and Evaluation (LREC 2006), Genoa, Italy, pp

9 Dialect geography Dictionary of the Brabantic Dictionary of the Limburgian Dictionary of the Flemmish Dictionary of the Zeelandish Dialects (WBD) Dialects (WLD) Dialects (WVD) Dialects (WZD)

10 Total research area

11 Form based dictionary: Dictionary of the Zeelandish Dialects (WZD)

12 Sense based dictionary: Dictionary of the Brabantic Dialects (WBD)

13 Map from Dictionary of the Brabantic Dialects (WBD)

14 Unification: Map showing unified data from WBD, WLD, WZD, WVD

15 Unification: Map showing unified data from WBD and WLD (CLARIN COAVA (Cornips et al. 2011)) Frog ( Kikker )

16 Towards a unified structure based on standards All dialect dictionary projects use the same core data types: form, sense and location.

17 WBD

18 WVD

19 Dictionary of the Zeelandish Dialects (WZD)

20 Mapping core data onto the LMF core model Although the data organisation is either form based or sense based, the core data types have the same heterarchical relation

21 LMF XML implementation for core data <LexicalResource name='"unified Dialect Lexicon"> <Lexicon name="wbd"> <LexicalEntry> <Form>ulling</Form> <Sense>Fret</Sense> <Location>Rosmalen</Location> </LexicalEntry> </Lexicon> <Lexicon name="wvd"> <LexicalEntry> <Form>voejerkuil</Form> <Sense>Groenvoerkuil</Sense> <Location>K136a</Location> </LexicalEntry> </Lexicon> <Lexicon name="wzd"> <LexicalEntry> <Form> aerdwurm</form> <Sense>dauwworm</Sense> <Location>Z.eil.</Location> </LexicalEntry> </Lexicon> </LexicalResource>

22 Additional information needed for specifying core data types Form Type: Lexical, Phonetic Alphabet: IPA, Genoveva, Latin Sense Type: Concept, Meaning Location Type: Placename, Area, Kloeke code Standardisation: Where possible convert data to the same standard (f.i. location: longitude/lattitude) Minimal: map type and alphabet labels to ISOCat

23 Additional information needed for classifying core data types Form Class: phonetic forms have a lexical classification in WBD, WLD en WVD Sense: Class: taxonomy Location Class: geopolitical taxonomy Unified classifications can be used to provide access to the unified data

24 Conclusions All dictionaries share the same core data types A unified structure built around these core data types will enable Using the data of the different dictionaries as one huge dataset. Organising the data based on either sense, form, or location. This will enable different perspectives on the data.

25 3 Dialectgebieden in Brabant. Geografische clustering op basis van de ruwe lexicale gegevens van het Woordenboek van de Brabantse Dialecten Folkert de Vriend, Jos Swanenberg, Roeland van Hout (2007). In: Taal en Tongval, themanummer 20, Dialectlexicografie, pp

26 Introduction Computational analysis of the data that were collected for WBD using cluster analyses Cluster analyses is defined in Jain and Dubes (1988) as the process of classifying objects into subsets that have meaning in the context of a particular problem. In a dialect geographic context the problem can be finding dialect areas. Aim: to see if we could find detailed dialect patterns in Brabant based on lexical data only The dialect patterns that we found were compared to the dialect map of Belemans and Goossens (2000)

27 Belemans and Goossens (2000): Represents a traditional view on the classification of the Brabant area Based on qualitative analyses of several types of data.

28 Data selection Only a subselection of the WBD data could be used: Part III of WBD Data collected in the Nijmeegse enquetete. These were collected for the whole research area and not just for subareas of Brabant. Only the data for the core data types: concept, lexical form and location Resulting data selection: a data matrix with 614,941 lexical forms for 4229 concepts and 639 locations.

29 Method 1) Lexical distances were computed for all location pairs in the data set. (RuG/L04 ) 2) Using these lexical distances the locations were grouped together using cluster analyses. (RuG/L04) 3) For interpreting the resulting groups of locations, these were then converted to a KML symbol map (cartographic software developed by Meertens and Radboud University) 4) This symbol map was overlaid onto the map of Belemans and Goossens so that (mis)matches between the two maps could be visually inspected. (Geobrowser)

30 Data characteristics that were problematic for the method Nijmeegse enquete covered the whole research area, but not for every concept a lexical form was recorded in each location. Result: Datamatrix was a huge gatenkaas (83.3% of the cells in the data matrix were empty) Distance matrix was also a gatenkaas. (Since often no distance could be calculated for a pair of locations.) This was problematic for the cluster algorithms we used since they cannot deal with missing distances.

31 Solution 1 Create plausible lexical distances for the empty cells using the lexical distance to locations that are geographically near ( imputation ). Cluster analyses set to yield nine clusters showed that it was possible to find dialect areas for Brabant based on lexical data.

32 But Although the resulting dialect maps showed some general resemblance with Belemans and Goossens (2000), the results were not very satisfactory. The dialect maps contained clusters that overlapped each other much and also clusters covering the entire research area.

33 6 overlapping clusters

34 3 clusters covering entire research area

35 Solution 2 Strongly reduce the percentage of empty cells in the data matrix by completely removing all concepts and locations with little or no lexical forms. Result: Data matrix was reduced from 614,941 lexical forms to 100,277 lexical forms. The percentage of empty cells in the data matrix was reduced from 83.3% to 20.3%. Now for every pair of locations a distance could be calculated (without using imputation)

36 Result Requiring the cluster analysis to return nine clusters resulted in a close resemblance to the nine main areas of the dialect map of Belemans and Goossens (2000). Also, the result did not contain clusters covering the entire research area anymore.

37 Final result based on 100,277 lexical forms

38 Conclusions We could find detailed dialect patterns in Brabant based on lexical data only. These detailed dialect patterns resembled the map of Belemans and Goossen (2000) closely. But for our computational method the dataset had to be manipulated extensively. Relevant for ehumanities perspective: The WBD dataset was collected with a typical humanities aim in mind: collecting as much variation as possible. For searching for general patterns using cluster analyses the gatenkaas character of the datamatrix suddenly was a problem.

39 4 Visualization as a Research Tool for Dialect Geography Using a Geobrowser Folkert de Vriend, Lou Boves, Roeland van Hout, Jos Swanenberg (2011). In: Literary and Linguistic Computing, 26(1), pp

40 Basic research chain for conducting dialect geography research Applies to dialect dictionary projects as well as to dialect atlas projects. Modelled very much as a pipeline with a unidirectional data and process flow.

41 Visualization as a research tool Visualization of map data does not have to be a static and final stage in the research chain. The basic research chain can be extended with support for using visualization as a research tool. What do we mean by that?

42 Scheidermans mantra Scheidermans mantra for designing advanced information visualization interfaces (Shneiderman, 1996) can also be applied to map data It formulates the basic principles as: overview first, zoom and filter, then details on demand. It can be regarded as a set of minimum requirements for using visualization as a research tool. A research chain that meets the requirements of Scheidermans mantra will need some form of dynamic visualisation.

43 Incorporation of independent Support for hypotheses about the processes underlying patterns in dialect variation might be found in independent diatopical data. Shattered block (Weijnen 1977) diatopical data (1)

44 Incorporation of independent diatopical data (2) By combining different types of non linguistic diatopical data with dialect data, one is able to explore hypotheses about relations between the structures found in the data sets. The basic research chain should be extended with the ability to combine visualizations of dialect data with visualisations of independent diatopical data, for example by overlaying maps. Ideally, research into relations between such independent diatopical and dialect data already starts in the interpretation stage. Therefore, in the extended research chain (next slide) incorporation of independent diatopical data is an additional input to the geographic interpretation stage.

45 Extended research chain with support for visualization as a research tool Original unidirectional basic research chain is turned into an architecture that supports an iterative process flow that aids exploration of multiple hypotheses about the data.

46 Support for visualization as a research tool We checked to what extent tools that are already available for dialect geography research support visualization as a research tool. The two main European workbenches for dialect geography research (RuG/L04 and VDM) are very sophisticated but they did (in 2010) not offer much support for using visualization as a research tool. Geobrowsers (like Google Earth or Nasa Worldwind) do support using visualization as a research tool: They fully adhere to Shneiderman s visual informationseeking mantra. For overlaying maps with independent diatopical data, Google Earth offers easy to use built in tools.

47 Demo geobrowser (Google Earth)

48 Conclusions With dynamic visualization and the ability to incorporate independent data the role of the map changes from a static presentation of research findings into a research tool that can be used to gain new insights about (dialect geographic) data. A first step for existing tools for computational analysis of dialect geographic data towards the full extended research chain described, would be to make them more interoperable with geobrowsers.

49 CLARIN MIGMAP (Bloothooft et al): KML output

50