Appendix 2 Back-end database and Website developments

Transcription

1 Appendix 2 Back-end database and Website developments Key to acronyms: RISM Repertoire Internationale des Sources Musicales: Manuscripts of Polyphonic Music c.1000 to 1550 Series B IV (5 vols. + Supplement) (Henle-Verlag, Munich, 1965 ff) CCM Census Catalogue of Manuscript Sources of Polyphonic Music (5 vols.) (American Institute of Musicology, Hänssler-Verlag, Stuttgart, 1979) MOTET the Motet Database, University of Florida: AHRC Arts and Humanities Research Council (UK) CNRS Centre Nationale de la Recherche Scientifique, Tours EXPANSION OF CONTENT AND BACK-END DATABASE. The scanning of the complete RISM and CCM and subsequent splitting into incipits as well as full pages gave rise to work, much of which was not anticipated in the original bid, but which has significantly enhanced usability and the value of the DIAMM content. An unforseen consequence of this extended content has meant that the layout currently in use for selecting countries and then cities has become unwieldy, since the number of countries and cities is too large for run-on lists to work well. Likewise, the number of sources listed under any particular library can be very large, and the facility to collapse the list (and preferably to have the default view collapsed) is highly desirable. The expanded information is for the most part not available online, since it would involve development considerably beyond the scope of the current grant. However, it forms the basis for future searchability and functionality. Expansion to the back-end Filemaker database: The item-level information listed below represents a deeper stage of granularity than had been possible, but is based currently only on the information that is available from RISM. The work currently under way in the MOTET database should add to this resource considerably. a. Item information has been transcribed for the whole of RISM vol. 2 as part of a scoping process, and is well under way for RISM vol. 1. The process of linking the item information to the relevant page images is complete where DIAMM page references match those given in RISM, however, there is a significant group of images and incipits for which linking will require more time and work by an expert, since the foliation does not match. At the time of writing, the database contains 5597 incipit entries, which are keyed to source descriptions and, where possible, to images. The text incipits for 4129 voices have been transcribed. Incipits remaning untranscribed have been keyed to the appropriate folios in the manuscripts to which they refer, the genre has been entered and the type of layout (score or parts) has been recorded. Where an incipit refers to a different entry, the relationship enables the second entry to be shown within the same record to facilitate comparison and transcription. Item information recorded: i. Folio(s) on which this item appears 1

2 Appendix 2: Website developments ii. iii. iv. v. vi. vii. viii. ix. x. part layout (score or parts) number of parts clef of each part text of each part (often different in each voice) language of text voice designation (e.g. motetus, tenor, contratenor) genre or piece title (e.g. ballade, motet, Sanctus) composer (original name and variant spellings) notes on the incipit (e.g. missing parts that may be located in other sources or description of notation peculiarities) xi. reference to a secondary source where missing parts or a fuller incipit may be found (a complex lookup allows this secondary image to be found and delivered alongside the original incipit) b. a composer database has been constructed, with links between the composers and the sources (via the item database) in which they appear. Each composer entry is accompanied by information such as variant name spellings and dates of birth, it also includes a comprehensive up-to-date bibliography for each composer, drawn from the bibliography database, and linked to it by relationship. CCM lists composers in its general description, and these have been keyed to the composer database in a representative number of sources. In RISM the composers are only listed at the incipit level, so that entries are keyed to composers only where the incipit data has been transcribed. This would however allow a user to generate a list of composers for a RISM source, since the item information is keyed both to individual images and to the main source description. c. a genre database has been constructed, with links between each genre and the source (via the item database) where they appear. CCM lists genres at sourcedescription level, so entries cannot be keyed to individual images, only to whole sources. RISM does not list genres (except occasionally in passing), but this information can often be extrapolated from the incipit by examining the text. Where 2

3 the genre is known, and where the incipit is in the database, this information has been keyed to the genre database and the image concerned, and hence to the source description. It is now possible, therefore, to find incipits and page images for any of the items in b. i-xi above in any combination, or to create worklists for composers (based on the manuscripts in the database) or lists of any other searchable item. It is also possible to create highly complex search results: e.g.1: find all tenor parts in c4 clef which include the word cuer : e.g.2: find all Motetus parts in Latin, using c2 clef: e.g. 3: create a complete worklist for the composer Chipre: Other searches could enable a user to create (e.g.) a list of all motets by a particular composer; all pieces of a particular genre in a particular language or from a particular date period or manuscript source; all french motets; all alleluia settings in three parts or by a single composer or group of composers; all ballades in the Bodleian Library etc. The imagekey number relates each entry to the relevant image record. d. A bibliography database, listing all the sources cited in RISM and CCM in short form, keyed to the complete citation. This will enable us to create complete-citation bibliographies in place of the cryptic form employed by both RISM and CCM. It will be relatively easy to create a display which shows the complete form of the entry either by mouseover or by generating a non-cryptic bibliography. Each bibliographical entry has been linked to every source description where it appears, and there are additional links between the composer database and bibliographical items relevant to each composer. At present the bibliography database is in a simple form: it contains 3164 citations, and has not been marked up beyond the surname of the author(s) and year of publication (although fields are in place to allow complete bibliographical markup eventually and a live link to JSTOR copies of journal articles where they exist). It has not been proofed for style (the entries have come from a number of different sources) or OCR spelling errors, but Dr Thomas Schmidt-Beste, one of the project directors of the MOTET project has undertaken to do this work, since it would be of value to his project and would save duplication of effort. 3

4 These facilities (a-d) can be made available to online users in a future phase of development. e. Fields listing the filenames for the relevant RISM and CCM catalogue page images have been added to the source database. The online view uses this information to allow users to call up the correct page images from the original catalogues. f. The content of the database has been massively expanded beyond the original group of manuscripts (those earmarked for photography), to include a much broader range of those listed in CCM, and all those in RISM. THE ENHANCEMENT OF THE CURRENT DIAMM DELIVERY SYSTEM Provision of tools to users for annotation and comment a. Image captions on the image-viewer are now formatted as live links to library and source information (issue raised by first usability trial) b. Public comment tool: this allows users to comment on individual images and for these comments to be available to any registered user of the online image collection. Previous comments appear below the new comment, with the oldest comment therefore appearing at the bottom of the list. Users' names and the date of their comment are automatically inserted. A range of formatting commands are available as buttons. CCH looked into Wiki markup, but since a) there is no standard markup language for wiki models and b) users could not be expected to know the markup language, and therefore there would in any case have to be some descriptive material to instruct the user. It was felt that formatting buttons would be familiar for users of word-processors, and would save space and user time (and frustration), while still getting the desired results. Users may only edit their most recent entry in Public Comments and Transcriptions, since these tools are accessed by all users, unless it has been superseded by a subsequent user. It is hoped that users of this facility will be self-moderating. 4

5 Appendix 2: Website developments c. Private comment tool: this appears below the public comment tool and functions in exactly the same way (markup etc) as the public comment tool, except that the contents are only visible to the registered user who made them. Unlike the public comment tool, the contents of this box may be edited by the user repeatedly. This tool will include a 'publish' button, which will automatically copy the contents of the box to the public comment tool. d. Text transcription tool. This has been implemented in the image-viewer alongside the commenting tools. As with the comment boxes, it can be collapsed to hide it from view, and it includes the same editing tools. e. In order to manage the proliferation of tools and buttons in the viewer, a new interface was introduced that allows the user to show or hide the tools available either individually or as a whole: The tools can be collapsed vertically individually, or horizontally (see close tab to left of tools palette) to maximize the use of the imageviewer space onscreen. 5

6 Appendix 2: Website developments f. 'MyDIAMM'. Users may now create persistent use-lists linked to their login account, which appears as a new link on the navigation bar ( MyDIAMM ). See below (personalization of work environment) for more detail. g. An 'image number' and a source number have been added to the captioning of images to allow users of the commenting tools to make abbreviated references to other images or sources instead of having to give the full citation or a cryptic reference that may not be understood by other users. This reference becomes a live link to the relevant image/source description when pasted into the comment or text transcription box, thus vastly improving fast access between groups of images and sources. h. All the volumes of RISM Series B IV and CCM have now been scanned, and the page images uploaded to the DIAMM server. Links to the pages relevant to source descriptions are accessed via buttons on each source description page and the image-viewer. The page-view allows the user to page back and forth through each book beyond the limits of the article for the source they are accessing. This encourages a browsability similar to using the printed book that would otherwise be lost. i. Some minor adjustments to content have been made in response to comments in the first usability report. j. Mechanisms to allow users access to secondary images (UV, detail, watermark and enhanced versions) are in place. k. Free-text search on the source description field has recently been implemented, and will act as a scoping tool for free text search on the text transcription field: The enhancement of existing image content through digital restoration and the provision of restored images alongside the originals This has now been implemented in the image-viewer. 6

7 Appendix 2: Website developments The availability of secondary images (UV, restored, watermarks etc) is signalled by the appearance of a new tab in this display, so the alternate image tab is invisible for the image above where there are none, but appears for the image below, where both UV and digitally restored images are available: Alternate or secondary images are displayed in the same viewer, alongside the original. Each image can be zoomed and panned independently. When shown this facility, users immediately asked to have the facility to juxtapose duplicate original images of the source in this way: it may be possible to implement this within the current phase. The view below shows an enhanced image. In this case the user is offered two alternative enhancements: 7

8 Appendix 2: Website developments There is no limit to the number or type of secondary images that can be displayed through this mechanism. If the user chooses a different image it will supersede the one currently in the window: at present only two images may be juxtaposed in this window at one time because of screen-width constraints, and the first image must always be the original one. This ensures that the user does not lose touch with the original condition of the source, and cannot mistake a restored image for the original, but there is no bar to the user opening a second viewing window for the same image, to view more than one secondary image at the same time as the original. The crowding of the images apparent in this view is corrected below by clicking on the close tab icon to the left of the tools palette, making the tools disappear: 8

9 At present this has disabled some of the functionality of the image-viewer, so that the image display frame is locked at a particular size, regardless of whether the window is resized. This will be repaired in the near future to allow users to maximise the window to the full size of their screen, enabling them to view a larger portion of the image as a result. (The window has been kept small in these views to fit the width of the report page.) Metadata for each manuscript listed in the database is now available online, and appears either in the complete MS listing for a library or, in more detail and under several headings, in the source description and list of thumbnails give when a MS shelf mark is clicked at the browse archive entry point. In order to support the OCR-ed text reading, and also to allow users to access entries beyond those that had been put online, page images of the two catalogues have also been brought online: these are accessed via a link on each source description, which opens, via a link, a new window displaying the catalogue page where the entry for a particular MS begins. The facility is extended to allow supplementary entries, or entries in another catalogue for the same MS, to be accessed at the same point e.g. Bodleian Library MS. Mus. c. 60 (two entries for RISM, one for CCM. The system allow for six different catalogue entries). In order to avoid losing the serendipity of finding data accidentally when browsing hard copy data, links to page images for the catalogues are not limited to the pages 9

10 specific to the manuscript, thus the user can turn the pages back and forth beyond or before the entry s/he was originally reading. (See Previous and Next links at the top of the display example below). JPG format was tried for page display, but even at high-quality compression the loss of mid-grays made the text very hard to read, since curves and fine lines became broken up. These pages are now presented in PNG format, which is compatible with all web browsers, and uses a lossless compression, so that gray values are not lost, and the text is clearly legible. The page size currently in use in this display makes the page extend beyond the maximum window-size for some users (768 px). We have experimented with smaller sizes to see whether the text will still be acceptably readable, and the smaller size will be compensated for the ability to fit the page onto the screen, thus avoiding scrolling, but resizing to 768 px diminishes readability considerably. Because of the length of some of the metadata entries (often extending well beyond the size of the browser window), users requested that they should be able to hide the description in order to see the image thumbnails without scrolling down. This has been implemented with a toggle phrase Hide/Show source description at the top of the entry. 10

11 11

12 Users were consulted on whether the material presented was in the correct order (i.e. description first with thumbnails below) and whether the description should be collapsed (i.e. hidden) by default on accessing that page. The consensus was that because of the nature of the content and the aims of the project, the description should be presented above the thumbnails, and should be showing rather than hidden, with the option to hide available. Discussion regarding the best way of presenting often very long data fields led to the proposal that perhaps a compromise could be reached: when a user first accessed a source description, the fields and the first line (or a little more) of the content should show, but the full description could be hidden. 12

13 Feedback from users shows that most users accessing the site are spending considerably more time consulting the catalogues or the image metadata than viewing images. THE ADDITION OF TEXT TRANSCRIPTIONS FOR THE EXISTING SOURCES; The practicability of commissioning specific source studies and updates to MSS where the description is inadequate has been very carefully considered by the project directors. The main finding here is that commissioning is hampered by two factors: the very limited number of scholars of a sufficient standing who could be called upon to create this data, and the fact that all of these scholars are extremely busy and are unlikely to be prepared to undertake this work without some specific incentive. Three courses have been suggested in order to achieve the results we require: a. The appointment of an international advisory board of senior scholars who will be encouraged to input data via commenting tool, or directly to the project manager, in particular (in the first instance) to bring the bibliographical entries up to date. b. The appointment of Elizabeth Leach, medieval musicologist (one of the next generation of scholars in medieval musicology), lecturer at Royal Holloway, and Nicolas Bell, Curator of Music Collections at the British Library as Associate Directors c. Creation of one or two studentships in medieval musicology to encourage students embarking on a PhD to work in this field, thus seeding the graduate population for the future. A number of related but not specific sources of text transcriptions have been collated and are shown in Appendix 1 (excel). There are a number of printed sources for chant, one of which has been digitized, and is probably the largest single source for predigitized text from this period. However it requires a costly subscription to access, and would not deliver the data within DIAMM. Pairing up the text with the source to which it is relevant, and then correcting it to match our own sources, will require considerable semi-expert clerical time, and may not be worth the effort: this source for text would provide only one form of the text, either the original transcription (diplomatic for spelling etc) from the chant source (i.e. not the polyphonic one) OR a standardizedspelling version, neither of which may exactly match the polyphonic version. The most obvious course for frequently-used texts would be to access digital versions of the vulgate for biblical texts, and the latin mass proper for those texts, and use these for the standardized transcriptions, and create our own diplomatic transcriptions where necessary. The most important of any of the sources for texts found so far (and probably the only one that is genuinely useful to this project) is undoubtedly the MOTET database ( which has considerable overlap with DIAMM content, and in fact has used DIAMM images online for the transcription of much of the text. This is a work in progress, but we have negotiated permission to share all information created by the MOTET database and place relevant text transcriptions alongside the images. This is a key musical resource with major relevance to the work of DIAMM. We would like to explore the possibility of making a closer collaboration with this project so that content could be delivered through a single portal. 13

14 The Motet database field list indicates the complexity of dealing with text transcriptions and other elements of source transcription that have hitherto not been relevant to DIAMM, where delivery of the images has been the priority. The list does not include container fields where incipits would appear (those marked * are already in use in the DIAMM database, those marked are incorporated into the general description field if that information is given in RISM or CCM): 01. Source* 02. RISM Sigla* 03. CC Sigla* 04. Startdate* 05. Enddate* 06. Intdate* 07. Date-Comments 08. Dedicatee 09. Dedicator 10. Dedication-text 11. Liminary text 12. Scribe/Publisher 13. Editor 14. Establishment/Patron 15. Source-City* 16. Source-Country* 17. Source-Region 18. Source-Citation 19. Contributor [project-specific reference field] 20. Source-Comments 21. Source type* 22. Composer-original 22a. Conflicting attributions 23. Composer-standard* 24. Catalog designation [project-specific reference field] 25. Order in source* 26. Foliation/Pagination 27. Musical genre* 28. Text type 29. Motet-source comments 30. Motet comments 31. Edition 32. Pars* 32a. Motet incipit - Standard 33. Text Incipit-Source* 34. Text Incipit-Standard 35. Voices* 36. Complete text - Original 37. Complete text - Standard 39. Musical incipit - Digitized Although created in a relational environment, the MOTET database is flat and therefore there is considerable repetition of data, particularly in listing the source and library, and data 14

15 which should be split further, in a related table (i.e. multiple texts in one piece, and/or multiple texts (verses) underlaid in one part), is contained in a single field. This may involve some re-engineering, but it is possible that DIAMM may be able to assist in re-structuring the MOTET database in order to facilitate data sharing. In addition to transcription of text, the MOTET project is also creating new diplomatic incipit transcriptions of each musical source, using the original notation rather than standardised notation as in RISM. (In RISM a source may refer to an incipit from another source for the transcription, regardless of whether the second source uses as completely different system of notation, as long as the piece is essentially the same. This prevents users from seeing at a glance information that is provided by the varying notations, although the actual notation may be deduced from reading the text description. This is nevertheless at best a misleading way in which to represent sources.) Some of the data required by MOTET and not yet input can now be supplied by DIAMM content. Part of the sharing of linked content has involved the bibliography database created by DIAMM being offered to the MOTET project through Prof Thomas Schmidt-Beste, who has offered to use some of his clerical assistance in correcting, formatting and marking up the bibliography to create consistent presentation from the very diverse originating texts. The text transcription tool accompanies the DIAMM image display, and text transcriptions from existing sources would be added here, while additional, or new transcriptions, may be added by users. The transcription tool includes sufficient formatting facilities to allow quite complex transcription content. 15

16 THE ENHANCEMENT OF THE DELIVERY SYSTEM TO ALLOW SOME DEGREE OF PERSONALIZATION OF WORK ENVIRONMENTS, TOGETHER WITH THE IMPLEMENTATION OF AN AUTHENTICATION SYSTEM TO MANAGE THIS; a. Users logged in have access to a MyDIAMM page which is automatically created when they create image annotations. The authentication system allows for the creation of multi-user accounts for (e.g.) teaching courses), with any number of users designated. Individuals in the group can also have personal accounts, and are able to move data between the two by logging out of one account and into the other. The implementation of MyDIAMM, which enables users to create pick-lists of images is a major enhancement to image access and the creation of working lists. Personalization of the work environment is currently limited to hiding and showing data or functionality add-ons, and the view selected does not persist between images or login sessions. Comments or transcriptions do of course persist between sessions. The proposed integrated workspace would enable custom workspaces to persist between login sessions and across images. b. Limited contextual help has gone online, and is being edited and updated in response to user needs. It can be turned on or off depending on the user s requirements New tools and features have been added gradually to the site, with usability and other feedback informing improvements and further development. There is currently help given in how to use these tools. Now that the image-viewer tools are in place, the front end web pages have been re-written to flag sources where secondary images are made available, and the presence of user tools is flagged. 16