MediaMosa A Drupal Distribution for Digital Asset Management

Transcription

1 MediaMosa A Drupal Distribution for Digital Asset Management Michel van de Ven (uid 28913) MediaMosa Project Management Committee

2

3 MediaMosa 101 MediaMosa is built with Drupal (7) MediaMosa is a Digital Asset Management System (DAM) Media Asset Management System (MAM) Video Distribution System Document Management System Generic distributed database with metadata MediaMosa is a back-end (!)

4 MediaMosa is a DAM Storage and retrieval of any digital asset Metadata functionality for any digital asset Native functionality for specific content types: video: transcoding, transcription, players audio: transcoding, transcription, players images: simple conversions pdf: pdf2swf, viewer

5 Assets, Mediafiles and Collections An asset in MediaMosa is an object with Standardised Metadata A collection of original mediafiles with technical metadata Transcoded / derived mediafiles with technical metadata Asset Metadata techn. Metadata Original mediafile Original mediafile... techn. Metadata transcoded file transcoded file transcoded file

6 Metadata Formats Supported standards: Dublin Core: video, images, web pages, and physical resources such as books and objects like artworks Qualified Dublin Core: refined and expanded DC IEEE LOM: Learning Object Metadata MAWG: W3C Ontology for Media Resources 1.0 CZP: local Dutch standard, deprecated New metadata formats can be added

7 Authentication MediaMosa authenticates applications, not users (!) DBUS protocol used to establish trust between front-end and back-end Challenge-Response with hashing, no encrypted link necessary Handshake yields cookie that allows APIs to be called Application ID Context + CookieID + ServerChallenge Challenge + sha1(serverchallenge + Challenge + cookie) Server can compute same hash, if OK: authenticated MediaMosa

8 Authorisation Authentication & Authorisation (ACL) Domains: e.g. mediamosa.org Realms: e.g. ASelect, OpenID, DigID, &c. Users: e.g. Groups of Domains / Realms / Users Restriction on start and end times Authorisation enforced on Mediafiles, not on assets (= metadata) Play / download Front-end must restrict access to metadata Pure access control, no separate ACLs for read / write / update &c (yet)

9 Authorisation -- behind the scenes Ticket system with symlinks for playing DRM / authorisation normally done by plugins in streaming server --> Streaming server lock-in MM does not need this: streaming server independent (!) Solution using expiring symbolic links /data streaming server wowza/red5/... /media symbolic link SAN/NAS MediaMosa

10 Authorisation -- behind the scenes Expiry time of tickets: 20 minutes Permanent links vs tickets only for unprotected content (!)

11 Search Contextual Query Language (CQL) A formal language for representing queries to information retrieval systems. dc.title any fish dc.title any fish or dc.creator any sanderson dc.title any fish sortby dc.date/sort.ascending Search/Retrieve via URL (SRU/SRW) Apache SOLR used for indexing / searching

12 OAI-PMH Support for Open Archives Initiative - Protocol for Metadata Harvesting (OAI-PMH) Provider is available, a harvester is on the roadmap (was built for NIBG / Ed*it platform, but not contributed yet) Exposes all asset metadata, but not the associated mediafiles.

13 MediaMosa Features & Services Transcoding of almost any video/audio format using FFmpeg Use of predefined transcoding profiles. Support for literally hundreds of (open) formats. Distributed Job Processing Architecture Master has knowledge of slaves and #cores / slave Master distributes jobs to slaves Slave runs FFmpeg with profile for transcoding / stills Uploads: POST, PUT, FTP

14 MediaMosa Features & Services Streaming solution is not part of MediaMosa This is by design: every institution has its own requirements for this, or even an existing solution. Out-of-the box HTTP progressive download is supported, so it just works Notification infrastructure Logging capabilities Statistics gathering and reporting Unit Tests across the board

15 MediaMosa API Online Also available /admin/mediamosa/browse/restcall RESTful API: URL in, XML out JSON / Atom output is on roadmap Schema validation in MediaMosa 3.5 API cleanup in 4.0 (2012Q3)

16 MediaMosa Apps MediaMosa Site Builder (demo) Drupal 7 front-end with direct Views and Media module integration to MediaMosa REST API Content Supplier Application Drupal 6 front-end for bulk operations Administrative Back-end Administer & Configure the MediaMosa service in a Drupal-like fashion

17 Operational Runs on standard LAMP stack: GNU/Linux, Apache, MySQL, PHP. But also FreeBSD, OpenBSD, Nginx, Lighttpd, MariaDB, Varnish, memcached, MongoDB, &c. Apply existing knowledge for scaling up and out, to achieve redundancy, availability &c. In production since 2008 Scales up from a laptop with a USB memory stick, to anything.

18 Architectural Principles Open Source (GPLv2) Open Standards Open Protocols Drupal used as Web Application Framework Drupal used to build front-ends But any front-end technology is possible: Java,.NET, Blackboard, Ruby on Rails, node.js, Lisp, Android, Apple, &c as long as it can do REST.

19 Architectural Principles Service Oriented Architecture (SOA) Representational State Transfer (REST) UNIX Design Principles org/docs/artu/ch01s06.html Worse is better org/doc/worse-is-better.html Modifiable & Self Documenting: "Use the Source, Luke."

20 Open Source Project Management Committee (PMC) Current members are: Surfnet, Kennisnet, Dutch Institute of Sound and Vision, University of Amsterdam, Inuits, OneShoe. PMC decides on project direction, roadmap, events, funding. MediaMosa Foundation to be created this year. Currently using a Cathedral model, will transfer to Bazaar model as community matures

21 Users, builders & thinkers Acquia: DAM offering and video services Archipel (BE): central DAM Boekentoren (BE): central DAM Surfnet (NL): original development, surfmedia.nl & research platform Uni Twente, Uni Amsterdam, Uni Groningen, Uni Leuven, Uni Gent, Uni Oslo NIBG: Ed*It platform, ed-it.nu Kennisnet: teleblik.nl, leraar24.nl, e-tv.nl, s- tv.nl

22 Users, builders & thinkers Cineca: linking 51 Italian universities with lecture recording system (Matteo Bertazzo) University of Berkely chose not to use MediaMosa for Opencast / Matterhorn, but still uses it as an architectural reference (and MediaMosa can be used as a back-end for the ingester anyway) accessibility.nl / webrichtlijnen.nl / Bartimeus: currently looking for funding to make MediaMosa WCAG2 compliant

23 Roadmap 3.5 (Q1 2012) Transcription: transform speech into text Useful for making video archives searchable Making meeting recordings searchable SURFconext (TERENA/NREN), Open Social meets Federated Identity Management. Flexible Workflow (Drupal Rules) Support for complex players (multistream) 4.0 (Q3 2012?) xml/json/atom output and parameter cleanup Cloud functionality: more Open API support & building a cloud of MediaMosa instances MediaMosa as a DAM for OpenCast setups Archives support (checksums, BagIt, PREMIS)

24 References Canonical source code repository Online version of the RESTful API reference guide MediaMosa ticket system and quick guides MediaMosa SDK The MediaMosa Software Development Kit: Connector, testbed, debug MediaMosa CK The MediaMosa Construction Kit: Drupal 6 & 7 modules (Media 2.x, Views 3.x) MediaMosa Site Builder

25

26 Q&A