Collaborative Open Source Software Production & APIs. Tom Cramer Stanford

Transcription

1 Collaborative Open Source Software Production & APIs Tom Cramer Stanford

2 Agenda Flavors of Open Source Software 3 Case Studies in Collaborative OSS The Power of APIs What about Web Archiving?

3 Different Models of Software Production regardless of the license 1.) Sole Source single developer 2.) Closed Source team or corporate dev; closed to add l contribs 3.) Club Source pooled resources commission solo/team/corporate dev 4.) Community Source community-based dev; direct & distributed participation 5.) Collaborative Open Source grassroots, democratic, meritocratic participation

4

5 What is Blacklight? Open source discovery application Search & faceted browsing Hundreds of adopters worldwide Works for many, many use cases Catalogs, digital repositories, digital collections, archives, media, etc.

6 Digital Collections General Purpose Discovery Plus Plugins Digital Repositories Blacklight MARC for Catalogs GeoBlacklight for spatial assets Spotlight for Exhibits

7 Blacklight Development

8 If you want to go far, go together.

9 A Digital Repository for All Types of Content ETDs (Theses) Books, Articles Images Audio- Visual Research Data Maps & GIS Documents Scalable, Robust, Shared Management and Preservation Services

10 One Body, Many Heads ETDs (Theses) Books, Articles Images Audio- Visual Research Data Maps & GIS Documents Scalable, Robust, Shared Management and Preservation Services

11 Hydra Partners and Known Users OR09 OR10 OR11 OR12 OR13 OR14 Now OR = Open Repositories Conference

12 Hydra UCSD, 2012 University of Virginia, 2008 LibDevConX^3, Stanford, 2012: (Hydranauts and Fellow Travelers)

13 First Worldwide Hydra UCSD, January 2014

14 This is not a conference it s a Connect. 2 nd Worldwide Hydra Cleveland, OH, September 2014

15 Hydra Partners are individuals, institutions, corporations or other groups that have committed to contributing to the Hydra community; they not only use the Hydra technical framework, but also add to it in at least one of many ways: code, analysis, design, support, funding, or other resources. Hydra Partners collectively advance the project and the community for the benefit of all participants.

16 Trending

17 Sustainability No animals were harmed in the making of this film.

18 Becoming a Community Hub

19

20 Fedora What is it? Flexible, Extensible, Durable, Object Repository Architecture Open source digital repository middleware for relating your objects and hooking them to services & storage More than 300 adopters worldwide 4 major software releases since 2000

21 2012: Growing Issues with Fedora 3 Technical Performance Fault tolerance Scalability Complexity Aging code base Organizational Small, fixed pool of developers Demands outstrip applied resources Lack of vibrancy

22 New Use Cases Research data management Traditional repositories Digital collections Heterogeneous content (especially media) Works for sites large to small

23 2013: Fedora 4.0 Project Launch! Preserve strengths of the architecture and community Address needs for robust and full-featured repository services Provide a platform in the repository ecosystem for the next 5-10 years

24 Annual $ Resourcing & Funding 63 Fedora members (up from 41 in 2013) 22 active developers (up from 3 in 2013) 28 leadership group members 925 fedora-community mailing list subscribers 350% increase in member-based funding 600, , , , , ,000 0 Fedora 4 init launched

25 How?

26 Collaborative Open Source Software Open source fundamentals Transparency, Inclusivity, Merit, Agility, Quality, Value Alignment of purpose, assumptions (both architecture & services), resources Build the community first; the code will come Community over code Agree on common APIs, shared architecture

27 The Power of APIs Componentize the problem: carve up the elephant Agree on standard interactions Interoperable, swappable components across platform institutions place time

28 (Sketchy) Web Archiving APIs Capture QA Store Extract Export/Import Index Search Preservation Playback Mining Monitoring Reporting

29 IIPC as a Community

30 IIPC as a Community List Number of Members Members 330 Access WG 93 Preservation WG 68 Harvesting WG 52 Collections WG 14 Steering 24 Curators (open list) 393 Open Wayback Contributors per Month

31 IIPC Software Acquisition ArchiveFacebook Heritrix HTTrack SiteStory WARCreate Warrick Wget Curator Tools Building Collections on the Web (BCWeb) CINCH NetarchiveSuite Web Curator Tool (WCT) Collection storage and maintenance HTTrack2ARC JHOVE2 MediaWiki Memento Extension SiteStory Web Archive Transformation (WAT) WarcManager WARC Tools Access and finding aids Memento Time Travel NutchWAX (Nutch with Web Archive extensions) WERA (WEb archive Access Wayback Machine Xinq (XML INQuire)

32 One of the main objectives of the IIPC has been to develop a set of high-quality, easy-to-use open source tools for setting up a web archiving chain. - Is this still accurate? How are we doing against this goal? Is a particular model of OSS production (closed, club, community, collaborative) best for IIPC? Would a well-codified set of reference APIs accelerate this? What else can/should we do?