PES. TWiki at CERN Service Evolution. Platform & Engineering Services. Terje Andersen, Peter Jones for IT-PES-IS Jan 2014

Transcription

1 PES Platform & Service Evolution Terje Andersen, Peter Jones for IT-PES-IS Jan 2014 CERN IT Department CH-1211 Geneva 23 Switzerland

2 PES Agenda Introduction Overview Current use Content maintenance Issues identified with SNOW New development (LDAP integration, search) TWiki web server configuration (current and future) TWiki Software future roadmap Questions CERN IT Department CH-1211 Geneva 23 Switzerland

3 TWiki CMS, an Open Source collaboration tool Installed in 2003 (requested by Atlas software developers) Required: apache, Perl and any browser. Initially installed on AFS with kerberos.

4 Who is using TWiki LHC experiments Groups / Sections Software projects Personal websites registered users 150,000+ topics 300+ webs 800,000+ attachments The trend shows a need to keep improving the service

5 worldwide use TWiki country distribution 2012 TWiki country distribution 2013 United Kingdom Germany Other countries United Kingdom Other countries France Unknown United States Germany France United States Switzerland Unknown Switzerland GeoIP lookup DB provided by

6 What are they doing Collaboration using Blackboards Dashboards Blogs Manuals or Workbook creation and PDF generation Taking of minutes of meetings Action lists FAQs Spreadsheets Photo albums Slideshows etc

7 TWiki use example

8 Content Maintenance TWiki webs and topics need care and attention wiki gardening Organise pages appropriately (e.g. move of Atlas Computing pages) Prune back the superfluous information Reshape topics into its most useful form

9 TWiki Gardening To aid performance Deleting or archiving unused topics (being investigated) Review backend store Use %VARCACHE% for faster page rendering To help with navigation Group pages together (static or dynamic) Provide better topic names Use menus, tags or plugins To keep content current and correct it is important to undertake regular reviews

10 TWiki navigation tools %TREEVIEW{web= MyWeb topic= MyTopic }%)

11 TWiki navigation tools continued %WEBMAP{lr="1" excludesystem="1"}%

12 TWiki navigation tools continued %SEARCH{ "." scope="topic" type="regex" nonoise="on" order="modified" reverse="on" header=" *Topic* *Last Edited By* *Date* " format="$percntcalc{$if($exact($wikiname, %WIKINAME%), $topic $wikiusername $date, <nop>)}$percnt"}%

13 PES Agenda Introduction Overview Current use Content maintenance Issues identified with SNOW New development (LDAP integration, search) TWiki web server configuration (current and future) TWiki Software future roadmap Questions CERN IT Department CH-1211 Geneva 23 Switzerland

14 TWiki and SNOW Improve the service by analysing the SNOW tickets Categorize the Incidents and Requests Objectives: create modules to decrease tickets by category. Example: LDAP integration, search optimization Tickets up until Sep 2013

15 TWiki development New development (LDAP integration, search)

16 Misc development Implementing a two-step logout in TWiki. Problem: The TWiki logout mechanism does not kill SSO session by default. Consequence: CERN TWiki logout links points directly to SSO logout, not killing the TWiki session. Solution: We created CernLogoutPlugin which intercepts a logout attempt, kills the local TWiki session before redirecting to SSO logout. Kill TWiki Session Kill SSO Session Go-back link

17 Misc development Short URLs Using apache rewrite rules Can be extended to other types of actions: /e/cms/pixelofflinesoftware for edit /pdf/cms/pixelofflinesoftware for generating pdf

18 Misc development 3-way search bar Using the search bar in the top right, a CERN TWiki user can choose to search for content using TWiki search, CERN search and Google search.

19 TWiki development Incorporating LDAP in TWiki

20 Incorporating LDAP in TWiki Get rid of the TWiki registration process. Used to tie a TWikiName to a CERN identity. Most often FirstnameLastname, like JohnDoe. Solution: Ask LDAP and build it automatically. Typical entry-point; A first-time visitor to a protected topic. SSO Authentication (Automatic) LDAP-Generated TWikiRegistration TWikiName (Manual) (Automatic) The protected topic

21 Incorporating LDAP in TWiki E-group availability on all TWiki scripts. Each TWiki action have a different script. TWiki is partly open to the public; some scripts not protected. In a ACL protected topic, we must redirect to read e-groups. Solution: Ask LDAP instead of redirecting the user. /view/web/topic /viewauth/web/topic LDAP

22 Incorporating LDAP in TWiki We re using LdapContrib, a TWiki community extension Offers identity management, authentication and authorization using LDAP. Ability to contribute code back to the TWiki developer community, as well as receiving free bug-testing and improvements. We chose it for identity management and authorization, since authentication is handled by Shibboleth (SSO). LdapContrib stores LDAP data in a local cache using a BerkeleyDB v1 to avoid querying the LDAP server unnecessary.

23 Incorporating LDAP in TWiki Issues solved in LdapContrib LdapContrib uses the Perl module DB_File for database operations which does not support locking. With the use of module DB_File::Lock we are able to set read and write locks to the cache.

24 Incorporating LDAP in TWiki Issues solved in LdapContrib The default TWiki user mapping is not preserved. In our case 3000 / registered users will in our case get a new WikiName if this is automatically calculated. When solve this by implementing that all these rules are loaded immediately after the cache first is created, taking precedence. Users removed from LDAP is not preserved in the LdapContrib cache This makes it possible for TWikiNames changing hands. Solution: Rewriting the package requiring users to be manually deleted once they are put in the cache.

25 Incorporating LDAP in TWiki Better e-group support; TWikiGroups in topic ACLs will be deprecated Still supported, but not reccommended. Use TWikiUsers directly for restricting access to individuals. For example for user topics. Long-term goal: Eliminate TWikiGroups from ACLs. ACL examples: TWikiUsers TWikiGroups

26 Incorporating LDAP in TWiki Additional plug-in: LdapContribAdminPlugin Creating an admin panel for administering the LdapContrib database Allows viewing, changing and deleting user and group records, viewing statistics. Created with JQuery making AJAX calls over REST to a Perl backend, reading from and writing to the cache. Target group: CERN TWiki administrators, and the second level support team for CERN TWiki. Code will be contributed back to the TWiki developer community.

27 Incorporating LDAP in TWiki

28 Incorporating LDAP in TWiki Additional plug-in: LdapUserInfoPlugin Makes all TWiki identities in a topic dynamic By hovering over them, you get a LDAP-generated summary about the user behind the TWikiName without having to do guesswork and go to the CERN phonebook. No need to look at a TWikiName, guess the name of the user, go to phonebook and then do a search. Information shown depends on the login status and access level. Created with JQuery and AJAX Code will be contributed back to the TWiki developer community.

29 Incorporating LDAP in TWiki

30 TWiki development Deployment forecast: 1. 3-way search bar Already implemented 2. LdapContrib End of January 3. LdapContribAdmin End of January 4. LdapUserInfoPlugin End of January +~ 1 week. 5. Two-step logout End of January +~ 1 week. 6. Compact URLs End of January +~ 1 week. After successful deployment, enhancements and new plugins will be contributed back to the TWiki developer community.

31 PES Agenda Introduction Overview Current use Content maintenance Issues identified with SNOW New development (LDAP integration, search) TWiki web server configuration (current and future) TWiki Software future roadmap Questions CERN IT Department CH-1211 Geneva 23 Switzerland

32 TWiki server changes Change of architecture Move off Dell Blades to an OpenStack VM cluster. More flexible TWiki Grizzly VM SLC6 machines created and managed by puppet. Load balancing using sticky sessions enabled. (mod_proxy) Tests made with 2 front end and 3 backend m1.xlarge machines Performance issues with NFS. Better with tuned-adm profile default Current production load time less that 1 sec VM load times between seconds and more tuning necessary TWiki User Load-balancing Frontend Backend

33 TWiki Backend Store Changes Backend = storage for topics and attachments Moved. DB to DSS NetApp filers Aug 2013 TWiki servers NFS mounted NetApp Filer Storage for TWiki topics and attachments Backup and snapshots available Results are good with a better and more stable response than before Aug For disaster recovery we a read-only offsite copy prepared.

34 Wikis other future options Checkout other wiki solutions for the future. Many wikis solutions available There are 143 listed in Examples: MediaWiki: GPL, php, Mysql, nonstructured (wikipedia) DokuWiki: GPL, php, files DB (small companies) TikiWiki: LGPL, php, Mysql (similar to Drupal wiki-centric) Confluence: Atlassian, Java, SQL, linkup with JIRA and MS. Foswiki: a TWiki Fork, Perl, files, RCS. We have imported data from other wikis into TWiki

35 TWiki Fork dispute between TWiki developers. FOSWIKI - a TWiki fork was created. Timing coincided with LHC startup CERN-IT stayed with TWiki for stability Same backend for both TWIki and Foswiki Further tests to be made on Foswiki We monitor development on both versions Number of contributors who made changes to the project source code each month.

36 TWiki Future Version released late 2013 Usability Enhancements Upgrade TinyMCE WYSIWYG Copy/clone topic function Improved SEARCH variable Conditional Skin Scalability Enhancements MetadataRepository Introducing web-level administrator Improved rename/delete Fast CGI compatible 50,000 Small Businesses, many Fortune 500 Companies, and Millions of People use TWiki

37 TWiki Roadmap Schedule for changes TWiki NetApp migration - August 2013 TWiki engine upgrade to end Sep 2013 Grizzly cluster preparations- Autumn 2013 Migration to the VM cluster 1Q 2014?? Migration to LDAP 1Q 2014 Test and upgrade to TWiki 6 1H 2014 Installation and tests of Foswiki and/or other wikis Investigate and test archiving options

38 PES Agenda Introduction Overview Current use Content maintenance Issues identified with SNOW New modules (LDAP integration, search) TWiki web server configuration (current and future) TWiki Software future roadmap Questions CERN IT Department CH-1211 Geneva 23 Switzerland

39 Questions Thanks For Listening. Terje Andersen, Peter Jones IT-PES-IS