CITI PROGRAM GCP TRAINING PROGRAM AND CITI PROGRAM LMS

Transcription

1 CITI PROGRAM GCP TRAINING PROGRAM AND CITI PROGRAM LMS Paul Braunschweiger Ph.D. Professor, Radiation Oncology CoDirector, Ethics Component, University of Miami CTSI CoFounder & CEO, CITI Program at the University of Miami University of Miami Miller School of Medicine Enhancing Clinical Research Professionals Training & Qualifica8on Project

2 What is The CITI Program v CITI Program is subscrip8on based, online, research ethics educa8on program >2,100 par8cipa8ng ins8tu8ons worldwide n >72,000 learners per month complete a CITI Course September 2014, 124,064 people /170,110 courses n 58% are first 8me learners to CITI n 65-70% are students and fellows n GCP Course 64% clinical, 30% students v Subscriber Profile Universi8es, colleges, medical centers, community hospitals, socie8es, government, networks (e.g., N2, HANC), commercial IRBs, industry, and interna8onal organiza8ons v Courses in all areas of research ethics Good Clinical Prac8ce

3 CITI Program GCP Course v v CITI ICH GCP Course is basic inves8gator GCP training CITI Basic Course in HSR + CITI GCP course provide a solid founda8on for a programma8c approach to clinical research ethics training

4 CITI Program GCP Course Stats v ~7,100 CITI GCP courses completed per month v 893 (43%) Organiza8ons use a CITI GCP Course v >65,300 CITI GCP courses completed 2014 YTD v All CTSIs use the CITI Program to meet Federal and ins8tu8onal HSR training mandates All CTSI have access to CITI GCP courses 56 CTSIs currently use the CITI Program GCP course

5 Top 20 CITI GCP Course Users 2014 YTD R # Organiza8on Quorum Review WIRB/CG UT Southwestern Medical Center Icahn School of Medicine at Mount Sinai Greater Cincinna8 Academic and Regional Health Centers Thomas Jefferson University The Society for Clinical Research Sites University of California, Davis University of California, San Francisco Sarah Cannon Research Ins8tute R # Organiza8on University of Southern California Emory University University of Pennsylvania Schulman Associates IRB, Inc St. Michael's Hospital University of North Carolina at Chapel Hill Department of The Navy Drexel University University of Pilsburgh University of Wisconsin - Madison

6 Top 20 CITI GCP Course Users 2014 YTD R # Organiza8on Quorum Review WIRB/Copernicus Group UT Southwestern Medical Center Icahn School of Medicine at Mount Sinai Greater Cincinna8 Academic and Regional Health Centers Thomas Jefferson University The Society for Clinical Research Sites University of California, Davis University of California, San Francisco Sarah Cannon Research Ins8tute R # Organiza8on University of Southern California Emory University University of Pennsylvania Schulman Associates IRB, Inc St. Michael's Hospital University of North Carolina at Chapel Hill Department of The Navy Drexel University University of Pilsburgh University of Wisconsin - Madison 12 of top 20 are CTSIs 3/20 Commercial IRBs, CRO Government Agency

7 Talking Points CITI Program Research Ethics Educa=on I. CITI GCP Training Program Characteris=cs II. CITI Program Learner Experience III. CITI Program Ins=tu=onal U=lity IV. CITI 2.0 Learning Management System Q&A

8 I. Training Program Characteris=cs A. Program format B. Course development C. Competency based learning assessment D. Module focus

9 I. Training Program Characteris=cs A. Program Format v CITI GCP courses are text based learning experiences delivered on the CITI 2.0 LMS v Balanced presenta8on including use of bullets and lists for easy reading and comprehension v Case studies, have recently been added to GCP modules v Quizzes v Links to regulatory and guidance documents as well as other resources

10 Example

11 Module Example Audits and inspec8ons

12 Example module Audits and inspec8ons

13 I. Training Program Characteris=cs B. CITI Program Course Development v Course development CITI Program Advisory Commilee (PAC) 12 members " Course/Module development proposal submiled " PAC reviews and approves development " Program Director recruits expert consultants to develop the course/module

14 I. Training Program Characteris=cs B. Course Development v Peer Review Process 2-3 experts recruited by Program Director Consensus is sought Copy editor Author approves changes Web designer posts materials Author and CITI staff approve Roll out

15 I. Training Program Characteris=cs B. Professional development v Professional enhancement Dr. Arango member of the CTTI GCP Working Group CITI ICH/GCP course meets the minimum TransCelerate criteria for GCP training for site personnel CITI content staff par8cipate in professional conferences PRIMR what s New at the CITI Program

16 I. Training Program Characteris=cs C. Competency- based learner assessment v Post module quizzes associated with each GCP module Quizzes considered a teaching exercise Modules are designed to require about minutes for the average reader Quizzes require 4-5 minutes Links for feedback to the developers

17 I. Training Program Characteris=cs C. Competency- based learner assessment v The Quizzes Quizzes based on the L.O.s Correct responses and explanatory feedback are provided aqer submission of each quiz Ins8tu8ons set the bar, 75% to 100% correct Learners may return to the module to review materials and retake the quiz to alain a higher score

18 I. Training Program Characteris=cs Quiz Ques=ons

19 I. Training Program Characteris=cs Quiz feedback

20 I. Training Program Characteris=cs D. Module focus - 3 courses 1. GCP Course for Trials With Inves=ga=onal Drugs and Biologics (ICH focus) n Meets TransCelerate Group requirements n English, Spanish & Portuguese 2. CITI Good Clinical Prac=ce Course (US FDA focus) n English, Spanish, Portuguese, Chinese, Thai, Khmer, Vietnamese 3. GCP Course for Clinical Trials Involving Inves=ga=onal Medical Devices 4. Interna=onal Courses n Canada GCP, CITI Canada (meets TransCelerate criteria) n India GCP, CITI India

21 II. Learner Experience A. Usability B. Accessibility C. User Support

22 II. Learner Experience A. Usability

23

24 II. Learner Experience B. Availability and accessibility Available world wide. 24/7/365 Browser Frequency Browser Chrome MSIE Safari Firefox Android Opera Silk others Total Usage 37.8% 27.8% 17.7% 16.1% 0.3% 0.1% 0.1% 0.1% 100.0%

25 II. Learner Experience C. Learner support v Excellence in technical support through the CITI Help Desk Learner support via . Chat line, phone calls Ins8tu8onal Coordinator support to: n Choose and set up curriculum n Modify curriculum\ add new courses to ins8tu8onal menu n Provide specialty reports

26 II. Learner Experience. C. Learner support - CME v CME and CEU available for all GCP courses v U of Miami Div. of CME Physicians, Cat 1 Nurses Psychologists

27 II. Learner Experience D. Learner Feedback to The Developers v Mul8ple opportuni8es for the learner v Links available to submit a comment: Aqer comple8ng a specific module Aqer comple8ng a course Via helpdesk, , phone, and chat sessions

28 II. Learner Experience D. Learner Feedback to The Developers v Course Learner Sa8sfac8on Survey Voluntary, Anonymous, Lime Survey tool, RR=~30% Pre- course Survey n Experience in HSR n Expecta8ons Post- course Survey n Follow up on expecta8ons n Naviga8on issues n U8lity of the course to learner n Value of the course to the learner

29 Value Example: Rate the CITI GCP Course (9 months,2014; n=19,317; RR =~ 33%) 35% Frequency of Response 30% 25% 20% 15% 10% 5% 0% 1% 1% 2% 2% 10% 10% 23% 29% 23% Poor 75% Outstanding

30 III. Ins=tu=onal U=lity A. Costs B. Ins8tu8onal Usability C. Customizability D. No8fica8ons University of Miami

31 III. Ins=tu=onal U=lity A. Cost v No Extra Cost for GCP if organiza8on is a CITI subscribing member organiza8on All CTSI organiza8ons are CITI subscribers v $3,000 annual subscrip8on, unlimited use basic package Includes: HSR, GCP, CoI, RCR, ACU v Course / module selec8ons and setup guided by CITI HD Tech

32 III. Ins=tu=onal U=lity B. Usability v CITI Integrates with local systems/databases for administra8ve/irb func8ons reducing administra8ve burden Incorporates training status into Electronic Protocol Submission and Tracking systems (e.g., Click Commerce) v SSO access available via Shibboleth (In Commons Ini)a)ve) v Manual or automa8c download of stored training data Customizable formats Available 24/7 v All organiza8ons have someone responsible for compliance training and CITI data review

33 III. Ins=tu=onal U=lity C. Customizability v All CITI courses are modular for customiza=on. Mandatory, elec8ve, and op8onal module selec8on op8ons All GCP course modules recommended as mandatory Seamless blending of organiza8on- specific modules to address: n Organiza8onal policies and procedures n Country- specific requirements (e.g., Canada, India) Custom training reports to the ins8tu8on n Customize data downloaded n Manual or automa8c downloads n SSO implementa8on

34 III. Ins=tu=onal U=lity D. No=fica=ons v CITI soqware sends training reports to: Learner n Automated and Manual Compliance / Training coordinators n Automated and Manual Ins8tu8onal training databases n Manual and automated downloads (automated queries, SSO) n Can be modified according to Ins8tu8onal policy/needs n 24/7/365

35 III. Ins=tu=onal U=lity D. No=fica=ons v Ins8tu8onal no8fica8ons updates to coordinators n New courses, updates, modifica8ons, new CITI procedures as they occur Conference exhibits and presenta8ons e.g., PRIMR What s New at the CITI Program Mass s to subsets of CITI learners n Ins8tu8on subsets and course groups (e.g. GCP) Quarterly CITI Newsleler n Ins8tu8onal Coordinators n CITI Program Homepage

36 IV. IT Pla`orm A. IT Overview B. Capacity C. Program improvements D. Security E. System safeguards

37 IV. IT Pla`orm A. IT Overview v CITI IT development group Dir. IT Development 1 Senior Systems Analyst 2 Senior systems programmers 2 web designers Contractors (e.g, M&K Tech Group) 10 help desk techs v Custom Playorm Soqware CRADA with the VA Easily modified Compa8ble with many presenta8on modes

38 IV. IT Pla`orm A. IT Overview Availability & Reliability v Servers located at University of Miami Datacenter and Commercial Datacenter (off site) share hos8ng environments v Intensive and extensive University network security (Firewalls, Intrusion Detec8on and Preven8on Systems IDPs) v Courses are available 24/7/365 v High capacity bandwidth Internet 2 and LambdaRail v Redundancy in all components

39 Production Web Server 1 Production Web Server 2 Production Web Server 3 Production Web Server 4 Reporting / Survey Server 1 Reporting / Survey Server 2 Prod Switch 1 Prod Switch 2 Prod SQL DB Server 1 Prod SQL DB Server 2 Storage Area Network (current database) INTERNET CITI Server configura=on at U of Miami Staging Web Server 1 Develop Web Server 1 Failover Web Server 1 Failover Web Server 2 Failover Web Server 3 Staging SQL Server 1 (recent database copy) Development-SQL Server 1 (recent database copy) Failover SQL Server 1 (sync d database copy) Failover Web Server 4

40 IT Overview B. Capacity Capacity More than 1.5 million courses per year At least 124,000 Learners per month At least 174,000 courses per month (9-14) Capacity for > 2,000 concurrent users Upgrades pending to boost >~3,000 n Current usage varies, max. about 1,200 to date

41 IT Overview C. Code upgrades and improvements v Ongoing ac8vity Make code more efficient and secure v SoP for development, tes8ng, implementa8on, and go- live E.G., New CME code in Aug 2014 n Makes obtaining CEU/CME much easier for learner n Much more efficient, more secure, and easier to maintain v Current CITI ver 2.0, August 2013 v Currently planning ver. 3.0 August 2016

42 IT Pla`orm D. Security and Availability v Who has DB access? Role based access to CITI admin components Role based access for ins8tu8ons to their site specific data Periodic internal and external audits of code Problem, Change and Project management in accord with the ITIL standards Log all user events (page views, quiz comple8ons) Forensics to assess fraudulent use when required

43 IV. IT Pla`orm D. Security and Availability v Extensive monitoring soqware for all aspects of opera8ons v Redundant components in all elements of system architecture v Excess produc8on capacity to meet peak loads v Separate development, tes8ng, produc8on and disaster recovery environments v Offsite backups for data archiving

44 Summary CITI Program at the University of Miami v 14 years delivering web based research ethics training to the research community v Academic NFP program v 3 GCP courses & interna8onal device course v Text and cases v Wrilen, peer reviewed and updated by recognized experts in GCP v High learner sa8sfac8on with the CITI GCP courses

45 Summary CITI Program at the University of Miami v ALL CTSIs have access to CITI Program Lille or no addi8onal cost to NCATS v Easy for learners to access and use v CITI playorm designed with the needs of the learner and coordinators in mind v Outstanding HD customer service and high service level Queries and reports easy to obtain as needed Sufficiently robust to accommodate >2,000 concurrent learners Program is scalable Addi8onal capacity easily added as needed

46 CITI Program At the University of Miami GCP TRAINING PROGRAM AND CITI PROGRAM LMS ACKNOWLEDGEMENTS

47 CITI GCP Authors and Peer Reviewers Name Degree GCP Years Affiliation GCP Consultant Authors Deborah Rosenbaum MS Clinical Research Consultant, Sarrison Clin. Res., Chair ACRP Academy Board of Trustees Cynthia Gates J.D. CIP Western IRB., Currently at UC Davis. GCP Peer Reviewers Karen Hansen BA IRB Director, The Fred Hutchison Cancer Center Paul Braunschweiger Ph.D CITI Program at the University of Miami Sally Mann MS IRB Director at University of Nebraska Medical Center Allison Handler MS Duke University Sue Avery MS Duke University Gary Chadwick Pharm. D University of Rochester Gwenn Oki MPH City of Hope, Van Andel Institute David Forester JD Western IRB Melody Lin Ph.D Deputy Director at OHRP, International Studies Ann Taylor BS 2011 Pfizer at the time Mary Jane Clark BS 2011 Pfizer at the time Chiu Lin Ph.D Lin & Associates, previously with FDA Holly Scott BA 2012 Currently with Biologics, formerly with the FDA Susan Miller MD The Methodist Hospital, Houston Diane Paul MSRN NCCN Jaime Arango Ed.D CITI Program at the University of Miami

48 Ques=ons and Answers Sugges=ons v List of GCP course module 8tles v New CITI Program courses under development CRC v CITI Program org. chart

49 CITI PROGRAM GCP TRAINING PROGRAM AND CITI PROGRAM LMS Paul Braunschweiger Ph.D. Professor, Radiation Oncology CoDirector, Ethics Component, University of Miami CTSI CoFounder & CEO, CITI Program at the University of Miami University of Miami Miller School of Medicine Enhancing Clinical Research Professionals Training & Qualifica8on Project

50 COURSE MODULES

51 GCP Course for Trials With Inves=ga=onal Drugs (interna=onal/ ICH focus). TransCellerate approved 1. The CITI Good Clinical Prac8ce Course for Clinical Trials Involving Drugs and Biologics 2. Conduc8ng Inves8gator- Ini8ated Studies According to FDA Regula8ons and GCP 3. Inves8gator Obliga8ons in FDA- Regulated Research 4. Managing Inves8ga8onal Agents According to GCP Requirements 5. Informed Consent in Clinical Trials of Drugs and Biologics

52 GCP Course for Trials With Inves=ga=onal Drugs (interna=onal/ ICH focus). TransCellerate approved 6. Monitoring Clinical Trials of Drugs by Industry Sponsors 7. Audits and Inspec8ons of Clinical Trials of Drugs and Biologics 8. Overview of New Drug Development 9. Overview of ICH GCP 10. Detec8ng and Evalua8ng Adverse Events

53 CITI ORGANIZATIONAL CHART

54 CITI Program Org. Chart University of Miami Office of the Provost UM Innovation EAB Dir. IT Dev. Web design CITI CEO PAC v v Assoc. Dir. for Program/Chair PAC Dir. interna8onal programs

55 NEW CITI COURSES COMING SOON

56 New CITI Program Courses Currently Under Development v GCP refresher v CRC Course v Phase 1 trials modules for GCP and HSR courses v The 3 modules for Community Based Par8cipatory Research/Community Engagement v GLP* v HECM* * For individual learners

57 CITI PROGRAM INTERNATIONAL DEVICE COURSE

58 GCP Course for Clinical Trials Involving Inves=ga=onal Medical Devices Interna=onal focus. 1. The CITI Good Clinical Prac8ce Course for Clinical Trials Involving Inves8ga8onal Medical Devices 2. Conduc8ng Inves8gator- Ini8ated Clinical Trials of Devices 3. Inves8gator Obliga8ons in FDA- Regulated Clinical Research of Devices 4. Managing Inves8ga8onal Devices According to GCP Requirements 5. Overview of U.S. FDA Regula8ons for Medical Devices 6. Informed Consent in Clinical Trials of Devices

59 GCP Course for Clinical Trials Involving Inves=ga=onal Medical Devices Interna=onal focus. 7. Monitoring Clinical Trials of Devices 8. Audits and Inspec8ons of Clinical Trials of Devices 9. Repor8ng Requirements for Device Studies 10. Comple8ng the CITI GCP Course 11. Repor8ng Serious Adverse Events in Inves8ga8ons of Drugs and Biologics 12. ICH Comparison Between ICH GCP E6 and U.S. FDA Regula8ons 13. Comple8ng the CITI GCP Course

60 The CITI Good Clinical Prac=ce Course for Clinical Trials Involving Drugs and Devices FDA Focus 1. Overview of New Drug Development 2. Overview of ICH GCP 3. ICH Comparison Between ICH GCP E6 and U.S. FDA Regula8ons 4. Conduc8ng Inves8gator- Ini8ated Studies According to FDA Regula8ons and GCP 5. Inves8gator Obliga8ons in FDA- Regulated Clinical Research 6. Managing Inves8ga8onal Agents According to GCP Requirements

61 MORE SURVEY DATA

62 Pre- Course Survey Data How do you feel about comple8ng this GCP course on line? n=1,080 85% 10% 5% Great! This permits me greater freedom to allocate my time wisely. I really don t know, this is my first experience with web based training and education. Not too good! I don t like web based learning. I would much rather spend the time (~ up to 4 or 5 hrs) in a traditional class room environment with an instructor.

63 Pre- Course Survey Data Please tell us about your background in GCP n % 30% 19% 1% The topic in the course I am about to take is all new to me I have some understanding of the topic but, it is rather superficial Although I have a good understanding of the topic, it is complex. I expect that this course will be a good review for me. I consider myself to be an expert in this topic. Comple8ng this course will likely be a waste of my 8me

64 Please tell the CITI developers about your learning experience in the GCP course that you just completed. N=2,511 49% 21% 21% 8% 1% Most of the modules that I read, enhanced my knowledge of the course topic. I knew a great deal about the GCP. However, this course was a valuable review for me. It was all new to me, every module that I read, enhanced my knowledge of the GCP. Only a few of the modules enhanced my knowledge of GCP. I learned nothing from this GCP course. It was a complete waste of my 8me.