U. S. Department of Energy Consolidated Audit Program Checklist 5 Laboratory Information Management Systems Electronic Data Management Revision 4.0 February 2014 Use of this DOECAP checklist is authorized only if the user has satisfied the copyright restrictions associated with TNI-EL-V1-2009 and ISO 17025:2005. DOECAP does not control or restrict the use of copyrighted standards that have been incorporated into this checklist; however, TNI and ISO do restrict use of their standards. Audit ID: Date:
Effective Date: February 2014 Page 1 of 15 Areas of Review During Audit Personnel Hardware LIMS Data Facilities Software Complaints Security Key: A = Acceptable U = Unacceptable NA = Not Applicable NO = Not Observed F = Finding O = Observation Referenced regulations are accessible at the following URLs: NOTE: https://doecap.oro.doe.gov/ When audit findings are written against site-specific documents (i.e., SOPs, QA Plans, licenses, permits, etc.), a copy of the pertinent requirement text from that document must be attached to this checklist for retention in DOECAP files. Fully document any deviation from the LOI or the requirements of the QSM. Refer to Page 15 for the record of revision.
Effective Date: February 2014 Page 2 of 15 1.0 Personnel 1.1 Do the LIMS and electronic data management support staff and users have adequate education, training and experience to perform the assigned LIMS functions? QSM, Rev. 5.0, Module 2, Section 4.2.3, a), ISO 17025. Clause 4.2.3, EPA 2185 GALP, Section 8.2.1, pg. 1-9 1.2 Has the technical staff demonstrated capability in the activities for which they are responsible? QSM Rev. 5.0, Module 2, Section 4.2.3, b, ISO 17025, Clause 4.2.3 1.3 Is the demonstration of capability for technical staff recorded? QSM Rev. 5.0, Module 2, Section 4.2.3, b, ISO 17025, Clause 4.2.3 1.4 Is the training for each member of the technical staff kept up-to-date (on-going)? QSM Rev. 5.0, Module 2, Section 4.2.3, c, ISO 17025, Clause 4.2.3 1.5 Does the training file for each employee contain a certification that the employee has read, understands and is using the latest version of the management system records relating to his/her job responsibilities? QSM Rev. 5.0, Module 2, Section 4.2.3, c i, ISO 17045, Clause 4.2.3 1.6 Are the QA personnel entirely separate from and independent of the LIMS personnel? ISO/IEC 17025, 4.1.5 I0, EPA 2185 GALP, Section 8.3.1 1.7 Do the QA personnel report directly to laboratory management? ISO/IEC 17025, 4.1.5 I0, EPA 2185 GALP, Section 8.3.1 1.8 Does the laboratory have a procedure to ensure individual user names and passwords are required for all LIMS users and that those passwords are changed at least once per year? QSM Rev.5.0, Module 2, Section 5.4.7.2, d), ISO 17025, Clauses 5.4.7.2, a - c
Effective Date: February 2014 Page 3 of 15 2.0 LIMS Data 2.1 Are periodic inspections (at least annually) of the LIMS operations performed by the QA unit to ensure the integrity of LIMS data? QSM Rev. 5.0, Module 2, Section 5.4.7.2; f, ISO 17025 Clauses 5.4.7.2, a - c 2.2 Does the QA unit maintain records of inspections and does QA submit reports to laboratory management noting any problems identified with LIMS data processing and stating the corrective actions taken? QSM Rev. 5.0, Module 2, Section 5.4.7.2; f, ISO 17025 Clauses 5.4.7.2, a - c 2.3 Does an SOP exist for the manual entry of raw data from analytical measurements when there is not a direct interface to the LIMS, e.g., double key entry, single entry with secondary review, etc.? ISO/IEC 17025, 5.4.7.1 See Checklist 1, LOI 19.11 2.4 Does an SOP exist for making changes to electronic data? QSM Rev.5.0, Module 2, Section 4.2.8.4, v.; ISO 17025 Clauses 5.4.7.2, a c, EPA 2185, GA:P, Section 8.4.5 See Checklist 1, LOI 19.11 2.5 Does an SOP exist for how electronic data are processed, maintained, and reported by the LIMS? QSM Rev. 5.0, Module 2, Section 4.2.8.4, w 2.6 Does an SOP exist for the retention of electronic data, documentation, and records pertaining to the LIMS? QSM Rev.5.0, Module 2, Section 5.4.7.2, i) v) See Checklist 1, LOI 19.11 2.7 Are the individual(s) responsible for entering and recording LIMS raw data uniquely identified when the data are recorded? EPA 2185 GALP, Section 8.4.2
Effective Date: February 2014 Page 4 of 15 2.8 Is the instrument transmitting LIMS raw data uniquely identified when the data are recorded? EPA 2185 GALP, Section 8.4.3, pg. 1-11 See Checklist 1, LOI 19.3 2.9 Are the time(s) and date(s) documented? EPA 2185 GALP, Section 8.4.3, pg. 1-11 See Checklist 1, LOI 19.4 2.10 Are the procedures and practices for making changes to LIMS raw data documented and does the documentation provide evidence of the change and preserve the original recorded documentation (see 2.8 and 2.9)? Documentation is dated? Documentation indicates the reason for the change? Documentation identifies the person who made the change if different? Documentation identifies the person who authorized the change? QSM Rev. 5.0, Module 2, Section 4.2.8.4, v, EPA 2185 GALP, Section 8.4.5 See Checklist 1, LOI 19.5
Effective Date: February 2014 Page 5 of 15 3.0 Software 3.1 Does an SOP exist for software development methodologies that are based on the size and nature of the software being developed? QSM Rev. 5.0, Module 2, Section 5.4.7.2, i) i) 3.2 Does an SOP exist for testing and QA methods to ensure that all LIMS software accurately performs its intended functions? Does the SOP include: acceptance criteria; tests to be used; personnel responsible for conducting the tests; records of test results; frequency of continuing verification of the software, and, test review and approvals? QSM Rev. 5.0, Module 2, Section 5.4.7.2, i) ii) 3.3 Does an SOP exist for software change control methods that includes instructions for requesting, authorizing, requirements to be met by the software change, testing, QC, approving, implementing changes, and establishing priority of change requests? QSM Rev. 5.0, Module 2, Section 5.4.7.2, i) iii) 3.4 Does an SOP for software version control methods exist that documents the LIMS software version currently used? QSM Rev. 5.0;Module 2, Section 5.4.7.2, i)vi) 3.5 Are data sets documented with the date and time of generation and/or the LIMS software version used to generate the data set? QSM Rev. 5.0; Section 5.4.7.2, i)vi) 3.6 Does an SOP exist for maintaining a historical file of software, software operating procedures, software changes, and software version numbers? QSM Rev. 5.0, Module 2, Section 5.4.7.2, i) v)
Effective Date: February 2014 Page 6 of 15 3.7 Are records available in the laboratory to demonstrate the validity of laboratory generated software? QSM Rev. 5.0, Section 5.4.7.2, j) 3.8 Does the facility Software Change Control documentation identify: persons requesting and authorizing software changes? requirements to be met by the change? measures for testing and QA? approving changes? implementing changes?; establishing priority of change requests? QSM Rev. 5.0, Module 2, Section 5.4.7.2, i) iii) See Checklist 1, LOI 19.6 3.9 Are records available to demonstrate the validity of laboratory generated software? Do the records include: software description and functional requirements? listing of algorithms and formulas? testing and QA records? and installation, operation, and maintenance records? QSM Rev. 5.0, Module 2, Section 5.4.7.2, j) 3.10 Do software historical files of all versions of software programs exist and include dates that software was placed into and removed from production? QSM Rev. 5.0, Module 2, Section 5.4.7.2, i) v) 3.11 Are the equations used in spreadsheets verified before initial use and after any changes to the equations or formulas? QSM Rev. 5.0, Module 2, Section 5.4.7.2, h) 3.12 Are software revision updates, and records available for review? QSM Rev. 5.0, Module 2, Section 5.4.7.2, h)
Effective Date: February 2014 Page 7 of 15 3.13 Are formula cells write-protected to minimize inadvertent changes to the formulas? QSM Rev. 5.0, Module 2, Section 5.4.7.2, h) 3.14 Do printouts from any spreadsheets include all information used to calculate the data? QSM Rev. 5.0, Module 2, Section 5.4.7.2, h)
Effective Date: February 2014 Page 8 of 15 4.0 Security 4.1 Upon employment, do employees receive initial training in computer security awareness and have ongoing refresher training on an annual basis? QSM Rev. 5.0, Module 2, Section 5.4.7.2, e; k) iii) See Checklist 1, LOI 19.10 4.2 Is the documentation of this training maintained and available for review? QSM Rev. 5.0, Module 2, Section 5.4.7.2, e; k) iii) See Checklist 1, LOI 19.10 4.3 Are the operating system privileges and file access safeguards implemented to restrict the use of LIMS data to users with authorized access? QSM Rev. 5.0, Module 2, Section 5.4.7.2, d, k) ii) See Checklist 1, LOI 19.7 4.4 Are system events, such as log-on failures or break-in attempts monitored? QSM Rev. 5.0, Module 2, Section 5.4.7.2, k) iv) 4.5 Is the electronic data management system protected from the introduction of computer viruses? QSM Rev. 5.0, Module 2, Section 5.4.7.2, k) v) 4.6 Do emergency, backup, disaster recovery, and contingency plans exist for the LIMS? EPA 2185 GALP, Section 8.6 Security 4.7 Do system backups occur on a regular and published schedule and can the system backups be performed by more than one person within the organization? QSM Rev. 5.0, Module 2, Section 5.4.7.2, k) vi), EPA 2185 GALP, Section 8.6, Security, Section V. Risk Management, pg. 2-84 2-85 See Checklist 1, LOI 19.1
Effective Date: February 2014 Page 9 of 15 4.8 Are tests of the system backups performed and recorded to demonstrate that the backup systems contain all required data? QSM Rev. 5.0, Module 2, Section 5.4.7.2, k) vii) See Checklist 1, LOI 19.2 4.9 Is the physical access to the servers limited by security measures such as locating the system within a secured facility or room, and/or utilizing cipher locks or key cards? QSM Rev. 5.0, Module 2, Section 5.4.7.2, k) viii) 4.10 Are fire extinguishers that are designed to avoid damage to computer equipment available and mounted in visible, accessible areas? EPA 2185 GALP, Section 8.6 Security, Section VI. Minimum Safeguards by Asset, Section C. Data Center Computing. 3. Physical and Environmental Safeguards, pg. 2-96 See Checklist 1, LOI 19.12
Effective Date: February 2014 Page 10 of 15 5.0 Hardware 5.1 Is a description of the LIMS design and capacity documented and maintained? QSM Rev. 5.0, Module 2, Section 5.4.7.2, j) i), EPA 2185 GALP, Section 8.7.1 5.2 Is an SOP established and maintained that defines the acceptance criteria, testing, documentation, and approval required for changes to the LIMS hardware and communications components? QSM, Rev. 5.0, Module 2, Section 4.2.8.5, xxv) & 5.4.7.2, i) vi), EPA 2185 GALP, Section 8.7.2 5.3 Is the documentation of the regularly scheduled maintenance for LIMS hardware and communications components maintained and does it include: a descriptions of operations performed? the names of the persons who conducted them? the dates operations were performed? the results? EPA 2185 GALP, Section 8.7.3 5.4 Does the documentation of non-routine maintenance include: a description of the problem? a corrective action? the acceptance testing criteria? the testing that was performed to ensure the LIMS hardware and communications components have been adequately repaired? EPA 2185 GALP, Section 8.7.3 5.5 Do SOPs exist for routine operations? EPA 2185 GALP, Section 8.7.3, pg. 1-13 5.6 Is documentation of routine operations maintained? EPA 2185 GALP, Section 8.7.3, pg. 1-13
Effective Date: February 2014 Page 11 of 15 5.7 Does the facility have a procedure to notify the customer prior to changes in LIMS software or hardware configuration that will adversely affect customer electronic data? QSM Rev. 5.0, Module 2, Section 5.4.7.2, g 5.8 Has a Disaster Recovery Plan been developed? EPA 2185 GALP, Section 8.6, Security, Section VI. Minimum Safeguards by Asset, Section C. Data Center Computing, 4. Backups, pg. 2-96 5.9 Has the Disaster Recovery Plan been tested on a regular and published schedule? EPA 2185 GALP, Section 8.6, Security, Section VI. Minimum Safeguards by Asset, Section C. Data Center Computing, 4. Backups, pg. 2-96
Effective Date: February 2014 Page 12 of 15 6.0 Facilities 6.1 Are the servers located in a temperature-controlled environment with adequate ventilation? EPA 2185 GALP, Section 8.6 Security, Section VI. Minimum Safeguards by Asset, Section C. Data Center Computing. 3. Physical & Environmental Safeguards 6.2 Are the LIMS and associated communications components protected through the use of surge protectors and connection to an uninterrupted power supply? EPA 2185 GALP, Section 8.6 Security, Section VI. Minimum Safeguards by Asset, Section A., Stand-Alone Computing, Section 3. Physical and Environmental Safeguards, pg. 2-89 6.3 Is environmentally adequate storage space provided for the retention of LIMS data storage media and hard copy records? EPA 2185 GALP, Section 8.10 Facilities, 2 LIMS Raw Data Storage, pg. 2-117 6.4 Are long-term archival copies of LIMS backup media stored in an offsite location with the same environmental control and security systems required of onsite storage facilities? EPA 2185 GALP, Section 8.10 Facilities, 2 LIMS Raw Data Storage, pg. 2-117
Effective Date: February 2014 Page 13 of 15 7.0 Electronic Data Deliverables 7.1 Does an SOP exist for how electronic data are processed, maintained and reported? QSM Rev. 5.0, Module 2, Section 4.0, 4.2.8.4, w; TNI EL-V1-2009, Section 4.2.8.4 d) 7.2 Does an SOP exist for verifying that electronic data deliverables match hardcopy report forms (for clients requiring both)? QSM Rev. 5.0, Module 2, Section 4.0, 4.2.8.4, x); TNI EL-V1-2009, Section 4.2.8.4 p) 7.3 Does an SOP exist for handling and documenting client-requested modifications to electronic data deliverable formats? QSM Rev. 5.0, Module 2, Section 4.0, 4.2.8.4, v) 7.4 Are the hardcopy data reporting forms and electronic data deliverables created from the same source? QSM Rev. 5.0, Module 2, Section 4.0, 4.2.8.4, s) aa); TNI EL-V1-2009, Section 4.2.8.4 a) r) 7.5 Does a corrective action plan exist for resolving discrepancies between electronic data deliverables and hard copy report forms? QSM Rev. 5.0, Module 2, Section 4.0, 4.2.8.4, t & Section 4.11; TNI EL-V1-2009, Section 4.2.8.4 l) n)
Effective Date: February 2014 Page 14 of 15 Notes:
Effective Date: February 2014 Page 15 of 15 Record of Revision for Checklist 5 Laboratory Information Management Systems and Electronic Data Management Revision Effective Date Reason for Revision Line of Inquiry 3.5 11/2009 Changed reference for SOP requirement for making changes to electronic data to 4.12.2.3. 2.3 3.5 11/2009 Changed reference for LOI to 4.12 DOE-4 2.9 3.5 11/2009 Add requirement that SOPs must be developed for the frequency of continuing verification of software. 3.2 3.5 11/2009 Users are trained on computer awareness security upon employment and thereafter, on an annual basis. 4.3 3.5 11/2009 Added periodic testing of LIMS backups to demonstrate that the backups contain all data and information. 4.10 3.6 11/2010 Added the requirement for the establishment of change control priority. 3.7 3.6 11/2010 Changed reference from 4.12.2.3 to QSAS, 5.4 DOE-4 3.7 3.7 11/2011 Added the following to the LOI Notes: Fully document any deviation from the LOI or the requirements for QSAS 2.7 Page 1 3.8 1/2012 Added the following to the LOI Notes: Fully document any deviation from the LOI or the requirements for QSAS 2.8 Page 1 3.9 11/2013 LOI s and references changed according to new requirements in the DoD/DOE QSM Rev. 5.0. All 4.0 2/2014 Minor revision following the first DOECAP audits All