Testing Professional Development Framework

Transcription

1 Testing Professional Development Framework Raising competency, or testing clones? Mike Jarred & Dr. Luke Avsejs

2 Agenda Objectives, and constraints IDBS Delivering Software to Regulated Environments What is the Testing PDF? Why did we do it? How have we implemented the PDF Some things we had to think about Mutual Benefit The good, the bad and the beautiful Closing thoughts Q&A Slide 2 IDBS 2012

3 Objectives and Constraints Objectives of this session It s a show and tell our journey to date, not an attempt to sell a generic approach Keep it interactive feedback and thoughts from other practitioners in the community Garner feedback and improvement suggestions. This is under continuous improvement, all ideas welcome Constraint Our framework is not publically available, but check out the BCS SFIA plus for a good starting point if interested Slide 3 IDBS 2012

4 Delivering Software to Regulated Environments IDBS is a global provider of innovative enterprise data management, analytics and modelling solutions Used by more than 200 pharmaceutical companies, major healthcare providers, global leaders in academic study, and high tech companies Our products enable increased efficiency, reduced costs and improved productivity of industrial R&D and clinical research Our platforms are enabling translational medicine personalised medicine (right drug, right patient, right time) Our customers work in highly regulated environments GxP (GLP, GCP, GMP etc ) FDA regulated, 21 CFR Part 11, EU Annexe 11 Customers audit IDBS to ensure risk of non-compliance introduced by our products is mitigated through robust software development practice The Test Group are under particular scrutiny as we provide the safety net... Slide 4 IDBS 2012

5 IDBS Testing Group IDBS operate a Test Centre of Excellence Standardised Testing Methodology, incorporating company best practices and tools Centralised resource pool providing risk mitigation across product portfolio 35 Team Members located globally - Guildford, London, Portland US, China Structure I report to VP of Product Delivery, who is part of the executive leadership team reporting to the CEO. Director of Test for corporate test strategy, and developing group capability 2 - Test Managers responsible for tactical delivery across products 4 - Test Leads specialising in product lines Technical roles for cross product service lines, Test Automation, Security Testing, Test Environments & DBA, Load & Performance Testing, Toolset Administration & Support Primarily science domain knowledge, 10 PhDs 4 - IT career testers to augment domain expertise Test follow a professional development framework to ensure necessary competencies / skills and certification (ISTQB) are available to support product development. Slide 5 IDBS 2012

6 So, what is the Testing PDF? It s a mechanism for specifying roles, career paths, competencies, skills, skill levels, training and qualifications required to deliver testing services to our projects Competencies and skills are categorised as Technical (what we want people to do) and Transferable (how we want people to be, inc. soft skills / behavioral attributes) Framework facilitates a consistent & logical approach to staff development through benchmarking of knowledge, experience, training and qualifications Under continuous improvement updated regularly as new competencies and skills are identified as the demands on testing change Lets take a look at it in a little more detail Slide 6 IDBS 2012

7 Career Path Test Manager ISEB Practitioner / ISTQB Advanced / PRINCE2 Practitioner Test Lead / Test Team Lead ISTQB Advanced Test Manager / ISEB Practitioner Automation Test Lead ISTQB Advanced Technical Test Analyst Security Tester ISTQB Advanced Technical Test Analyst / CSTA Test DBA ISTQB Advanced Technical Test Analyst / Test Analyst Senior Tester ISTQB Advanced Technical Test Analyst / Test Analyst Automation Test Engineer ISTQB Advanced Technical Test Analyst Load & Performance Tester ISTQB Advanced Technical Test Analyst Tester 1/2 ISTQB Foundation Tester 0 ISTQB Foundation Career Level Project Experience Training Senior Testing Experience should include participation of Test Strategy definition and tactical implementation of Testing Processes on at least one large project and two medium projects. Writing Test Strategies Writing Test Plans Web Application Security Tester Slide 7 IDBS 2012

8 Example Competency Skills Matrix Skill Level (0-5) Competency Skill Tester 1 Tester 2 Senior Tester Example Evidence / Indicators Environment & Data Administration / Coordination Test Design & Maintenance Able to define test environment hardware, software and data requirements Able to support test team with Oracle issues relating to strategy and infrastructure Able to maintain Oracle instances Able to give environment management advice Able to assist developers with Oracle related problems. Able to assist with database enhancements Programming with Java Test Script Creation Test Script Maintenance Test Model Creation and Implementation Test Environment Plan Updated Project Issues Log Fully Accepted Test Script Checklists Test Coverage Report Use of Oracle, SQL and PL/SQL within test scripts Updated Regression Test Packs Skill Level Description 1 - Basic Basic knowledge has been acquired through limited training. Very limited experience, through classroom and/or case studies, or on-the-job training. Regular direction and assistance is required to perform duties 2 - Intermediate Broader base of training has been received. A few successful experiences have been completed. 3- Senior Extensive training has been received. Several successful experiences have been completed. Performs almost all actions associated with this skill without assistance - may occasionally require assistance from a more senior individual for only the most difficult aspects of this skill. 4 - Expert Very extensive training has been received. Training may be provided by this individual within their organisation. Many repeated successful experiences have been completed in a variety of complex settings. Performs actions associated with this skill without assistance. Recognised within the organisation as "the person to ask" when difficult problems arise. 5- Master Very extensive training has been received. Is sought after to train others. Many repeated successful experiences have been completed in the most complex settings. Performs actions associated with this skill without assistance. Is sought after, from both within and outside their organisation, when the most difficult problems arise. Slide 8 IDBS 2012

9 So, why did we do this? Organisational Drivers Demonstrate to customers how we holistically maintain a competent testing group (we are always asked under audit) Demonstrate internally that we are driving professionalism / increasing capability in test group Enable clear objective setting and structured appraisal reviews Provide clarity regarding competencies and skills needed in the test group Demonstrate the Career Paths open to individuals in the Test Group there was an org structure, but no visibility of how to progress Assisting in formalising competency based recruitment Slide 9 IDBS 2012

10 How have we applied the PDF? Awareness & Training Overview of PDF, benefits, risks and pilot implementation plan provided to Exec team secured approval and sponsorship Test Management contributed and reviewed PDF content Implementation Team members skill levels for competencies are validated through the appraisal mechanism. Evidence of skill levels required Outputs from appraisal are linked to Personal Development Plans Test Group received PDF overview as part of change management Initial 6 month pilot to capture improvement suggestions Experience, skills and qualifications are continuously reviewed as part of personal development meetings Once a level on the PDF is established, further opportunities for development will be provided where possible as part of project delivery and departmental initiatives Slide 10 IDBS 2012

11 Some things we had to think about Slide 11 IDBS 2012

12 Mutual benefit Benefits to Testers A more structured & professional approach to career development Supports Continuing Professional Development (CPD) Visibility to all staff of how to progress within testing Benefits to Company Increased sales and retention of existing business through successful audits Employee retention / recruitment advantage Increase skill levels and broaden experience Gain Industry recognized qualifications Commitment from IDBS to their development Consistent appraisals across the department, equitable & quantitative scoring Increased awareness of training needs to budgeting and planning. Identification of skills gaps, succession planning, coaching & mentoring Opportunities to cross skill into a different framework e.g. Development (once rolled out to other groups) Assurance that recruitment targets area of greatest value Slide 12 IDBS 2012

13 The Good, the Bad and the Beautiful Lots of positive feedback (within, and outside the Test Group) Training economies of scale Competency based recruitment, and succession planning Increasing team flexibility Some scepticism over value of PDF and our ability to support career aspirations Evidencing skill levels has proven time consuming, but is a one off activity consensus is it s worth investing in Evidencing transferable skills is difficult Heat map / Pie Charts (management information is beautiful) Slide 13 IDBS 2012

14 Example heat map & pie charts COMPETENCY SKILL Job Role 1 Job Role 1 Job Role 2 Job Role 2 Competency 1 Competency 2 Competency Skill Competency Skill Competency Skill Competency Skill Competency Skill Competency Skill Competency Skill Competency Skill Competency Skill Competency Skill Competency Skill Test Team Competency Tester 2 Competency 18% 24% 52% Attaining 51% Attaining 30% Not Meeting Exceeding 25% Not Meeting Exceeding Slide 14 IDBS 2012

15 Closing thoughts from our team I find that it opens the discussion about what your line manager expects of you in your role, and allows me to identify areas where I could improve prior to having that meeting. It also provides a good backing for the appraisal, which is pretty invaluable, as you tend to forget the things you ve done throughout the year It provides a structure in line with the company organisation to allow us to see the different paths for professional development; giving a clear yet open idea about career progression within the team I think it is useful. The only downside is that the number of fields does make it lengthy to complete, it seems some fields could be consolidated or perhaps a few removed that don t seem relevant to a testers role Slide 15 IDBS 2012

16 Some things we had to think about (Q&A) Slide 16 IDBS 2012