Assessment Guidance for Mentors Expert Witness Advisors (EWA) Assessors and Internal Verifiers

Transcription

1 UPSTREAM OIL & GAS INDUSTRY TECHNICIAN TRAINING PROGRAMME (UOGITTP) Assessment Guidance for Mentors Expert Witness Advisors (EWA) Assessors and Internal Verifiers A.G. Revision D October 2013

2 Page Page 6 Page 6 Amendments From Revision C to D: October 2013 Amendment Removed paragraph regards SQA document Added bullets ;- be a discipline expert with typically a minimum of 2 years relevant experience register or be registered with OPITO (Appendix 4) participate in briefing session with Assessor Page 7 Changed defined competence of an assessor to align with new OPITO Assessment Strategy Page 9 Changed defined competence of an Internal Verifier to align with new OPITO Assessment Strategy Page 9 Bolding of during Interim reviews Addition of Interim Verification Reports and Internal Verifier Final Summary Reports must Page 9 be included and examples can be found in appendices 12 and 13 Page 15 Changed from minimum of 5years to minimum of 2 years Page 15 Added same bullets as page 6 Appendix 12 Addition of an exemplar Interim Verification Report Page Amendments From Revision B to C: October 2012 Amendment Front Cover UOGITTS Tag line & Opito logo added Page 2 Contents Added Assessor and IV's final summary reports to Appendix 11 and 12 Added Appendix references for Exemplar EWA observation report and EWA summary Page 5 feedback report Page 7 Added bullet - Provide "An Assessors Final Summary Report" (Appendix 11) Page 8 Added "and complete an IV Final Summary Report" (Appendix 12) Page 10 Page 13 Added Assessor and IV's Final Summary Reports and amended the appendixes accordingly Addition of 7.7 Final Summary Report section Page 22 Signature changed for consistency (David Gannon not Jones). Changed dates to align Page 23 Changed 26(1)/26(1) to 26(1)/27(1) Signature change as to keep an electrical assessor and not the mechanical one from earlier Page 29 examples Changes Made By: J Fraser Changes Checked By: F Tyreman 11 th October th October of 37

3 Contents 1.0 Introduction Roles of Key Personnel Mentor Expert Witness Advisor (EWA) Assessor Internal Verifier OPITO Reviewers Assessment Organisational Chart Assessment Materials Planning the Training Delivering the Training Assessing Competence Observation Product Evaluation Questions Personal Statements Witness Testimonies Simulation Final Summary Reports Appendix 1 - Expert Witness Advisor Strategy Document Appendix 2 - Trainee Technician Assessment and IV Appendix 3 - Assessor Information Form Appendix 4 - Expert Witness Registration Form Appendix 5 - Internal Verifier Information Form Appendix 6 - Exemplar Assessment Plan Appendix 7 - Exemplar Trainee Personal Statement Appendix 8 - Exemplar Expert Witness Advisor Observation Report Appendix 9 - Exemplar Expert Witness Advisor Summary Feedback Report Appendix 10 - Exemplar Assessor Observation Report Appendix 11 - Exemplar Assessor Final Summary Report Appendix 12 Exemplar Interim Internal Verifier Report Appendix 13 - Exemplar Internal Verifier Final Summary Report Appendix 14- Exemplar Witness Testimony Appendix 15 - Exemplar Record of Questions & Answers Appendix 16 - Guidance on Requirements for Contents of Trainee Maintenance Technician Portfolios Appendix 17 - Guidance on Requirements for Contents of Trainee Process Technician Portfolios Revision D October of 37

4 1.0 Introduction Throughout the two year work placement of the Trainee Technician Training Programme, trainees are required to gain experience of the various work activities which occur within their specific discipline (Mechanical/Electrical/Instrument/Production) on the Installation. During this time, trainees must record the activities they have been involved in and, by this means, build up a portfolio of work experience evidence. This ensures that trainees follow their training plans as well as playing a significant part in the Assessor s decision as to the competence of the trainee. Assessment is about measuring a candidate s competence against a national standard and collecting evidence that proves the standard has been met. As a Mentor, Expert Witness Advisor, Assessor or Internal Verifier, you have a key part to play in the training or assessing process and the purpose of this document is to help you understand how you can play your part to the full. The workplace is a new experience for most Trainees and they may well be overwhelmed by the sights and sounds of the working environment. The role of experienced colleagues in the workplace is to try to ensure that the transition from training centre to the workplace is as smooth as possible, allowing trainees to settle quickly in to this new environment. In this way trainees will feel at ease with more experienced colleagues and be able to achieve their full potential in the workplace. It is important that Mentors, Expert Witness Advisors, Assessors and Internal Verifiers understand the requirements of the Training and Assessment Programme and the specific SVQ Unit Standards on which the competences to be achieved are based. Trainees are provided with TWO handbooks i.e. Programme Operation and Building your Evidence Portfolio. These handbooks provide detailed information of the Training and Assessment Programme and give guidance on how the evidence of competence should be generated and presented to an Assessor. Mentors, EWAs, Assessors and Internal Verifiers should refer to the trainee s copies of these handbooks as necessary. Trainees are also provided with a CD which contains the Unit Standards for the both the Process Engineering Maintenance and Process Operations: Hydrocarbons Level 3 SVQs Assessors have to interpret standards and judge candidate performance competently and consistently. They would normally be involved at all stages of the assessment from the planning and timing discussions that take place in the early stage of the assessment process, reviewing of evidence which is generated as part of that process, through to the feedback that is given to candidates after their assessment has been completed. Assessors have to determine that sufficient evidence exists for the performance evidence and knowledge evidence requirements of the assessment routes. Assessors will require support and input from Mentors and some cases, Expert Witness Advisors (EWAs) to assist them make appropriate assessment decisions. This document will provide detailed guidance on the roles and responsibilities of Mentors, Expert Witness Advisors, Assessors and Internal Verifiers. Internal verification is necessary to ensure the integrity of the assessment process has not been compromised in any way. Internal verifiers ensure consistency of the assessment process by the assessor for different types of candidate and ensure standardisation of assessment between assessors. Internal Verifiers must be fully familiar with the standards against which assessments are to be carried out, the awarding bodies systems and criteria for qualifications. OPITO reviewers are also involved in providing guidance and support to trainees as they progress through their training and assessment programme. Revision D October of 37

5 2.0 Roles of Key Personnel 2.1 Mentor MENTORS have a key role in the training process acting as the direct training supervisor for trainees. Mentors should use their knowledge and experience of the tasks and equipment that trainees will be working on to provide advice and guidance as they progress through their training and assessment programme. Mentors should hold regular meetings with their trainees to discuss any issues arising and to review their progress against the agreed plan. One of the initial tasks of Mentors is to help their trainees produce a Training and Assessment Plan incorporating a planned schedule for completion. Where possible this should be done in conjunction with the trainee s assessor. The training plan will allow comparisons to be made between the actual progress over the training period against the agreed planned schedule. It will also allow Mentors the opportunity to check that their trainees are achieving their targets and plan the remaining training activities accordingly. Mentors should work with the trainee s assessor, especially in the latter stages of the training period, to carry out regular reviews of the training plan and make appropriate adjustments where necessary. Mentors are also required to sign off the Personal Statements written by the Trainee after a training or assessment activity has taken place. These Personal Statements are simply written accounts of the activity with which the Trainee has been involved and provide details of the nature of the activity and the extent and level of involvement of the Trainee. Trainees are also required to provide copies of any documentary evidence such as logs, permits, risk assessments, isolation certificates, etc. that support their specific involvement in the activity. By signing a trainee s personal statement, Mentors are confirming that it is an accurate account of the trainee s involvement in the activity. Mentors are also confirming that the Trainee has completed the relevant task to the required standard and has gained the experience stated in the Personal Statement. Mentors should also work with the trainee s assessor to advise when they consider that a trainee has demonstrated sufficient skill and experience to be assessed against specific units. In summary, the key responsibilities of Mentors are to: Ensure that trainees have undergone site safety and relevant emergency response training Ensure that trainees have knowledge of the general rules and practices applying to the facility Induct trainees in the safety procedures and culture of the facility Understand the requirements of the training and assessment programme Plan with the Trainee a programme of training to cover a suitable range of activities across a range of appropriate equipment/systems/processes Ensure that suitable and appropriate tasks are assigned to trainees Ensure that trainees are given instruction in the skills and knowledge applying to the activities and equipment/systems/processes in question Supervise the work of trainees, ensuring that they work safely and are not exposed to any unnecessary risk Provide guidance to trainees on the content and accuracy of Personal Statements and sign them off when complete Be responsible for deciding when trainees are able to carry out activities unsupervised and when they are ready for formal assessment Maintain a close and effective relationship with the trainees assessors Revision D October of 37

6 Ensure that the welfare needs of trainees are met throughout the placement period Provide OPITO with information on progress of trainees as and when required Trainees will meet regularly with a Training Reviewer appointed by OPITO (see section 2.5). These meetings are extremely important and give opportunities for the progress of training to be reviewed by a person external to the workplace. Mentors should be encouraged to participate in these review meetings and to contribute to the discussions on the progress of trainees. The option of conference call or similar facility should be considered to allow the mentor to participate in these meetings. 2.2 Expert Witness Advisor (EWA) Where it is not possible or practical to have an assessor available in the same location as a candidate, an EXPERT WITNESS ADVISOR (EWA) may be utilised to assist in on-the-job observation for a candidate. Mentors may also act as Expert Witness Advisors. Sponsoring organisations are required to provide Expert Witness Advisors (EWA) who are occupationally competent and who will work with the trainee and assist with the process of observing the trainee in the work place. The Expert Witness Advisor should only be used if it is not reasonably practical for an Assessor to be present during an observation. The Assessor however is still responsible for all assessment activities and will use the Expert Witness Advisor report as an important piece of evidence in judging competence of trainees. It is expected that an EWA would participate in a briefing session to ensure that they are familiar with the Standard(s) being assessed and that they understand their role to observe the Candidate and record observations and comments details of briefing sessions must be available for review by External Verifiers/Auditors at External Verification visits/audits. The following extract from the EWA Strategy document ( Appendix 1) provides a summary of the assessment process with EWA involvement. 1 Observations would be carried out by an Expert Witness with a qualified Assessor continuing to be responsible for assessment planning, providing feedback and for all other assessment activities e.g. review of paperwork, knowledge questioning. 2 An Expert Witness would be required to: be a discipline expert with typically a minimum of 2 years relevant experience register or be already registered with OPITO (Appendix 4) Participate in briefing session with Assessor agree a time with the Candidate for the observation to take place and advise the Assessor; observe the Candidate carrying out normal work tasks/activities ( Appendix 8); record details of each task/activity observed and confirm its completion according to the required standard; authenticate any supporting documentation/job paperwork (for example, a Candidate s name may not always appear on the job paperwork so his/her involvement should be confirmed); comment on the Candidate s technical ability, knowledge of equipment, team work, safe working practices, etc.; make a recommendation to the Assessor on the Candidate s ability to carry out the task/activity,( EWA Final Summary Report Appendix9) 3 The Assessor will review all the evidence provided by the Candidate, including the observation by the Expert Witness, and make a judgement on the competence of the Candidate. Revision D October of 37

7 2.3 Assessor ASSESSORS of the appropriate discipline will be provided in the workplace wherever possible by the sponsoring organisations to support and assess trainees working towards their SVQ Level 3. Where assessors cannot be provided in the same workplace as the trainee, sponsoring organisations may consider utilising the services of third party external assessors, who with the assistance of an EWA, carry out assessments of trainees for their SVQ Level 3.The Sponsor, with assistance from OPITO, should ensure that 3 rd party Assessors are familiar with the overall assessment and training strategy and with the specific requirements that apply to UOGITTS trainees. The OPITO Assessment and Verification Strategy requires that Assessors must meet the following requirements: be technically competent in the discipline being assessed this could be demonstrated in a number of ways, for example: they are an experienced practitioner or supervisor in the same discipline; they have been assessed as competent for the relevant Standard(s) and/or have achieved an equivalent qualification; they have previously performed or supervised the activities defined in the Standard(s) and can demonstrate that they have maintained their technical expertise and knowledge of current processes and practices; hold (or be working towards) a recognised Assessor s qualification (e.g. L&D9DI, L&D9D, A1, A2, D32, D33). Assessors must also have extensive knowledge of the tasks and equipment the Trainee will be working with. Assessors who are not normally employed in the same location as the trainee should make every effort to familiarise themselves with the specific requirements of the worksite to which the trainees is assigned. E.G. permit process; procedure use; safety standards; work management; reporting etc. Assessors in these circumstances should seek assistance and input from the EWA to provide location specific expert knowledge to support 3 rd party assessors by way of observations, testimonies and authentication of documentary evidence provided by trainees. The assessor s role is to assess the evidence that Trainees provide whilst carrying out their work and to make a judgement on whether or not they have met the appropriate Standards. Assessors will also question the Trainees on various aspects of the plant/equipment and activities, to ensure that they have sufficient knowledge and understanding of the underpinning principles and concepts on which the Standards are based. The following bullet points and example questionnaires provided in Appendix 12 provide further guidance on assessing knowledge and understanding of trainees. Knowledge & Understanding (K&U) are key components of the assessment process however it is unlikely that observing performance will provide sufficient evidence in this area. Where the trainee s K&U is not apparent from performance evidence then it must be assessed by other means and be supported by suitable evidence. Questioning can be a useful way of confirming what candidates know and understand. Trainees have to show that they can meet the knowledge specifications for the SVQs. For these SVQs, K&U is specified for each (Unit/PC). Questions can be asked in a variety of forms, such as oral questions, short answer written questions, and multiple-choice. Assessors need to ensure that the method of questioning does not go beyond the competence required for the SVQ and become a barrier to fair assessment. For example, some candidates will feel more comfortable with oral questions than written. Revision D October of 37

8 To assist in the decision-making process, assessors will be able to inspect the Personal Statements written by the Trainee on completion of various work activities and the Supporting Evidence (Witness Testimonies/Work Authorisations/Risk Assessments/Work Permits/Isolation Certificates/Daily & Handover Logs/Etc.) gathered in support of these activities. This documentation will be presented in an SVQ Portfolio which the Trainee will have compiled. This Portfolio will help the assessor establish an overall view of the level of competence of the Trainee in various aspects of their work. Only when the assessor is satisfied that the Trainee has met all the performance criteria and knowledge requirements to meet the standards will they sign off the relevant assessment documentation. Trainee Maintenance Technicians will be required to provide additional personal reports and assessment material for maintenance activities over and above that required for their SVQ. Trainees will retain this evidence in Section 2 of their Training Portfolio. Assessors will be required to review the evidence provided in Section 2 of the Training Portfolio and assess it to the same standard as the SVQ. The assessor and EWA (where applicable) will sign off this assessment material to confirm trainee competence as appropriate. A more detailed description of this process is provided in a document titled Guidance on Requirements for Contents of Trainee Maintenance Technician Portfolios. A copy of this document is provided in Appendix 13. In summary, the key responsibilities of Assessors are to: Fully understand the Standard(s) for the relevant VQ(s) Plan with the Trainee and the Mentor (and EWA where appropriate) when formal assessment against the Standards of the SVQ is to take place Formally assess the Trainee s performance and knowledge for each task in their assessment plan Give feedback to the Trainee on their performance and knowledge Decide whether or not the Trainee is competent against appropriate standards in a particular task Sign the assessment documentation for that task confirming that the Trainee is competent Provide an Assessors Final Summary Report (Appendix 11) In assessing the Trainee s competence, assessors should use the following sources of evidence: Direct Observation of the Trainee carrying out the task. (This is considered to be the optimum method of assessment and should be carried out by the assessor wherever possible or by EWA where required) Inspection of the Trainee s Personal Statements. Direct Questioning of the Trainee to ensure that they have an understanding of the task in question Inspection of Supporting Evidence gathered by the Trainee. Evaluation of previous work the Trainee has carried out Witness Testimonies provided by appropriately experienced/qualified person and aligned to specific performance standards. It should be noted that the weight of the evidence will vary, depending on the knowledge and expertise of the person providing the witness testimony. Only once the assessor has judged the Trainee competent in ALL the tasks required for the SVQ, will they sign-off the Unit Progress Record and Unit Achievement Records. Revision D October of 37

9 2.4 Internal Verifier INTERNAL VERIFIERS (IV) will be provided by the sponsoring organisations to support assessors and carry out internal verification of trainee s SVQ portfolios. Where a sponsoring organisation is unable to provide internal verifiers then third party internal verifiers will be appointed to carry out the internal verification process. Internal verifiers should be familiar with the specific requirements that apply to UOGITTS trainees. The OPITO Assessment and Verification Strategy requires Internal Verifiers to: have sufficient occupational expertise in the broad discipline area covered by the relevant Standard(s) to permit valid judgements about assessments and appeal decisions typically this would be demonstrated by the Internal Verifier having worked at either operational or supervisory level in the broad discipline area; hold (or be working towards) a recognised Internal Verifier s qualification (e.g. L&D11, V1, D34). The role of the IV is to verify that the assessment process has been carried out correctly and with the rigour demanded by the scheme. This will be carried out during interim reviews of candidate portfolios and at the end of the training period, before the Trainee s evidence is presented to the OPITO office for External Verification. Interim Verification Reports and Internal Verifier Final Summary Reports must be included and examples can be found in appendices 12 and 13 The main function of the IV is to review the SVQ Portfolio compiled by the Trainee, including all supporting evidence and assessment records, to ensure that the assessment process has been carried out in a proper manner and with the rigour demanded by the SVQ programme. They must also ensure that the integrity of the assessment process has not been compromised in any way. When satisfied that proper assessment of all the evidence has been carried out and the decisions of the Assessor are correct, the IV must sign off all appropriate documentation contained in the trainee s portfolio and complete an IV Final Summary Report ( Appendix 12). The SVQ Portfolio will then be submitted to the OPITO office and made available for External Verification. NOTE: There is no requirement for IVs to carry out internal verification of any assessment material held in Section 2 of the maintenance trainee technicians Training Portfolio. The focus for internal verification must be on the evidence contained in the trainees SVQ Portfolios. 2.5 OPITO Reviewers OPITO utilise their own reviewers to support trainees and review trainees portfolios as they have been doing since the Technician Training Scheme was introduced. Where reviewers identify discrepancies in Training or SVQ portfolios, they will in the first instance communicate with the sponsoring organisation s OPITO Technician Training Scheme Focal Point. The Focal Point may then advise that the reviewer communicates direct with the assessor and EWA to resolve these issues. In certain circumstances it may also be decided that the reviewer should communicate directly to the trainee s mentor or supervisor on other issues not necessarily related to the SVQ portfolio. Mentors should be encouraged to participate in these review meetings and to contribute to the discussions on the progress of trainees. The option of conference call or similar facility should be considered to allow the mentor to participate in these meetings. OPITO Reviewers should endeavour to provide Mentors with notice of planned review meetings. Revision D October of 37

10 3.0 Assessment Organisational Chart It is essential that the Sponsor organisations document the actual training and assessment process to be followed for individual trainees. This can be done utilising the documents titled Programme Operation Handbook and Building Your SVQ Evidence Portfolio issued to trainees. The following simplified organisation chart illustrates the interfaces and communication lines between the different roles in the assessment process and can be used by Sponsor organisations as a model for creating their own location specific version. Internal Verifier Assessor Assessor OPITO Reviewer EWA/Mentor EWA/Mentor Mentor Mentor OPITO Reviewer Candidate Candidate Candidate Candidate Candidates in some locations may not have regular, direct access to an assessor therefore they will be assigned an Expert Witness Advisor (EWA) who will provide an interface between the candidate and the assessor. (Refer to Section 2.2 for more detail on the EWA role). It is likely that the Mentor and the EWA will be the same person and this is reflected in the diagram. This diagram also illustrates that assessors may work with more than one candidate. Coaching sessions should be held wherever necessary involving all parties concerned including trainees, mentors, EWAs, Assessors, Internal Verifiers and OPITO Reviewers. These coaching sessions should be specific to each sponsoring organisation so that local issues around communication between OPITO reviewers, assessors, EWA, supervisors, mentors, trainees and Internal Verifiers can be explored and addressed. These sessions can be run using conference calls, video conferencing or Net Meeting as appropriate. (Refer to additional guidance on coaching detailed in the EWA Strategy document in Appendix 1) 4.0 Assessment Materials OPITO will provide the trainees with templates for all of the forms and reports that require to be generated as part of the assessment process. Examples of some of the wider used reports are provided in the Appendices to this document as follows: Appendix 3 Appendix 4 Appendix 5 Assessor Information Form Expert Witness Registration Form Internal Verifier Information Form Revision D October of 37

11 Appendix 6 Appendix 7 Appendix 8 Appendix 9 Exemplar Assessment Plan Exemplar Trainee Personal Statement Exemplar Expert Witness Advisor Observation Report Exemplar Expert Witness Advisor Summary Feedback Report Appendix 10 Exemplar Assessor Observation Report Appendix 11 Exemplar Assessor Final Summary Report Appendix 12 Exemplar Interim Internal Verifier Report Appendix 13 Exemplar IV Final Summary Report Appendix 14 Exemplar Witness Testimony (For use by witness other than the EWA) Appendix 15 Exemplar Record of Questions & Answers NOTE: These examples are provided in this document for information only. Forms from the trainee technicians work books should always be used to ensure that the latest version is used. 5.0 Planning the Training This section is a summary of the detailed information contained in the following Appendices: Appendix 13 - Guidance on Requirements for Contents of Trainee Maintenance Technician Portfolios Appendix 14 - Guidance on Requirements for Contents of Trainee Process Technician Portfolios Trainees will arrive at the workplace with suitable PPE, two Handbooks: Programme Operation and Building your Evidence Portfolio, a CD containing the Unit Standards, report forms etc. Trainees will be provided with evidence folders to be used as follows: Trainee Maintenance Technicians 1 Training Portfolio to be used to retain copies of all relevant training evidence including activity reports not referenced in the SVQ Portfolio 2 SVQ Portfolio to be used to retain all relevant SVQ documentation and evidence. All evidence retained in this portfolio must be cross referenced to the appropriate SVQ standards. Trainee Process Technicians 1 A single portfolio to be used to retain copies of all relevant training evidence with all relevant SVQ documentation and evidence retained in a separate section. All evidence retained in SVQ section must be cross referenced to the appropriate SVQ standards. Following site induction, the Mentor and the Trainee will meet together to commence planning the Training Programme. The plan will require to be structured and flexible, and reviewed regularly to ensure that the trainees achieve the required outcomes of the UOGITT scheme. At this meeting the Trainees will inform their Mentors of their training/education to date and will confirm the occupation/discipline they are to follow. This being established, the Training Plan will be drawn up. The activities listed on the Plan must conform to the requirements of the scheme and will depend on the facilities/ plant/equipment/systems & processes available on-site and the tasks likely to be undertaken. Examples of the Proforma to be used in the production of the Training Plan are included in the handbook Programme Operation provided to the trainees. Revision D October of 37

12 6.0 Delivering the Training Mentors and other experienced personnel involved in delivering the training should have an empathy with the Trainees and be able to communicate effectively with them. They should establish a rapport with the Trainee and be approachable. The Trainee should be able to ask questions of a technical nature and expect to be given relevant and meaningful answers. The training being delivered should reflect the activities agreed in the Training Plan and the nature and scope of the work must ensure that the Trainee has opportunities to gather evidence relevant to the competences required by the SVQ Standards. The Mentor must monitor the delivery of training against the Training Plan activities to ensure that the Plan is being achieved. Where it is proving difficult for a planned activity to be realised, this should be identified at an early stage in the training and the Training Plan amended accordingly. It should be noted that the time allocations presented on the Training Plan are estimates, by the Mentor, of the amount of time the Trainee needs to be exposed to an activity to achieve competence. This time period need not be delivered in one block but may be spread throughout the training period as relevant tasks present themselves. Trainees should be given clear instructions relating to each task including: An explanation of the purpose and the operation of the item of plant/equipment/system. A demonstration of how to operate and/or maintain the item A step-by-step instruction on how to execute the task In addition to this the Mentor should question the Trainee to ensure that they know and understand the process and should confirm by observation that the Trainee is able to perform the task to the required standard of competence. The tasks allocated to the Trainee should increase in complexity and demand over time. 7.0 Assessing Competence Competence can only be judged from evidence provided by trainees that meets the standards defined in the relevant Level 3 SVQ. Although assessors will review material from additional activities carried out by maintenance technician trainees, it may well be that there is insufficient evidence of competence arising from some of the training activities. Assessors must ensure that the evidence provided reflects the trainee s competence and not just the achievement of a training programme. Various methods may be used by assessors to assess competence over the range of Activity Items trainees have listed in their Training Plan. Some of the most commonly used methods are: Observation Product evaluation Questioning Personal statements Witness testimony Simulation 7.1 Observation Observation is considered to be the most valid and reliable method of assessment. It can be undertaken by the Assessor: Revision D October of 37

13 Working alongside the Trainee OR Arranging to visit the Trainee when the work activities occur naturally OR Arranging for work activities to take place with an EWA carrying out the observations NOTE: Refer to Appendix 10 for an example of an Observation Report 7.2 Product Evaluation Product Evaluation may be used as an assessment method where evidence in the form of products of work is available for inspection. The nature of this evidence can vary widely depending on what the task entails, but examples of product evidence include Risk Assessments, Work Permits, Isolation Certificates, Maintenance History Records, Daily logs, Reports, Hand-over documents etc. 7.3 Questions The Assessor will use Questioning as an essential part of the assessment process to show that the Trainee can meet the knowledge specifications for the N/SVQs. The required knowledge and understanding is specified for each Unit. Much of the Trainee s knowledge and understanding will be apparent from what is done or produced as part of the work activities, but this will not always be the case, and questioning can be a useful way of confirming knowledge and understanding. The Assessor may ask questions in a variety of ways, such as oral questions, short answer written questions, and multiple-choice. The Questions asked by an Assessor and the answers given by the Trainee should be recorded on the Proforma provided in the Programme Operations Handbook. NOTE: Refer to Appendix 12 for examples of question and answer formats. 7.4 Personal Statements Personal Statements are reports produced by the Trainees giving a detailed description of each activity they complete as they gain experience towards competence on a particular Unit. Personal Statements should record what was done, how it was carried it out and why it was decided to do the work in this particular way. Personal Statements must be written in such a way that it is obvious which parts refer to each Performance Statement for each specific Unit being covered. It is not necessary for a Personal Statement to cover the whole task from start to finish, as evidence may have already been provided for part of the activity in another Statement e.g. for maintenance units, the preparation work (Units 13, 69 and 70). For instance, if the task involves the removal and replacement of components (Units 72 and 73), then the report should only follow the guidelines given for these Units. Great detail about the preparatory work carried out is not required in this instance since this will be covered when writing Statements for the tasks used to satisfy Units 13, 69, 70. The report should only detail information relevant to the Unit/s it is intended to cover. A Proforma for Personal Statements is provided in the Programme Operations Handbook issued to all trainees. NOTE: Refer to Appendix 7 for an example of a personal statement Revision D October of 37

14 7.5 Witness Testimonies Witness Testimonies are reports produced by technically competent persons who have observed the Trainee undertake a particular activity. These accounts may be used to support the Trainee s Personal Statement. The value placed on a Witness Testimony by an Assessor will depend on the knowledge, expertise and credibility of the person providing the testimony. A Witness Testimony alone is unlikely to be sufficient in itself for a competency decision to be made, and the Assessor would normally supplement this by asking questions of the Trainee. Witness Testimonies must be capable of being authenticated. To authenticate a Witness Testimony, a record must be available showing clearly the name and status of the Witness, their relationship to the Trainee, (e.g. supervisor, client) their contact details and the date. A Proforma for Witness Testimonies is provided in the Programme Operations Handbook issued to all trainees. NOTE: Refer to Appendix 11 for an example of a witness testimony. 7.6 Simulation Simulation is any structured assessment exercise involving a specific task which reproduces real-life situations. The Assessor must ensure that any Simulated assessment exercise is carried out under conditions which reflect a realistic work environment and activity characteristics as closely as possible. Assessors should refer to the OPITO Assessment and Verification Strategy for more information on the use of simulation. 7.7 Final Summary Reports The final action of EWAs, Assessors and Internal Verifiers will be to provide written final summary reports. EWA Summary Feedback Reports should outline how the observations etc have been carried out, provide an overview of the trainee s development and state the EWA s recommendations to the assessor. (Appendix 9) Assessor Reports should provide an overview of the trainee s development during the training period and of the assessment process used, i.e. direct observation, inspection of work etc. The reports should also state the Assessor s final conclusion on the trainee s competence (Appendix 11) Internal Verifier Reports should provide an overview of how the assessment process was carried out throughout. The IV will also comment on the content of the Evidence Portfolio in relation to the referencing of evidence, the organisation and labelling of the document itself and the ease with which evidence is traceable. It should also state whether or not the IV agrees with the assessor s judgement. (Appendix 12) Revision D October of 37

15 Appendix 1 - Expert Witness Advisor Strategy Document Expert Witness Advisor Strategy Assessment is the process of determining a Candidate s competence against national standards (i.e. skills defined by the National Occupational Standards). The assessment process involves observing a Candidate carrying out normal work activities, reviewing associated job paperwork and assessing a Candidate s knowledge through questioning. Typically it would be expected that all assessment activities would be carried out by qualified a Assessor. A qualified Assessor would be expected to have achieved a national qualification such as A1, L&D9DI or D33. Where it is not possible or practical to have a qualified Assessor in the same location as a Candidate, the following strategy outlines how Expert Witnesses can be used to support the assessment process by carrying out on the job observations. The requirements for an Expert Witness are outlined in the following paragraphs. 1 Observations would be carried out by an Expert Witness with a qualified Assessor continuing to be responsible for assessment planning, providing feedback and for all other assessment activities e.g. review of paperwork, knowledge questioning. 2 An Expert Witness would be required to: be a discipline expert with typically a minimum of 2 years relevant experience; register or be already registered with OPITO; participate in briefing session with Assessor; agree a time with the Candidate for the observation to take place and advise the Assessor; observe the Candidate carrying out normal work tasks/activities; record details of each task/activity observed and confirm its completion according to the required standard; authenticate any supporting documentation/job paperwork (for example, a Candidate s name may not always appear on the job paperwork so his/her involvement should be confirmed); comment on the Candidate s technical ability, knowledge of equipment, team work, safe working practices, etc; make a recommendation to the Assessor on the Candidate s ability to carry out the task/activity,( EWA Final Summary Report Appendix9) 3 The Assessor will review all the evidence provided by the Candidate, including the observation by the Expert Witness, and make a judgement on the competence of the Candidate. Expert Witnesses Expert Witnesses are discipline experts who would generally be expected to have a minimum of 2 years relevant experience. Expert Witness CVs should be available for review by the External Verifier. It is the responsibility of the Sponsor Organisations to ensure that all Expert Witnesses participate in a briefing/coaching session to ensure that they are familiar with the standards and qualification being assessed and that they understand their role to observe candidates and record observations and comments. Revision D October of 37

16 Expert Witness Strategy (Continued) The following are some key points which must be covered in briefing/coaching sessions with Expert Witnesses. 1 Assessing competence against occupational standards What does competence mean? What is an occupational standard? How does assessment relate to the standard? The specific standards and qualification to be used for the observation. 2 Carrying out observations What can be assessed by observation? How to prepare for observation. Good practice during observations. 3 Making recommendations and providing supporting evidence Confirming that job paperwork is available, can confirm the activities and that these can be attributed to the Candidate. Use of reports to record observations made and any comments. Providing a clear recommendation to the Assessor on the Candidate s performance and ability to carry out the task/activity. Revision D October of 37

17 Appendix 2 - Trainee Technician Assessment and IV Trainee Technician Assessment & IV Process START 1. Assessor Briefs EWA/ Mentor and Trainee 2. Assessor Conducts Briefing with Trainee 3A. Assessor Agrees Assessment Plans with EWA/Mentor and Trainee 3B. EWA/Mentor Agree Assessment Plans with Trainee and Assessor 3C. Trainee Agrees Assessment Plans with EWA/Mentor and Assessor 4 Trainee Gathers Evidence with Observations by EWA 5. EWA Writes up Observation Reports for Trainee s Portfolio 6. Trainee Writes up Personal Statements and Builds Portfolio 7. Assessor Provides Interim Review of Portfolio Trainee Submits Portfolio to OPITO Reviewer for Quarterly Reviews 8. IV Carries out Interim Review of Portfolio 9. IV Provides Feedback to Assessor 10. Assessor Provides Feedback to Candidate and EWA/Mentor 11. EWA Carries out any Further Observations Arising from Assessor Feedback 12. Trainee Updates Portfolio as Identified by Assessor Feedback 13 Trainee Gathers Remaining Evidence with Observations by EWA 14. Assessor Meets with Trainee and Completes Final Assessment of Portfolio 15. Assessor Signs off Assessment 16. Assessor Submits portfolio for Internal verification 17. IV Carries out Final Internal verification of Portfolio Trainee Submits Portfolio to OPITO Reviewer for Review Prior to Submission for Final Assessment by Assessor 18. OPITO Reviewer Carries out Final Check Revision D October of 37 FINISH Process

18 Appendix 3 - Assessor Information Form Full Name Date of Birth Current Employer Assessor Qualifications Assessor Information Form Other Relevant Qualifications Centre to see proof of certificates Details of Occupational Competence as required by the Assessment Strategy Details of Continuous Professional Development (CPD) If you hold previous versions of Assessor qualifications such as D32/D33 or A1 please show CPD in relation to achieving knowledge/ experience of current Assessor Standards Occupational areas(s) you will be assessing Assessor: I am satisfied that the information I have provided on this form is correct and I submit my relevant certificates with this form Signed: Printed: Dated: OPITO Representative: I have checked that the information given on this form is accurate Signed: Printed: Dated: Revision D October of 37

19 Appendix 4 - Expert Witness Registration Form Expert Witness Registration Form Full Name Date of Birth Current Employer Discipline you will be Witnessing Final Assessment will be carried out Discipline Relevant Qualifications A In-House Assessor B External Assessor (Please State) Other Relevant Qualifications Details of Occupational Competence Details of Continuous Professional Development (CPD) Occupational Areas in which you have worked I am satisfied that the information I have provided on this form is correct and I submit my relevant certificates with this form Signed: Printed: Dated: OPITO Representative: I have checked that the information given on this form is accurate Signed: Printed: Dated: Discipline Approved: Mechanical, Electrical, Instrument, Process Revision D October of 37

20 Appendix 5 - Internal Verifier Information Form Full Name Date of Birth Current Employer Verifier Qualification(s) Internal Verifier Information Form Other Relevant Qualifications Centre to see proof of certificates Details of Occupational Competence as required by the Assessment Strategy Details of Continuous Professional Development (CPD) If you hold previous versions of IV qualifications such as D34 or V1 please show CPD in relation to achieving knowledge/ experience of current IV Standards Occupational areas(s) you will be verifying Verifier: I am satisfied that the information I have provided on this form is correct and I submit my relevant certificates with this form Signed: Printed: Dated: OPITO Representative: I have checked that the information given on this form is accurate Signed: Printed: Date: Revision D October of 37

21 Appendix 6 - Exemplar Assessment Plan Units: 13, 69 & 70 Elements: Activities Performance Criteria Method of Assessment/Sources Of evidence Preparing the work area, equipment and materials in order to carry out the maintenance of plant and equipment 13 (2), (3), (4), (7) 13 (1) 13 (1), (5), (6) Observation Witness Testimony Work Permit Date of assessment 24/10/2011 Evidence already available Ref 66 (1) Ref 68 (2) Links to other Performance Criteria 69 (1), (2), (3), (6), 70 (1), (2), (3, (4) 69 (1), a, 70 (1) 69 (1), (4), (5), 70 (1), (5) Questioning for knowledge and understanding not apparent from performance to be identified from 2 nd review 13 (1), (6) Questions 26/10/ (1), (6); 70 (1), (6) Assessor/EWA Name(s) Trainee s Name Date of agreement David White 1 ST Review Date 22/10/2011 David White Assessor/EWA (Assessor) Signature(s) Duncan Campbell Trainee s Signature Duncan Campbell 2 ND Review Date N/A 20/10/2011 Date of completion 26/10/2011 Revision D October of 37

22 Appendix 7 - Exemplar Trainee Personal Statement Plant/Equipment Activity Item No: A1 Date of Evidence Details of statement Activity Index number 21/10/11 A1-1 This statement provides evidence of my Competence to Adjust mechanical plant and Equipment to meet operational requirements. This was achieved by carrying out the task of adjusting the speed control on an emergency generator diesel engine. Evidence Ref No. A1-1 Title: Diesel Engines Links to other Evidence (enter numbers) Unit, Performance Criteria covered 71 I carried out a task risk assessment which was reviewed by the Area Authority following discussions with myself. The risk assessment highlighted the hazards of working with heavy mechanical plant and the need to prepare the work site and to wear personal protective clothing A work permit was then issued I obtained specifications from the Vendor s maintenance manual for this engine, to ensure that I had the correct information and settings for this task including the manufacturer s recommended speed. Since I am not yet considered competent to carryout detailed maintenance work on the plant, my only task here was to check the speed control mechanism and adjust it as required. To do this I started the engine and allowed it to run for five minutes before checking the idling speed. This is the procedure stated in the Vendor s Manual. I noted that the speed was about 100 rpm faster than the recommended 1000 rpm, so I adjusted the throttle control mechanism until the speed was at the correct 1000 rpm. As requested in the maintenance manual, I allowed the engine to run for another 5 minutes before checking the speed again to ensure that the adjustment was correct. During this 5 minute period, the speed crept back up to the original 1100 rpm; therefore I attempted to adjust the throttle mechanism once again. During this operation I noticed that the threads on the adjusting screw were worn this being the likely reason that the adjustment was slipping. I replaced the adjusting screw, adjusted the speed to 1000 rpm and observed over a further 5 minute test period. The speed was held constant at 1000rpm. A1.1a (Risk Assessment), A1-1b (Work Permit) A1.1c (Page from Vendor manual) A1.1d (Job Card) A1.1c (Page from vendor manual) 71 (1) 71 (1) 71 (2) 71 (3) 71 (4) 71 (5) Revision D October of 37

23 Exemplar Personal Statement (cont) Plant/Equipment Activity Item No: A1 Date of Evidence Details of statement Activity Index number Title: Diesel Engines Links to other Evidence (enter numbers) 21/10/11 A1-1 After completing the task, I made sure that the work site was tidy and that no tools or materials were left in the area. I also cancelled the work permit Evidence Ref No. A1-1 Unit, Performance Criteria covered 71 (1) I outlined the task in the daily log and recorded my findings and the test results in the maintenance history records. I made mention in both these documents that I had problems with the adjusting screw and had to replace it and that this resolved the problem A1.1e (Copy of maintenance history report) A1.1f (Copy of daily log) 71 (6/7) 71 (6/7) Signed: (Trainee) Duncan Campbell Signed: (Mentor) David Gannon Date: 23/10/2011 Date: 23/10/2011 Assessment Statements: a. I confirm that I observed this activity and that this is an accurate report. ASSESSOR Sig/Date EWA Sig/Date MENTOR Sig/ Date George Black (b) I did not observe this activity, however I am satisfied that this is an accurate account. 23/10/2010 David White David Gannon 24/11/ /10/2011 Note: If the Mentors are also the EWA or Assessor, then it is only necessary for them to sign in the EWA or Assessor box as appropriate. Revision D October of 37