DOCTRINE AND TRAINING SERIES ADFP THE DEFENCE TRAINING MODEL

Transcription

1 ADFP DOCTRINE AND TRAINING SERIES ADFP THE DEFENCE TRAINING MODEL Australian Defence Force Publication is issued for use by the Australian Defence Force and is effective forthwith. A.G. HOUSTON, AO, AFC Air Chief Marshal Chief of the Defence Force Australian Defence Headquarters Canberra ACT July 2006

2 Commonwealth of Australia 2007 This work is copyright. Apart from any use as permitted under the Copyright Act 1968, no part may be reproduced by any process without prior written permission from the Australian Government Department of Defence. Announcement statement may be announced to the public. Secondary release may be released to the Australian Government Department of Defence and its equivalent in the United States of America, United Kingdom, Canada, and New Zealand. All Defence information, whether classified or not, is protected from unauthorised disclosure under the Crimes Act Defence information may only be released in accordance with the Defence Security Manual and/ or Defence Instruction (General) OPS 13 4 Release of Classified Defence Information to Other Countries, as appropriate. The Commandant of the Australian Defence Force Warfare Centre is the approving authority for the release of Unclassified joint doctrine publications to countries not covered by the secondary release statement. ADFP First edition 2006 Sponsor Chief of Joint Operations Headquarters Joint Operations Command Developer Commandant Australian Defence Force Warfare Centre RAAF Base WILLIAMTOWN NSW 2314 Publisher Defence Publishing Service Department of Defence CANBERRA ACT 2600 Defence Publishing Service DPS: March/2007

3 ADFP FOREWORD 1. Australian Defence Doctrine Publications (ADDP) and Australian Defence Force Publications (ADFP) are authorised joint doctrine for the guidance of Australian Defence Force (ADF) operations. ADDP are pitched at the philosophical and high-application level, and ADFP at the application and procedural level. Policy is prescriptive as represented by Defence Instructions and has legal standing. Doctrine is not policy and does not have legal standing, however, it provides authoritative and proven guidance which can be adapted to suit each unique situation. 2. We have an unquestionable mandate from the nation to maintain and enhance Defence capability to meet whatever challenges lay ahead. Quality training is the foundation from which we will develop the people we need today with the capability to meet tomorrow s challenges and uncertainty. The Defence training model (DTM), for the first time, provides us with a common approach to quality vocational education and training (VET) across all the Services. 3. The DTM is based upon an update of the tried and proven systems approach to training with additional enhancements to better enable our trainers to deal with the increased tempo and uncertainty that are likely to remain features of our environment. These enhancements include a Rapid Development and Risk Management Process to better facilitate the timely availability of training resources when short notice training requirements arise. 4. The DTM is also aligned with the National Training Framework (NTF) so that Defence can take full advantage of national education and training systems and initiatives while reinforcing our already strong focus on quality assurance and continuous improvement. Alignment with the NTF also supports national recognition for ADF VET thereby aiding the recruitment, retention and eventually, transition back into civilian employment of our people. 5. This publication is the result of the efforts of many subject matter experts and trainers across Defence. The DTM is a significant milestone in our continual quest to improve the quality and efficiency of the training that is so vital to our work at home and abroad. 6. ADFP embraces complex but complementary subject matter. The reader should therefore consult the following publications, with which ADFP is congruous: ADDP D Foundations of Australian Military Doctrine, July 2005; and ADDP 7.0 Doctrine and Training, January iii

4 ADFP Every opportunity should be taken by the users of this publication to examine constructively its contents, applicability and currency. If deficiencies or errors are found, amendment action should be taken. Australian Defence Force Warfare Centre (ADFWC) welcomes any assistance or input from whatever source in order to improve this publication. iv

5 ADFP AMENDMENT CERTIFICATE Proposals for amendment of ADFP may be initiated in either of the following ways: By Minute to: Director Doctrine and Training Australian Defence Force Warfare Centre RAAF Base WILLIAMTOWN NSW 2314 By directly entering comment into the Joint Doctrine Development Environment (JDDE) found on the ADFWC Defence Restricted Network (DRN) web site (see VCDFweb/sites/adfwc/). Select JDDE on the ADFWC homepage and open either the ADDP or ADFP block as required. Open the relevant publication and utilise the Add Comment function button of the summary page for each publication. Note The second option is an addition to encourage feedback from the wider ADF, as well as encouraging use of the JDDE in general. Amendment Effected Number Date Signature Date This publication is current as at July This publication will be periodically reviewed and amended. The latest version of this publication is available on the ADFWC DRN web site v

6 ADFP DOCTRINE PUBLICATION HIERARCHY The hierarchy of ADDP and ADFP and the latest electronic version of all ADDP and ADFP are available on the JDDE found on the ADFWC DRN web site located at vi

7 ADFP Foreword Amendment Certificate Doctrine Publication Hierarchy Contents List of Figures List of Tables CONTENTS Page iii v vi vii xv xvii Paragraph CHAPTER 1 INTRODUCTION OVERVIEW OF THE DEFENCE TRAINING MODEL Introduction 1.1 Phases of the Defence training model 1.5 Management of the Defence training model 1.12 GOVERNANCE AND MANAGEMENT The role of the Defence training model in the management of Defence training 1.13 Application of the Defence training model 1.15 Governance responsibilities 1.16 Management responsibilities 1.17 THE ACCELERATED APPLICATION OF THE DEFENCE TRAINING MODEL Risk management 1.26 A RAPID DEVELOPMENT PROCESS CONTINUOUS IMPROVEMENT AND QUALITY ASSURANCE PROCURING TRAINING AND RESOURCES EXTERNAL TO DEFENCE Annex: A. An example of a rapid development process CHAPTER 2 THE ANALYSE PHASE Introduction 2.1 Starting point 2.3 Processes of the Analyse phase 2.4 Continuous improvement 2.12 Key outputs and responsibilities 2.13 Use of an external agency 2.15 CONDUCT NEEDS ASSESSMENT Inputs, process and products 2.24 vii

8 ADFP Step 1: Establish the Terms of Reference 2.25 Step 2: Determine information requirements 2.26 Step 3: Plan and gather information 2.28 Step 4: Collate and analyse information to make recommendations 2.31 Step 5: Report and record 2.32 ANALYSE THE JOB Inputs, process and products 2.38 Step 1: Establish scope of job analysis 2.39 Step 2: Determine information requirements 2.41 Step 3: Plan and implement information collection 2.45 Step 4: Collate and analyse information 2.50 Step 5: Develop job specification 2.51 Step 6: Gain authorisation of job specification and disseminate 2.53 ANALYSE TARGET POPULATION Inputs, process and products 2.59 Step 1: Determine the target population 2.60 Step 2: Determine information requirements 2.61 Step 3: Plan and gather information 2.62 Step 4: Collate and analyse information 2.66 Step 5: Develop a Target Population Profile 2.67 DETERMINE PERFORMANCE GAP Inputs, process and products 2.72 Step 1: Determine scope of performance gap 2.73 Step 2: Determine scale of performance gap 2.74 Step 3: Prepare a Performance Gap Statement 2.75 CONDUCT FEASIBILITY ANALYSIS Inputs, process and products 2.82 Step 1: Identify options for ensuring job holder competence 2.83 Step 2: Investigate options for ensuring job holder competence 2.89 Step 3: Compare options for ensuring job holder competence 2.91 Step 4: Identify options for assessing competence 2.94 Step 5: Investigate and compare options for assessing competence 2.96 Step 6: Report the feasibility analysis and file 2.99 SPECIFY TRAINING REQUIREMENT Inputs, process and products Step 1: Identify and apply existing training documentation Step 2: Customise existing training documentation viii

9 ADFP Step 3: Develop new competency standards Step 4: Setup competency details in Personnel Management Key Solutions Step 5: Determine training context and accreditation action Step 6: Develop a Training Requirement Specification Step 7: Gain formal approval of the Training Requirement Specification, action and file Annexes: A. Flow chart showing the Analyse phase of the Defence training model B. Defence training model purpose, inputs, process and products of the Analyse phase C. Defence job specification template D. Target Population Profile template E. Target Population Capability Record template F. Performance Gap Statement template G. Defence Feasibility Report template H. Training Requirement Specification template CHAPTER 3 THE DESIGN PHASE Introduction 3.1 Key product and responsibilities 3.4 Starting point 3.6 Processes of the Design phase 3.8 Continuous improvement 3.18 Use of an external agency 3.19 ANALYSE LEARNING AND ASSESSMENT ENVIRONMENTS Introduction 3.20 Inputs, process and products 3.26 Step 1: Determine information requirements 3.27 Step 2: Plan, prepare and implement information collection 3.30 Step 3: Collate and analyse information 3.32 Step 4: Develop and confirm environments profile 3.33 ANALYSE PROSPECTIVE LEARNERS Introduction 3.35 Inputs, process and products 3.40 Step 1: Determine information requirements 3.41 Step 2: Plan, prepare and implement information collection 3.43 ix

10 ADFP Step 3: Collate and analyse information 3.45 Step 4: Develop and confirm learner profile 3.46 ANALYSE COMPETENCIES Introduction 3.48 Inputs, process and products 3.53 Step 1: Analyse competencies to complete Competency Analysis Sheets 3.54 Step 2: Check completed Competency Analysis Sheets 3.66 GENERATE LEARNING OUTCOMES Introduction 3.67 Inputs, process and products 3.72 Step 1: Collect inputs 3.73 Step 2: Determine required performances 3.75 Step 3: Complete learning outcomes and assign course learning outcome levels 3.83 Step 4: Write outcome assessment methods 3.89 Step 5: Check learning outcomes and file 3.92 DESIGN ASSESSMENT STRATEGIES Introduction 3.94 Inputs, process and products Step 1: Determine assessment needs and draft the strategy Step 2: Select assessment methods Step 3: Refine the assessment strategy Product input to a training and assessment strategy document DESIGN TRAINING STRATEGY AND PRODUCE THE TRAINING AND ASSESSMENT STRATEGY DOCUMENT Introduction Inputs process and products Step 1: Select training delivery methods Step 2: Apply other principles of learning theory Step 3: Select associated media Step 4: Combine training and assessment strategies Step 5: Develop course map Step 6: Complete training and assessment strategy Step 7: Gain approval COMPILE CURRICULUM Introduction Inputs, process and products Step 1: Compile curriculum Step 2: Confirm curriculum x

11 ADFP DESIGN TRAINING REVIEW AND EVALUATION STRATEGIES Introduction Inputs, process and products Step 1: Determine need for training review and evaluation Step 2: Determine information requirements Step 3: Select information sources and collection methods Step 4: Record the training review and evaluation strategies COMPILE DRAFT TRAINING MANAGEMENT PACKAGE Introduction Inputs, process and products Step 1: Compile Training Management Package Step 2: Gain formal endorsement and approval of draft Training Management Package Step 3: Submit for accreditation Annexes: A. Flow chart showing the Design phase of the Defence training model B. Defence training model purpose, inputs, process and products of the Design phase C. National assessment principles and their essential features D. Factors to consider when selecting appropriate training modes E. Some criteria for selecting appropriate training activities F. Guidance for estimating the costs and times of developing new training materials G. Model course map and explanation H. Environments profile template I. Learner profile template J. Competency analysis sheet template K. Training and assessment strategy template L. Training review and evaluation strategies template M. Training management package indicative template xi

12 ADFP CHAPTER 4 THE DEVELOP PHASE Introduction 4.1 Key products and responsibilities 4.3 Starting point 4.5 Processes of the Develop phase 4.6 Continuous improvement 4.12 Use of an external agency 4.13 CHECK EXISTING TRAINING MATERIALS AND EQUIPMENT Introduction 4.14 Inputs, process and products 4.18 Step 1: Identify requirements 4.19 Step 2: Research existing Defence materials and equipment 4.22 Step 3: Research external sources 4.23 Step 4: Decide and record whether to acquire existing or develop new materials and equipment 4.24 OBTAIN MATERIALS AND EQUIPMENT FOR ASSESSMENTS AND TRAINING Introduction 4.25 Inputs, process and products 4.28 Step 1: Obtain training materials and equipment 4.29 Step 1.1: Obtain existing materials and equipment 4.31 Step 1.2: Customise materials and equipment 4.32 Step 1.3: Produce materials and equipment 4.35 Step 1.4: Formatively evaluate materials and equipment 4.38 Step 2: Add materials to draft Training Management Package 4.42 PREPARE MAJOR TRAINING RESOURCES PILOT ASSESSMENTS AND TRAINING Introduction 4.47 Inputs, process and products 4.52 Step 1: Plan the pilot assessment and/or training 4.53 Step 2: Plan and prepare the evaluation of the pilot assessment and/or training 4.55 Step 3: Implement the assessment and/or training 4.61 Step 4: Review, report and follow-up the pilot assessment and/or training 4.63 Step 5: Approval of Training Management Package 4.66 Step 6: Set up the course 4.67 Annexes: A. Flow chart showing the Develop phase of the Defence training model xii

13 ADFP B. Defence training model purpose, inputs, process and products of the Develop phase CHAPTER 5 THE CONDUCT PHASE Introduction 5.1 Key product and responsibilities 5.4 Starting point 5.6 Processes of the Conduct phase 5.7 Continuous improvement 5.16 Use of an external agency 5.17 ADMINISTER TRAINING PREPARATION Introduction 5.18 Inputs, process and products 5.21 Step 1: Promote the training 5.22 Step 2: Training nominations and panelling 5.25 Step 3: Prepare major resources 5.28 Step 4: Prepare materials and equipment 5.30 Step 5: Manage course sessions 5.32 IMPLEMENT PRE TRAINING ASSESSMENTS/TESTS Introduction 5.36 Inputs, process and products 5.38 Step 1: Assess/test trainees 5.41 Step 2: Evaluate the evidence 5.44 Step 3: Instigate appropriate action 5.45 CONDUCT TRAINING Introduction 5.46 Inputs, process and products 5.50 Step 1: Prepare training activity 5.53 Step 2: Deliver training activity in accordance with the plan 5.55 Step 3: Monitor trainees progress and adjust training as required 5.57 CONDUCT ASSESSMENT Introduction 5.58 Inputs, process and products 5.62 Step 1: Prepare summative assessments 5.63 Step 2: Conduct summative assessments 5.66 Collect the evidence 5.67 Evaluate the evidence 5.69 Confirm the results 5.71 Step 3: Evaluate assessment conduct methodology in accordance with evaluation plan/strategy 5.72 Moderation 5.79 xiii

14 ADFP POST COURSE ADMINISTRATION Introduction 5.80 Inputs, process and products 5.82 Step 1: Record and report outcomes 5.83 Step 2: Instigate training failure management 5.84 Step 3: Housekeeping 5.85 Step 4: Issue course attendance certificates/records of attainment 5.86 CONDUCT TRAINING REVIEW (INTERNAL EVALUATION) Introduction 5.88 Inputs, process and products 5.91 Step 1: Plan and prepare training reviews 5.95 Step 2: Collect, collate and analyse training review information 5.99 Step 3: Report training review Annexes: A. Flow chart showing the Conduct phase of the Defence training model B. Defence training model purpose, inputs, process and products of the Conduct phase CHAPTER 6 THE EVALUATE PHASE Introduction 6.1 Key products and responsibilities 6.4 Steps and products of the Evaluate phase 6.7 Step 1: Plan and prepare the evaluation 6.9 Step 2: Collect and collate evaluation information 6.21 Step 3: Analyse and synthesise evaluation information/data 6.24 Step 4: Report evaluation 6.27 Annexes: A. Flow chart showing the Evaluate phase of the Defence training model B. Defence training model purpose, inputs, process and products of the Evaluate phase Glossary Acronyms and Abbreviations xiv

15 ADFP LIST OF FIGURES Figure Title Page 1 1 Phases of the Defence training model Analyse phase of the Defence training model Hierarchical structure of work Algorithm illustrating the decision-making process Design phase of the Defence training model Indicative relationships between jobs, competencies, learning outcomes, and training Training review, evaluation and the training process Kirkpatrick s model for evaluating training programs The Four Levels G 1 Course map 3G 1 3K 1 Course map 3K Develop phase of the Defence training model Conduct phase of the Defence training model Critical questions in the review of assessment processes Evaluate phase of the Defence training model 6 1 xv

16

17 ADFP LIST OF TABLES Table Title Page 2 1 Inputs, process and products of the conduct needs assessment process Inputs, process and products of the job analysis process Inputs, process and products of the analyse target population process Inputs, process and products of the performance gap process Inputs, process and products of the feasibility analysis process Inputs, process and products for specifying the training requirement Inputs, process and products of a competency analysis Inputs, process and products of a learner analysis Inputs, process and products of a competency analysis Inputs, process and products of learning outcomes Course learning outcome levels Inputs, process and products for designing an assessment strategy Inputs, process and products for designing a training strategy Inputs, process and products for compiling curriculum Inputs, process and products for designing training review and evaluation strategies Inputs, process and products for compiling a draft Training Management Package L 1 Training review strategy 3L 2 3L 2 Evaluation strategy 3L Inputs, process and products for checking existing training materials and equipment Inputs, process and products for obtaining training materials and equipment Inputs, process and products for pilot assessments and training 4 12 xvii

18 ADFP Inputs, process and products of administering training preparation Inputs, process and products for implementing pre-testing/assessments Inputs, process and products of conducting training Inputs, process and products for conducting assessment Inputs, process and products for post-course administration Inputs, process and products for conducting training review 5 21 xviii

19 ADFP Chapter 1 CHAPTER 1 INTRODUCTION 1 OVERVIEW OF THE DEFENCE TRAINING MODEL 1.1 Introduction This chapter provides an overview of the Defence training model (DTM). In addition it introduces the phases of the model and significant features of their management. 1.2 Terminology. Much of the terminology used in this publication is national terminology, defined by the Department of Education, Science and Technology (DEST) for application within the National Training Framework (NTF). 1.3 Purpose. The purpose of the DTM is to achieve training that meets work force capability requirements effectively and efficiently. It does this by providing a common approach to work force performance problems, an approach that: a. directs training towards workplace performance needs; b. employs training strategies that reflect best practice, are feasible, and make the best use of available resources; c. supports management of training implementation; and d. bases decisions to improve training effectiveness and efficiency on evaluation information. 1.4 Underlying principles. The delivery of quality training in Defence and the content of the DTM are based on six principles: a. Principle 1. All Defence personnel, particularly leaders and managers, are responsible for enhancing Defence capability by developing their people and themselves. b. Principle 2. Defence training, education and other development experiences, develop Defence capability by preparing the work force for the roles, duties and tasks assigned and anticipated. c. Principle 3. Defence training uses common terminology, processes and methods wherever appropriate. 1 1

20 ADFP Chapter 1 d. Principle 4. Defence uses a systems approach that ensures training: (1) is directly related to improving the skills, knowledge and attitudes required for Defence capability; (2) is the most cost-effective intervention; (3) uses learner-centred approaches to delivery; and (4) is subject to continuous improvement 1. e. Principle 5. Defence training and assessment applies competency-based training and assessment (CBTA) in accordance with the NTF 2 where practicable and cost-effective to do so. f. Principle 6. Defence training and assessment is based on current, nationally endorsed training packages, where these are available and meet Defence requirements. Phases of the Defence training model The phases of the DTM are represented in figure 1 1. The figure shows a cyclic process structured to enable quality assurance and continuous improvement. 1 Continuous improvement processes should normally accord with the quality requirements of the Australian Quality Training Framework (AQTF) Standards for Registered Training Organisations (RTO). 2 The Defence application of CBTA in accordance with the NTF is described in Defence Instruction (General) (DI(G)) PERS Implementation of the National Training Framework within Defence vocational education and training. 1 2

21 ADFP Chapter 1 Figure 1 1: Phases of the Defence training model 1.6 The model represents the Defence approach to training. The realities of training are that: a. the entry point for the cycle depends on the aspect of training under consideration; b. work flows, particularly during the Design and Develop phases, can be two-way; c. phases may overlap and may be addressed concurrently; and d. the processes of continuous improvement overlap and link all phases. 1.7 Analyse phase. The Analyse phase of the DTM is when the need for training (new or revised) to improve human performance is analysed and specified. This phase incorporates six main processes. To begin, a needs assessment determines that training is likely to be required for modifying workplace performance. With this established, a job analysis identifies the characteristics of the job performance required, and a target population analysis identifies the characteristics of the group of people who are to undertake that performance. A comparison of the two sets of characteristics identifies the existing performance gap. A feasibility analysis is then undertaken to identify the best solutions for closing this gap. If training is a recommended solution, details of the required workplace performances are recorded in a Training Requirement Specification (this document becomes the starting point of the Design phase of the DTM). If other solutions are recommended, they are reported to appropriate authorities. 1 3

22 ADFP Chapter Design phase. The Design phase of the DTM is when the assessment and learning strategies for training are selected, together with the strategies for evaluating the whole process of preparing for and conducting training. The object of these strategies is to ensure that the learning is achieved as efficiently and effectively as possible, and that it meets the original requirement. This phase begins with three analyses that may be undertaken concurrently. The competencies required for the job are analysed for their constituent skills, knowledge and attitudes, and the characteristics of both the likely learners and the circumstances in which assessment and learning could occur are identified. The products of these three analyses are used to generate the intended learning outcomes of the training. Once this end point is understood, strategies for both the assessing the achievement of these outcomes and the conduct of training are developed. These strategies are then combined to create a curriculum the document that provides the specifications of the training to be conducted. With the curriculum established, strategies for the Training Review and Evaluation (TR&E) of the training can be developed. Finally, the strategies are combined to form the draft Training Management Package (TMP). 1.9 Develop phase. In the Develop phase of the DTM, the materials and equipment needed to conduct training and evaluate trainee performance are prepared in accordance with the agreed strategies, and a trial of the whole program/course is conducted. Important activities within this phase are the development of assessment and training materials, the compilation and development of learning materials and equipment (including technology-based aids), and the preparation of TR&E materials. The optimal sequence of learning activity is also mapped and learning facilitators/trainers and assessors are appropriately prepared. When ready, the developed training is tested and, where necessary, revised. The major product of the Develop phase is a fully developed and tried program/course presented as a TMP Conduct phase. The Conduct phase of the DTM involves the ongoing implementation of training and associated assessment, and the training review (sometimes referred to as internal evaluation) processes. Fully developed training courses are programmed, administered, conducted, assessed and recorded using the materials and equipment prepared in the Develop phase (notwithstanding this, within the parameters of the training documentation, learning staff may adapt the materials to suit their situation). Training may be conducted in training units, the workplace or other locations. The Conduct phase can incorporate recognition processes to determine whether individuals already have the required competencies, proficiencies or other attributes. 1 4

23 ADFP Chapter Evaluate phase. Evaluation is the process of checking how well training has prepared learners for the workplace and whether the original capability requirement has been met. Evaluation identifies under- and over-training, the strengths and shortfalls of training, and makes recommendations for improvement. These recommendations thereby become inputs to the Analysis and other phases for the next cycle of the DTM. Evaluation incorporates four processes: planning the work, collecting information from the job environment, analysing and synthesising that information to identify ways in which training can be improved, and reporting to the appropriate authorities. Management of the Defence training model Training application and implementation responsibilities. The DTM applies to all Defence training (individual and collective training 3 ), whether developed or being developed, and whether sourced internally or from an external agency. Responsibility for applying the model rests with the authorities designated by the Defence organisation with the controlling interest in the workplace performance and/or the training. These authorities are responsible for managing the respective DTM processes and for approving and maintaining their products. GOVERNANCE AND MANAGEMENT 1.13 The role of the Defence training model in the management of Defence training The role of the DTM is to provide a common philosophy and consistent approach to the development of training within Defence. Head Defence Personnel Executive, as the Policy Owner (PO) for Human Resources within Defence, has delegated the Director-General Defence Education, Training and Development to act on their behalf to develop and coordinate Defence-wide education and training policy. The model is sponsored and managed by Defence Education, Training and Development as part of that responsibility Individual training is the responsibility of each Defence Group and each PO. This is achieved by each Service Training Command or equivalent operating as an RTO within the NTF. The Defence Learning Services Network operates as an RTO and is managed by the Defence Personnel Executive (DPE). 3 Although this publication is focused on individual training, the general principles have application to collective training. Further details on the relationship between individual and collective training is contained in Australian Defence Doctrine Publication (ADDP) 7.0 Doctrine and Training. 1 5

24 ADFP Chapter 1 Application of the Defence training model Defence training policy 4 requires that the DTM is to be applied to all Defence training, whether developed or being developed, and whether sourced internally or from an external agency. The policy acknowledges that the wide range of circumstances of Defence training is such that the DTM cannot always be fully applied. Where deviation from the DTM is necessary, it states that commanders and managers should ensure that the principles underpinning it should be followed as closely as possible. Governance responsibilities The governance of the DTM requires a whole-of-defence approach and is managed by the DPE Training Committee System. At the highest level are the Defence Education and Training Committee (DETC). At the working level, the subordinate committee which supports the DETC determinations is the Training Policy and Projects Group. Management responsibilities As the management of individual training is the responsibility of both Group and PO, each Group and PO should have a suitable organisational structure with clear lines of authority and accountability. At or near the top of this structure should be a position that designates the five authorities responsible for implementing the DTM The five authorities that oversee the phases of the DTM are the: a. Analyse Authority, b. Design Authority, c. Develop Authority, d. Conduct Authority, and e. Evaluate Authority Each authority is responsible and accountable for managing the processes within their respective phase. This involves ensuring that suitable resources are available and that appropriate procedures and methods are followed. The authorities are also responsible for the accuracy, reliability, feasibility and affordability of the outputs from DTM work. They are to ensure that all relevant factors and all feasible options have been considered. Finally, they are also responsible for the quality assurance and continuous improvement of the processes and outputs for which they have responsibility. 4 DI(G) PERS Implementation of the Defence Training Model in Defence Vocational Education and Training. 1 6

25 ADFP Chapter The close association of training design and development processes requires a close working relationship between the respective authorities. For this reason, the same person may undertake the roles of the Design and Development Authorities. To provide objectivity and rigour it is preferred that the Analyse and Evaluate Authorities are not the same as the Design, Develop and Conduct Authorities Policy Owner responsibilities. Under the governance arrangements for business skilling PO have responsibility for the training requirements for whole of Defence policy domains. Within these arrangements, PO have been specified by the Defence Committee as the Analyse and Evaluate Authorities for their domain, as well as having the responsibility for authorising the training strategy for personnel working in their domain. They are also responsible for ensuring that consultation and coordination is undertaken and maintained with other stakeholders, and sourcing the funding for the training programs required to support domain policy changes and new initiatives The PO may delegate a Business Process Owner (BPO) or another agency with the responsibilities of Design, Development and Conduct Authority for their domain. The BPO (or delegated authority) will negotiate with service providers (whether they are within Defence or an external organisation) for the provision of design, development and/or conduct services Rationalised Defence training. Where training is to be rationalised across Services and/or Groups and is not the responsibility of a PO, a Manager of Joint Training (MJT) is appointed. The MJT has the responsibility for forming a Training Advisory Group, comprising the major stakeholders. The MJT is responsible for ensuring that the DTM is applied to the rationalised Defence training. 1 7

26 ADFP Chapter 1 THE ACCELERATED APPLICATION OF THE DEFENCE TRAINING MODEL It is preferable for Defence training and assessment to be designed and developed in accordance with all the phases, processes and steps of the DTM. However, in response to an urgent or unforeseen requirement, it may be necessary to expedite the processes, which means that the DTM steps will need to be accelerated. This acceleration is to be based on a documented risk assessment. The decision to accelerate the DTM rests with the commander of the force 5, the Analysis Authority or the BPO. No matter what level of command or management is applying the DTM, a commander or manager must ensure the responsible staff are clear on the degree of risk that they are prepared to accept In considering the time available to plan, the relevant commander or manager should consider: a. the provision of specific direction limiting options and focusing training development staff on those planning aspects the training authority feels are most important; b. limiting the number of training options to be developed in order to achieve a workable plan that meets the workplace requirements in the time available; c. maximising parallel planning with the early issue of outputs, and the sharing of all information with other training authorities and stakeholders. While this is an extremely effective method of increasing tempo, it must be balanced against the risk that the efforts of the other authorities will be wasted. Risk management The DTM is a tool that allows for the management of risk within the training environment. Risk management is the systematic application of procedures and practises for analysing, assessing, controlling and monitoring risk and is applied throughout the DTM. Rather than limiting activities through restrictive safety concerns, risk management allows the full potential to be achieved through managing rather than avoiding risk. The identification of 5 During the mission analysis phase of the joint military appreciation process (JMAP), a review of own forces is completed. This review involves an examination of own forces to determine the capabilities the available assets can bring to the operation. This assessment examines issues such as readiness levels, training levels achieved and morale of personnel. The own forces review seeks to determine whether forces are at an appropriate level of readiness, with sufficient resources to conduct operations within a specified time frame. If they are not then the commander may require the DTM to be applied to bring those assets personnel to the required level of capability. Further details on the JMAP are in ADFP Joint Military Appreciation Process (to be issued) 1 8

27 ADFP Chapter 1 various methods of delivery allows the commander or manager to manage and at times ameliorate, the risk associated with the prosecution of a training activity. At all stages throughout the execution of the DTM the application of risk management allows a training authority to detail the cost of the planned options. This cost can be measured in different ways depending on the circumstances and direction from the higher levels. Risk management is concerned with balancing identified threats against the harm those threats may inflict to various outcomes The objectives of risk management 6 as they apply to the DTM are to: a. enhance operational effectiveness, and b. ensure training is conducted in a manner that ensures force preservation The enhancement of operational effectiveness is done by: a. facilitating employment of the most effective training techniques and procedures while managing the risks that are inherent in the conduct of training, b. ensuring that risk treatment measures can be tailored to meet training circumstances, and c. avoiding a risk averse approach to managing training Force preservation is achieved by: a. achieving a rigorous assessment of benefit against cost as a basis for decision-making; b. the acceptance of risk at an appropriate level of command or management; c. reinforcement of awareness of risks and the need for appropriate treatment measures (including policy and procedures) in the planning and conduct of training; and d. implementation of a process to identify risks and derive risk treatments that will mitigate risks as far as practicable The main point of risk management in training is that risks are best managed in the planning stages of an activity and in the development of policy regarding training management. 6 Staff undertaking risk management in the application of the DTM should consult the Defence policies on risk management found at the Defence Enterprise Risk Management web site see and seek specialist advice when required. 1 9

28 ADFP Chapter 1 A RAPID DEVELOPMENT PROCESS In those circumstances where there is a great urgency in having a training product there is the possibility of using a rapid development process (RDP), supported by a risk management plan. Using the RDP involves accelerating the DTM. Making such judgments requires a higher level of expertise than normally required for full implementation of the DTM. However, the decision to proceed with an RDP rests with the commander of the force, Analysis Authority or the BPO Guidelines for the acceleration of the DTM through an RDP are: a. the extensive use and customisation of existing learning, assessment and evaluation materials (if no such materials are available, the appropriateness of using the RDP process must be queried); and b. combining some of the processes across phases of the DTM. Annex A provides an example of how these guidelines are applied to the DTM The RDP described is not the only such approach available. However, the use of any RDP should facilitate the full application of the DTM at a subsequent time if circumstances justify it. CONTINUOUS IMPROVEMENT AND QUALITY ASSURANCE The DTM embodies the principles of continuous improvement and quality assurance. This accords with AQTF standards for RTO that state: a. the RTO must collect and analyse stakeholder and client feedback and satisfaction data on the services it provides, and b. the RTO must use the information mentioned to review its policies and procedures The DTM enables continuous improvement of Defence training through: a. a structure that provides for links across all phases, processes and steps; b. integrated formal quality assurance checks; 7 DEST Standard 1.9 a and b (08 June 2001). 1 10

29 ADFP Chapter 1 c. establishing and maintaining auditable records from all processes and products; and d. being a dynamic and transparent process The DTM is presented as a closed loop to represent its cyclical nature. The application of the DTM will result in training that is continually reviewed and improved. This continuous improvement is achieved through cooperative interaction between stakeholders which relies on communication that is timely, responsive and supportive Quality assurance checks. The principal quality checks incorporated in the DTM include the internal review processes Reporting. AQTF standards require RTO to document any actions taken to improve the quality and consistency of assessment and learning 8. Formal reporting of findings from quality assurance checks should be provided to stakeholders. Methods for tracking the progress of recommended actions should be devised for each phase Record keeping. It is essential that accurate and comprehensive records are maintained in accordance with Commonwealth and AQTF audit requirements. PROCURING TRAINING AND RESOURCES EXTERNAL TO DEFENCE A Defence decision to acquire a product or service by contract is made when the product or service cannot be gained internally. All or any of the DTM processes may be sourced externally. While Defence can outsource all or any part of the DTM processes, under national and Defence guidelines, the designated authority has responsibility for all outputs and outcomes. Particular attention should be given to the Analyse and Evaluate phases if these are outsourced. Under Defence guidelines, the products developed must be subject to review by an agency independent of the developer prior to acceptance. 8 DEST Standards 9.2 ii and 9.3 i (08 June 2001). 1 11

30 ADFP Chapter Outsourcing and nationally recognised qualifications. Due care must be taken if outsourced training leads to the award of a nationally recognised qualification under the Australian Qualifications Framework. If the commercial training provider is awarding the qualification, the provider must be an RTO with the qualification listed under their scope of registration. If the qualification is to be awarded under the scope of a Defence RTO, then the contract governing the service must explicitly state the responsibilities for meeting required AQTF standards Guidelines and instructions. The Commonwealth and Defence provide guidance and instructions to manage outsourcing. The Commonwealth Procurement Guidelines and Best Practice Guidance clarify what is required and expected in Commonwealth purchasing. Advice on the procedures for the procurement of goods and services in Defence can be found on the Defence Materiel Organisation web site see The capability and operational outcomes of Defence are dependent on the performance of its people. The DTM is a systematic approach to developing the competencies of Defence personnel. The DTM ensures a common and consistent approach to the analysis, design, development, conduct and evaluation of training. It incorporates quality assurance and continuous improvement. The DTM is sufficiently general for application to all Defence training requirements To support the successful implementation of the DTM there must be in place appropriate governance and management processes as well as adequate resources. This chapter has described how these two elements might be addressed. Finally, a methodology has been discussed for making training available in an expeditious manner. Adopting this approach involves special circumstances and the development of a thorough risk analysis. Annex: A. An example of a rapid development process 9 DEST Standard 1.6 for RTO (08 June 2001) states that: a. The RTO must have, and comply with, a written agreement with each organisation that provides training and/or assessment on behalf of the RTO. b. The agreement must specify how each party to the agreement will discharge its responsibilities for compliance with the Standards for RTO. c. The RTO must maintain a register of all agreements made under Standard 1.6.a. 1 12

31 Annex A to ADFP Chapter 1 AN EXAMPLE OF A RAPID DEVELOPMENT PROCESS Phase Process The application of the rapid development process Analyse Analyse the job Identify available job specifications. Analyse the target population Determine the performance gap Conduct feasibility analysis Using either focus groups or interviews with key personnel, identify the performance gap statement, possible training and non-training solutions to the performance gap, and gather data for the Target Population Profile. If training forms part of the agreed solution, document the requirement in the form of a Training Requirement Specification (TRS). The job/policy/business Process Owner is to formally agree to the TRS. This agreement, together with the TRS is to be placed on the program/course file. A 1A 1

32 ADFP Phase Process The application of the rapid Design Analyse prospective learners Analyse learning and assessment environments Generate learning outcomes Design assessment and training strategies development process In the rapid development process, this is best done in conjunction with the Analyse phase. Identify, seek out and assemble existing, relevant learning outcomes; assessment, training, Training Review and Evaluation (TR&E) strategies; assessment guidance; materials and equipment for: assessment, training and learning, TR&E. Do these resources address the requirement? If so, use them. If the materials are not immediately useable, can they be readily customised to address the requirement? If so, customise them. Are additional resources needed? If yes, determine what is needed and decide how best to acquire them. Action this determination. Determine evaluation requirement Plan and develop evaluation Decide the minimum acceptable level of evaluation and draft an outline evaluation plan and program. Note: the outline evaluation plan and program and the development of required materials can then be completed during the Conduct phase. The evaluation plan is to be placed on file as it is developed. 1A 2

33 ADFP Phase Process The application of the rapid Develop Conduct Evaluate Produce/procure training material and equipment (commercial-off-theshelf should be considered in the first instance) Prepare/brief administration staff, trainers and assessors Administer assessment and training preparations development process Compile the training, learning and assessment materials into a logical learning order for the Conduct phase. Write the learning outcomes based on the materials assembled and prepared. Confirm the validity of the learning outcomes against the TRS. Amend, redevelop as necessary and place a copy on the program/course file. Keep stakeholders informed throughout the development process. Use existing, proven staff, procedures and processes as much as possible. The Conduct phase should not be condensed or abbreviated. It is to be implemented in accordance with guidance provided in the Defence training model (DTM). The Evaluation phase should not be condensed or abbreviated. It should be implemented in accordance with guidance provided in the DTM. 1A 3

34

35 ADFP Chapter 2 CHAPTER 2 THE ANALYSE PHASE 2 Introduction The Analyse phase is the first of five phases constituting the Defence training model (DTM), as shown in figure 2 1. It is during this phase that the need for training is analysed and specified. Figure 2 1: Analyse phase of the Defence training model 2.2 As with all phases of the DTM and their constituent processes and steps, the level of application should match the circumstance. The guidance provided in this, and the other process manuals, is applicable to situations of high significance (eg the introduction of training in support of a major new Defence capability). Abbreviated processes may be appropriate in some circumstances. Starting point The DTM is applied to produce new, or to modify existing training. There are three circumstances in which its first phase, the Analyse phase, is used: a. A new capability requirement. New training may be necessary if job holders are required to contribute to a new or modified capability requirement. The requirement might arise from the purchase of new, or modifications to existing 2 1

36 ADFP Chapter 2 equipment, software, or technology; or a change in employment area, legislative requirements, product requirements, or employment specifications. b. Continuous improvement and evaluation feedback. Feedback from continuous improvement processes and evaluation can indicate that the training presently conducted is failing to meet workplace needs and/or it can be improved. c. Periodic review of competency standards. Changes to competency standards are likely to require corresponding amendments to training. Periodic reviews of competency standards may be initiated either from within or external to Defence. (Competency standards forming part of nationally endorsed training packages are reviewed, on average, every three to five years.) Processes of the Analyse phase The Analyse phase is made up of six processes: a. needs assessment; b. job analysis; c. target population analysis; d. determine performance gap; e. feasibility analysis; and f. Training Requirement Specification (TRS). The first of these, needs assessment, might be considered external to the DTM however, some form of needs assessment precedes all applications of the DTM. 2.5 The flow chart in annex A illustrates the processes and outputs of the Analyse phase of the DTM, and annex B presents the purpose, inputs and products of each of the processes in tabular form. 2.6 Needs assessment. The DTM is applied in response to an identified need, where a need is the difference between the present and a required performance in a workplace. Before the DTM as a whole, or any of its parts is applied, some form of needs assessment is completed and training is identified as a possible solution to the problem. 2 2

37 ADFP Chapter Job analysis. A job analysis identifies the performance requirement, both current and foreseen, for a given employment or job classification (such as task, job, or career/occupation). The requirement is recorded in terms of the duties, tasks, sub-tasks and elements that constitute the job. A job analysis may involve any one or combination of: a document search; direct observation; survey; and/or consultation with Policy Owners (PO), Business Process Owners (BPO), capability and trade sponsors, supervisors, managers and job holders. It also involves research of relevant existing competency standards and units and other proficiencies. Determining the training requirement requires understanding of what the job consists of, the standards of performance required, the context of performance, and how performance is judged. This information is recorded in a job specification. 2.8 Target population analysis. To identify what training is required, how much, and how it might best be conducted the characteristics and qualities of the target population must be understood. This is achieved through a target population analysis. The first step is to determine who comprises the target population (possibly the present job holders). The current competencies of its members are then identified, together with their typical characteristics and qualities (such as academic record and ability, employment experience, age, gender, physical characteristics, learning preferences, special aptitudes and motivation). This information is obtained from workforce records and plans, and from education and training records; other elements can be gained from surveys of likely learners and their supervisors/managers, or by direct assessment. Finally the information is recorded in a Target Population Profile. 2.9 Identification of the performance gap. A comparison of the competencies required to perform the job and those of the target population is used to identify the performance gap the difference between current and required competence. The difference is recorded as a Statement of Performance Needs Feasibility analysis. There are several ways in which a deficiency in workplace competency can be met, of which training is just one. A feasibility analysis involves the determination of the best option for meeting the need. The analysis should include a cost/benefit analysis of training and non-training options, and should conform to current policies and instructions. Non-training options may include recruitment, automation, changing the job, re-allocating the job, and modifying the workplace. Three categories of training options should be considered: in-house, outsourced and a combination of these. For each option several factors must be reviewed, including the existence of similar training; the potential for using attendance, distance, and on the job methods; and the availability of training facilities and staff. The product of an analysis is a Feasibility Report that, when approved, is presented to the appropriate authorities. 2 3

38 ADFP Chapter Training Requirement Specification. Having identified the workplace performances for which training will be conducted, the final process of the Analyse phase is to formally document the requirement in a TRS. This specification records the requirement in terms of the people (numbers required, rates of throughput, any time limitations, and the context and recognition of their training) and the competencies they need to undertake the job 1. In preparing the document, relevant existing training is investigated and decisions made on whether it might be used unchanged, or whether it might be customised to meet Defence needs. If no existing competency standards apply, new ones are written. The TRS lists the competency standards requiring training together with other important factors defining the training requirement. It provides the baseline against which subsequent training activity is measured. Continuous improvement While working on any of the processes of the Analyse phase, stakeholder input is sought for help in assuring the quality of the product. A consequence is that the product is likely to require repeated revisions before acceptance. When each process and step is completed, an accurate and comprehensive record of the decisions contributing to it should be maintained. This record is likely to be referred to during all subsequent development of the training, in the near and long-term. Key outputs and responsibilities While each process of the Analyse phase has outputs, the two key products of the phase are the job specification and the TRS Training Analysis Authority. The PO/BPO is the Training Analysis Authority for approving the documents of the phase, particularly its key outputs, in accordance with their responsibility for overseeing the work standards of an occupation (category/employment category number/ mustering etc). The sponsor is also responsible for retaining key documentation for audit, quality assurance and continuous improvement purposes. Notwithstanding the responsibilities, the Analyse phase is performed jointly by representatives of both the training authority and the job/occupational sponsor. 1 The use of competency standards, units and elements of competence should accord with the guidelines of the National Training Framework (NTF described in Defence Instruction (General) (DI(G)) PERS Implementation of the National Training Framework within Defence vocational education and training. 2 4

39 ADFP Chapter 2 Use of an external agency In accordance with national guidelines, the quality assurance of all training by, or on behalf of Defence (by Defence staff, contractors, professional service providers, or other Registered Training Organisation (RTO)) remains the responsibility of Defence. As the Analyse phase is one of the two key phases of the training cycle 2, it should not be contracted out except where the scale of the project is beyond the in-house capacity and/or the necessary expertise is not available within Defence. CONDUCT NEEDS ASSESSMENT The DTM is a cyclical system. It is applied to a workplace situation when a difference is perceived between present and desired performance, and training is perceived as a way of resolving the difference. The workplace performance, or job, might be new as when new equipment, systems or procedures are introduced or the desired improvement might arise from policy changes, or simply when deficiencies in current performance are thought to exist Before the DTM is applied, the nature of the required performance should be identified and some assurance gained that training can significantly contribute to achieving it. This is accomplished through a needs assessment Training is not the only way of rectifying or improving workplace performance. In fact, training is often only one component of a total performance solutions package. In such situations all components are important, and training by itself may not solve the over-all performance deficiency. Other components to a solution package may include: a. improving motivation by providing incentives; b. employing staff with proven competence in the job, reorganising the job to make better use of existing staff; c. provision of good quality job aids; d. removal of obstacles preventing good performance; e. automating all or parts of the job; f. applying more or enhanced resources to it (including time); and g. improving the quality of materiel used in the job, or the environment in which it occurs. 2 The other key phase is the Evaluate phase. 2 5

40 ADFP Chapter Any one or combination of these options might be the most cost-effective means of improving workplace productivity/performance A needs assessment is used to justify the initial application of the DTM to a performance problem and, at a lower level, its application for updating and amending existing training. The resources applied to the process should be proportional to the situation and its associated risks Purpose. The purpose of needs assessment is to scope the nature of the performance problem or opportunity, and to determine whether the DTM should be applied Responsibility. Responsibility for managing the Analyse phase of the DTM lies with the Analysis Authority of the organisation owning the job performance. The Authority, or their delegate, should be satisfied that an adequate needs assessment has been conducted before authorising work on subsequent processes of the phase Triggers. The triggers for conducting a needs assessment include: a. new or modified job/employment requirements (possibly caused by new equipment and/or processes); b. changes to a national training package; or c. feedback from the Evaluation phase. Inputs, process and products The inputs, process and products of the conduct needs assessment process are summarised in table 2 1. Inputs Process Products Directive, operational report, new or modified job or training package data, Evaluation Report, existing training. Step 1: Establish the Terms of Reference (TOR). Step 2: Determine information requirements. Step 3: Plan and gather information. Step 4: Collate and analyse information to make recommendations for further action. Step 5: Report and record. A Needs Assessment Report that provides a clear indication of the nature of the performance problem/opportunity, including its causes, and recommends further action, possibly including application of the DTM. Table 2 1: Inputs, process and products of the conduct needs assessment process 2 6

41 ADFP Chapter 2 Step 1: Establish the Terms of Reference To ensure that subsequent work focuses on an agreed requirement, a TOR for a needs assessment should be produced. The terms should clearly establish the scope of the job/workplace performance being considered and, if relevant, how it relates with associated work in the organisation. It should present the background that gave rise to the perception that workplace performance can be improved. This background might describe any relevant history, the current position, and any foreseen developments. The terms should present any pertinent time factors, indicate any urgency involved, identify the agency responsible for the conduct of the assessment, and any specific reporting requirements. Step 2: Determine information requirements The information needed to assess possible solutions to a perceived workplace performance problem will be context specific, as directed by the TOR. In most instances information will be required about: a. the current and desired workplace output, the difference between them, and what caused this difference (including history, directives, any benchmarks); b. the factors that contribute, or could contribute to the output (including the physical factors of environment, equipment, materiel; and the human factors of work force size, location, capability, management, and motivation; and any policy or funding factors); and c. the stakeholders involved with the employment and training of the relevant work force Consideration of these types of information will give rise to suggestions for achieving the desired levels of performance. These suggestions will themselves need to be researched so that their validity and value can be assessed. Step 3: Plan and gather information The planning and gathering of information for the needs assessment should identify the method to be used, the resources required, the means of acquiring the resources and the time frames involved. Methods of collecting information can include: a. workplace observation; b. focus groups; 2 7

42 ADFP Chapter 2 c. communication with individuals (using recorded interviews and s); d. group surveys; and e. research of documents and records The method(s) used and the scope of the needs assessment will determine what information collection materials should be prepared and, possibly, tested The emphasis given to planning and collecting the information required should be proportional to its significance. The work should involve representatives of all stakeholders in the outcome (workplace performance). This includes the job or policy/bpo, a sample of any existing job holders, those responsible for resources and policies contributing to the achievement of the required performance, and the customers of that performance. Existing training providers and reference groups should also be considered. Step 4: Collate and analyse information to make recommendations Once gathered, the information is collated and analysed. Particular attention is to be paid to identifying all options for achieving the required performance, not just the option of training people. The options need to be considered individually and in combination. They should be considered against such factors as: a. the extent to which the performance is required (including the number/percentage of people involved); b. the consequences of error if the performance is inadequate (threats to life and/or security); c. the consequences of task delay (should the performance be automatic, instinctive or considered?); d. the resource implications of implementing each solution; e. the timeliness of the solution (including the scope for short-term and more strategic solutions); f. integrating solutions; and g. access issues (geographic locations, mobility etc). From such consideration arguments for and against the implementation of each option are developed. These arguments will commonly cover the benefits, costs, and associated risks in relation to the expected performance 2 8

43 ADFP Chapter 2 improvement. Logical consideration of these factors leads to recommendations being derived on which of the option(s) should be implemented. Step 5: Report and record The needs assessment should be recorded, reported and formally acknowledged as the basis of pursuing options to improve workplace performance. The Defence paper format detailed in Australian Defence Force Publication (ADFP) Defence Writing Standards (currently ADFP 102) should be used for presenting the Needs Assessment Report The scale and scope of training needs assessments vary, and so therefore should the administrative effort applied to them. For a workplace output of low risk the effort might be minimal, but for high risk projects the effort required, and the detail of the analysis involved, would be greater. ANALYSE THE JOB Job analysis is the process of determining/confirming the workplace performance requirement, both current and foreseen, for a given domain or employment classification (task, job, or career/occupation). This performance requirement is recorded in a job specification. This document provides a benchmark for all subsequent related work Purpose. The purpose of a job analysis is to formally describe the career/job/task capability requirement in terms that can be commonly understood and used for training and other purposes Structure of jobs 3. A job specification may be limited to one job within a career or occupation, or might cover all jobs within a specific career, occupation or employment group. Alternatively it can also be applied to describing the requirements of a particular domain (such as finance) which crosses employment classification boundaries. To understand workplace performance requirements, especially those that contribute to careers/occupations, it is useful to view jobs as part of hierarchical structure of work. The following diagram illustrates such a structure: 3 The DTM uses the term job generically to describe the whole hierarchy of tasks, duties, jobs and careers. 2 9

44 ADFP Chapter 2 Note Figure 2 2: Hierarchical structure of work Simple definitions of the terms used in the diagram are provided in the following table: Term Definition Example Career or occupation Job Duty Sequence of related jobs requiring developing qualifications and/or experience A post of employment comprising a number of duties and tasks A major area of job responsibility comprising a grouping of closely related tasks Nurse Pilot Electronics technician Casualty nurse Seahawk pilot Diesel engine maintainer For diesel engine maintainer: maintain and repair engine cooling systems tune diesel engines Task A discrete unit of work For diesel engine maintainer: conduct periodic services report major malfunctions 2 10

45 ADFP Chapter 2 Term Definition Example Sub-task A step in achieving a task For conduct periodic services: inspect/clean filters check/top-up coolant level Task element A step in achieving a sub- task For check/top-up coolant level: visually check level obtain coolant top-up coolant 2.37 Trigger. The trigger for this process is the approval of a needs assessment s recommendation, as signified by the Analyse Authority (or their delegates) sign off on the Needs Assessment Report. Inputs, process and products The inputs, process and products of the job analysis process are summarised in table 2 2. Inputs Process Products Existing or related job specifications, job duty statements and selection criteria, Australian Public Service work level standards, process manuals current competency standards, project requirements, exercise/operation reports, observations, research, surveys. Consultation with BPO, capability and trade sponsors, supervisors/managers and job holders and legislative and licensing requirements and organisational policies. Step 1: Establish scope of job analysis. Step 2: Determine information requirements. Step 3: Plan and implement information collection. Step 4: Collate and analyse information. Step 5: Develop job specification. Step 6: Gain authorisation for the job specification and disseminate. An authorised job specification that provides a baseline description of the human performance requirements together with associated information. Table 2 2: Inputs, process and products of the job analysis process 2 11

46 ADFP Chapter 2 Step 1: Establish scope of job analysis Before work begins on a job analysis, the scope of the task should be defined, understood, agreed and recorded by key stakeholders 4, possibly using a TOR that refers to any associated needs analysis. The importance of the task should be understood and any risk and resource (including time constraints) identified In establishing the scope it is important to determine whether a new job specification is required or whether an existing one will suffice. This requires locating existing documentation, comparing it with the problem/opportunity and deciding whether a new job specification is required. Step 2: Determine information requirements Decisions affecting Defence jobs should be based on knowledge and understanding of those jobs and their relationships with others. The knowledge needed for a job analysis is common to all situations, and can be classified as information about the requirement for the job, and the characteristics of the job: a. Job requirement: (1) number of job positions (which may change over time); (2) location of positions; (3) anticipated staff turnover; and (4) importance. b. Job characteristics: (1) classification (ie how they fit in organisational structures); (2) components (ie contributing duties, tasks, and sub-tasks); (3) required performance levels for the tasks and sub-tasks; (4) inter-relationships with other jobs (including pathways between jobs and key impacts on the abilities of other jobs to be performed effectively); 4 Key stakeholders are the sponsor or BPO for the job, the staff undertaking the analysis, existing training providers/managers, and possibly representatives of both the job holders and their supervisors who are likely to be most affected. 2 12

47 ADFP Chapter 2 (5) required equipment and materiel; (6) physical conditions and social environment (including locations, levels of teamwork and operational circumstances); (7) physical and mental skills and demands (that relate to the conditions); (8) need for job holder attributes/qualifications; and (9) links with employment and career factors The currency of the information should be considered when determining information requirements. Any anticipated changes should be researched and recorded to inform decision-making Where there are no previous records to work with or there are substantial changes to job requirements the information requirements of a job analysis can be large. Where records of preceding analysis exist, or there are records of associated analyses, the information in these may be current, and checking, updating and/or supplementing those records may only be required The information gathered by a job analysis has utility for other fields of human resource management apart from training. Consideration should therefore be given to advising managers of related responsibility areas (eg work study, organisational structures, pay and conditions) with a view to sharing the products of the analysis and the feasibility of tailoring the information requirement to meet their needs. Step 3: Plan and implement information collection When the nature and extent of the job information requirement is known, its collection can be planned and implemented. Planning should incorporate: a. selecting sources of information (which may involve sampling); b. selecting collection methods; c. costing data collection; d. gaining, where necessary, formal endorsement of the plan; e. preparing for the collection, including the planning of visits, observations; 2 13

48 ADFP Chapter 2 f. meetings and the preparation of any data collection tools; and g. consideration of any confidentiality, validation and security implications affecting the process Likely information sources about a job include: a. existing job specifications; b. position duty statements and selection criteria; c. job process and machinery manuals; d. exercise/operation reports; e. observation of the work (direct or recorded); f. surveys of and consultation with job owners, PO or BPO, capability and trade sponsors, supervisors/managers, and job holders; g. occupational analyses and surveys; h. legislative, regulatory and licensing requirements; i. organisational policies and procedures; j. existing course evaluations and reviews; k. training continuums; l. accredited courses; and m. training packages Collection methods should be selected on the basis of balancing effectiveness against resource costs (including time). Particular consideration should be given to the reliability of collection methods and tools, and where possible they should be trialed to check this. Methods may involve record searches, workplace visits with observations, interviews and group process methods such as expert panels and focus groups. The latter can be particularly valid, but requires time and administrative effort to conduct. Trade-offs will often be required between the quality of the information and the effort required in gaining it on the basis of associated risk If planned well, the collecting of job information should be straightforward. The time allocated for collection should allow for modifications to the plan in accordance with strengths and weaknesses that become apparent during implementation. 2 14

49 ADFP Chapter Whilst collecting information about the job, other information relating to training may be found. This can include feedback about past training, the existence of relevant competencies and training courses and materials. Such information should be collected, as it may be useful to the later training development work. Step 4: Collate and analyse information As information is gathered it will require both collation and review for its validity. It may be necessary to refer back to sources to add to, or check information. The need for this will vary with the criticality of the information and the reliability of the source. Once data has been determined to be valid, analysis of the data can be undertaken. Analysis, particularly of job component data, should be based on set criteria which may include the percentage of job holders who perform the task, the criticality and difficulty of the task, the requirement to identify sub-tasks and the extent or level to which tasks will be broken into sub-tasks. Ultimately the analyst should be able to substantiate and justify all the information included in the job specification. Step 5: Develop job specification A job specification records the product of a job analysis. Annex C provides a template that suggests a structure for the document, together with guidance on its content. The scope and depth of content of a job specification should be appropriate to its context. The key components of all job specifications are the information that describes the job: the duty task inventory (DTI) that details the work performed and the required performance standard, the conditions under which the work is conducted, and any information affecting the job-holder s performance For formative evaluation purposes the draft job specification should be reviewed by authoritative stakeholders as it is being developed and revised as necessary. Step 6: Gain authorisation of job specification and disseminate Job specifications must be authorised by the Analysis Authority the sponsor or BPO for the job, or their delegate. Approval is a significant decision, not just for the training implications, but because the contents of job specifications may be used for other purposes. Job information may be used, for instance, for personal reporting, and as a basis for performance management and development pathways When authorised, the job specification should be disseminated to all pertinent authorities. These could include career and personnel management staff and senior job supervisors. The specification should also be filed with the appropriate training development documents. 2 15

50 ADFP Chapter 2 ANALYSE TARGET POPULATION The group of people from which trainees are selected is called the target population. The development of quality training requires a detailed and clear understanding of this group. The role of a target population analysis is to help develop this understanding A full target population analysis is undertaken for major training projects, particularly when no relevant training records exist. For less significant problems and/or where relevant training documentation applies, a limited process might be applied Purpose. The information gained by analysing the target population serves two purposes. Knowledge of the target population s current competencies, when compared with those required in the workplace, is used to determine the performance gap that training might fill. Other information about the target population supports decisions about the strategies for assessing and conducting the training Trigger. The trigger for a target population analysis is a decision, resulting from a needs assessment, to apply the DTM to a workplace performance problem. Inputs, process and products The inputs process and products of analysing the target population are summarised in table 2 3 and explained in the following paragraphs. Inputs Process Products Directive with parameters Job specification Work force records and plans Recruiting data Education and training records Surveys of likely learners and their supervisors/managers Step 1: Determine the target population. Step 2: Determine information requirements. Step 3: Plan and gather information. Step 4: Collate and analyse information. Step 5: Develop a Target Population Profile, report and file in accordance with policy. Target Population Profile of likely learners. (The document may be an attachment to a formal report of the process.) Table 2 3: Inputs, process and products of the analyse target population process 2 16

51 ADFP Chapter 2 Step 1: Determine the target population The composition of the target population will normally be determined by the career or personnel managers, although training specialists might also provide input. The target population may consist of people already in the organisation, including those who might already be undertaking complementary work, or new recruits who will be trained for the work. The group may therefore be large and heterogeneous, or small and homogenous. The individuals within it may be located close to or distant from the intended workplace and possible training venues. The criteria used to classify the target population (eg career/employment grouping, location) are recorded on the Target Population Profile. Step 2: Determine information requirements Two types of information about the target population are required: that which defines the group as a whole, and that which describes individuals within the group. At the group level, its size, general levels of educational and/or professional competence, and its geographic distribution are factors that can contribute to decisions concerning training strategies. At the individual level, information contributes to decisions about the training required and the methods to be used for it. Knowledge of the typical individual s competencies is critical, and is used for deriving the performance gap. Other useful information about the typical individuals can include their: a. academic record and learning ability; b. employment and training background; c. age, gender, physical characteristics, learning preferences, and special aptitudes; d. attitudes to the intended change to working practice; e. workplace environment (in terms of suitability for assessment and training); f. any factors limiting availability for training; and g. special or unusual group characteristics. 2 17

52 ADFP Chapter 2 Step 3: Plan and gather information Good planning ensures efficiency in the gathering of information. Such planning should incorporate the sources to investigate and the methods to be employed. They might include: a. research of the Defence Personnel Management Key Solution (PMKeyS) database, recruiting data, performance appraisal, and training documentation; and b. survey, observation, interview and group processes involving a mix of sources including members of the target population, recruitment, career, and personnel staff The planning should also consider any potential for efficiency by using the information search for collecting other related information, such as that concerning the working and learning environments The collection strategy developed should balance the quality of the information against the effort required to gather it, the whole being proportional to the scale of the training development task. Sampling will usually be necessary, and the process should be demonstrably valid. The larger the sample the more valid will be the generalisations derived from, and decisions based on it. Once the strategy is determined, the planning needed to implement it should be undertaken. This can include arranging visits and interviews, and preparing aids such as questionnaires and data collection forms If planned well, gathering the required information should be straightforward. Meeting with likely future learners might also provide a useful opportunity to promote the workplace change to which training may contribute. Step 4: Collate and analyse information As the information on the target population is likely to come from a number of sources it will require collation. This process could identify inconsistencies that need resolution and gaps that require more information to be gathered. This work enables the development of an accurate picture of the likely learners. Step 5: Develop a Target Population Profile A Target Population Profile records the information of the analysis. Annex D provides a template. The scope of a Target Population Profile and the depth of its content should be appropriate to its context. The job classification data incorporated must match that of the job specification. 2 18

53 ADFP Chapter 2 DETERMINE PERFORMANCE GAP The performance gap is the difference between the performance required for a job and the capabilities of the target population: the people likely to perform it. The gap has two dimensions: scope and scale. The scope of the gap describes the work that the target population cannot presently perform to the required workplace standard. The scale describes the significance of this problem in terms of its extent, impact and importance to the organisation Documenting the performance gap further defines the performance problem addressed in the needs assessment. It is recorded on a Performance Gap Statement Purpose. The purpose of determining the performance gap is to specify the difference between the performance required for a job and the competencies of the target population Trigger. The Analysis Authority instigates a performance gap analysis following formal acceptance of the job specification and the Target Population Profile. Inputs, process and products The inputs process and products of the performance gap process are summarised in table 2 4 and explained in the following paragraphs. Inputs Process Products Job specification. Target Population Profile. Step 1: Determine scope of performance gap. Step 2: Determine the scale of the performance gap. Step 3: Develop a Statement of Performance Need. Performance Gap Statement. Table 2 4: Inputs, process and products of the performance gap process 2 19

54 ADFP Chapter 2 Step 1: Determine scope of performance gap The first step in deriving the performance gap for a job is to determine which of the tasks and subtasks listed in the DTI (an element of the job specification) cannot presently be performed by members of the target population. A simple comparison of the proficiencies of the target population (as recorded in the Target Population Profile) is made against the tasks and subtasks listed in the job s DTI. This determines whether members of the target population can satisfactorily achieve the task, or whether a gap in required performance exists. The Target Population Capability Record template in annex E 5 is an aid for this work. Step 2: Determine scale of performance gap While comparing the scope of the performance gap, and by reference to the job specification, its scale should also be identified. This information describes the significance of the gap and indicates how difficult its closure is likely to be. Identifying the scale requires analysis of the extent to which the target population cannot perform the listed tasks, and whether they have the necessary attributes to do so. The affect of the gap on the organisation should also be considered. How important is the deficiency to Defence capability, both now and in the future? What risks are involved in failing to close the gap in any relevant time frames? Step 3: Prepare a Performance Gap Statement The third step in determining the performance gap is to develop a Performance Gap Statement. This document has two parts. The first collates the task performances defining the scope of the gap. The second presents information about the scale of the gap. It gives context to the need in terms of how many people lack the proficiencies, when they will need them, and their significance. Annex F presents a suitable template Both the Target Population Capability Record and the Performance Gap Statement are working documents. Although they do not have to be formally approved by higher authority, informing stakeholders of their content can be worthwhile, and they should be retained for reference purposes. 5 The Target Population Capability Record form is a working document. It is not approved by a higher authority, but should be retained for reference. 6 The Performance Gap Statement is also a working document that should be retained for reference. 2 20

55 ADFP Chapter 2 CONDUCT FEASIBILITY ANALYSIS A feasibility analysis is the DTM process used to determine the best option, or mix of options, for ensuring the competence of job holders. It furthers the work of the needs assessment, which identified that job holders are needed to achieve the required workplace performance Ensuring the competence of job holders has two components: adopting measures to achieve job holder competence, and then assessing that the required competence level has been reached. A feasibility analysis addresses both components, with its results and recommendations being recommended in a Feasibility Report Depending on the situation, a feasibility analysis can take many months of intensive work by a large team, or it can be completed quickly and simply by a single person Purpose. The purpose of a feasibility analysis is to identify the most effective, efficient and, where required, flexible means of ensuring the required competency of job holders Trigger. The feasibility analysis process follows the determination of the performance gap as recorded in a Performance Gap Statement. Inputs, process and products The inputs, process and products of feasibility analysis are summarised in table 2 5 and explained in the following paragraphs. Inputs Process Products Performance Gap Statement; research data relating to options, including organisational policies and any regulatory or legislated requirements. Step 1: Identify options for ensuring job holder competence. Step 2: Investigate options for ensuring job holder competence. Step 3: Compare options for ensuring job holder competence. Step 4: Identify options for assessing competence. Step 5: Investigate options for assessing competence. Step 6: Report the feasibility analysis and file. A Feasibility Report that recommends training and non-training options for meeting the performance need and how competency should be assessed. Table 2 5: Inputs, process and products of the feasibility analysis process 2 21

56 ADFP Chapter 2 Step 1: Identify options for ensuring job holder competence To identify options for ensuring job holder competence, both the performance gap and the circumstances that led to it must be understood. This information is contained in the Performance Gap Statement the product of the preceding process. Important information includes the number of job holders or positions lacking the required competence, the turnover of people, any time frames involved, and the significance of the problem. This information should help determine whether the job tasks and the target population can be considered homogenously, or whether they should be grouped to meet special needs or characteristics There are many separate but complementary ways by which an organisation can achieve competent job holders working to required performance levels. For established workplace situations, management-based options such as supervision, motivation, and planning may suffice. Such options may also be useful where the workplace performances have changed or are new, but in these circumstances the following options are more commonly appropriate: a. modifying the workplace, equipment, and tools; b. process re-engineering; c. recruiting competent staff; d. reallocating work; e. automating the performance; f. informal on the job coaching; and/or g. workforce training Each of these options may have two or more variations. This is particularly true for training, where options can include: a. structured and unstructured on-the-job training; b. structured off-the-job training that can be residential or distance, in-house or outsourced training; or c. any mix of these (and each of these options might be applied in more than one location and situation) Furthermore, the optimum choice may consist of a mix of any number of the options. For example, new partially competent recruits are employed in a new organisation, receive induction training in one location and are trained 2 22

57 ADFP Chapter 2 at another location using a mix of off-the-job and on-the-job strategies. Their training involves the use of tools that are new to the organisation to achieve a new workplace performance Investigating all possible options would be unrealistic, and judgments are required to identify those that are sensible. In making such judgments, the options must be clearly appropriate to the performances listed on the Performance Gap Statement and should be checked for consistency with current policies and instructions It would be nugatory to identify and investigate all conceivable options for achieving competent job holders. Instead, those considered more effective and efficient should be selected. Such judgment is improved if key stakeholders are involved. This practice also assures acceptance of the outcomes of the feasibility analysis. Step 2: Investigate options for ensuring job holder competence To compare the options, information about the options and the situation must be gathered and considered. The required information is likely to include, but not necessarily be limited to, time frames for achieving the desired workplace performance, resource and policy limitations, recruitment data, availability of target population for training, minimum training throughput, and any applicable regulatory or legislated requirements. As the comparison will largely be based on cost, benefit (measured in money terms) and/or effectiveness (considered in monetary and other terms), such information is also required. Costing of options is often a lengthy and difficult process. Cost estimates should therefore be prepared only for otherwise strong options. Obtaining suitable information is rarely easy, not least because the effectiveness of an option can rarely be quantified. Consequently it is usually necessary to provide qualifying or explanatory notes, and to recognise and identify where judgments are subjective As the investigation of options progresses, some may prove inappropriate, in which case no more time should be given them. Conversely, new options may become apparent, in which case they should be investigated and reported. It is also likely that options will be better understood in later phases of the DTM (as when the cost of training are clarified when strategies for it are investigated in the Design phase). In such instances, reviewing the feasibility analysis and revising its report may be worthwhile. 2 23

58 ADFP Chapter 2 Step 3: Compare options for ensuring job holder competence To determine the best option, or mix of options, for designating competent job holders the possibilities are subjected to an evaluation process. The type of process should suit the circumstance. In many situations it can be useful to determine and apply essential and desirable evaluation criteria such as: a. policy; b. resources (knowledge, human, financial, facilities and equipment); c. time; and d. risk It can also be useful to consider the required performances in terms of: a. the number (percentage) of the target group performing the work; b. the consequence of error if the skill is inadequately performed; c. the learning difficulty if a competency is difficult to learn it is more likely to be candidate for formal training; and d. the task delay tolerance, for if a performance is required instinctively eg fire response, it is more likely suitable for formal training In many situations alternative or complementary methods involve the use of a cost/benefit analysis, a cost/effectiveness analysis, or both. 7 In situations of significant risk, conducting risk analyses might also be worthwhile: for all the options, or just the recommended one(s). 7 The difference between the two types of analysis is that a cost benefit analysis uses monetary information to determine the return on investment, while a cost/effectiveness analysis uses a more relaxed approach towards the measurement of benefits as when options are ranked in accordance with risk, or a training solution is ranked in order maximum possible throughput. 2 24

59 ADFP Chapter 2 Step 4: Identify options for assessing competence Assessment of job holders level of competence can be achieved in several ways. The basic methods are: a. on-the-job measurement and evaluation of performance in either: (1) real situations, or (2) exercise situations; b. off-the-job measurement and evaluation of performance (such as in a training or an assessment centre); c. desktop evaluation of qualifications (possibly against standardised measures); and d. a combination of the methods listed. The first three of these methods are listed in the order of their validity (how closely they might measure the required performance) In considering which options should be considered for more detailed investigation, not only their validity, but their suitability (how feasible and at what cost?) and reliability (how consistently are their results likely to be?) should be broadly assessed. Another consideration that should be made is whether the assessment is likely to be recognised by the award of an Australian Qualification Framework (AQF) unit of competence (in which case Australian Quality Training Framework (AQTF) assessment guidelines are to be applied 8 ), or whether recognition will be organisational specific, without links to the NTF. Step 5: Investigate and compare options for assessing competence Having identified possible ways of assessing job holders competence, an investigation of the strengths and weakness of each should be undertaken. This involves gathering detailed information about, and making judgments on the validity, suitability and reliability of each option in relation to the performances listed in the Performance Gap Statement. In this process the job tasks might be considered homogenously, or they might better be grouped to meet the needs of special characteristics. 8 AQTF Learning and Assessment Strategies How to Kit. 2 25

60 ADFP Chapter The hierarchy of validity being fixed, the comparison of options is made from the most valid to the least, with judgments being made about their suitability and reliability. For each field the judgments might consider the following fields: a. Suitability: availability of workplace when required, disruption to normal work, costs involved, time constraints; and b. Reliability: consistency of conditions (including human inputs) The basic judgments to be made are whether an option is acceptable or not acceptable. If two or more at the same level of validity exist then comparisons of levels must be made. Step 6: Report the feasibility analysis and file The results of the two parts of the feasibility analysis, including the comparison of options, conclusions and recommendations, are reported to the Training Analysis Authority in a Feasibility Report. The form of this report should match the scope of the task, but for most occasions a Defence options paper format in two parts should be used (a template is provided in annex G) In most cases training is likely to form part, if not the complete solution for ensuring job holder competence. Where a mix of options is selected for both this and the assessment of competence, the report should clearly link each option with the performance(s) to which it relates The Feasibility Report should be staffed through all significant stakeholders, particularly those outside of training that might be affected by its content. Appropriate levels of agreement should be assured before action arising from the recommendations is undertaken. SPECIFY TRAINING REQUIREMENT Introduction. The early processes of the Analyse phase identified that training could contribute to achieving the required workplace performance. The final process of the Analyse phase ensures the requirement is formally documented in a TRS. This requirement is expressed in terms of, or relates to, competencies: ie the abilities to perform tasks and duties to expected standards Purpose. The purpose of specifying the training requirement is to provide the baseline for subsequent phases of the implementing the DTM. 2 26

61 ADFP Chapter Trigger. The trigger for beginning work on a TRS is acceptance of a Feasibility Report recommending that formal training be used to achieve a required workplace performance. Inputs, process and products The inputs process and products for specifying the training requirement are summarised in table 2 6 and explained in the following paragraphs. Inputs Process Products Endorsed Feasibility Report. Job specification. Occupational analysis data. Existing competency standards. Data from Industry Skills Councils (ISC). Products of compulsory and higher education. Step 1: Identify and apply suitable existing training documentation. Step 2: Customise existing training documentation. Step 3: Develop new competency standards. Step 4: Set up competency details in PMKeyS. Step 5: Determine training context and accreditation action. Step 6: Develop a TRS. Step 7: Gain formal endorsement of the TRS, action, and file. TRS. Table 2 6: Inputs, process and products for specifying the training requirement Step 1: Identify and apply existing training documentation When developing new or revising established Defence training, consideration should be given to using any existing training documentation, particularly any that uses competency standards/units of competency 9 from established training packages or accredited courses. If relevant qualifications, competency standards or other Defence competencies exist, they are to be used and/or customised in preference to developing new ones. 9 Department of Education, Science and Technology s (DEST) Training Package Development Handbook, together with other training documents, uses competency standards and units of competency synonymously. The only difference is that a competency standard consists of a number of individual units of competency that are closely associated and linked. DI(G) PERS 05 29, annex D refers. 2 27

62 ADFP Chapter A search for existing training documentation should begin by investigating local and then Defence records in training establishments and with training authorities. For competency standards, other important sources are the Defence accreditation cells (within Defence Education Training and Development and individual Services) and the Defence Sector Component of the Public Safety Training Package. A wider electronic search can be undertaken using the Competency Standard folder of the DEST s National Training Information Service (see This site has a key-word/phrase search function for identifying existing units of competency. Existing training packages and competencies can also be found through the Australian Training Products Ltd database (see and national and/or state or territory industry training advisory bodies (ITAB 10 ). The ITAB incorporating Defence and serving Defence needs is the Public Safety ITAB Having identified existing training documentation, its suitability for meeting the required performances is evaluated. In making this evaluation, consideration should be given to ensuring that all Defence training documentation uses the levels and terminology of competency-based training, whether or not the training is nationally recognised within the AQTF. Step 2: Customise existing training documentation If existing training documentation closely matches the requirement it can be customised. This might involve writing performances in competency terms and can lead to the national recognition of those competencies within the NTF. If it already uses competency terminology, customisation is typically limited to the range of variables, the evidence guide and the addition of enterprise-specific (ie Defence) performance criteria. Customisation adds specific information to endorsed competency standards or units to reflect the Defence environment. It is not an open-ended process, for the integrity, or meaning of the standards must not be altered. Further details on the extent to which competency units and standards can be customised are available in DI(G) PERS 05 29, annex C, appendix 2 and in DEST s Training Package Development Handbook. When competencies have been appropriately customised, they can be listed in the TRS. 10 Note: at the time of publication the ITAB are being reorganised into ISC. 2 28

63 ADFP Chapter 2 Step 3: Develop new competency standards Where the performance requirement is not written in competency terms or existing competencies are unsuitable for use, even with customisation, new competencies must be developed. Whether these new competencies are to be nationally registered or not, they should be developed in accordance with the format used within the NTF. This work is detailed in DI(G) PERS and, to a large extent, in part 2 of DEST s Training Package Development Handbook. These references should be used for the work. When developed, new competencies have seven appropriately worded components: a. Unit code a unit s identifying reference can either be registered as part of a recognised training package and awarded a unit code in accordance with the national coding system, or it might be an industry specific reference (those for Defence have DDD as a prefix), DI(G) PERS 05 29, annex C, appendix 1 refers. b. Unit title a succinct statement of the broad area of competency covered expressed in terms of the outcome. c. Unit descriptor an expansion of the title providing additional information relating to its purpose and relationship with other units of competency. d. Elements of competency the functions, described in demonstrable and assessable outcome terms that are the basic building blocks of a unit of competency. e. Performance criteria statements specifying what is to be assessed and the required level of performance to verify competent performance. f. Range statement a description of any contextual variables that will be used or encountered when applying the competency in a workplace situation. g. Evidence guide direction of how to assess the unit or learning outcome, with advice on the appropriate context and scope of the competency in its application. 2 29

64 ADFP Chapter Key competencies. Key competencies are the generic, underpinning, transferable skills/qualities essential for effective participation in work 11. Seven key competencies have been defined: collecting, analysing and organising information; communicating ideas and information; planning and organising activities; working with other s and in teams; using mathematical ideas and techniques; solving problems, and using technology. Units of competency in a training package can identify which key competencies are relevant and should specify one of three performance levels: a. Level 1 undertakes tasks effectively using routine approaches; b. Level 2 manages tasks selecting from routine approaches; and c. Level 3 evaluates and reshapes tasks to establish new approaches. Step 4: Setup competency details in Personnel Management Key Solutions When the competency units required for workplace performance are agreed, they are entered in the Defence human resource database, PMKeyS. When there, they can be linked to the data on Defence personnel in the system and a competency profile developed. This profile is used for the award of proficiency s (qualifications) and for the management of Defence people The setup of competency units in PMKeyS involves entering: a. the criteria underpinning elements; b. the elements, and then attaching the relevant criteria; and c. the competency units, and then attaching the relevant elements. Step 5: Determine training context and accreditation action At this stage of the Analyse phase, just before its primary output is documented, it is appropriate to consider whether the training will be recognised by award of an AQTF qualification, how it will be recognised in the Defence system, and how it relates to the learner s career development. 11 Related terminology for key competencies are generic skills and employability skills. DEST is presently undertaking several studies in how to use these concepts. 2 30

65 ADFP Chapter A learner s achievement of competence is recognised by the award of a qualification or statement of attainment (SOA) recognised within the AQF 12, or an organisation-based qualification. There are three types of AQF qualification: a. a qualification that is listed as part of an endorsed training package and is endorsed by the National Training Quality Council; b. an industry specific qualification that is accredited by a State or Territory Training Authority (Defence uses the Australian Capital Territory Accreditation Authority), and either: (1) uses competencies from one or more training package(s) and/or groups of endorsed national competency standards; or (2) is not linked to either a relevant training package or endorsed competency standards. (ie a training need has been identified and competency standards have not, or are not likely to be developed in the area covered). Organisation-based qualifications may not be recognised by authorities outside Defence Where a combination of training and workplace experience equates to one or more competencies that can be mapped against the requirements of a nationally recognised qualification, then this should be done. The RTO (Navy, Army, Air Force, the Defence Learning Services Network or contracted RTO) is then approached to ensure that those units/qualifications are within their scope of registration and to determine any RTO requirements for their delivery and assessment Where training and workplace experience do not directly map to existing competencies or qualifications, the Analyse Authority, together with the RTO, must decide on whether to : a. add new units or qualifications to an existing training package (such as the Defence Sector to the Public Safety Training Package); b. apply to develop a new accredited course 13 ; or 12 An AQF qualification recognises achievement (through education, training, experience etc) of a discrete package of competencies, and an SOA is a certificate issued by an RTO that recognises that part of a qualification has been attained. 2 31

66 ADFP Chapter 2 c. issue an organisation specific qualification, which may be an SOA Full details of how to package competencies into AQF qualifications are contained in section 2.4 of DEST s Training Package Development Handbook and in its Australian Quality Training Framework Guidelines for Course Developers: A guide to developing VET courses for accreditation under the Australian Quality Training Framework, July The guidance in these references should be followed when establishing or amending qualifications Incorporating new or amending existing qualifications in the AQF can be a lengthy process. It may be implemented concurrently with work on other phases and processes of the DTM If the new training is likely to affect the career development of learners as individuals or groups, particularly Australian Defence Force members, then their career manager should be consulted to ensure that any implications are properly managed. Figure 2 3: Algorithm illustrating the decision-making process 13 An accreditation submission requires information available in draft form at the end of the Design phase and confirmed following piloting at the end of the Develop phase. 2 32

67 ADFP Chapter 2 Step 6: Develop a Training Requirement Specification A TRS specifies the training requirement in terms of the people needing training (the required throughput, and the characteristics of the target population), the required product (the competencies the learners should gain), specifications for the assessment and training (including anticipated major resource requirements), and how achievement of the learning will be recorded. It contains information collected, analysed and developed throughout the Analyse phase. A template for a TRS is in annex H. Step 7: Gain formal approval of the Training Requirement Specification, action and file The job or BPO (for the PO) has the expertise and responsibility that, as the Analysis Authority, approves the TRS. The approved document is a key audit and internal self assessment reference of the training system. Consequently it must be appropriately filed. Annexes: A. Flow chart showing the Analyse phase of the Defence training model B. Defence training model purpose, inputs, process and products of the Analyse phase C. Defence job specification template D. Target Population Profile template E. Target Population Capability Record template F. Performance Gap Statement template G. Defence Feasibility Report template H. Training Requirement Specification template 2 33

68

69 Annex A to ADFP Chapter 2 FLOW CHART SHOWING THE ANALYSE PHASE OF THE DEFENCE TRAINING MODEL A 2A 1

70

71 Annex B to ADFP Chapter 2 DEFENCE TRAINING MODEL PURPOSE, INPUTS, PROCESS AND PRODUCTS OF THE ANALYSE PHASE B Analyse PHASE PURPOSE INPUTS PROCESS PRODUCTS To scope the nature of the performance problem/opportunity and determine whether the Defence training model should be applied To describe the career/job/task capability requirement Communication/direction proposing training intervention, subject matter expert stakeholders, related documentation, job situation Job specifications, duty statements, selection criteria, process manuals and current competency standards and units. Project requirements, exercise/operation reports. Observations, research, surveys and consultation with Business Process Owners, capability and trade sponsors, supervisors/managers and job holders. Legislative and licensing requirements and organisational policies. Needs assessment Needs Assessment Report Job analysis (of current or foreseen job, or work group) Job specification 2B 1

72 ADFP PURPOSE INPUTS PROCESS PRODUCTS To describe the characteristics of the groups/individuals who are to perform the job To specify the human dimensions of the workplace performance problem To determine and recommend the best way of achieving the career/job/task capability requirement To formally record the training requirement in competency terms Work force records and plans. Education and training records, surveys of likely learners and supervisors/managers, testing of likely learners. Job specification and Target Population Profile Statement of Performance Needs, data on existing vocational education and training, higher education, professional development, and other current and possible solutions Feasibility Report, job specification, occupational analysis data, existing competency standards, industry training advisory bodies, compulsory and higher education outputs Target population analysis Target Population Profile of likely learners Determine performance gap Performance Gap Statement Feasibility analysis Feasibility Report, with training and non-training solutions, together with estimates of through-life resource and costs estimates Training Requirement Specification Training Requirement Specification 2B 2

73 Annex C to ADFP Chapter 2 DEFENCE JOB SPECIFICATION TEMPLATE 1 C JOB TITLE 1 Introduction 1 1. Job classification. The job title is classified in Defence as follows: a. Job code number. Provide the applicable job identification or reference descriptor. b. Job designation. Provide a context for the job (eg Service core/branch/category, civilian group/occupational group) 2. c. Job description. Provide a short description 3 of the role of the job 4. d. Function. Describe the function of the job 5 (preferably in a single sentence) 6. e. Sponsor. Insert details of the job sponsor/business Process Owner (BPO) 7. 1 This template is provided as a guide. The scope and content of a job specification should be appropriate to its context. 2 For example a crewman Australian Light Armoured Vehicle (ASLAV) is a Royal Australian Armoured Corps soldier, who may be either a permanent force or Reserve member. 3 For example a crewman operates allocated equipment and maintains the ASLAV as a member of an armoured fighting vehicle crew. Participates in the conduct of reconnaissance and surveillance in support of operations in either a mounted or dismounted role. 4 If the job spans a number of ranks and grades that cannot be summarised in a single description statement, include sub-paragraphs for each level. 5 For example to drive, operate and maintain an ASLAV. 6 Again, if the job spans a number of ranks and grades that cannot be summarised in a single function, include sub-paragraphs for each level. 7 Most Service job positions will have either a career or functionally-based sponsor. Generic work, as applicable to most civilian positions, is more likely to be managed by a BPO. 2C 1

74 ADFP f. Rank/level. If a job embraces a range of ranks/levels, indicate the range, such as PTE to WO1. If specific ranks/levels/pay groups carry out the job, list them. g. A possible layout is shown below: Job code number 8 Operator PQR Provisional PTE PG 1 Job code number Operator PQR Grade 1 PTE LCPL PG 2 Job code number Operator PQR Grade 2 LCPL CPL PG 3 Job code number Supervisor PQR SGT WO2 PG 3 Job code number Manager PQR WO2 WO1 PG 4 2. Job requirement. The Defence requirement for the job title has the following specifications: a. Numbers. State, as accurately as possible, the number of job positions in Defence. If changes are anticipated, this should be made clear. b. Throughput. An estimate of the demand for trained people as expressed in numbers against time (either specific calendar dates, units of time or both). c. Where employed. Describe the geographical locations and/or organisational units in which the job normally occurs (including Reserve units for Service positions). 3. Job analysis background. Provide some background to, and explanation of the analysis that led to this specification. Reference to preceding specifications might be made, as well as the reason for the analysis, when and how it was conducted, and any limitations that apply. JOB CHARACTERISTICS The following sections are indicative. They and/or others should be omitted or incorporated in accordance with their significance. Duty task inventory The duties, tasks and sub-tasks that comprise the job, together with details about the required performance levels for each unit, are listed in the duty task inventory (DTI) in appendix 1. Each listed element of the job should begin with an action verb and be described concisely. 8 If the entire career is in one category, the job code letters will be consistent (eg ECN 074 Clerk Administration and ECN 078 for Clerk Supervisor). 2C 2

75 ADFP Existing units of competencies that apply to the job are listed in appendix 1. Current units of competency that apply to parts of the job should be listed as they are identified. This work complements and does not replace any completion of the DTI. It reduces the risk of unnecessary duplication later in the development process. Responsibilities Subordinates. State the supervisory requirement, indicating (where applicable) numbers of Service personnel and Defence civilians by ranks and levels, and/or employment groupings. 8. Delegations. Describe any delegations that relate to specific roles eg supervisors are likely to hold personnel delegations, and particular job holders could have financial delegations. 9. Classified matter. Describe any responsibility for classified matter, including the highest level associated with the job. 10. Stores, equipment and finance. Describe the level of supervision required for management of equipment and/or accounting procedures. Where materiel is of an attractive or valuable nature this is to be stated, as well as the level of supervision for demanding, receiving and issuing these items. Describe any responsibilities for financial matters such as bookkeeping or maintenance of accounts. 11. Assessment and training. Describe the responsibility for assessment and training, for on-the-job training (OJT) and formal training in the organisation, together with any requirement for specific qualifications. 12. Consequence of error. The consequence of error can involve the following: a. Safety of others. Describe the responsibility for safety of others associated with this job, considering the type of work performed and the levels of risk involved. b. Effect on the organisation s mission. Describe the effect of error on the organisation s mission. Be specific and avoid broad descriptions. Include a description of any consequential effects on other parts of Defence. 2C 3

76 ADFP Equipment List of equipment. List the major equipment used (specifically) in this job. Refer to generic equipment types rather than specific items and exclude minor items of equipment, complete equipment schedule and tools. Environmental conditions Working conditions. The working conditions for the job are as follows: a. describe the conditions under which work is usually performed, with particular regard to exposure to climatic conditions, ventilation, working space and duration of work; and b. differentiate between the conditions pertaining to any possible circumstances (such as exercise or combat, base and field). 15. Most demanding job conditions and hazards. Summarise the most demanding/highest duty conditions and any associated hazards. These must be stand-out duties. Be succinct: a. Non-combat. Identify the non-combat or industrial conditions and hazards associated with the job. When the degree of risk requires qualification indicate this accordingly. b. Combat. Indicate the degree to which any Service person is involved in combat, together with an assessment of the associated risks. If non-combat hazards are intensified in combat, indicate this also. 16. Team/social environment. The social environment is as follows: a. Consider the need for and level of personal contact in undertaking the job and/or associated with it. Describe the type usually encountered and the most demanding. b. Indicate the frequency of contacts with other Service personnel and civilians; and the requirements for handling these situations; the need for tact as part of a small team; and the ability to communicate, supervise, advise, answer questions and interview. 2C 4

77 ADFP Physical demands. The physical demands of the job are as follows: describe both the normal and the highest physical demands placed on the job eg physical or environmental discomfort. 18. Physical skills. The physical skills required by the job are as follows: describe the physical skills required in the job, eg coordination and precision in operating equipment or manual skills. 19. Mental demands. The mental demands of the job are as follows: a. Complexity and judgment. Illustrate the complexity of the work to be performed and the degree of judgment required by the job holder. b. Scope of action. Explain the scope of action or the requirement for independent action, resourcefulness and ingenuity that is part of the job. c. Communication skills. Describe the requirement for written and oral expression in the job. 20. Distastes. State any features of the job or duties which could cause certain job holders to be ineffective in their duties. Do not list factors that are accepted as being a normal requirement of military life. 21. Immediate supervisor. State the rank/level to whom the job holder is normally responsible. Job holder specifications Medical. State any relevant medical requirements from Service or other regulations (including medical categories). When issued electronically, this paragraph will contain a link to the parent document. 23. Psychological. State any specific psychological requirements for the job. 24. Security classification. State the minimum level of security classification required for the job Other special requirements. State any other special requirements not covered above such as whether the job holder has to be a volunteer, any restrictions relating to physical size and the ability to swim. 9 This is not to include security clearances that are related to specific appointments or roles that may be undertaken by only some members. 2C 5

78 ADFP Employment and career factors Prerequisite employment. List any career pathways, jobs or positions that are prerequisites for this job. 27. Prior training, qualifications, licenses and experience. State all formal and OJT, qualifications, civil licensing or other mandatory requirements, and/or experience known to be required before either starting in, or beginning training for the job. 28. Recognition of current competency or prior learning. Identify any recognition of current competency or recognition of prior learning issues. 29. Career profile and flow chart. The career profile to which the job relates, is in appendix 2. Relate progression from entry to exit in the career stream specific to that the job only. The career flow chart illustrating this career profile is in appendix 3. It must illustrate diagrammatically the details shown in the career profile. Any relevant training management system Personnel Management Key Solution career logic flow chart should be included as an annex to this document. 30. Follow on categories. List career pathways that the job holder can follow as direct progressions from this job. 31. Applicable units of competency. The existing units of competency applicable to this job are listed in annex H, appendix 2. This information is an incidental product of the job analysis, recorded at the time of data collection, but not necessarily researched. The competencies requiring training are identified in full when the training requirement is specified. 32. Accreditation. Provide details of any civil accredited qualifications that job holders might receive on attainment of appropriate units of competency (Defence Instructions refer, and details can be found on the accreditation web site see The information may also cover any civilian licences awarded. 33. Authorisation. Provide position and contact details of the job sponsor/bpo authorising this document. This information provides context to the signature block. 2C 6

79 ADFP Miscellaneous Miscellaneous information. List any information considered to be of relevance but not covered elsewhere in the job specification. Examples include the need to maintain skills and comply with ethical practices and any specific values/codes of conduct. Prepared by: 10 Signature Authorised by: Signature Name Name Rank Rank Position Position 10 Date Appendixes: 1. Duty task inventory 2. Career profile 3. Career flow chart Date 10 Finalised job specifications are to include the following signature blocks. They are to be forwarded to designated group authorities including those for job/position management and training: a. the individual who prepared the specification, normally a career/trade manager, but possibly a training specialist; and b. final sign off is by the sponsor/bpo of the job. This should not to occur until all major stakeholders have reviewed and provided feedback about the specification. 2C 7

80

81 Appendix 1 to Annex C to ADFP Chapter 2 DUTY TASK INVENTORY 1 1 DUTY: JOB/JOB TITLE 1 DTI Ref TASK: SUB-TASKS: 2 Job composition Dif Imp Freq Required performance criteria 2 TASK: SUB-TASKS: Key Dif difficulty Imp importance Freq frequency E easy L low D at least daily S straightforward M medium W weekly D difficult H high M monthly VD very difficult C critical R rarely Note (a) The document describes work performed, it does not list the competencies that have some relevance to that work. Existing units of competency applicable to this job are listed below. 1 A separate inventory will be required for each rank/grade. 2 To identify sub-tasks satisfactorily it may be necessary to identify the task elements that enable that performance. 2C1 1

82 ADFP APPLICABLE UNITS OF COMPETENCY 2.2 Name of job 2 2. The units of competency 3 known to be required for the (job title) job are listed below beside the associated task identifier from the duty task inventory (DTI) of the job specification. DTI ref number Unit reference Unit title 3 Only the units of competency are to be listed. The elements of competency, performance criteria, range of variables and evidence guide will be described in the related competency standard contained in the training package. 2C1 2

83 Appendix 2 to Annex C to ADFP Chapter 2 CAREER PROFILE 2 Job code number Rank/ level Career development/ promotion courses Professional/trade courses Number required Note (a) The career profile and the components listed should be included if relevant. 2C2 1

84

85 Appendix 3 to Annex C to ADFP Chapter 2 CAREER FLOW CHART 3 One or more simple linear flow charts with amplifying notes may be provided to support the career profile introduced in annex C, appendix 2. 2C3 1

86

87 Annex D to ADFP Chapter 2 TARGET POPULATION PROFILE TEMPLATE 1 D JOB TITLE 1 General 1 1. Job classification. The job for which the target population is described is classified as follows: The information listed here should match that provided in the job specification. a. Designation. Provide any job identification or reference descriptor. b. Situation. Provide a context for the job (eg Service core/branch/category, civilian group) 2. c. Job description. Provide a short description 3 of what the job involves 4. d. Function. Describe the function of the job 5 (preferably in a single sentence) 6. e. Owner. Insert details of the job or Business Process Owner (BPO). 1 This template is provided as a guide. The extent of its use and the depth of the content it covers should be appropriate to the context. 2 For example a crewman Australian Light Armoured Vehicle (ASLAV) is a Royal Australian Armoured Corps soldier, who may be either a Permanent Force or Reserve member. 3 For example a crewman operates allocated equipment and maintains the ASLAV as a member of an armoured fighting vehicle crew. Participates in the conduct of reconnaissance and surveillance in support of operations in either a mounted or dismounted role. 4 If the job spans a number of ranks and grades that cannot be summarised in a single description statement, include sub-paragraphs for each level. 5 For example to drive, operate and maintain an ASLAV. 6 If job spans a number of ranks and grades that can not be summarised in a single function, include sub-paragraphs for each level. 2D 1

88 ADFP Target population group Classification. The target population for the job title is defined by the following factors: The list might include: a. career, trade, rank/level, experience; and b. vocational education and training and educational qualifications. The section may contain reference to the homogeneity of the group and highlight any significant disparities in its members. 3. Education and training prerequisites. Both mandatory and/or desirable education and training prerequisite qualifications might be listed. 4. Professional experience prerequisites. Both mandatory and/or desirable prerequisite employment experiences might be listed. 5. Group characteristics. Briefly state the characteristics of the work group people required to undertake the job, including: the numbers required in any pertinent time frames, any categories or classifications, any sub-groups evident, the locations of the work and any special factors affecting it. 6. Geographic distribution. Identify the present locations of the target population. Indicate the extent to which its members are collocated (as in a barracks or training establishment), or dispersed. Target population characteristics Proficiencies. The following proficiencies are characteristic of individuals in the target population that are relevant to the job. The qualifications and records (eg certificates and licences) indicating outcomes of education, training and employment and other experiences of relevance to the job situation are listed. 8. Other factors. Listed below are other factors that might influence decisions about training strategies and which are not covered elsewhere in the Target Population Profile. Possible fields include: a. academic record and learning ability; b. employment and training background; c. age, gender, physical characteristics, learning preferences and special aptitudes; 2D 2

89 ADFP d. attitudes to the intended change to working practice; e. workplace environment (in terms of suitability for assessment and training); f. any factors limiting availability for training; and g. special or unusual group characteristics. Prepared by: 7 Signature Authorised by: 8 Signature Name Name Rank Rank Position Position 78 Date Date 7 The individual who prepared the specification, normally a career/trade manager, but possibly a training specialist. 8 Final sign off is by the sponsor/bpo of the job. This should not to occur until all major stakeholders have reviewed and provided feedback about the specification. 2D 3

90

91 Annex E to ADFP Chapter 2 TARGET POPULATION CAPABILITY RECORD TEMPLATE E JOB/JOB TITLE 1 1. This form is a job aid rather than a formal record of the Analyse phase. 2. The composition of the job title is described by the performances listed in the following table. Any unit of competency known to match specific tasks is identified. For each task sub/task performance, the target population either has the necessary skills, knowledge and attributes that enable satisfactorily performance, or is lacking competence. This determination is indicated by a in either the currently able or lacking competence column. 12 Duty: Job composition Ref No TASK: SUB-TASKS: 2 Unit of competency reference (if any) Target population capability 1 currently able lacking competence TASK: SUB-TASKS: 1 Using information from the Target Population Profile, the classification should not be contentious. If necessary, footnotes explaining decisions can be incorporated. 2 To identify sub-tasks satisfactorily it may be necessary to identify the task elements that enable that performance. 2E 1

92

93 Annex F to ADFP Chapter 2 PERFORMANCE GAP STATEMENT TEMPLATE 1 F JOB TITLE 1 1. This document specifies the performance gap the difference between the performances required for a job and the abilities of members of the target population. The Performance Gap Statement has two parts: the first lists the job tasks that the target population cannot presently perform, and the second presents information explaining the scale and significance of the performance problem. 2. The following table lists the tasks in which members of the target population lack workplace competence: Duty: Task number Task description Unit code of any associated competency Related considerations The following factors indicate the scale of the performance gap and, thereby, its significance: a. List, possibly using a table, the number of current and/or anticipated job holders/vacancies that lack competence. Where significant the dates and details of any changes to this specification should be provided. b. Provide an indication of the importance/criticality of the gap, and therefore the need to fill it. c. Identify any jobs/performances that are dependent on the gap being closed, together with any urgency to the need. d. As relevant identify the location of the jobs and/or members of the target population. 1 This template is provided as a guide. The extent of its use and the depth of the content it covers should be appropriate to the context. 2F 1

94

95 Annex G to ADFP Chapter 2 DEFENCE FEASIBILITY REPORT TEMPLATE 1 G FOR DESIGNATING, AND ASSESSING THE COMPETENCY OF JOB HOLDERS FOR THE: JOB TITLE INTRODUCTION 2.1 Background 1 1. This report presents an analysis of options for: a. ensuring job holder competence for the job title job; and b. assessing a job holder s competency to undertake this work. 2. It identifies the strengths and weaknesses of each option and recommends the best for implementation. 3. Provide a brief description of the situation that required the Analyse phase of the Defence training model and led to the definition of the performance gap. The nature and scope of the gap must be explained, for which the relevant Performance Gap Statement might be included as an annex. Any Terms of Reference and/or limitations should also be stated. 4. The depth and extent of the investigation behind the paper should be described and any assumptions acknowledged. 5. The background should identify the authorities consulted in the preparation of the report. Structure Explain the structure of the report, and the way in which the evidence is presented and compared to derive the conclusions and related recommendations. The report may, for example, consider the tasks of the Statement of Performance homogeneously or they might be divided into two or more groups in accordance with particular characteristics. Aim State that the aim of the report is to recommend the best solutions for meeting the required job performance. 1 This template is provided as a guide. The extent of its use and the depth of the content it covers should be appropriate to the context. 2G 1

96 ADFP OPTIONS FOR ENSURING JOB HOLDER COMPETENCE List all the appropriate existing or potential options that might be used for ensuring that Defence personnel are positioned to undertake the performances listed in the Performance Gap Statement. If current policies, time or resource constraints limit the options or their implementation, this should be explained. 9. For each option: a. describe the option; and b. state the advantages and disadvantages of implementing the option, including estimates of effectiveness (in monetary and other units of benefit), costs and deployment considerations (such as time and major resource implications). The aim of this section is to present data that can be compared (for training options this may require seeking estimates from potential contractors, consideration of instructional facilities and equipment, accommodation and catering facilities, and the opportunity costs of using current capacity. Other likely factors include instructor profiles: current numbers, rank/level categorisation, their competencies and capacity; and the capacity to develop courses). Where estimates and judgments are used, this should be identified. Comparison of options Present a structured, logical means of comparing the options that clearly identifies the stronger and weaker options or sets of options. If the performances constituting the gap are being treated homogeneously, only one set of comparisons will be required. If they have been divided into groups, a comparison will be needed for each separate group. The text should clearly illustrate progress towards a logical conclusion. A decision matrix or paired comparative analysis can help in this. Conclusion The conclusion presents the best option(s) for designating the Defence personnel who will become undertake the performances listed in the Performance Gap Statement. 2G 2

97 ADFP OPTIONS FOR ASSESSING COMPETENCE Present the options for assessing the competence of Defence personnel to undertake the required performances. Do so in the approximate order of their validity. If current policies, time or resource constraints limit the options or their implementation, this should be explained. 13. For each option: a. provide explanatory detail; and b. state the advantages and disadvantages of implementing the option, including estimates of effectiveness (in monetary and other units of benefit), costs and any deployment considerations (such as time and major resource implications). The aim of this section is to present data that can be compared (eg availability of environments, staff, facilities and equipment, accommodation and domestic facilities, and any significant opportunity costs). Where estimates and judgments are used, this should be identified. Evaluation of options Present a structured, logical evaluation of the options for assessing competency. The process is one of selecting the option or mix of options that has the greatest validity, suitability and reliability in relation to the costs involved. The text should clearly illustrate progress towards a logical conclusion. Conclusion The conclusion presents the best option for assessing the competence of the Defence personnel designated to undertake the performances listed in the Performance Gap Statement. It should also identify any implications arising from implementing this option. 2G 3

98 ADFP Recommendations The recommended course of action follows from the two conclusions is presented. The section should state who would be responsible for implementing the recommendations, together with any pertinent time frames and resource requirements. Prepared by: 2 Signature Authorised by: 3 Signature Name Name Rank Rank Position Position Date Date Completed Feasibility Reports are to be forwarded for the attention of the stakeholders of the performance requirement The individual who prepared the Feasibility Report, normally a career/trade manager. 3 Final sign off is by the sponsor of the job. This should not to occur until all significant stakeholders have reviewed and provided feedback about the report. 2G 4

99 Annex H to ADFP Chapter 2 TRAINING REQUIREMENT SPECIFICATION TEMPLATE H 1. Job classification 1. JOB TITLE 1 a. Designation. Provide any job code or reference number. b. Job description. Provide a short description of the role of the job. c. Function. Describe the function of the job (preferably in a single sentence). d. Sponsor. Insert details of the job sponsor. Background Briefly provide an account of the circumstances giving rise to the requirement. 3. Supporting documentation. This Training Requirement Specification (TRS) is based on analysis work that can be referred to in the documentation listed below: Document Location Dated Report of needs assessment Job specification Target Population Profile Performance Gap Statement Feasibility Analysis Report Training requirement people Training throughput. Specify the numbers of trained people required, and any specified or preferred time frames, starting and/or completion dates. 5. Trainee profile. List the significant target population characteristics identified in the Target Population Profile. Include information how the training relates to individuals or groups career profiles. 1 The details of this section should match those of the job specification. 2H 1

100 ADFP Training requirement competencies Those competencies requiring training for the job title are summarised in the table in appendix 1. Those competencies that have prerequisite or co-requisite competencies are identified. The type(s) of training deemed appropriate for each competency 2 is/are also identified. 7. Summary of competencies. A summary of the competencies requiring training is provided in appendix 1. The format conforms to the nationally consistent system used by Australian industries and training organisations. 8. Competency details. Details of each of the competencies requiring training are provided in appendix 2. They include the: a. elements that combine to enable the competency, b. key competencies that underpin the competency, c. performance criteria by which successful achievement of competency elements are measured, d. range statement that specifies the conditions under which the competency is performed, and e. evidence guide that states how assessment of competency will be achieved. Training and assessment specifications This section should broadly specify how the training will be conducted (eg formal program(s), course(s), on the job, contractor provided) and how the required competency will be assessed (eg on-the-job, off job, through exercises, in training environment etc). It should also specify the environments in which these processes occur and any major resource implications, this information having been considered and reported in the Feasibility Report. The information is important, as it provides the major parameter for determining the assessment and training strategies in the Design phase. 10. Any other known constraints or limitations that might influence the design and conduct of training (such as timing and resources) should also be identified in this section. 2 This information is derived from the feasibility analysis that considered the options of course training, distance-based course training and on-the-job training. 2H 2

101 ADFP Learning accreditation Describe how successful achievement of the training will be recognised (eg award of an Australian Quality Training Framework qualification, a statement of attainment, or some other proficiency award) 3. Prepared by: 4 Signature Authorised by: 5 Signature Name Name Rank Rank Position Position 45 Date Appendixes: 1. Summary of competencies to be trained 2. Details of competencies to be trained Date 3 Finalised TRS are to include the following signature blocks. 4 The individual who prepared the specification, normally a career/trade manager, but possibly a training specialist. 5 Final sign off is by the sponsor/business Process Owner of the job. This should not to occur until all major stakeholders have reviewed and provided feedback about the specification. 2H 3

102

103 Appendix 1 to Annex H to ADFP Chapter 2 SUMMARY OF COMPETENCIES 1 TO BE TRAINED1 Reference: code for unit/element of competency, or other Duty/task number(s) Title of unit or element Prerequisite competencies Co-requisite competencies 1 Competencies may be either nationally recognised and within the Australian Qualifications Framework or Defence-specific. Type of training required 2H1 1

104

105 Appendix 2 to Annex H to ADFP Chapter 2 DETAILS OF COMPETENCIES TO BE TRAINED 2 1. Each competence for which training is required should have the following detail listed: a. Classification (1) Reference. Provide a reference code, either one provided when the competency is endorsed and registered in accordance with the national coding system, or one that is Defence specific in accordance with Defence Instruction (General) PERS Implementation of the National Training Framework within Defence vocational education and training, annex E. (2) Competency title. Provide the title that identifies the general area of competency encompassed by the unit. The area may relate to the duty level of job classification. (3) Competency descriptor. Provide a short statement that provides added explanation to the competency title and which helps clarify the scope and intent of the competency. b. Details (1) Elements. List the elements that are the building blocks of the competency. They describe the specific outcomes or functions encompassed in the unit that can be individually demonstrated. They relate to the task level of job classification. (2) Key competencies. Describe the generic, underpinning, transferable skills or competencies essential for effective participation in work. The mix of seven Key competencies applies to work generally, rather than being specific to work in a particular occupation or industry. (3) Performance criteria. Define the specified performance for the individual elements of the competency. They focus on the evidence required to show that a person can undertake the activity to a level acceptable in the workplace. 2H2 1

106 ADFP (4) Range of variables. Set out the context and conditions under which competent performance is expected in achieving the unit as a whole. (5) Evidence guide. Outline the type and scope of evidence required to assist in determining whether competency have been achieved. The evidence guide identifies a number of relevant factors to be taken into account, including making a judgment about the achievement of competency. Any prerequisite or co-requisite units of competencies and key underpinning knowledge should be identified. 2H2 2

107 ADFP Chapter 3 CHAPTER 3 THE DESIGN PHASE 3 Introduction The Design phase is the second of five phases constituting the Defence training model (DTM), as shown in figure 3 1. In this phase, the assessment and learning strategies for training are selected, together with the strategies for evaluating the whole process of preparing for and then conducting training. The object of these strategies is to ensure that the learning is achieved as efficiently and effectively as possible, and that it meets the original requirement. Figure 3 1: Design phase of the Defence training model 3.2 The Design phase involves making decisions about what is to be done, when and where, by whom, with what resources, and how the outcomes are to be measured and evaluated. 3.3 As with all phases, processes and steps of the DTM, the extent to which the DTM is applied should be appropriate to the circumstance. In situations of high significance (eg the introduction of training in support of a major new Defence capability) the DTM should be applied rigorously. Abbreviated processes (such as the rapid development process in chapter 1 Introduction ) may be appropriate in lesser circumstances based on the outcomes of a mandatory risk analysis. 3 1

108 ADFP Chapter 3 Key product and responsibilities The final and principal product of the Design phase is a draft Training Management Package (TMP). A TMP contains a curriculum that describes what is to be learned, how it is to be delivered, and how the learning will be checked; and strategies for Training Review and Evaluation (TR&E). At the end of the Design phase a TMP is a draft document because it is likely to be amended during work in the Develop phase, and because it needs the learning materials of the training program/course to be added. 3.5 For the Design phase, responsibility for shaping/designing the learning process shifts from the job sponsor/business Process Owner (BPO) (the Analyse Authority), to a trainer (the Design Authority). In the Australian Defence Force (ADF) the trainer with that responsibility is generally at the Training Command level. In the Department, responsibility for design generally lies with the sponsor. In joint training the responsibility is held by someone agreed by the stakeholders. Starting point The Design phase follows-on from the Analyse phase and is based on the latter s major product, a Training Requirement Specification (TRS). 3.7 Because the Design phase of training is concerned with broad strategies, it is most commonly applied to develop new training. Once training is established, the phase may be employed to check and revise the reasoning behind the strategies, and the strategies themselves. Processes of the Design phase The Design phase has nine major processes: a. analyse learners, b. analyse learning environments, c. analyse competencies, d. generate learning outcomes, e. design assessment strategy, f. design training strategy and produce the training and assessment strategy document, g. compile curriculum, 3 2

109 ADFP Chapter 3 h. design TR&E strategies, and i. compile a draft TMP. Not all of these processes have to be implemented sequentially as is indicated in the flow diagram in annex A. Paragraphs explains each of the processes and annex B summarises the purpose, inputs and products of each one in table form. 3.9 Analyse learners. Good training is appropriate to the characteristics and circumstances of prospective learners. To help ensure this, the first process of training design is the analysis of those characteristics and qualities. A key area of this analysis is the prospective learners existing competencies and capabilities, as it helps minimise over-and under-training. The output of the process is a learner profile a document that describes the existing competencies and capabilities of prospective learners, their preferred learning style, special aptitudes and motivation Analyse learning and assessment environments. The training and the assessment of job competency skills should be suited to the environments in which they are conducted. A learning and assessment environment analysis helps ensure this by studying the things or conditions in the environments that could affect learning and assessment. The information gained is recorded in an environments profile, and it is used to inform the design of training and assessment strategies Analyse competencies. Most competencies comprise complex performances that are best learned by identifying their components and then training each component in a preferred sequence. In analysing competencies, therefore, the skills (mental and physical), knowledge and attitudes/attributes that a person must acquire in order to perform each element of, and therefore the whole of a competency are identified. This work leads to the development of learning outcomes that serve as the objectives of training. As such, the product of the process, completed Competency Analysis Sheets (CAS), are working documents in the design of training Generate learning outcomes. Learning outcomes are the building blocks of training and learning. They clearly and concisely describe what a learner will be able to do on successful completion of a unit of training, the conditions under which they will do it and to what standard. They are therefore the objectives of the training system. Learning outcomes are generated from the skills, knowledge and attitudes listed on CAS, and using knowledge of the prospective learners and the environments in which training and assessments will be conducted. To be complete, each learning outcome should have an associated method for assessing its accomplishment. 3 3

110 ADFP Chapter Design assessment strategy. Assessment is the process of collecting evidence and making judgments about something. When applied to training, assessment can diagnose learners abilities (ie for placement, selection, and identification of strengths and weaknesses), motivate and guide their progress (formative assessment), and determine mastery of something learnt (summative assessment). It can be applied before, during, or on completion of training. For units of competence, it can also be applied following other developmental experience. In designing an assessment strategy, a broad plan is developed for using assessment to support learning and to determine whether intended learning outcomes have been achieved. A comprehensive strategy should state the purpose and approach to be used for assessment, present detail on the methods and sequence to be used, and indicate the likely resource requirements. This information is used to guide the development and conduct of assessment in a training program/course Design training strategy and produce the training and assessment strategy document. The term training describes the development of skills, knowledge, attitudes, competencies etc through instruction or practice. A DTM training strategy is therefore a broad methodology for the conduct and management of training conduct. It provides guidance on the why, what, where, when and how of training. It states the purpose, methods, location, activities, media, recommended or prescribed tasks, formative assessment, sequence and likely resource requirements that training will involve. In designing a training strategy, alternative methods of achieving each of the learning outcomes are considered against an understanding of the learner and the learning environments. The purpose of this is to identify and select the option, or mix of options that most effectively and efficiently assess the learning outcomes. While focused on the required learning outcome, a training strategy must complement its associated assessment strategy. To help ensure this, the two strategies are combined for presentation in a training and assessment strategy document Compile curriculum. A curriculum presents the specifications for a training program, course or subject (module). It describes all the learning experiences a learner undergoes, the intended learning outcomes, the training and assessment strategies, the content, teaching methodology, and prescribed or recommended assessment tasks. This information is determined in the preceding processes of the Design phase, which means that the process of compiling a curriculum is simply a process of collating the products of these processes. A curriculum is the major component of a TMP. 3 4

111 ADFP Chapter Design training review and evaluation strategies. For the purpose of maintaining the effectiveness and efficiency of training, strategies for two forms of continuous improvement process are produced in the Design phase: a. Training review. The evaluation and reporting of the conduct of training. b. Evaluation. The evaluation and reporting of how well training has contributed to meeting the original performance requirement. As with the other strategies in the Design phase, the two strategies for these processes provide broad guidance on what, where, when and how. They cover the methods to employ, when to employ them and the resources required. Contained in a TMP, the two strategies are used in the Develop phase for producing the materials for the TR&E document and in the Conduct phase for ensuring the processes are properly administered Compile draft Training Management Package. The final process of the Design phase is the compilation of a TMP. A complete TMP is the authoritative reference for the conduct of a training program or course. It contains the curriculum of a program/course, its TR&E strategies, and the materials developed for conducting the training. At the end of the Design phase a TMP is in draft form and incomplete. This is partly because the strategies contained are likely to change consequent to work during the Develop phase, and partly because the materials for conducting a program/course are added during that phase. Continuous improvement While working on any of the processes of the Design phase, stakeholder input is sought to help assure the quality of the product. A consequence of this is that products are likely to be revised many times before acceptance. To keep track of the developments and to enable reference during later development work, an accurate and comprehensive record of development activity should be maintained. Use of an external agency In accordance with national and Defence guidelines, the quality assurance of all training by, or on behalf of Defence (by Defence staff, contractors, professional service providers or other Registered Training Organisations (RTO)) remains a Defence responsibility. External agencies may, however, be used to undertake design work. When this is being considered, the potential for packaging design and development work in a single contract should be looked at. 3 5

112 ADFP Chapter 3 ANALYSE LEARNING AND ASSESSMENT ENVIRONMENTS 3.20 Introduction For training and assessment of job competency to be effective and efficient they must be tailored for the environments in which they are to occur. It follows that the design of training and assessment strategies requires an understanding of these environments. Such understanding is gained by analysis Analysis of learning and assessment environments involves evaluation of the environmental influences that might affect the processes. The results are recorded in an environments profile and are used to inform the design of training and assessment strategies An analysis of learning and assessment environments complements the feasibility analysis work of the Design phase. The scope and scale of an analysis will depend on the comprehensiveness of that work, as reflected in both the TRS and the Feasibility Report. Those documents should specify the intended training and assessment environments. Where they do not, a selection will have to be made. The environments profile provides a sound basis for this judgment Triggers. An analysis of learning and assessment environments is most commonly undertaken for the development of new training. In this circumstance the trigger for the work is approval of a TRS. Analysis can also be worthwhile when changes to environments offer potential improvements for current assessment and training, or when changes to either the training requirement or the learners arise. When existing processes might be redeveloped, the environments profile should be reviewed and, if necessary, updated an outcome that might necessitate corresponding amendments to dependent training documentation A learning and assessment environments analysis is one of three processes that may be undertaken concurrently at the beginning of the DTM s Design phase: the others being analysis of the learner and the competencies that require training Purpose. The purpose of analysing learning and assessment environments is to help ensure that the associated strategies are appropriate. 3 6

113 ADFP Chapter 3 Inputs, process and products The inputs, process and products of a competency analysis are summarised in table 3 1 and explained in the following paragraphs. Inputs Process Products TRS, Feasibility Report, current policies and instructions, workplaces, training organisations and schools, and the staff of these, resource specifications. Step 1: Determine information requirements. Step 2: Plan, prepare and implement information collection. Step 3: Collate and review information. Step 4: Develop and confirm environments profile. A detailed environments profile that identifies the strengths and weaknesses of the environments for assessment and learning. Table 3 1: Inputs, process and products of a competency analysis Step 1: Determine information requirements The first step in analysing learning and assessment environments is to determine the information requirements. For this, the TRS is reviewed to identify the learning and assessment environments that are to be analysed. Ideally the TRS describes specific environments (such as on-the-job, Service/Defence facility, contractor facility, home and/or other). Where it does not, and it is necessary to select environments for use (as when the TRS specifies an ADF training school, but not which one), a comparative analysis may be necessary to identify the best option Having determined which environments are to be analysed, reference should be made to the Feasibility Report from the Analyse phase. That should contain, or refer to, information about those environments. The currency and comprehensiveness of this information is checked to determine the need for gathering new information. 1 Detailed work in analysing learning and assessment environments can reinforce or challenge the content of the Feasibility Report and dependent TRS. If a problem is apparent, the related analyse processes might need revision. 3 7

114 ADFP Chapter The analysis requires information on the features of each of the environments that can influence learning and assessment. Although information requirements are situation specific, the following categories are commonly relevant: a. Environment description. For reference purposes, a broad description of the workplace/school/college etc. Its management arrangements might be relevant and the contact details of authorities should be sought. Any support facilities needed (eg accommodation, medical or social) should be identified. b. Location. The location in relation to the workplace, learners accommodation, associated training facilities, external training sites and transport facilities might be considered. The relative isolation of the proposed training environment or its proximity to a major city may also be significant for the learners and staff. c. Access and availability. The availability of the facility now and in the future should be checked. Ideally, design and development determine the length of a course and the assessment processes (work in the Design phase should assume this). However, the duration of training may be determined by the availability of training environments. d. Facility security. Identify the level of security that the facility offers and/or the capacity to upgrade security. This is particularly important where the proposed training may have a security classification that restricts general access or where current facility activity and the proposed training activity have different security requirements. e. Spaces for activity. Various aspects of spaces must be considered, dependent on the type of space (classroom, workplace, practical area). These may include size and suitability of area, power and water resources, computer network access, storage, floor type, washing facilities, heating, lighting, and positioning of teaching and learning aids. f. Training resources. Use of existing training resources can reduce costs. The availability of equipment, tools, computers, the types and numbers of desks/benches projectors, printed materials, screens, photocopiers and other electrical equipment could be pertinent. 3 8

115 ADFP Chapter 3 g. Occupational health and safety. Defence, through its leaders and managers, owes a duty of care to ADF members and civilian staff. This translates to providing a safe workplace with all the necessary protective equipment and supervision in place. h. Impact on the environment. The impact of new or changed training requirements on the proposed environment and the local population can affect the planned learning and assessment activities. This is particularly important to consider where the new training is likely to involve large numbers of personnel, extensive training activity or a significant change in local activity eg the introduction of field exercises or adventurous training. i. Staff. The availability of suitable training staff for both administration and the conduct of instruction and assessment is important. The numbers available and/or the ability to attract them there are significant factors. Similarly, the motivation of the staff and any need to prepare them for new responsibilities must be considered. Both the present and anticipated situations should be considered. j. Finances. The availability of funds for establishing and conducting training and assessment in the environments might be relevant (they can influence consideration of modifying the environment, and procuring equipment and other resources that might be used for training and assessment). k. Policy. Policies at many levels can influence the design of courses and assessment processes. At a local school/establishment level they will influence decisions on routines, organisation and the allocation and use of resources. At a branch or Service level, policies on such matters as career structures, staffing levels affecting the ability to undertake on-the-job training (OJT), and civil accreditation can affect course design. In the work environment they can influence the allocation of resources (particularly time) for training and assessment purposes. 3 9

116 ADFP Chapter 3 Step 2: Plan, prepare and implement information collection To gain the required information suitable sources must be considered and the most appropriate selected. Much information is likely to already be documented within Defence, but may require research. To collect the remainder, interview and first-hand direct observations may be appropriate methods. Planning of such collection is required, not least to avoid a disproportionate use of time, and it may be necessary to produce collection instruments such as questionnaires and check sheets. The arrangement of visits, observations and interviews can be necessary, requiring consideration of: a. the precise definition of what information is required from the analysis; b. the geographic distribution of likely respondents or interviewees; c. the ease of access to relevant personnel, particularly where operational or exercise activity is likely; d. the availability of competent personnel to design survey instruments, conduct interviews or observations and analyse data; e. the resources required for the collection, collation and analysis of data; f. time available for the analysis; and g. contingency plans if survey or interview schedules cannot be met Meeting the learning and assessment staff employed in these environments provides an opportunity for promoting the changes that will accompany the conduct of the new training. Step 3: Collate and analyse information Information about learning and assessment environments is normally single-sourced, and the task of collating it should therefore be straightforward. Some information may, however, prove contentious, and checks of it can be worthwhile. If any inconsistencies or gaps are discovered, further collection and collation will be needed. 3 10

117 ADFP Chapter 3 Step 4: Develop and confirm environments profile Each environments profile is a working document specific to a unique situation. There is no set format, although a template is suggested in annex H. All profiles should clearly identify and evaluate the factors that can affect learning and assessment in the nominated environments When developed, the environments profile should be checked against the TRS. As the document is a supporting reference, it does not require high-level authorisation. Its value is in the support it provides to designing assessment and training strategies. ANALYSE PROSPECTIVE LEARNERS 3.35 Introduction The Design phase of the DTM marks a shift in the preparation of training from the definition of its requirement to the processes of its assessment and conduct. For the processes to be effective, they must be tailored to the individuals who will experience them. Understanding the characteristics and qualities of the prospective group of learners is a prerequisite for such tailoring. Analysing the learners develops this understanding, which is recorded in a learner profile An analysis of prospective learners builds-on similar work undertaken in the target population analysis process of the Analyse phase. The scope and scale of a learner s analysis will depend on the comprehensiveness of that earlier work, as reflected in both the TRS and the Target Population Profile Purpose. The purpose of analysing the prospective group of learners is to ensure that the selection of assessment and training strategies is appropriate to their characteristics and circumstances Triggers. The major trigger for analysing prospective learners is the design of new training. In this circumstance the process follows on from approval of a TRS. However, changes in organisation and/or staffing can upset the validity of an analysis. In such cases, the Learner Profile should be reviewed and, if necessary, updated, possibly necessitating corresponding amendments to dependent training documentation A learner analysis is one of three processes that may be undertaken concurrently at the beginning of the DTM s Design phase, the others being analysis of the learning and assessment environments, and the competencies that require training. 3 11

118 ADFP Chapter 3 Inputs, process and products The inputs, process and products of the prospective learner analysis are summarised in table 3 2 and explained in the following paragraphs. Inputs Process Products TRS, Target Population Profile (including learners current competencies and experience levels), Department and workplace data about the learners (including age, sex, classification, location, access and equity requirements, language, literacy and numeracy requirements, relevant experience and education), current policies and instructions, data from Defence Recruiting, job sponsors and career managers, survey results and national literacy standards. Step 1: Determine information requirements. Step 2: Plan, prepare and implement information collection. Step 3: Collate and review information. Step 4: Develop and confirm learner profile. A detailed learner profile that describes the preferred learning style, special aptitudes and motivation of likely learners. This information helps ensure the suitability of the assessment and learning strategies. Table 3 2: Inputs, process and products of a learner analysis Step 1: Determine information requirements The first step in analysing learners is to determine the information requirements. These requirements depend on the comprehensiveness and currency of the trainee profile contained in the related TRS 2. If comprehensive and current, the requirement for gathering new information is minimal. If it is not, then additional information will be required Typical characteristics of learners that, because of their potential to affect the training and learning processes, should be recorded include: a. Employment classification. Understanding the representative rank/level of the typical learner, the employment specialisation/category and any particular stream or skill is a foundation for work in the phase. b. Proficiencies. Any relevant proficiencies ie qualifications, certificates, licences and employment experience related to the job or course. 2 This profile was derived from the target population analysis, where it should be accessible in full. 3 12

119 ADFP Chapter 3 c. Personal characteristics. List any characteristics of composition by gender, age, culture and location. Gender can be important where a mix requires appropriate domestic facilities. The age span of the learner group can be relevant to learning styles and the capability to learn new concepts, (research has shown that the older a person the more difficult it is to learn skills dealing with numerical functions. Conversely, a mature person is generally more responsible and can more readily accept the onus for personal development/learning). d. Competencies. List any competencies that are likely to be held by learners, related to the required job performance, but are not covered by any identified proficiency. e. Academic ability. Where relevant, indicate level of academic ability, especially in English and Maths. f. Physical characteristics. Is height, eyesight, hearing, dexterity, ability to discriminate colours or any other physical characteristic important enough to be defined as an entry standard for the course? For courses with a special fitness dimension, the normal and likely minimum fitness levels of the new learner should be identified. g. Learning styles. Any favoured or problematical learning styles should be identified, such as experience with self-paced, discovery and computer-based learning. h. Special aptitudes. Are there any particular mental or physical characteristics of the job or course that require particular aptitudes and which cannot be identified with factors in subparagraphs c. e. f. and g. above? i. Motivation. How motivated are the learners likely to be, as a result of perceptions about the job or course? Are the rewards for passing the course likely to motivate learners or does passing mean longer hours, a dirtier or dangerous job etc, without corresponding gains. Such factors will influence decisions on design strategies and tactics. Step 2: Plan, prepare and implement information collection For required information not available from Analyse data/documents/records, suitable collection sources need to be considered and the most appropriate selected for use. The selection of the collection methodology should be determined by what sources, time, expertise and other resources are available and by the nature of the data to be collected. 3 13

120 ADFP Chapter 3 Much of the required information is likely to already be documented by Defence in the form of career/trade/recruitment data, but may require further research. To collect any remainder, interview, survey or direct observation and measurement can be used. This work requires planning, not least to ensure that a disproportionate time is not spent on it. Factors to be considered in the planning stage are: a. a clear understanding of what information is required from the analysis; b. the geographic distribution of and ease of access to likely respondents or interviewees (particularly where operational or exercise activity is likely); c. the availability of competent personnel to design survey instruments, conduct interviews and analyse data; d. the resources required for the collection, collation and analysis of data; e. the time available for the learners analysis; and f. contingency plans if survey or interview schedules cannot be met The task of collecting information can be aided by use of instruments such as questionnaires and check sheets. These, as appropriate, should be developed and tested. Other possible preparation tasks include the arrangement of visits, observations and interviews. Step 3: Collate and analyse information As the information on the target population is likely to come from a number of sources it will require collation. This process could identify inconsistencies that need resolution and gaps that require more information to be gathered. This work enables the development of an accurate picture of the likely learners. Step 4: Develop and confirm learner profile There is no set structure for a learner profile. It should serve the situation of its requirement. Most, however, should identify the course and the classifications of typical learners; together with the qualifications and proficiencies they should have; and their levels of relevant language, literacy and numeracy. A suggested template for a learner profile is in annex I. 3 14

121 ADFP Chapter When developed, the learner profile should be checked against the TRS and by reference to personnel managers and workplace authorities for suitability and accuracy. As the document is a supporting reference, it does not require high-level authorisation. ANALYSE COMPETENCIES 3.48 Introduction The analysis of competencies identifies the skills (mental and physical), knowledge and attitudes/attributes that a person must acquire in order to perform each element of a competency. Most competencies comprise complex performances that are best trained by identifying their components and focusing on each one in a preferred sequence. By this means, components that require greater or special attention can receive it, and those that require little or no training can be treated accordingly The analysis of competencies is a preliminary process in developing the learning outcomes that serve as the objectives of training. As such, the products of the process, completed CAS are working documents in the design of training Purpose. The purpose of analysing competencies is to identify the basic skills (physical and mental) knowledge and attitudes required to achieve competencies. This knowledge is an important basis for determining the intended outcomes of training Trigger. The analysis of competencies is only required when training is initially designed or, subsequently, when a change is made to a competency. The primary trigger for undertaking the process is therefore authorisation of a TRS and the start of the Design phase of a training development project A competency analysis is one of three processes that may be undertaken concurrently at the beginning of the DTM s Design phase, the others being analyses of the learner, and the learning and assessment environments. 3 15

122 ADFP Chapter 3 Inputs, process and products The inputs, process and products of a competency analysis are summarised in table 3 3 and explained in the paragraphs that follow. Inputs Competencies and related information from the TRS, job-holders, logistic support analysis (LSA) task lists, manufacturers manuals and equipment specifications. Process Step 1: Analyse competencies. Step 2: Check completed CAS. Products CAS. Table 3 3: Inputs, process and products of a competency analysis Step 1: Analyse competencies to complete Competency Analysis Sheets There is only one substantive step in analysing competencies, and it can be considered the first step in generating learning outcomes Identifying the knowledge, skills and attitudes of elements of competency requires the combined expertise of job holders experienced in the job and an analyst competent in the process. Analysis can be based on observation of the work and questioning of job holders, or by questioning alone. Ideally, more than one job holder will be involved in the process, as this helps ensure that all aspects of the work are considered. Where major equipment acquisition is involved or new capabilities are being implemented, LSA task lists and manufacturers manuals and equipment specifications may provide a useful information source The first step in the process is to transcribe the competencies and their elements from the TRS to CAS. A template for a CAS containing sample data is in annex J (this particular example focuses on one element) For each element of competency in turn the steps required for its performance are listed and numbered. As this is done, consideration is given to whether the learners already have the ability to perform the work. If they can, it is not necessary to analyse the performance any further. If not, then the CAS columns for skills, knowledge, attitudes, training notes and difficulty are completed (note the following explanatory paragraphs) Physical skills. Physical skills are the learned capability of performing actions in an organised and fluid manner. Their outcomes are reflected in the speed, accuracy, force or smoothness of bodily movement. These skills are learned in connection with such common human activities as driving a car, 3 16

123 ADFP Chapter 3 using a keyboard and climbing a hill. Many complex physical skills can be broken-down into smaller self-contained sub-skills that can be learned separately before being combined for whole performance Mental skills. Mental skills are the learned capability of reacting with the environment by using symbols. They are characterised by knowing how, as contrasted with the knowing what of knowledge (eg converting fractions to decimals, analysing intelligence). They enable adequate responses to entire classes of natural phenomena. Mental skills may be divided into several sub-categories that can be ordered hierarchically, with complex skills requiring the prior learning of simpler skills Knowledge. Knowledge involves the recall of facts or sets of events being able to communicate information. It is seen as a necessary prerequisite of all other learning and therefore a critical constituent of the learning process. Increasingly, with the application of information technology, the mental skill of accessing required knowledge is more important than owning it Attitudes. An attitude is an acquired mental state that influences the choice of personal actions and is reflected in those actions. They are evident in tendencies to respond positively or negatively to another person, group of people, an object, idea or situation. Attitudes may be rational or irrational. They are related to motives, beliefs, values, personality, emotions and the groups of people Difficulty. When the skills knowledge and attitudes columns have been completed, judgments can be made on the difficulty of learning the performance described in the element of competency. Guidance in this can be gained from current job holders and learners. This information will indicate areas for which training might usefully incorporate practice or incremental learning activities Training notes. Although the work of competency analysis focuses on job performance, thoughts and ideas related to training are likely to arise. A column of the CAS is provided for noting these points which are only suggestions, as competency analysis does not formally consider the subsequent training implications The information gathered during the competency analysis process is vital to the development of training, for it provides the content of, and a guide to the structure of training. Information on the complexity or difficulty of the task will indicate the need for practice or the need for incremental learning activities. Some aspects of work may indicate the need for more than one individual to be involved and therefore the requirement for group learning activities. 3 17

124 ADFP Chapter Potential learning outcomes are derived by analysing all the data on the completed CAS (ie all the data in paragraphs 3.58 to 3.63). This process, (sometimes referred to as task analysis) although not an exact science, involves identifying the major concepts. These major concepts in turn form the basis of the potential learning outcomes. In addition, the process involves eliminating skills, knowledge and/or attitudes that have already been taken account of, so that the final list of learning outcomes represents a consolidated list. Finally, other data on the CAS that is related to a larger concept is identified for inclusion as a potential assessment criteria or content. Step 2: Check completed Competency Analysis Sheets On completion of the CAS it is useful to have them checked by an independent subject matter expert (SME) and someone with training development expertise. Although a working document, it is an important one. When checked, the completed CAS is used for generating learning outcomes. GENERATE LEARNING OUTCOMES 3.67 Introduction Generating learning outcomes is the process of establishing the intended results of training and therefore the objectives of the training system. It involves translating workplace performances into performances for the training situation and optimally structuring them to enable learning. Learning outcomes therefore clearly and concisely describe the intended results of training in terms of what a learner will be able to do having successfully completed a unit of training, the conditions under which they will do it and to what standard. Each learning outcome is also associated with a prescribed method for assessing its accomplishment In order to facilitate the development of curriculum documentation the DTM specifies two types of learning outcomes. The use of the two levels is merely to make one aware that in most instances there will be a hierarchy of Los. The first, the course learning outcome (CLO) is expressed in broad terms to specify the intended result of the course or program. The second, module learning outcomes (MLO) specify the intended outcome of chunks or components of learning. MLO are supported by Assessment Criteria, which indicate what the student must do and how well they are to perform. In combination, MLO and assessment criteria state the performance and standards, while the specific conditions under which the outcomes are to be performed are provided in an accompanying statement. 3 18

125 ADFP Chapter Learning outcomes are the building blocks of training and learning. They are a major factor in the training strategy developed in the Design phase, the production of training and learning materials in the Develop phase, and the goals of all activity in the Conduct phase of the DTM Purpose. The purpose of generating learning outcomes is to specify the objectives for the work of trainers and learners Trigger. In developing new training, learning outcomes are generated following the analysis of competencies. The trigger for the work is, therefore, completion of the CAS recording that analysis. When reviewing and updating existing training, the records from the original design work should be accessed, checked and amended. Inputs, process and products The inputs, process and products of generating learning outcomes are summarised in table 3 4 and explained in the paragraphs that follow. Inputs Process Products CAS, existing industry and Defence training/learning materials, any regulatory or legislated requirements, and the learner and learning and assessment environment profiles. Step 1: Collect inputs. Step 2: Determine required performances. Step 3: Complete learning outcomes and assign CLO Levels. Step 4: Write assessment criteria for each of the MLO. Step 5: Check learning outcomes and file. Agreed learning outcomes with assessment method. Table 3 4: Inputs, process and products of learning outcomes Step 1: Collect inputs The first step in generating learning outcomes is to obtain the required documentation (CAS, any regulatory or legislated requirements, and the profiles of learners and environments) together with any existing training documentation that might be used to save development time (Defence, other military, Government or industry) While use of relevant existing training documentation can save development time, care must be taken to ensure that any material used either meets or is customised to meet the specified need. 3 19

126 ADFP Chapter 3 Step 2: Determine required performances In generating learning outcomes the primary focus is on the performance elements (noting that the variables of performance frequently have a controlling influence and should also be considered). The competencies and their elements listed in TRS approximate to the two types of learning outcome performance (CLO and MLO), while the skills, knowledge and attitudes identified in competency analysis provide input to such documents as lesson plans (which do not form part of the formal curriculum documentation). The indicative relationships between the components of jobs, competencies, learning outcomes and units of training are illustrated in figure 3 2. Figure 3 2: Indicative relationships between jobs, competencies, learning outcomes, and training 3.76 In writing the performances of learning outcomes consideration is given to how they might be structured to: a. meet the required workplace performance; b. efficiently use particular training methods and learning environments; and c. form a course/program of training that is appropriate to the learners Some workplace performances can be replicated in the prescribed training environment, in which case they can be used as learning outcomes. Others cannot, for safety or cost reasons, in which case the performances, conditions, and standards of the learning outcomes are written to be as valid and appropriate as practicable. 3 20

127 ADFP Chapter Each performance should describe a single, observable behaviour that can also be measured. Its key term is an action verb. The description should be kept as short as possible. Qualifiers, explanations and expansions, when necessary, should be included in the conditions area of the outcome In this process thought is given to the sequencing of the outcomes. Having identified the required performances for all the competencies listed in the TRS, the list is rationalised to: a. remove any that the learner profile identifies as being existing learner attributes; b. avoid duplication and minimise overlap; and c. group any that will be more easily learned by association (groups may be based on related performances or training methods) Sequencing learning performances. The sequencing of training is important for effectively and efficiently achieving the desired outcomes. A number of factors should be considered: a. dependent outcomes (identifying prerequisites and/or co-requisites so that dependent learning outcomes are sequenced accordingly); b. learning principles that suggest sequencing from: (1) simple to complex; (2) whole-part-whole; (3) concrete to abstract; and (4) theory before practical; c. modularisation (ensuring that all necessary assessment criteria are contained within a module); d. training and qualifications frameworks (enabling entry and exit points from courses/programs in accordance with learner s needs); and e. personnel, resource and administrative limitations. 3 21

128 ADFP Chapter These factors are considered in relation to the following alternative bases of sequencing training: a. job performance: the training sequence is the same as the job sequence; b. critical sequence: the sequence is determined by the priority of the training; c. spiral: similar outcomes are revisited with an increasing degree of complexity, or to illustrate different perspectives; d. chronological: the sequence follows the order in which events occur in time; e. simple to complex: outcomes are presented in stages of increasing difficulty or complexity; or f. mixture: a mix of the preceding bases is normally used in designing training Once the list has been rationalised and sequenced, a numbering system is applied to enable tracking of the learning outcomes. Step 3: Complete learning outcomes and assign course learning outcome levels Having developed the performance element of the learning outcomes, the related conditions and standards can be added, and CLO levels can be assigned Conditions. To ensure that training consistently meets the workplace requirement, the conditions and limitations under which the learner carries out the performance must be defined. There are six types of conditions that should usually be considered: a. resources (including facilities, tools and equipment); b. special job aids and manuals (eg fault finding algorithms, cue cards, procedural check lists); c. environmental conditions (extremes of temperature, humidity, noise, motion etc); d. simulated situations (by use of simulators, role-play, case studies and exercises); 3 22

129 ADFP Chapter 3 e. physical location (where it influences performance, like when a marine engine is repaired in a workshop or in a ship s engine room); and f. associated personnel (eg supervisors, other operators and members of team/group) When the performance is based on a nationally recognised or previously documented unit of competency these conditions are derived from the associated range of variables and checked using the job specification. To promote the transfer of learning to the workplace the conditions in the learning outcomes should match those of the job as closely as possible Standard. Assessment criteria list the standards which a participant will be judged against, to determine whether or not they have achieved a learning outcome. The standards of learning outcomes are the benchmark against which satisfactory performance is assessed. They should match the standards expected in the job situation. The standards are, therefore, derived from the performance criteria of the TRS. Standards can be set from the areas of quality, quantity, consistency and time; and any performance may have one or more of such criteria. In writing learning outcomes it is important to retain a focus on what the learner will be doing in the workplace and not be distracted by what might be easy to achieve in the training environment Assign course learning outcome levels. The purpose of assigning a level to CLOs is to provide a shorthand/easy method of indicating the meaning of CLOs to stakeholders, particularly the employers of recent training graduates. Four levels are used, as illustrated in table 3 5. Level Description of graduate in relation to prescribed standard 1 Trained, but not against a prescribed standard. 2 Partially trained against prescribed standard. 3 Fully trained and ready for assessment against prescribed standard. 4 Trained and assessed competent against the prescribed standard. Meaning Background or familiarisation training. Additional training and/or experience required before assessment. Employment under supervision only. Assessment required. Employment under supervision until assessed. Ready for employment. Table 3 5: Course learning outcome levels 3 23

130 ADFP Chapter The product of writing learning outcomes is a numbered list of outcomes of two types: CLOs and MLO with the first of these described in terms of levels. This list contributes to the development of a training strategy and is documented in the training curriculum. It is, therefore, a working document. A suggested template is in annex K. Step 4: Write outcome assessment methods A statement of how the performance is to be assessed accompanies each learning outcome. Various methods 3 can be used and include assessment of: a. demonstration of workplace performance (normal or on specified exercise tasks); b. output sample from test/exercise or workplace; c. performance in training exercise (individual or team); d. simulation (individual or team, and including role-plays and hypotheticals ); e. oral presentation; f. interview; g. written test/exercise (open or closed book); and h. third-party evidence Selecting assessment methods involves evaluating the validity, suitability/practicality, and reliability of options. Validity is the extent to which the assessment measures what is intended. To be suitable/practicable, assessment should be administratively feasible and cost-effective. Reliability refers to the consistency with which it measures what it is supposed to measure If, in the course of writing assessment methods, it seems that a listed performance cannot be assessed, then it is probably necessary to amend the performance. 3 Many of the methods can be implemented using information technology, in which case they might be referred to as computer-based testing. 3 24

131 ADFP Chapter 3 Step 5: Check learning outcomes and file To prevent nugatory work it is important to check the completed learning outcomes against the relevant competency and the workplace to confirm that all performances and performance criteria have been adequately and properly addressed When the learning outcomes have been checked they are to be retained, together with documents showing how they were generated, on the departmental project/program/course file for audit and continuous improvement purposes, and to maintain a history of course development. DESIGN ASSESSMENT STRATEGIES 3.94 Introduction Assessment is the process of collecting evidence and making judgments about something. When applied to training, assessment can diagnose learners abilities (ie for placement, selection, and identification of strengths and weaknesses), motivate and guide their progress (formative assessment), and determine mastery of something learnt (summative assessment). It can be applied before, during, on completion of training and for units of competence, following other developmental experience In the DTM an assessment strategy is a broad plan for using assessment to support learning, and to determine whether intended learning outcomes have been achieved 4. A comprehensive strategy should state the purpose and approach to be used for assessment, present detail on the methods and sequence to be used, and indicate the likely resource requirements. This information is used to guide the development and conduct of assessment in a training course/program The design and implementation of an appropriate assessment strategy is a standard required of RTO To retain a focus on training outcomes, the design of an assessment strategy normally precedes the design of a training strategy. It is important, however, that the two strategies complement each other (the timing and methods of assessments can influence the design of the training strategy 4 The ultimate outcome for the DTM is workplace performance and learning experiences other than training that contribute to it. The assessment of workplace performance is more properly the responsibility of the job/bpo than trainers, but the latter are usually better qualified to design the processes and prepare the materials for the task. 5 Currently described by Standard 9 of the Australian Quality Training Framework (AQTF). 3 25

132 ADFP Chapter 3 and, in some circumstances, training methods can influence the design of the assessment strategy). For this reason the two strategies are presented together in a training and assessment strategy document Purposes. The two purposes of designing an assessment strategy are to guide the use of assessment for supporting and measuring learning in a training course/program, and to determine the achievement of workplace competence Trigger. An assessment strategy is designed for any new training when it is known what is to be trained, who is going to receive it and in what broad circumstances. The primary source of this information is the TRS, but the necessary depth of information is contained in the list of Learning Outcomes, and the learner and environment profiles. Checks and revisions of a completed training and assessment strategy document are made whenever TR&E processes indicate the inputs have changed. Inputs, process and products The inputs, process and products of the process for designing an assessment strategy are summarised in table 3 6. Inputs Process Products Assessment specifications listed in the TRS, learner and environments profiles, learning outcomes, and any relevant existing assessment resources. Step 1: Determine assessment needs and draft the strategy. Step 2: Select and add assessment methods. Step 3: Refine the assessment strategy. Input to a training and assessment strategy document. Table 3 6: Inputs, process and products for designing an assessment strategy Although the following steps are presented as discrete tasks, they usually interrelate and overlap not only with each other, but with preceding and subsequent steps. In consequence the process is highly iterative National assessment principles. A background understanding of the National Assessment Principles and their essential features as shown in annex C, should underpin the design of all Defence training assessment. The National principles, have been identified by the Department of Education, Science and Technology (DEST), in the context of training packages. Where some are criteria checked for registration as an RTO. However, they are applicable to any assessment of human performance against a required standard, and should therefore be considered and incorporated in Defence assessment strategies and tools. 3 26

133 ADFP Chapter Occupational health and safety. A background understanding of occupational health and safety (OHS) issues should also underpin consideration strategies and methods for assessment. Defence is bound by the Occupational Health and Safety (Commonwealth Employment) Act Although there is a special provision for the ADF that recognises the potentially dangerous nature of the work which a national defence emergency may make acceptable or necessary, it does not allow for exposing members to dangerous situations whilst training. Where doubt exists, professional assistance should be sought through the OHS branch. Step 1: Determine assessment needs and draft the strategy To determine what assessment a training course/program should have, the types and functions of assessment are considered in relation to the course as represented by its learning outcomes. To retain a strategic focus, this work is limited to the higher two levels of outcome, the CLO and MLO 6. The work requires judgment, and must take into consideration the characteristics of the learners, the environments available, and the resources that can be used Primary types and functions of assessment. Different forms of assessment serve a range of functions in support of training. They include the following: a. Recognition of prior learning/recognition of current competencies/recognition is a type of assessment normally offered before the conduct of training. It is used to grant status or credit for related competency already held, thereby avoiding overtraining. b. Diagnostic pre-tests are normally administered before the conduct of training for identifying specific learning needs. c. The most common, and frequently the most important, function of training assessment is to determine progress towards, and achievement of intended learning outcomes. This formative assessment is used during a course to provide feedback on learner progress. d. At the end of a course and/or other learning experience summative assessment is used to check overall achievement of competence. Defence training is directed at the summative assessment of workplace competency, which may be conducted on- or off-the-job. 6 Assessment at the level of assessment criteria learning outcomes is considered during the Development phase of the DTM. 3 27

134 ADFP Chapter 3 e. Holistic assessment. Holistic assessment is a particular form of assessment which determines in an integrated manner the way a range of competencies are combined to successfully complete practical workplace tasks. It is primarily used in workplace contexts when the specified learning outcome requires the demonstrated integration of a number of separate learning outcomes or competencies. It is characterised by: (1) whole-of-work observation of workplace activity; (2) synthesis of multiple sources of evidence in making a judgment; (3) opportunity to gather evidence on more than one unit of competency at the same time; and (4) close adherence to the National Assessment Principles The longer, more complex and difficult training is, the greater the need for both formative and summative assessments Other functions of assessment. Other functions of assessment that might be incorporated in an assessment strategy include: a. the pre-course diagnosis of learners competencies for placement (as when selecting individuals with mathematical abilities for technical courses); b. confirmation and/or reinforcement of course prerequisite learning; c. identification of learners preferred learning styles 7 (individual, group, self-paced, text, pictorial, theory or practical etc); and d. the post-course job placement of course graduates The implementation of all assessment processes also serves to motivate or demotivate learners. These effects, particularly the negative effects, can be powerful. The design of an assessment strategy should therefore attempt to maximise motivational effects and minimise the negative effects Draft the assessment strategy. In drafting the assessment strategy the needs identified are applied to the list of learning outcomes to determine when assessments will be conducted and for what purpose. The summative 7 This can be of value in situations where choices of learning methods are incorporated in training conduct. 3 28

135 ADFP Chapter 3 assessments of intended learning outcomes are determined first, thereby retaining an outcome focus. Work on formative assessment processes and assessment for other purposes then follows Where existing training documentation with proven assessment strategies and materials is of relevance to the training need, use of this material is encouraged, as it promotes efficiencies and facilitates standardisation of assessment practices. However, care should be exercised to ensure that adapted material is appropriate for the purpose and that any modification of the existing material does not adversely affect its validity The outcome of the step is a draft plan that combines the learning outcomes of a training course/program with a broad plan for the conduct of assessments. Step 2: Select assessment methods Once the basic assessment strategy is determined, the method(s) for conducting each of the assessments identified is selected. This decision usually also involves determining where the assessment should take place. Again, this work builds on, and can be informed by guidance provided in the TRS and thoughts recorded whilst generating learning outcomes Methods of assessment. The range of assessment methods that might be employed in a course or program of training includes: a. demonstration of workplace performance (normal or on specified exercise tasks); b. output sample from test/exercise or workplace; c. performance in training exercise (individual or team); d. simulation (individual or team, and including role-plays and hypotheticals); e. oral presentation; f. interview; g. written test/exercise (open or closed book); and h. third-party evidence. 3 29

136 ADFP Chapter Selecting methods. The principal consideration in selecting the optimum assessment method, is the learning outcome. The verb chosen for the performance of the outcome indicates suitable methods of assessment 8. Other considerations include the training methods employed; the location and suitability of the training and alternative venues; the costs involved; and the qualities of validity, reliability, flexibility and fairness Check. Before expending resources in refining the assessment strategy it can be worthwhile to check that: a. all the CLO and MLO of the training course/program, (particularly any CLO that equate to units of competency), have an associated assessment process; b. all the performance criteria and requirements for competencies listed in the TRS are appropriate to the workplace, training environments and learners; and c. relevant requirements from the AQTF Standards for RTO have been addressed. Step 3: Refine the assessment strategy As the assessment strategy is refined, more detailed issues are addressed and determined. These are likely to include: a. the location of the assessments (on- or off-the-job); b. the resources the assessments will require (including qualified assessors); c. their recording/administration (including log progress, Defence Personnel Key Solutions (PMKeyS) data entry, archiving etc); d. the management of failure to meet standards; and e. the processing of appeals against assessment decisions Administration of assessments and results. Administrative procedures for the conduct of assessment should be practicable, as should the management of the results obtained. This is likely to be straightforward where the normal practices of the organisation are followed, but can be problematical where new systems are introduced. In Defence, consideration 8 The term describe, for example, indicates the appropriateness of written or oral methods, whether off or on-the-job; and an objective of tune a diesel engine suggests assessment by practical methods in a workshop setting. 3 30

137 ADFP Chapter 3 should be given to the practical difficulties of workplace assessment for ADF units on operational service, and to the existence of PMKeyS for programming and panelling courses, and for managing course results Training progress management. In situations requiring it, assessment strategies are to include provisions for managing instances where standards are not met. Provisions might include control points restricting progress; and the resources, including time, for retesting. The requirement will derive from personnel management policies and its implementation will be governed by the training progress management policy of the Service or organisation conducting the training Programming of assessments. The programming of assessments is another task requiring professional judgment. The primary determinant of the sequence and timing of assessment activities is the training strategy, but the following factors should also be considered: a. the value of assessment in reinforcing learning; b. the motivating/demotivating influence of assessment; c. the risks of pursuing training processes without confirmation of learning and achievement; d. the implications of learners failing to achieve the required performance levels; and e. the size/length and cost of assessment process in relation to the outcomes and training processes Estimate resource requirements. Each of the judgments and decisions of earlier steps in designing an assessment strategy have involved consideration of the resource implications. Where appropriate, the strategy should summarise these implications, particularly those involving significant: time for administration and conduct, facilities, equipment, personnel, funds, and organisation. If the estimates exceed the expectations of the organisation, appropriate action must be taken National principles of assessment. Final refinements of the assessment strategy may be achieved by ensuring that its content accords with DEST s National Principles of Assessment (annex C). Of these, principle 8 is of significance to all training and assessment situations. Product input to a training and assessment strategy document The product of designing an assessment strategy is a broad plan that is merged with a training strategy and recorded in the training and assessment strategy document, an example template is in annex K. This plan 3 31

138 ADFP Chapter 3 is likely to require amendment, first as it is combined with the training strategy and later as work on developing materials to implement the strategies suggests the need for change. DESIGN TRAINING STRATEGY AND PRODUCE THE TRAINING AND ASSESSMENT STRATEGY DOCUMENT Introduction Training describes the development of skills, knowledge, attitudes, competencies etc through instruction or practice. In the DTM a training strategy is a broad methodology for the conduct and management of training. A training strategy should therefore provide guidance on the why, what, where, when and how of training. It should state the purpose, methods, location, activities, media, recommended or prescribed tasks, formative assessment requirements, sequence and likely resource requirements that training will involve While focused on the required learning outcome, a training strategy must complement its associated assessment strategy. To help ensure this, the two strategies are presented together in a training and assessment strategy document Purposes. The purpose of designing a strategy for training is to ensure that training is conducted effectively and efficiently, consistent with relevant policies and learning theories. The purpose of producing a training and assessment strategy document is to formally record the guidelines for developing and conducting training Triggers. When preparing new training, the trigger for designing a strategy is knowledge of what is to be trained, who is going to receive it, and in what broad circumstances. The primary source of this information is the TRS, but the necessary detail is contained in the list of learning outcomes, and the learner and environment profiles. Checks and revisions of a completed training and assessment strategy are triggered by feedback from TR&E processes. 9 While this chapter refers to a single training strategy, some situations may require multiple layered or sequential strategies (eg linked strategies for introducing and then maintaining a new capability, or short, medium and/or long-term strategies that meet the requirements of a changing environment). 3 32

139 ADFP Chapter 3 Inputs process and products The inputs, process and products of designing a training strategy, and producing the training and assessment strategy document are summarised in table 3 7. The first three steps listed in the table are indistinguishably combined in practise. They are separated here for purposes of explanation. 10 Inputs 10 Training and assessment specifications listed in the TRS, learner and environments profiles, learning outcomes, assessment strategy, and relevant existing assessment and training resources. Process Step 1: Select training delivery methods. Step 2: Apply other principles of learning theory. Step 3: Select associated media. Step 4: Combine training and assessment strategies. Step 5: Develop course map. Step 6: Complete training and assessment strategy. Step 7: Gain approval and file in accordance with policy. Products Training and assessment strategy. Table 3 7: Inputs, process and products for designing a training strategy As is common in the DTM, the steps of designing the training strategy take the products of earlier phases and processes and develop them to a greater level of detail. The greater understanding this achieves often leads to revisions of the earlier work. Step 1: Select training delivery methods The first step of designing a training strategy involves the selection of a training method for each of the MLO previously generated 11. A training delivery method comprises a mode, activity and setting. Selecting the training delivery method is based on consideration of the practical considerations, the estimated costs and times of development, and the potential effectiveness of alternatives in these fields Training modes. Training can be trainer-centred, learner-centred or a mix of the two. These training modes can be viewed as opposite ends of a continuum where the mix of methods, media and learning theory can be used 10 A supplementary resource for this DTM process is the DEST, 2003, Learning and Assessment Strategies Part 1: How to Kit, and Learning and Assessment Strategies Part 2: Resource Guide that supports Standard 9 of the AQTF Standards for RTO. 11 Training methods for course learning outcomes were effectively set during the feasibility analysis of the Analyse phase. 3 33

140 ADFP Chapter 3 to achieve the most effective blend of trainer-centred and learner-centred modes. A predominantly trainer-centred mode has been the most commonly used by organisations, as it enables groups of trainees to be trained simultaneously. Catering for diverse trainees is, though, difficult, and delivery costs can be high. Learner-centred training has a growing potential to be the more efficient mode, as the rapid development of computer and network technology is enabling learning content, times and pace to be designed and developed for individual needs. Development of good materials is, however, difficult and costly, and some trainees respond poorly. A table showing factors to consider when selecting a training mode is in annex D Training activities. The selection of the most appropriate training activity will depend on the setting, the mode, what is to be learnt, its complexity, learner abilities and commitment, the position of the activity in relation to the course, the class size and the course/module/session duration A training strategy can employ any of a large variety of training activities, most of which can involve individual or group work, and many of which can be conducted either face-to-face or by distance learning mode. The more common activities include: a. lecture; b. demonstration; c. lesson (theory and practical); d. on-the-job work experience; e. discussion; f. independent study; g. exercise; h. tutorial; i. simulation; j. role-play; k. case study; l. mentoring; m. games; n. playlet; 3 34

141 ADFP Chapter 3 o. sport; p. adventure training; q. seminar; r. project; s. debate; and t. visit A table mapping relevant selection criteria against appropriate training activities is in annex E Training settings. Training may be delivered on-the-job or off-the-job, in residential or day schools or, for distance learning, in various settings. Any one or mix of these may be used for a course/program. Where the learning outcomes depend on a workplace context eg operating large, expensive equipment, then OJT will be the obvious choice. For other learning outcomes where the workplace environment is not so critical, then off the job training may be more cost-effective and less disruptive to the workplace. Learner choice of setting, as occurs in distance learning, is appropriate for learner-centred training where group interaction and time are relatively insignificant Practical considerations. In addition to the advantages and disadvantages of possible training modes, activities and settings, the selection of training methods must consider practical factors such as: a. the appropriateness of the method to the learners; their numbers, location, interests, abilities and level of development as recorded in the learner profile; b. the media requirements of the selected method; c. the availability of environments, facilities, instructors and physical resources (equipment, tools, job and instructional aids); and d. the time and cost of developing and implementing training methods If significant costs or time are involved, then the need should be identified and managed early to ensure the required allocations. 3 35

142 ADFP Chapter Costs and times of development. The cost and time of acquiring and developing the resources necessary to implement a mode of training are difficult to accurately detail. Estimates are necessary though, for informing selection processes and for budgeting training. The feasibility analysis process in the Analyse phase provided broad estimates. An additional level of detail, and therefore accuracy is provided in the Design phase. That detail should consider: a. initial acquisition or development costs and times of facilities, materials and equipment (annex F contains guidance for estimating the costs of and time required for developing new training materials); b. delivery costs, including trainee salaries and support costs; c. staffing costs, including acquiring, training and supporting staff; d. ongoing operating and maintaining costs over the anticipated life of the training, including the costs of any service agreements; e. depreciation/replacement costs; f. administrative costs; and g. any anticipated revenue, where this is applicable. Step 2: Apply other principles of learning theory Either following, or concurrent with selecting training methods related principles of learning theory should be applied. Some such principles were applied when designing the assessment strategy. Some are covered in step 5, generate learning outcomes, dealing with sequencing learning. Other principles that should be considered include: a. opportunities for practice and feedback (learning is enhanced when performance is promptly followed by formative assessment, and personal and valid feedback); b. variety in methods (variety in methods can be stimulating, promoting interest and learning efficiency; too much or too little variety can be counterproductive); and c. activities promoting retention and transfer (relating to previous learning; repetition and practice, particularly in differing contexts, promotes learning retention; training methods that replicate workplace activity). 3 36

143 ADFP Chapter 3 Step 3: Select associated media The term media refers to all training/learning aids that assist the learning process. Because some media are more appropriate to some methods than others, the two fields are often considered concurrently. However, for any delivery method, one or more media might be appropriately employed The choice of training/learning media includes: a. instructor, supervisor, mentor; b. SMEs (individuals and communities of practice); c. computers (stand-alone or networked); d. job aids (includes multipurpose devices); e. real objects (including smart objects); f. simulators and emulators; g. boards (white, chalk, magnetic etc); h. visits and field trips; i. books (test, reference, programmed learning); j. wall charts; k. video and audio (compact disc or tape); l. models; and m. television (broadcast or closed circuit) Each of these media has advantages and disadvantages, some of which are liable to change consequent to rapid technological development. In consequence selecting the media for a training method requires current knowledge. Media selection models may provide such knowledge and also assist with decision making. If used, their currency and basis for outputs should be understood Factors affecting media selection. Several factors need to be weighed when choosing which media should support a learning method. They include: a. the learning outcomes (the media should enable the performance of the activity described in the learning outcome); 3 37

144 ADFP Chapter 3 b. the characteristics of the learners (including experience, motivation, levels of English and computer literacy, numeracy, numbers, age and maturity); c. the characteristics of the instructors, (can and will the instructor use the media well?); d. constraints (eg unavailability due to cost; insufficient acquisition, development or course time; insufficient support staff; course schedule; class/group size; and rank/ability/gender of learners); e. time required to develop/prepare; f. availability of the selected media (eg simulators); g. supportability of new media through life (eg maintenance); h. life expectancy of the training and/or the media technology; and i. possible spin-offs into other training areas Decisions on media are often a compromise between the best solution to a learning method and resource management consideration. Once made, those decisions are liable to be changed as new understanding is gained, not infrequently as a result of work in the Develop phase of the DTM. Step 4: Combine training and assessment strategies It is important that the strategies for training and assessment be combined in a way that: a. enhances learning, b. promotes efficiency, and c. enables continuous improvement If both strategies are concordant with the outcomes specified in the TRS, and the principles of learning theory described in preceding paragraphs are applied, the amalgamation should be straightforward and achieve these aims On completion of the merger it is timely to review and, if necessary revise the work to ensure it is valid (covering all that is required), suitable (covering it in the most effective and efficient ways), and administratively feasible within identified constraints. It can also be appropriate to review the 3 38

145 ADFP Chapter 3 indicative time needed for the training to be completed. Course/program lengths cannot be ascertained until completion of the Develop phase, but a good estimate at this stage of design can be useful for managing the pilot course process and for planning the subsequent conduct of training. Step 5: Develop course map When merged, the training and assessment strategies are likely to have a sequence arising from the generation of learning outcomes. In most situations alternative pathways for completing the course/program may be used. Alternatives that might be followed without detriment to training may be presented in a course map. A course map enables flexibility of scheduling and presents options for the day-to-day programming of training A course map may be produced either manually, or through use of a software tool (eg Syllabus + and Microsoft Project). The course map is derived by discovering the dependencies between learning outcomes/modules, taking into account prerequisites and co-requisites. Whatever means employed the resultant map should be easy to produce and use. A model course map with associated explanations is provided in annex G. Step 6: Complete training and assessment strategy To complete the training and assessment strategy document the material developed in the previous steps is prepared for incorporation in the draft curriculum and as a resource for designing strategies for TR&E. Although effectively a working document, the training and assessment strategy can be transferred directly into the draft curriculum. A template for a training and assessment strategy is in annex L Training and assessment strategy documents must meet appropriate standards. They are 13 : a. audience focus: (1) appropriate to industry and training context, (2) consistent with the range of needs of potential users, and (3) relevant materials ensured for different modes and pathways. 13 Training Package Development Handbook, DEST, pages

146 ADFP Chapter 3 b. sufficiently clear, relevant content for the intended audience: (1) clear links to, and consistency with the competency outcomes of qualifications; and (2) support the development of applied knowledge and skills, including workplace communication, language, literacy and numeracy. c. written in an attractive style that is useable by the intended audience: (1) written in plain English, avoiding training jargon; and (2) having a user-friendly format and presentation. d. encourage flexibility in training and assessment: (1) provide for maximum flexibility in training and assessment delivery, (2) facilitate gaining qualifications through a range of pathways, and (3) promote integrated assessment. e. are cost-effective to produce and maintain: (1) incorporate generic and cross-industry material where appropriate to support portability with and across industry sectors; and (2) avoid replication of existing resources where these meet training package requirements. Step 7: Gain approval A training and assessment strategy is a major input to the curriculum. Because there may be significant resource implications associated with implementing the strategies, their approval is critical, especially where only broad cost estimates have been identified in a feasibility report. Consequently, gaining approval of the document from the Design Authority is necessary before proceeding to development of the curriculum. 3 40

147 ADFP Chapter 3 COMPILE CURRICULUM Introduction A curriculum presents the specifications for a training program, course or subject (module). The curriculum describes the learning experiences a learner undergoes during training, the intended learning outcomes, the strategies used for training and assessment, the content covered, the training methods and prescribed or recommended assessment tasks. A curriculum is the major component of a TMP the complete set of documentation necessary for managing a training course or program The contents of a curriculum are the products of preceding processes of the DTM Purpose. The purpose of compiling a curriculum is to produce a training document, which includes the specifications, strategy, content and resources required to conduct a training course/program. This document is the major specification for the Develop phase of the DTM Trigger. A curriculum for a training course/program is compiled following design of a training and assessment strategy. Inputs, process and products The inputs, process and products for compiling curriculum are summarised in table 3 8. Inputs Process Products TRS, training and assessment strategy, CAS. Step 1: Compile curriculum. Step 2: Confirm curriculum. Curriculum. Table 3 8: Inputs, process and products for compiling curriculum Step 1: Compile curriculum A curriculum builds-on the training and assessment strategy document by adding the content of the training program/course. The required training content is contained in the CAS that were the product of previous processes in the Design phase. A training management package template for a curriculum is in annex M. 3 41

148 ADFP Chapter 3 Step 2: Confirm curriculum As a component of a TMP, the curriculum is approved by the Design Authority, at the end of the Design phase. In preparation for that, training staff associated with the work (including those involved with the Design and Conduct phases of the training) should be invited to review its contents and confirm their suitability and practicability. DESIGN TRAINING REVIEW AND EVALUATION STRATEGIES Introduction The effectiveness and efficiency of a training program/course are maintained by applying continuous improvement processes. Two forms of the processes are planned in the Design phase: a. Training review (internal evaluation). The evaluation and reporting of the Conduct phase of the DTM; and b. Evaluation. The evaluation and reporting of how well training has contributed to meeting the original requirement The two processes are largely independent of each other, but both relate to the structure and content of the course, as illustrated in figure 3 3. Figure 3 3: Training review, evaluation and the training process Strategies for the TR&E of a course/program provide broad guidance on the what, where, when and how of the processes. They cover the methods to employ, when to employ them and the resources required. This information, contained in a TMP, is used in the Develop phase for producing the materials to be used and in the Conduct phase for ensuring the processes are properly conducted Purpose. The purpose of designing TR&E strategies is to ensure that effective and efficient continuous improvement processes are applied to training. 3 42

149 ADFP Chapter Trigger. As the TR&E strategies are applied to the structure and content of a course, they are designed following completion of its training and assessment strategy. Inputs, process and products The inputs, process and products for designing TR&E strategies are summarised in table 3 9. Inputs Process Products Training and assessment strategy. Step 1: Determine need for TR&E. Step 2: Determine information requirements. Step 3: Select information sources and collection methods. Step 4: Record the TR&E strategies. TR&E Strategies document. Table 3 9: Inputs, process and products for designing training review and evaluation strategies Defence uses the concepts of Donald Kirkpatrick s model for evaluating training programs in its TR&E strategies 14, it being a standard reference. The levels of assessment referred to in this chapter are those identified in the model. They are illustrated in figure 3 4 and explained in subsequent paragraphs. Figure 3 4: Kirkpatrick s model for evaluating training programs The Four Levels The four levels identified by Kirkpatrick represent a sequence of ways to evaluate training. Each level is important, but the processes of evaluation become more difficult and time consuming during the progress from level one to level four. The information gained from higher levels is, however, more valuable. The four levels identified by Kirkpatrick are explained below. 14 Kirkpatrick D. L., 1994, Evaluating Training Programs The Four Levels, Berret-Koehler Publishers, San Francisco. 3 43

150 ADFP Chapter Level 1: Reaction. As the word implies, evaluation on this level measures how the trainees react to a training program/course. These reactions are usually recorded using end-of-course questionnaires, or by individual or group debriefs. The evaluation focuses on the efficiency of training as judged by the trainees and training staff, as its effectiveness cannot be realistically measured until the trainee has spent some time in the workplace, applying the skills, knowledge and attitude they have learned Level 2: Learning. Kirkpatrick defines learning as the extent to which participants/trainees change attitudes, improve knowledge, and/or increase skill as a result of attending the program. These outcomes are measured by assessments conducted on completion of a training program/course/module Level 3: Behaviour. To be effective, learning resulting from training needs to be applied in the performance of workplace tasks. Kirkpatrick s third level addresses the extent of this transfer. In the DTM the level is an important focus of the Evaluation phase, with information gained being of importance to trainers Level 4: Results. Results are the effects of training on the organisation. Desired results can include enhanced capability, improved quality, decreased costs, increased sales, and reduced frequency and/or severity of accidents. There may also be undesired results. This level is also assessed during the Evaluate phase, but at this level the information gained is of most relevance to the job or process owner. Step 1: Determine need for training review and evaluation All training should have its effectiveness and efficiency reviewed. The extent and rigour of such reviews should depend on the risks to those qualities. Assessing the risks requires judgment. If a course is new, its inputs untried and the course is high in significance the need for TR&E is likely to be high. Conversely, if a training course/program is well-established, of proven worth, and its inputs and processes do not change, the need will be less. Circumstances differ, but the following points provide guidance in making appropriate judgments Strategies for TR&E have two dimensions: their application to a single course/program and their application to a series of courses/programs over an extended period Most training courses/programs should incorporate level 1, reaction checks, at their conclusion and, if long or involved, at suitable points throughout. The frequency of such checks can vary. Some circumstances justify a higher frequency than daily and others being at greater than monthly intervals. 3 44

151 ADFP Chapter Most training situations will require level 2 training review. This level is the subject of checks incorporated in the assessment strategy. It is usually programmed for the end of training, but may also be used formatively at key points during its conduct Level 3 and level 4 evaluations (see chapter 6 The Evaluate phase ) are usually conducted concurrently, as the information required is similar and the same methods and processes can serve both purposes. They can be planned as comprehensive reviews, routine checks, or reviews targeted at specific components or aspects of a course/program. All new training should be subjected to a comprehensive evaluation. Once the training is established, routine checks at a more superficial level may suffice. Targeted reviews are planned when a specific need is identified, possibly arising from another continuous improvement process Develop plan of Training Review and Evaluation activities. The product of determining the need for TR&E is an indicative plan of TR&E activities to be applied to a training course/program. The plan should list both the training review activities of a single course/program and the intended schedule of evaluation activities to be applied over the life of a course/program. Step 2: Determine information requirements The information requirements of routine TR&E tend to be broad and enduring. They can therefore usefully be incorporated in the TR&E strategies document that informs the development of information collection instruments Level 1 training reviews commonly seek feedback on the efficiency and effectiveness of: a. the purpose/concept of the training; b. training administration; c. locations/venues and facilities used; d. pre-course requirements; e. training content; f. assessment strategies, instruments and conduct; g. methods, media and material selected together with their use; and h. the training program. 3 45

152 ADFP Chapter Level 2 training review seeks to determine what trainees have learned. This work is also a function of the assessment strategy, the design of which is focused on measuring and supporting trainees progress. In training review the focus is more on the improving the training process. Towards this end, information may be sought on all things learned by trainees, random sample areas or specifically targeted areas The information common to level 3 evaluation (see chapter 6) assesses how well training has prepared learners for the workplace and whether the original capability requirement has been met includes: a. the required job performances for which training was not provided; b. the required job performances at which training graduates are performing below the required standard; c. the competencies trained but not used in the workplace; d. the competencies trained to a standard higher than required in the workplace; e. the competencies in which graduates experience difficulty in adapting from off-the-job training to the workplace; f. reflections on the design and conduct of training received; g. any workplace performance requirements that are assessed as inadequate; h. any recent changes to the organisation of work, its procedures or equipment used; and i. the attitudes of graduates to their work, training and their organisation Level 4 feedback (see chapter 6) is less commonly sought and more likely to be situation-specific. The information requirements are likely to relate to the effects of the training program/course on the organisation and the training staff. They might include attempts to cost the training. Step 3: Select information sources and collection methods The sources of information on the conduct of TR&E are limited. For training review they are the trainees, their instructors, training administration staff and the results of assessments. For evaluation the principal sources are the graduate trainees, their supervisors and, occasionally, ex-training staff employed in the workplace. Each source is likely to have differing, yet valid perspectives of training. Where possible, therefore, all should be used albeit that sampling may be employed. 3 46

153 ADFP Chapter Many methods can be used to collect TR&E information. Only the major ones that require the allocation of resources and the development of collection instruments are considered during the Design phase. Commonly used methods, which may be applied individually or in combination, include: a. questionnaires; b. individual or group structured interviews/discussions/workshops; c. workplace observation and the use of check sheets; d. examination of job product; and e. competency testing Assessing what guidance on information sources and collection methods the TR&E strategies document should include requires judgments on the resource costs and likely benefits. As was noted in step 1, these factors will change over the life of a course/program. TR&E strategies should reflect this. Step 4: Record the training review and evaluation strategies The strategies developed for TR&E are recorded in a form enabling them to be incorporated in a TMP. The template in annex M provides a suggested format for TR&E strategies. The document will enable information collection instruments to be prepared during the Develop phase, and the conduct of TR&E to be administered appropriately in the pilot course process and Conduct phase. COMPILE DRAFT TRAINING MANAGEMENT PACKAGE Introduction A TMP presents the complete set of documentation necessary for managing the conduct of a training program or course. It is the output of training preparation and the central resource for the conduct of training A complete TMP contains the curriculum of a program/course, its TR&E strategies, and the materials developed for conducting training. At the end of the Design phase a TMP is in draft form, and incomplete. This is partly because the strategies contained are likely to change consequent to work during the Develop phase and partly because the materials for conducting a program/course are added during that phase. 3 47

154 ADFP Chapter Despite being incomplete, a draft TMP has all the content needed for a program/course to be accredited by a State or Territory course accrediting body. If such recognition has been deemed desirable, a submission may be made at this stage of the DTM, although waiting until the completion of the pilot process of the Develop phase is less risky Purpose. The purpose of compiling a draft TMP is to establish the basis of a single authoritative reference point for a training program/course Trigger. As the summative record of the Design phase of the DTM, the draft TMP is compiled following design of the curriculum and the TR&E strategies. Inputs, process and products The inputs, process and products of compiling a draft TMP are summarised in table As with other processes in the DTM, the steps are not necessarily sequential and some are likely to be undertaken concurrently. Inputs Process Products Curriculum, training and assessment strategies. Step 1: Compile draft TMP. Step 2: Gain formal endorsement and approval of the draft TMP. Step 3: Submit for accreditation (if required). Draft TMP. Table 3 10: Inputs, process and products for compiling a draft Training Management Package Step 1: Compile Training Management Package A TMP builds on a curriculum by adding course management information and the TR&E strategies. The requirements of a TMP will vary with the needs of the organisation and of the training course/program. An indicative template for a cover sheet, contents page and Training Management Information section is in annex M. This template incorporates the main features of a DEST national template designed to assist the development of courses for accreditation under the AQTF, that lead to an Australian Qualification Framework qualification or Statement of Attainment Further details about this template can be found in the DEST booklet, Guidelines for Course Developers, July

155 ADFP Chapter 3 Step 2: Gain formal endorsement and approval of draft Training Management Package Albeit in draft form and incomplete, the draft TMP is endorsed by the Design Authority when they are satisfied that within the context of the situation to which they apply: a. the curriculum meets the requirement of the TRS; b. the training and assessment strategies are valid, suitable, reliable and practicable; and c. the TR&E strategies are appropriate and practical At this point the approval of the customer of training, the job/bpo, is formally sought and obtained. This action serves as a useful check that the requirement of the TRS remains valid. On receipt of this approval, the TMP can properly be used as the basis for work in the Develop phase of the DTM. Step 3: Submit for accreditation If the Analyse Authority determined that the course/program should be recognised by the award of a National qualification, the application may be undertaken when the design of a course is complete. By this time all the information required by the related State or Territory accrediting body is drafted (note: if changes to the documentation are likely to be made consequent to the piloting in the Develop phase, it might be preferable to delay the submission until the Develop phase has been completed) Broad guidelines and a template for submitting a program/course for accreditation are described in Guidelines for Course Developers, July 2002 produced by DEST and accessible on its web site (see Detailed advice on making a submission should, however, be obtained from the State or Territory accrediting body If a proposed program/course is accredited, the State or Territory course accrediting body will provide an accreditation document, such as a letter or certificate that includes the title and national code for the accredited course, and the start and end date of the accreditation. The body will also record the accredited course on the National Training Information Service. Annexes: A. Flow chart showing the Design phase of the Defence training model B. Defence training model purpose, inputs, process and products of the Design phase C. National assessment principles and their essential features D. Factors to consider when selecting appropriate training modes 3 49

156 ADFP Chapter 3 E. Some criteria for selecting appropriate training activities F. Guidance for estimating the costs and times of developing new training materials G. Model course map and explanation H. Environments profile template I. Learner profile template J. Competency analysis sheet template K. Training and assessment strategy template L. Training review and evaluation strategies template M. Training management package indicative template 3 50

157 Annex A to ADFP Chapter 3 FLOW CHART SHOWING THE DESIGN PHASE OF THE DEFENCE TRAINING MODEL A 3A 1

158

159 Annex B to ADFP Chapter 3 DEFENCE TRAINING MODEL PURPOSE, INPUTS, PROCESS AND PRODUCTS OF THE DESIGN PHASE B Design PHASE PURPOSE INPUTS PROCESS PRODUCTS To ensure that the selection of assessment and training strategies is appropriate to learners characteristics and circumstances To help ensure that the associated strategies are appropriate Training Requirement Specification (TRS), Target Population Profile, current policies and instructions, data from Defence Recruiting, job sponsors, survey results and national literacy standards TRS, Feasibility Report current policies and instructions, information from workplace supervisors and managers, training organisations and schools, resource specifications Analyse prospective learners Analyse learning and assessment environments Learner profile Environments profile 3B 1

160 ADFP PURPOSE INPUTS PROCESS PRODUCTS To identify the basic skills (physical and mental) knowledge and attitudes required to achieve competencies To specify the objectives of training To guide the use of assessment for supporting and measuring learning in a training course/program and for determining achievement of workplace competence To ensure that training is conducted effectively and efficiently, consistent with relevant policies and learning theories Competency standards from the TRS, existing industry and Defence training/learning materials, and the learner profile CAS, learner and learning environment profiles Competency units from the TRS, learner and environments profiles, learning outcomes, relevant existing assessment resources, training failure management policies Learning outcomes, the competency assessment strategy, and the learner and training environment profiles, existing training materials Analyse competencies Competency Analysis Sheet (CAS) Generate learning outcomes Outcomes 2 types: course learning outcomes and module learning outcomes Design assessment strategy Training and assessment strategy, including: Design training strategy and produce the training and assessment strategy document location, methods; continuous improvement processes; media, sequence; resource requirements. 3B 2

161 ADFP PURPOSE INPUTS PROCESS PRODUCTS To ensure that effective and efficient continuous improvement processes are applied to training To produce a training document specifying the specifications, strategy, content and resources required to conduct a training course/program To collate in one package all course documentation/materials The TRS, learner profile, learning outcomes, learning strategy and the competency assessment strategy, estimates of available resources Training and assessment strategy, CAS Curriculum, training review and evaluation strategies Design training review and evaluation strategies Training review and evaluation strategies Compile curriculum Curriculum Compile and gain approval of draft Training Management Package (TMP) Draft TMP 3B 3

162

163 Annex C to ADFP Chapter 3 NATIONAL ASSESSMENT PRINCIPLES AND THEIR ESSENTIAL FEATURES PRINCIPLE Endorsed industry/enterprise standards are the benchmarks for assessment, where they exist. Assessment guidelines within endorsed training packages provide the framework for assessment of the units of competence within that industry, industry sector or enterprise. Endorsed industry/enterprise standards form the basis of qualifications and Statements of Attainment in the vocational education and training sector, where they exist. ESSENTIAL FEATURES 1.1 Endorsed industry/enterprise competency standards are the benchmarks for all assessment activity conducted for the purposes of national recognition, regardless of whether this is through a training and assessment pathway or assessment only pathway. 1.2 Where national industry/enterprise standards do not exist, specified competencies agreed to by the appropriate industry, enterprise, professional or community group are the benchmarks for assessment. 2.1 Where a relevant endorsed training package exists, assessment will be conducted in accordance with the assessment guidelines and competency standards of the endorsed training package. 3.1 Qualifications within the vocational education and training sector of the Australian Qualifications Framework (AQF) are based on endorsed packages of national industry/enterprise competency standards. 3.2 Where national industry/enterprise endorsed standards do not exist, qualifications or Statements of Attainment (SOA) must be based on specified competencies agreed to by the appropriate industry, enterprise, professional or community group. 3.3 A qualification of an SOA under the AQF may be attained through a variety of pathways, including training and assessment pathways as well as assessment only pathways. C 3C 1

164 ADFP PRINCIPLE Assessment conducted for the purposes of national recognition should lead to a qualification or SOA under the AQF. Assessment should be undertaken by, or auspiced through, an RTO. Assessment for national recognition purposes shall be conducted within a quality assurance framework. ESSENTIAL FEATURES 4.1 All assessment activity that is conducted for the purposes of national recognition, regardless of whether this is through a training and assessment pathway or an assessment only pathway, should lead to the issuing of a qualification or SOA under the AQF. 4.2 A qualification of SOA, recognised under the AQF, should be issued when an individual is assessed as competent against endorsed competency standards. Assessment for national recognition purposes is underwritten by effective mutual recognition arrangements under the Australian Recognition Framework (ARF) which requires all Registered Training Organisations (RTO) throughout Australia to fully recognise and credit qualifications and SOA issued by all other RTO. 5.1 Assessment for national recognition purposes should be undertaken by, or auspiced through, an RTO. 6.1 Assessment for national recognition purposes, regardless of whether this is through a training and assessment pathway or an assessment only pathway, should be undertaken by qualified assessors, involve the use of industry-endorsed assessment guidelines and be underpinned by an appeals process. 3C 2

165 ADFP PRINCIPLE ESSENTIAL FEATURES Assessment against competencies in a training package will be carried out in accordance with the endorsed assessment guidelines within the training package. The endorsed assessment guidelines include the necessary qualifications for those conducting assessment and provide for those situations where more than one person may contribute to the assessment and where the required technical and assessment competencies may not all be held by any one person. 6.2 All assessments require the application of the following endorsed competency standards: The Assessment Competency Standard: Conduct Assessment in accordance with an Established Assessment Procedure and Extension Unit: Plan and Review Assessment from the Competency Standards for Assessment. The relevant vocational competencies, at least to the level being assessed. 6.3 Assessment carried out by or under the auspices of an RTO will by subject to audit, consistent with ARF requirements. External audit procedures should focus on improving both the operation and outputs of assessment processes. 3C 3

166 ADFP PRINCIPLE ESSENTIAL FEATURES Responsibility for assessment 7.1 Responsibility for assessment resides with the body that issues the should reside with the body that qualification or SOA under the AQF. issues the qualification or SOA. 7.2 Partnership arrangements involving RTO can be developed to establish assessment arrangements that enable individuals to gain national recognition under the AQF. Assessment processes shall be valid, reliable, flexible and fair. 8.1 Assessment processes must cover the broad range of skills and knowledge needed to demonstrate competency. 8.2 Assessment of competence should be a process, which integrates knowledge and skills with their practical application. 8.3 During assessment, judgments to determine an individual s competence should, wherever practicable, be made on evidence gathered on a number of occasions and in a variety of contexts or situations. 8.4 Assessment processes should be monitored and reviewed to ensure that there is consistency in the interpretation of evidence. 8.5 Assessment should cover both onand off-the-job components of training. 8.6 Assessment processes should provide for the recognition of competencies no matter how, where or when they have been acquired. 8.7 Assessment processes should be made accessible to individuals so that they can proceed readily from one competency standard unit to another. 8.8 Assessment practices must be equitable for all groups of individuals. 8.9 Assessment procedures and the criteria for judging performance must be made clear to all individuals seeking assessment. 3C 4

167 ADFP PRINCIPLE Assessment systems must incorporate mechanisms for recording, storing and accessing assessment outcomes. Assessment recording systems should indicate the units of competence that the individual has achieved. Assessment systems should incorporate ongoing monitoring and review processes. Assessment processes shall provide for the recognition of current competencies regardless of where these have been acquired. ESSENTIAL FEATURES 8.10 There should be a participatory approach to assessment. The process of assessment should be jointly developed/agreed between the assessor and the candidate Opportunities must be provided to allow individuals to challenge assessment decisions with provision for reassessment. 9.1 Assessment systems must incorporate mechanisms for recording, storing and accessing assessment outcomes. 9.2 The body that is responsible for issuing the qualification or SOA under the AQF should also be responsible for maintaining and operating a secure, permanent and reliable system for recording and storing assessment outcomes Qualifications and SOA issued under the AQF should indicate the units of competency that the individual has attained Assessment reporting systems should conform to any national nomenclature developed for reporting assessment outcomes SOA issued under the AQF should have a nationally consistent format and include common categories of information Assessment systems should incorporate ongoing monitoring and review processes Recognition of prior learning (RPL) focuses on identifying the endorsed industry/enterprise competency units currently held by individuals as a result of formal and informal training, not how, when or where the learning occurred. 3C 5

168 ADFP PRINCIPLE ESSENTIAL FEATURES 12.2 RPL underpins any system of competency based training. It is essential that RTO have a demonstrable commitment to recognising the prior learning of individuals RPL shall be available to all potential applicants RPL shall involve processes that are fair to all parties involved RPL shall involve the provision of adequate support to potential applicants. 3C 6

169 Annex D to ADFP Chapter 3 FACTORS TO CONSIDER WHEN SELECTING APPROPRIATE TRAINING MODES FACTORS TO CONSIDER WHEN SELECTING LEARNING MODE Factors to consider Trainer-centred Learner-centred 1. Subject matter is predominantly: dynamic X static X simple X complex X conceptual X procedural X 2. Lessons require: repetition X constant delivery X detailed record keeping and score X tracking instructor observation and evaluation X self-evaluation by learner X 3. Target group is: small X X large X X of varied backgrounds and X X capabilities geographically scattered X network connected X X 4. Feedback required is: specific X open or varied X 5. Development time is: limited X flexible X 6. Financial resources are: limited X flexible X 7. Simulation is required X X 8. Learner s education background is: extensive X intermediate X X poor X 9. Human resource limitations are: few instructors X X plentiful instructors X X D 3D 1

170

171 Annex E to ADFP Chapter 3 SOME CRITERIA FOR SELECTING APPROPRIATE TRAINING ACTIVITIES E Lecture Lesson Demonstration Performance Discussion Independent study Exercise Tutorial Simulation Role-play Case-study Study text Visit Drill Games Instruction-based x x x x x x x Experience-based x x x x x x x x x Skill-based x x x x x x x x x Knowledge-based x x x x x x x x x x x x High complexity x x x x x x x x x x x x x Low complexity x x x x x x x x x x x x x x High resistance x x x x x Low resistance x x x x x x x x x High interaction x x x x x x x x x Low interaction x x x x x x x x x Start of course x x x x x x x x x Middle of course x x x x x x x x x x x x End of course x x x x x x x x x x x x x x Large class x x x x x x x x x x x Small class x x x x x x x x x x x X x x High ability x x x x x x x x x x x Low ability x x x x x x x x x x Short time frame x x x x x x x x x x x x Long time frame x x x x 3E 1

172

173 Annex F to ADFP Chapter 3 GUIDANCE FOR ESTIMATING THE COSTS AND TIMES OF DEVELOPING NEW TRAINING MATERIALS 1. The cost of, and time required to develop new training materials are closely linked, as the major cost involved is that of the personnel required for the work. The length of time required, the numbers of people involved, and the levels of expertise and experience required of those people will vary with the circumstance, each of which is unique. 2. Listed below are the indicative times required for developing the materials course development for an hour of course time. (For example, a ratio of 15:1 means that 15 hours of development are required for each course hour). For each method of training a range of values is presented. The range allows for the specifics of various training situations, including the expertise of the practitioners: a. Seminar 15:1 to 30:1. b. Audio tapes or audio compact disc 1:1 to 5:1. c. Lecture 2:1 to 10:1. d. Televised lecture 2:1 to 5:1. e. Video 20:1 to 50:1. f. Interactive video 50:1 to 100:1. g. Face to face (F2F) instruction 2:1 to 30:1. Examples are as follows: (1) adapting extant material 2:1 to 15:1; and (2) new material developed (from scratch) approximately 15:1 to 30:1. h. Online learning/web-based 1 7:1 to 1000:1 depending on complexity eg adapting a simple page format could take only 7:1. Other examples are: (1) Converting F2F material 48:1. This assumes that: F (a) (b) a library of graphics is available; subject matter experts (SME) are available; 1 Defence Flexible Learning Solution advice, F 1

174 ADFP (c) (d) (e) content authors and instructional designers are available; four hours F2F coverts to approximately one hour of online courseware of approximately 200 screens; and testing, refinement and trialing are additional. (2) New material (from scratch ) approximately 70:1. This assumes that: (a) (b) (c) (d) a library of graphics is available; SME are available; content authors and instructional designers are available; and testing, refinement and trialing are additional. (3) Web-based developed within a template 2 approximately 40:1 to 300:1 as follows: (a) text only with limited interactivity 40:1 to 100:1; (b) text and graphics 100:1 to 150:1; (c) (d) text, graphics and animation with moderate interactivity 150:1 to 200:1; learning object-based dynamic web page 60:1 to 300:1; and (e) online help system 3:1 to 10:1. (4) Web-based material developed (from scratch ) approximately 100:1 to 1000:1 as follows: (a) text only and limited interactivity 100:1 to 150:1; (b) text and graphics 150:1 to 200:1; 2 Web-based data adapted from Kapp, K.M. How Long Does it Take? Estimation Methods for Developing E-Learning in Learning Circuits ASTD s Online Magazine All About E-Learning, 14 July F 2

175 ADFP (c) text, graphics and animation with moderate interactivity 250:1 to 400:1; (d) televised lecture 2:1 to 5:1; (e) (f) fully interactive text, graphics and animation with full interactivity 400:1 to 600:1; and fully interactive virtual reality simulation and full interactivity. i. Computer managed learning 40:1 to 60:1. j. Distance broadcast (satellite) 15:1 to 30:1. 3F 3

176

177 Annex G to ADFP Chapter 3 MODEL COURSE MAP AND EXPLANATION 1. A course map is shown below. Each horizontal line represents a course event (course introduction, module of instruction, major assessment, visit etc). Written above each line is an event identification letter, and under it is the time required to complete the event. In this example there are eight events, A H, which take a total of 22 hours to conduct. The instructional sequence demands that A, B and C must occur before D, which, together with E must be completed before F. G can occur at any time after A the course introduction. G Key: Figure 3G 1: Course map Code A B C D E F G H Event Course introduction Module 1 name Module 2 name Module 3 name Module 4 name Module 5 name Visit Competency assessment 2. Each elongated vertical loop represents a point-in-time relevant to the starting time for the course. The two numbers in the loop are the latest event time the maximum time required to reach that point-in-time, and the earliest event time the minimum time required to reach that point-in-time. 3. No event to the right of a loop/point-in-time should be started until all the events to the left of that loop are completed. Module time includes instructional time as well as any linked assessment time. There is no significance in the differing sizes of the loops or in the differing lengths of the lines. 3G 1

178

179 Annex H to ADFP Chapter 3 ENVIRONMENTS PROFILE TEMPLATE H COURSE TITLE 1 Course: Provide any course title, code or reference number Job: Title. Provide the name or description of the job for which the course is designed. Designation. Provide any job code or reference number. 1. This document presents the features of the environments to be used for: a. training for the competencies listed in the Training Requirement Specification, and b. assessing achievement of the competencies required for the work of job title. 2. It identifies the features or conditions of those environments that can help or hinder the processes. Learning environment(s) Facility description. Provide a broad description of the workplace/school/college. If relevant, detail its management arrangements. Include the contact details of authorities. If support facilities (eg accommodation, domestic) are needed for using the facility, the need and how it will be met should be identified. Planned modifications to the facility that can influence training should be noted. 4. Access and availability. Note any limitations concerning access to and the availability of the facility now and foreseen. 5. Location. If significant, describe the location in relation to the workplace, learners accommodation, associated training facilities and transport facilities. 1 Where more than one environment is to be used, each should be considered separately. 3H 1

180 ADFP Activity spaces. Describe aspects of the spaces for activity that might influence training. Possible factors include size and suitability, power and water resources, computer network access, storage features, floor type, changing/washing facilities, heating, lighting, positioning of teaching and learning aids and screens. 7. Resources. Any available training resources should be listed, including equipment, tools, computers, the types and numbers of desks/benches projectors, screens, printed materials, photocopiers and other electrical equipment. 8. Staff. The availability of suitable training staff for both the administration and conduct of training must be covered. Their numbers for both training and administration and their readiness for such work should be described. Similarly, the motivation of the staff and any need to prepare them for new responsibilities should be noted. 9. Finances. Note any financial aspects of the environment that might be relevant (beware of the commercial sensitivity of such information). 10. Policy. Note any policies of possible significance at the local school/establishment, branch or Service levels. Assessment environment(s) Facility description. Provide a brief, broad description of the environment (workplace, exercise or training area). If relevant, detail its management arrangements. Include the contact details of authorities. Planned modifications to the environment that would influence assessment should be noted. 12. Access and availability. Note any limitations concerning access to and the availability of the environment now and foreseen. This is particularly important for assessment in working environments. Any security requirements for students and instructors should also be noted. 13. Location. If significant, describe the location in relation to the workplace, associated training facilities, accommodation and transport facilities. 14. Activity spaces. Describe any aspects of the spaces for activity that might influence the assessment. Possible factors include size and suitability, power and water resources, storage features, floor type, changing/washing facilities, heating, lighting. 3H 2

181 ADFP Resources. Access to any resources that could be used for the assessment should be listed, including equipment, tools, computers, the types and numbers of desks/benches, electrical equipment etc. 16. Staff. The availability of suitable staff for conducting the assessment must be considered. Their numbers and readiness for the task must be described together with their motivation and any need to prepare them for the new responsibilities. 17. Policy. Note any policies at the environment that might have significance, particularly any which might constrain assessment in unusual circumstances. 3H 3

182

183 Annex I to ADFP Chapter 3 LEARNER PROFILE TEMPLATE I COURSE TITLE 1 Course: Provide any course title, code or reference number Job: Title. Provide the name or description of the job for which the course is designed. Designation. Provide any job code or reference number. Profile of typical prospective learner Employment classification. Describe the individual by rank/level, specialisation, stream skill. 2. Proficiencies. List any relevant proficiencies ie qualifications, certificates, licences, employment experience. 3. Aptitudes. List any common aptitude standards. 4. Personal characteristics. List any characteristics of composition by gender, age, culture, location. The age span of the learner group can be relevant. 5. Competencies. List any units of competency not covered by a proficiency and which relate to the course. 6. Academic ability. Where relevant, indicate level of academic ability, especially English and Maths. 7. Physical characteristics. For courses with a special fitness dimension, identify the normal and probable minimum fitness level of the typical learner. 8. Learning styles. Identify any favoured or problematical learning styles, such as experience with self-paced, discovery, on the job training and computer-based learning. 9. Motivation. Identify any significant factors of motivation towards the course and/or the job. 3I 1

184

185 Annex J to ADFP Chapter 3 COMPETENCY ANALYSIS SHEET TEMPLATE J Job: Title: Vehicle maintainer Designation: ABMT Competency: Unit title: Carry out maintenance operations Unit code: AUR00208B 3J 1

186 ADFP Element number Competency element and performance criteria Physical and mental skills 208B/02 Identify and use correct lubricants and fluids Information required for identifying and confirming the lubricants and fluids for particular applications is accessed and correctly interpreted. Reading fluid level indicators. Complete service logs. Use a thermometer and hydrometer (batteries). Interpret thermometer and hygrometer readings. Lubricants and fluids to be used for appropriate systems are identified and applied in accordance with manufacturer s specifications. Use of manufacturer s specifications. Selecting correct lubricants/fluids. Knowledge Attitudes Training notes Safety regulations and rationale. Servicing periods and routines. Types, properties and uses of lubricants and fluids. Sources of fluids. Mixing procedures for battery acids. Dangers with acid. Disposal of old lubricants/oils. Locations of manufacturer s specifications. Risks of inappropriate servicing. Choose and comply with safety regulations, including dress, naked flames. Cleanliness personal and environmental. Consistent with those for the job. Practical use of workshop and vehicles. Occupational health and safety considerations close supervision, small groups. Supply and disposal of materials. Check net for updates/modifications. Use of job aid. 3J 2

187 Annex K to ADFP Chapter 3 TRAINING AND ASSESSMENT STRATEGY TEMPLATE COURSE NAME TABLE OF CONTENTS K Introduction Incorporate an introductory statement identifying in broad terms the reasoning behind the approach taken to training and assessment with reference to the learner, environment and competency analyses. Course event Page Number Course introduction 1 Module 1 title 2 Module 2 title 4 Module 3 title 7 Module 4 title 9 Module 5 title 11 Module 6 title 14 Mid course assessment 16 Module 7 title 18 Module 8 title 19 Module 9 title 22 Module 10 title 24 Module 11 title 26 End of course assessment 28 Approval date Course name and reference COURSE INTRODUCTION Duration 1 Off-the-job training: On-the-job training: Job 3K 1

188 ADFP Content summary Learner pre-course preparation RPL or pre-course testing Summary of resource requirements Human: Physical: $ Cost: References Text: Web links: MODULE 1 Module title Module code Duration COURSE LEARNING OUTCOME: No: Level: Performance Conditions Standard Linked competency as listed in the Training Requirement Specification Prerequisite and/or Co-requisite modules 1 All times listed in this document will be estimates when first entered during the Design phase. They will be confirmed following the pilot course process of the Develop phase. 3K 2

189 ADFP Content summary This module covers the following topics: Assessment method: Training method: Summary of major resource requirements Human: Physical: $ Cost: References Text: Web links: 3K 3

190 ADFP Module learning outcomes 3.3 Module learning outcome 1 Conditions Assessment criteria 1.1 Training method Assessment method Module learning outcome 2 Conditions Assessment criteria 2.1 Training method Assessment method K 4

191 ADFP Figure 3K 1: Course map 3K 5

192

193 Annex L to ADFP Chapter 3 TRAINING REVIEW AND EVALUATION STRATEGIES TEMPLATE L Course reference: COURSE NAME Training review strategy 3.1 The program of training review activities listed in table 3L 1 should be applied to the standard name training course. 3L 1

194 ADFP Activity Information requirement Collection method Pre-course/program Describe each activity and the source to which it refers to eg pre-course assessments, and introduce training review and evaluation processes to: 1. trainees 2. instructors and staff During course/program Describe each activity and the source to which it refers eg: 1. daily feedback from trainees 2. end of module feedback from instructors End of course/program Describe each activity and the source to which it refers eg: evaluate assessments of competency Summary of major resource requirements Human: Physical: Table 3L 1: Training review strategy Collection instrument Duration resources required 3L 2

195 ADFP Evaluation strategy 3.2 The program of evaluation activities listed in table 3L 2 should be applied throughout the period in which the name training course is conducted. Activity Frequency Information requirement Comprehensive review Regular review Targeted review Collection method Collection instrument Duration resources required Summary of major resource requirements Human: Physical: Table 3L 2: Evaluation strategy 3L 3

196

197 Annex M to ADFP Chapter 3 TRAINING MANAGEMENT PACKAGE INDICATIVE TEMPLATE Date of issue: TMP approval date Training Development Authority: Location: Conduct of Training Authority: Location: TRAINING MANAGEMENT PACKAGE PROGRAM/COURSE NAME Ref No Curriculum approval reference: Copy No (Optional) M 3M 1

198 ADFP Name and reference number of program/course TABLE OF CONTENTS SECTION 1 TRAINING MANAGEMENT INFORMATION Course purpose Course description Course learning outcomes Pathways Course awards Accreditation, recognition and licensing Articulation and credit transfer Course reporting Copyright and licensing Special instructions SECTION 2 CURRICULUM Course events Course introduction Module 1 title Module 2 title Module 3 title Module 4 title Module 5 title Module 6 title Mid course assessment Module 7 title Module 8 title Module 9 title Module 10 title Module 11 title End of course assessment Course map SECTION 3 TRAINING REVEW AND EVALUATION Training review strategy: 3 1 Evaluation strategy: 3 2 3M 2

199 ADFP SECTION 4 MAJOR RESOURCE REQUIREMENTS Human resources: instructors: assessors: Physical resource requirements: facilities and infrastructure: accommodation: utilities and electronic networks: Major Defence units: Financial requirements: SECTION 5 MODULE PACKS (Contain the training materials used to conduct training, developed in the Develop phase of the DTM) Course introduction Module 1 etc Training Management Plan approval date: Name and ref of course Section 1 1 COURSE TITLE TRAINING MANAGEMENT INFORMATION Course purpose. A brief statement of the need for and purpose of the training. Course description Course aim. Course location. Course design strategy. Training prerequisite requirements. Duration. Phase Hours Pre-joining preparation Off-job components Training modules 3M 3

200 ADFP Non-instructional hours On-job components Total Course learning outcomes On completion of the course, the graduate is able to achieve the following: Level , 2, 3 or Pathways. Identify alternative pathways eg recognition of prior learning, other courses (eg overseas), workplace learning. Course awards. List qualifications awarded for successful completion of the course. Accreditation, recognition and licensing Course accrediting body. Name of state or territory course accrediting body. AVETMISS information. Provide classification codes for accredited courses. Period of accreditation. Inserted after accreditation. 1 Course learning outcome levels and their meanings. 3M 4

201 ADFP Level 1 2 Description of graduate in relation to prescribed standard Trained, but not against a prescribed standard. Partially trained against prescribed standard. Meaning Background or familiarisation training. Additional training and/or experience required before assessment. Employment under supervision only. 3 Fully Trained and ready for assessment against prescribed standard. Assessment required. Employment under supervision until assessed. 4 Trained and assessed competent against the prescribed standard. Ready for employment. Recognition. State any recognition given by professional or industry bodies. Licensing. State, if applicable, the extent to which the course satisfies licensing/regulatory requirements. Articulation and credit transfer. Provide details of articulation and/or credit transfer arrangements Course reporting. Describe where and how the results of the course will be reported and recorded. Copyright and licensing Copyright. Detail any copyright restrictions on the documentation and identify the owner of the course. Licensing. State any requirements relating to licensing and/or the franchise of the course for use by other providers. Provide useful contact details. Special instructions. Provide any special information about the course. 3M 5

202

203 ADFP Chapter 4 CHAPTER 4 THE DEVELOP PHASE 4 Introduction The Develop phase is the third of five phases constituting the Defence training model (DTM), as shown in figure 4 1. During the Develop phase the materials and equipment used to deliver training and assess outcomes are obtained, all the resources contributing to training are prepared, and a trial of the whole program/course is conducted. Figure 4 1: Develop phase of the Defence training model 4.2 As with all phases, processes and steps of the DTM, the extent to which the Develop phase of the DTM is applied should match the circumstance. In situations of high significance (eg the introduction of training in support of a major new Defence capability) the guidance provided should be applied more closely and carefully. Key products and responsibilities The major product of the Develop phase is a fully developed and practised program/course presented as a Training Management Package (TMP). A draft TMP was prepared in the Design phase. In the Develop phase it is completed by the addition of supporting materials and equipment. 4 1

204 ADFP Chapter Responsibility for the Develop phase and for approving TMPs lies with an organisation s Development Authority. The cooperation of the Conduct Authority and of Defence authorities responsible for major training resources is required. Starting point The processes of the Develop phase are applied to develop new or, more commonly, to improve existing training. When applied to new training, development work follows-on from the approval of a draft TMP that signifies completion of the DTM s Design phase. The trigger for re-developing existing training materials and equipment is recognition that cost-effective enhancements can be made. Processes of the Develop phase The Develop phase of the DTM has four processes: a. checking existing training materials and equipment, b. obtaining training materials and equipment, c. preparing major training resources, and d. piloting the training. 4.7 The flow chart in annex A illustrates these processes, together with their products. Annex B presents the purpose, inputs and products of the processes in table form. The following paragraphs introduce each of the processes and subsequent chapters of this manual provide guidance on the steps for undertaking them. 4.8 Check existing training materials and equipment. The first step in the Develop phase is to check whether suitable materials already exist. If they do, and may be acquired, then a decision is made as to whether they should be used or whether new resources should be developed. 4.9 Obtain training materials and equipment. Obtaining the materials and equipment required for making assessments, conducting training, and implementing Training Review and Evaluation (TR&E) covers a wide range of simple and complex activity, and may involve major expenditure. The resources can be obtained either by purchase or through production. 4 2

205 ADFP Chapter Prepare major training resources. The major resources needed to implement training include facilities (such as training venues, military platforms and accommodation); infrastructure (such as power, water and communication networks); and people (such as trainees, training and assessment staff, workplace supervisors and training administrators). Preparation of such resources ensures they are available and ready for implementing a training program/course Pilot the training. Training is piloted when, newly developed, it is tested under operational conditions and subjected to an in-depth training review. The process is recognition that designing and developing good training is an iterative process, and the best way of identifying possible improvements is to test the program/course in circumstances that are as real as possible. If the training review identifies needed enhancements to the content, strategies, methods, media or conduct of training, they are reported for action. Continuous improvement While working on any of the processes of the Develop phase, stakeholder input is sought to help assure the quality of the product. A consequence of such formative evaluation is that many revisions may be necessary before products are accepted. To keep track of the developments and to enable reference during later development work, an accurate and comprehensive record of development activity should be maintained. Use of an external agency In accordance with national and Defence guidelines, the quality assurance of all training by, or on behalf of Defence (by Defence staff, contractors, professional service providers or other Registered Training Organisations) remains the responsibility of Defence. Notwithstanding this, external agencies may be used to undertake work in the Develop phase. Where this is being considered, synergies might be achieved by packaging design and development work in a single contract. CHECK EXISTING TRAINING MATERIALS AND EQUIPMENT 4.14 Introduction The Develop phase of the DTM is when the materials and equipment needed to conduct, assess and evaluate training are produced and/or prepared. The first step in the Develop phase is to check whether suitable materials and equipment already exist. If they do, and may be acquired, then a decision is made as to whether they should be used, or whether new resources should be developed. 4 3

206 ADFP Chapter The work of checking existing training materials and equipment is a continuation of: a. the feasibility analysis (when existing training was first investigated); b. the analysis of learning and assessment environments (in which available resources were listed); and c. the design of strategies (which used knowledge of and/or assumptions about resources) Purpose. The purpose of checking the existence of training and assessment materials and equipment is to minimise development costs and to standardise items used, where appropriate Triggers. The start of the Develop phase triggers the checking of existing training materials and equipment. This follows on from acceptance of the draft TMP at the end of the Design phase. Inputs, process and products The inputs, process and products of checking existing training materials and equipment are summarised in table 4 1. Inputs Process Products Draft TMP, Feasibility Report, environments profile, training references. Step 1: Identify requirements. Step 2: Research existing Defence materials and equipment. Step 3: Research external sources. Step 4: Decide whether to acquire existing and/or develop new materials and equipment. Decision to either acquire existing and/or develop new training materials and equipment. Table 4 1: Inputs, process and products for checking existing training materials and equipment Step 1: Identify requirements Before determining the suitability of existing materials and equipment, the requirement must be understood. This understanding is developed in the Analyse and Design phases and presented in section 4 (course resource requirements) of a TMP. It is refined in the Develop phase. 4 4

207 ADFP Chapter The requirement should be considered from several perspectives. At a macro level it consists of a whole course, one or more modules, and events and activities. At a micro level it consists of each item of material and equipment used during a program/course, together with the guidance that links them in the learning process. The requirement consists of one or more lists of the materials and equipment needed to conduct assessments, implement the training, support learning, review the training and conduct an evaluation. For each listed item, required specifications should be stated. Once the contents of the list have been drafted, the questions of how much of each item is needed, and when, should also be considered The primary references for the requirement are strategies described in a curriculum (section 2 of a TMP) and for TR&E (section 3 of the TMP). These strategies might make direct reference to the materials and equipment required. Alternatively, judgments will need to be made on the requirement. Such judgments need to consider the: a. required learning outcome; b. characteristics of the learner and learning environments; c. contribution/value to learning required; d. cost of resource through-life (acquisition and maintenance); e. ease of update; f. time to acquire; and g. alternative uses. The step is complete when a detailed list of all the materials and equipment required to implement the program/course in accordance with the draft TMP has been produced. Step 2: Research existing Defence materials and equipment The search for materials and equipment begins with the interrogation of Defence resource databases. Single-Service and Group policies and procedures will provide the detailed procedure for this step. Step 3: Research external sources Where resources are unable to be sourced from within Defence, support from external agencies may be considered. Single-Service and Group policies and procedures will provide the detailed procedure for this step. 4 5

208 ADFP Chapter 4 Step 4: Decide and record whether to acquire existing or develop new materials and equipment When existing training materials and equipment are found, their suitability and likely effectiveness are weighed against their affordability, reliability, availability, maintainability and supportability and need for and ease of customisation. This judgment is then compared with the options of developing a new resource, or progressing without the resource and a decision to pursue one of the three courses is made. OBTAIN MATERIALS AND EQUIPMENT FOR ASSESSMENTS AND TRAINING 4.25 Introduction The process of obtaining materials and equipment for making assessments, conducting training and TR&E, covers a wide range of activity and resource expenditure. Materials and equipment may either be purchased directly or produced. In either situation, processes may be complex or relatively simple dependent on the circumstances of the acquisition Purpose. The purpose of obtaining training materials and equipment is to gain cost-effective resources for implementing a program or course Trigger. The work of obtaining resources to implement training follows on from the decisions about the requirement for and means of, gaining the required materials and equipment. Inputs, process and products The inputs, process and products for obtaining training materials and equipment are summarised in table 4 2. Inputs Process Products Draft TMP, facilities, raw materials and equipment, finances, internal and external staff, training facilities and infrastructure. Step 1: Acquire training materials and equipment for assessing and implementing training, and for TR&E. Step 1.1: Obtain existing materials and equipment. Step 1.2: Customise materials and equipment. Step 1.3: Produce new materials and equipment. Step 1.4: Formatively evaluate materials and equipment. Step 2: Add materials to draft TMP. Draft TMP completed by addition of materials and equipment for assessing and conducting training, TR&E. Table 4 2: Inputs, process and products for obtaining training materials and equipment 4 6

209 ADFP Chapter 4 Step 1: Obtain training materials and equipment The materials and equipment used can be obtained by either acquiring existing resources in which case they may require customisation to meet Defence needs or producing new resources. Whichever method is used, formative evaluation should be used to ensure the resources function as required. In most situations the number of items needing to be obtained is such that all four of the processes comprising the step will be managed concurrently There are four processes involved in this step. They are: a. obtain existing materials and equipment, b. customise materials and equipment, c. produce new materials and equipment, and d. formatively evaluate materials and equipment. Step 1.1: Obtain existing materials and equipment Existing materials and equipment can be obtained in three basic ways, either by an organisation alone or in partnership with another. They can be: a. borrowed or reproduced at no cost from internal or friendly sources (eg other Defence organisations, Government departments, or military services overseas); b. purchased, either outright, or under a license to use, from commercial or other sources; and c. leased/rented/hired from commercial or other sources. Step 1.2: Customise materials and equipment Once acquired, materials and equipment are likely to need customisation to meet the requirements and standards of the organisation. Customisation can involve additions, deletions, updates and corrections; as well as amending the design and presentation of the resource to achieve the desired branding The work of customising existing resources to meet the requirement is little different to the work of producing new resources described in Process 3. As it progresses, care needs be taken when changing anything that might affect the external accreditation of the course. For this reason, customisation is often best carried out by the organisation that initially developed the materials and/or equipment. If these are being procured, customisation should be addressed in any contractual agreement. 4 7

210 ADFP Chapter Copyright and patent. Copyright and patent laws provide protection to the originators of materials and equipment from unauthorised use. If training resources that are subject to those laws are to be customised then written permission must be obtained from the owner, and a legal copy of the documentation should be secured. Similarly, if Defence is involved in the production of materials/equipment over which it wishes to exert copyright or patent, such action may be pursued. These actions will involve time and money, and require expertise. Guidance on copyright and patents is contained in the Defence Procurement Policy Manual and can be accessed from the Defence Legal Office. Step 1.3: Produce materials and equipment The costs of producing new equipment for training are such that it is normally purchased commercially rather than being developed or manufactured by Defence. This is not the case for materials, which are more easily and efficiently produced specifically for a known requirement The challenge in producing materials for use by learners, trainers, assessors, training administration staff and quality control staff is to make them: a. achieve what is intended so they should meet 100 per cent of the requirement; b. understandable so they should be presented clearly, at the appropriate level; and c. attractive so their use is encouraged. These qualities are the products of effective planning that applies the resources available to the subject content, its structure and detailed learning strategies, its presentation, and related technical issues Subject content. In the iterative process of drafting and checking learning materials, attention must be paid to ensuring that the content matches the requirement specified in the curriculum, and that it is accurate and current. This could be the first time that the detailed contents, or the references where detailed content can be found, are committed to a program or course (the curriculum describes content rather than states it). Having subject matter expertise to provide or check content is therefore important. 4 8

211 ADFP Chapter 4 Step 1.4: Formatively evaluate materials and equipment Formative evaluation should be applied to materials and equipment as they are developed. This helps avoid unplanned delays and costs at the end of the process. Checks and tests of the materials and equipment during their development should evaluate their: a. validity the extent to which they meet the specified requirement; b. suitability how well they can be used by learners and training staff in the environments and circumstances they are designed for; c. reliability how consistently do they achieve their outputs; and d. cost-effectiveness with a view towards improvement A mix of stakeholders should contribute to this continuous improvement work. Subject matter experts representing the job/business Process Owner (BPO) should check the accuracy and validity of what is being trained/learned. The training staff who will implement the training in the Conduct phase should be used to check the suitability of the materials and equipment. Learning/training technologists with expertise in the processes of the DTM can make judgments on the reliability of resources, and all these stakeholders can contribute to judgments on the cost-effectiveness of the materials and equipment No matter how much work is invested in acquiring, customising, and producing the materials and equipment required to implement training, there is the potential to do more and better. Continuous improvement is highly applicable to this area of the DTM, but the resources used should be limited to the potential benefit to be gained The procedures and policies for the execution of these four processes are managed in accordance with single-service and Groups direction. Step 2: Add materials to draft Training Management Package By adding the materials to the draft TMP produced during the Design phase, the draft TMP is completed. 4 9

212 ADFP Chapter 4 PREPARE MAJOR TRAINING RESOURCES Preparing major resources to be used for a training program or course is either undertaken before the introduction of new training or when major changes to existing training are made. It is distinct from the more focused work of preparing to conduct a single training program or course. The process describes the work of ensuring the availability of, and making ready the major resources required to implement the program/course. Such resources normally include: a. facilities and infrastructure including schools, training rooms, workshops, accommodation and related buildings for traditional training; and electronic networks and software that enable e-learning; b. people including staff for training management, administration, and implementation (assessors, instructors, mentors, workplace supervisors); and c. major Defence units and platforms such as ships, aircraft, major simulators and units The major resources required for the training should have first been identified during the feasibility analysis process of the Analyse phase and recorded in the Training Requirement Specification. As soon as the need for major training resources is known, their acquisition and preparation can be planned and implemented. This work can involve Defence organisations other than training, including: materiel, estate, personnel (including work force planning, and recruitment), budget and finance, and infrastructure. As such, it may be outside the direct scope of the DTM and the responsibilities of people working within it. If this is the case, the training staff involved with the DTM should retain an active involvement in that work, helping to ensure that the resources are prepared and ready when needed Even though not responsible for providing the major resources, the Development Authority will have responsibility for coordinating action that ensures they are ready: firstly for piloting the program/course and, subsequently, for conducting the training. This work might involve contributing to: a. securing the facilities and infrastructure; b. acquiring and fitting furnishings and equipment; c. conducting acceptance testing of major simulators and equipment; 4 10

213 ADFP Chapter 4 d. ensuring the long-term availability of any required domestic and social accommodation; e. advertising for and recruiting, and/or assigning staff for training, administration, training conduct and evaluation; f. coordinating the ongoing provision of resource inputs from other organisations (such as the Services); and g. budgeting and allocating the funds required for the training When the major training resources have been prepared, the facilities, infrastructure, people, equipment and other significant management inputs are ready for the training to be piloted and implemented. PILOT ASSESSMENTS AND TRAINING 4.47 Introduction The purpose of the pilot is to identify any factors that prevent or restrict the implementation of the resources and materials that have been designed and developed. If the process identifies changes to the content, strategies, methods, media or conduct of either the training or assessment developed, they are reported for corrective action A report of the pilot process indicates whether or not the materials developed are ready for implementation. If the Development Authority is satisfied that it is, the TMP is approved. If not, then the necessary changes are made and, possibly, piloted again While management of the pilot process is the responsibility of the Development Authority, its success requires the coordinated efforts of staff from the Conduct, Design and Analyse Authorities Purpose. The purpose of piloting developed training and assessments materials is to evaluate their readiness for implementation Trigger. Training and assessment materials are piloted when a draft TMP, or a component of one, has been completed and all major resources have been prepared. 4 11

214 ADFP Chapter 4 Inputs, process and products The inputs, process and products for pilot assessments and training are summarised in table 4 3. Inputs Process Products Draft TMP; training administration, instruction and assessment staff; trainees; learning and assessment facilities, equipment and prepared materials; and training review materials. Step 1: Plan the pilot assessment and/or training. Step 2: Plan and prepare the evaluation of the assessment and/or training. Step 3: Implement the assessment and/or training. Step 4: Review, report and follow-up the assessment and/or training. Step 5: Approval of TMP. Step 6: Set up the course(s). A report on the pilot process recommending revisions, amendments to the TMP and, finally, an approved TMP. Table 4 3: Inputs, process and products for pilot assessments and training Step 1: Plan the pilot assessment and/or training Planning the conduct of a pilot assessment and/or training (whether a program, course, module or lesser unit) is a responsibility of the Development Authority, but the work is undertaken in partnership with staff responsible for assessing competencies, and conducting training and training reviews. The work involves considering, answering and then responding appropriately to the following questions: a. When and where will the assessment/training pilot be conducted? b. Who will administer and conduct the activity? c. Who will evaluate the activity? d. What preparation of staff and other resources is required? (It may be necessary to provide training for the assessment and/or instructional staff. The staff may need instructional and/or assessment competencies in addition to the technical competencies relating to their occupation). 4 12

215 ADFP Chapter 4 e. How much time should be allocated to conduct the activity? (A contingency must be allowed for faulty estimates in the time required, to support learners whose learning suffers through no personal fault, and to conduct a training review that will require more time than for subsequent courses). f. How much revision of the resources can be achieved during the piloting process? g. How long after the pilot activity will be required to complete this revision? h. What special resources will be required to conduct the pilot process? 4.54 The timing of the pilot activity is determined partly by the schedule to complete development of the training and partly by the organisation s need for qualified people. If the assessment/training is be piloted shortly after the required resources are obtained and prepared, preparatory planning should begin early in the Develop phase. Another consideration is the allowance of sufficient time between the pilot process and the use of the resources to enable necessary revisions to be made. Step 2: Plan and prepare the evaluation of the pilot assessment and/or training While development staff ensure it occurs, the work of evaluating a pilot assessment or training should be undertaken by experienced training review staff that are independent of the inputs involved The breadth and depth of the evaluation should match the significance of the resources being piloted. In most instances training review staff should develop a plan containing an aim, objectives and the means of achieving those objectives. Gaining the Development Authority s approval of the plan is recommended, as it confirms the evaluation addresses agreed requirements and helps authorise its implementation Reviews of assessment and training processes usually incorporate, or draw from, a common set of objectives: a. to evaluate course documentation for: (1) information accuracy; (2) consistency and contiguity; (3) conformance with set formats; and (4) usefulness to learners, trainers, assessors and administrators; 4 13

216 ADFP Chapter 4 b. to evaluate the effectiveness and efficiency of teaching strategies; c. to evaluate the conduct of training and other programmed activities for consistency between intended and actual training events (eg time, sequence, content, facilities and equipment, administration); d. to evaluate all assessments for validity, suitability and reliability; e. to evaluate the suitability of the facilities and infrastructure used; f. to evaluate the utility of materials and equipment used; g. to establish the time required for training by timing each component; h. to evaluate the efficiency of course administration; and i. to evaluate assessment processes and results in relation to learning objectives In determining what data is needed to inform the evaluation work, consideration needs be given to how data will be collected, analysed and presented. This level of detail should be documented and agreed by the major stakeholders A wide range of methods and sources may be used to collect data and information for the evaluation. They include: a. checks of the TMP, facilities and infrastructure; b. pre-course assessment of learners; c. observations, evidence from and results of assessment and training activities; d. computer software and training devices that collect and record performance results; e. learners written formative critiques and/or diaries; f. end-of-course critique and review; and g. interviews/discussions with learners, instructors, and other training staff (types in accordance with purpose). 4 14

217 ADFP Chapter Most of the methods listed use materials that need design and preparation. Some materials will be those previously designed and developed for ongoing training review (in these instances their design should also be a subject to evaluation). Other materials will need preparing specifically for use in the pilot process. Step 3: Implement the assessment and/or training Once planning is complete, the assessment and/or training being piloted and the collection of evaluation data can begin. To gain maximum benefit from the pilot process, it is important that everyone involved cooperates towards achieving both the desired outcomes of the activity and the objectives of its evaluation. Learners and individuals being assessed should both fulfil the requirements of the assessment/training and contribute to the evaluation process. Staff responsible should implement the assessment/training in accordance with the draft TMP, but also be mindful of achieving the required outcomes and of contributing to the evaluation. Training review staff should follow the plan for the review, aiming not to affect the activity. Meanwhile, the Development Authority coordinates the conduct and evaluation processes and, as required, provides guidance on the TMP The pilot assessment/training should be administered and conducted in accordance with both the draft TMP and the normal management procedures of the organisations responsible for them. Step 4: Review, report and follow-up the pilot assessment and/or training To review the pilot assessment/training: a. data and information collected in accordance with the evaluation plan is collated and considered in relation to the plan s objectives; b. conclusions are drawn; c. options for overcoming any identified problems are evaluated; and d. preferred options are presented as recommended courses of action. This process and its products are formally documented and reported so that the satisfactory elements of the TMP are confirmed, corrective action can be taken where needed and a record of the whole process is maintained. 4 15

218 ADFP Chapter The report of the assessment and/or training piloted should be addressed to the Development Authority, with other stakeholders being informed. The Development Authority should then request appropriate authorities to remedy any deficiencies described in the report, setting any applicable deadlines Although the review, report and follow-up step is shown sequentially after implement, the two work processes are likely to be undertaken in parallel. This means that following completion of an assessment or training activity, evaluation and revision of it can occur whilst subsequent activities are being conducted. Adopting this approach means the period between completing the pilot process and finally approving the developed training resources is minimised. Step 5: Approval of Training Management Package When following revision and additional checks of any revised assessment and/or training resources, the Development Authority is satisfied with the TMP, the package of documents/materials is ready for approval. As the customer of the training, the approval of the policy owner or job/bpo is formally sought. Gaining such approval signifies that training is ready for implementation in the Conduct phase. Step 6: Set up the course A final administration task of preparing new Defence training may involve the set up of the new course(s) in the Defence human resource database, Personnel Management Key Solutions (PMKeyS). By entering the course(s) into PMKeyS the training is made visible to Defence personnel The setup process for new training requires course information to be entered on PMKeyS. The required information should all be recorded in the training specification section of a TMP. Annexes: A. Flow chart showing the Develop phase of the Defence training model B. Defence training model purpose, inputs, process and products of the Develop phase 4 16

219 Annex A to ADFP Chapter 4 FLOW CHART SHOWING THE DEVELOP PHASE OF THE DEFENCE TRAINING MODEL A 4A 1

220

221 Annex B to ADFP Chapter 4 DEFENCE TRAINING MODEL PURPOSE, INPUTS, PROCESS AND PRODUCTS OF THE DEVELOP PHASE B Develop PHASE PURPOSE INPUTS PROCESS PRODUCTS To minimise development costs and standardise resources used. To gain cost-effective materials and equipment for implementing a training program or course. Draft Training Management Package (TMP), training references. TMP, existing training resources, financial and other resources including human expertise, Defence instructions governing acquisition policies and processes. Check availability of existing materials and equipment for: assessment, training and learning, Training Review and Evaluation (TR&E). Obtain existing; and/or produce new training materials and equipment. Decision to either acquire existing and/or develop new training resources. Draft TMP complete with materials, and equipment for making assessments and conducting training, and TR&E. 4B 1

222 ADFP PURPOSE INPUTS PROCESS PRODUCTS To enable the implementation of a training program or course. To prove the efficiency and effectiveness of the program/course. Designated administration, assessment and instructional staff; learning and assessment strategies; curriculum; training infrastructure; equipment and TR&E resources; administration; system; finance. TMP, training and assessment staff, target learner population, learning and assessment facilities, equipment and prepared materiel, TR&E materials. Prepare major training resources. Facilities, infrastructure, people, and other major resources are ready to implement training. Pilot the training. Report on the outcomes of the pilot process leading to an approved TMP and training procedures. 4B 2

223 ADFP Chapter 5 CHAPTER 5 THE CONDUCT PHASE 5 Introduction Following their design and development, training and assessment are implemented in the Conduct phase of the Defence training model (DTM) as shown in figure 5 1. Each of the processes comprising the Conduct phase will have been piloted and proven during the Develop phase of the DTM. In the Conduct phase they become normal workplace practice. Successful completion of the phase signifies that the needs specified in the Training Requirement Specification have been met. Figure 5 1: Conduct phase of the Defence training model 5.2 The Conduct phase involves preparing for training and/or assessment, the conduct of those activities and, on completion, the conduct of a training review aimed at improving subsequent iterations. 5.3 As with all phases of the DTM, the extent to which the processes and steps of the Conduct phase are applied should be appropriate to the circumstances. In situations of high significance (eg training and assessment closely associated with combat capability) the processes and steps should be applied rigorously. Abbreviated processes may be appropriate in lesser circumstances. 5 1

224 ADFP Chapter 5 Key product and responsibilities The key products of the Conduct phase are trained people who, through assessment, are recognised as having the proficiencies required by the organisation. These are people who can be employed to achieve the required workplace performances identified in the Analyse phase of the DTM. Not all training immediately leads to competence. Additionally, on-the-job training (OJT) and/or experience may be required before competency assessment is undertaken. 5.5 Responsibility for implementing training and assessment rests with the Conduct Authority. The Conduct Authority oversees off-the-job and OJT, training conducted internally and by external contractors. In the Australian Defence Force such authority is generally vested in, and delegated by the Training Command (for Army and Air Force) or Training Authority (for Navy), with responsibility for OJT and assessment being shared with the job owner. Joint training is the responsibility of a nominated manager of joint training. In the Department, responsibility for implementing training generally lies with the related policy, business process or job owner. Starting point A training course/program or an assessment activity is initially implemented following its design, development, and formal acceptance in a Training Management Package (TMP). Following this start, programs/courses and/or activities are undertaken as necessary and may be conducted concurrently, consecutively or intermittently. Processes of the Conduct phase The Conduct phase has five major processes which are illustrated in the flow diagram in annex A: a. administer training preparation; b. implement pre-training assessments/tests; c. conduct training; d. conduct assessment; and e. conduct post-course administration and training review (internal evaluation). 5.8 The flow chart in annex A illustrates these processes, together with their products. Annex B presents the purpose, inputs and products of the processes in table form. The following paragraphs introduce each of the processes, and provide guidance on the steps for undertaking them. 5 2

225 ADFP Chapter Administer training preparation. Administration involves the coordination, provision and readiness of the required inputs, the trainees/assessees, staff, major resources, materials and equipment. This work must be started at an appropriate time before the activity is scheduled Implement pre-training assessments/tests. Not all Defence training programs and courses have pre-course tests and assessments. Where used, however, they can be used to: a. determine recognition of prior learning (RPL); b. check the prerequisite requirements of enrolled trainees; and/or c. customise training to meet trainees particular needs The requirements for pre-course tests/assessments are contained in TMPs. The steps involved in implementing them are common to any assessment and can be summarised as preparation, conduct, evaluation of results and instigation of action appropriate to their original purpose Conduct training. The conduct of training is the process of implementing designed and developed activities for achieving specified learning outcomes that develop individuals competencies and related skills, knowledge and attitudes. The range of training activities is great, including instruction, distance learning, e-learning, coaching, individual and team/group work, and formative assessment. The activities may take place OJT, or off-the-job in training centres or other locations. The training and assessment activities to be used for programs/courses are specified in TMPs, and should be suitably prepared and implemented by qualified trainers/facilitators. To achieve quality training these staff members must apply appropriate learning principles and theories. The product of such work is people who have achieved the outcomes of the training. In due course, they can have their new competencies and/or attributes assessed and recognised Conduct assessment. Summative assessment, usually conducted on completion of training, determines the extent to which desired learning outcomes have been achieved 1. Summative assessment may also be conducted in the workplace, sometime after completion of OJT or off-the-job training. 1 A form of summative assessment, RPL, may be conducted without reference to training, or before a training program/course, to avoid unnecessary training. 5 3

226 ADFP Chapter Summative evaluation is used with most Defence training to confirm that trainees have achieved required competencies, proficiencies or attributes. Summative assessments are conducted using steps common to all other assessments: preparation, implementation, the evaluation of evidence and administration of results Post-course administration and training review (internal evaluation). The two linked processes of post-course administration and the conduct of a training review are undertaken on completion of a program/course and the assessment of trainees. Post-course administration commonly involves the issue of qualifications (or, if required, the instigation of remedial measures); the recording of course outcomes; the housekeeping tasks of restoring the training resources; and contributing to training reviews. It closes one course and enables the conduct of subsequent courses. Training review is a continuous improvement process aimed at making those subsequent courses as efficient and effective as possible. In the context of the DTM the term training review is also referred to as internal evaluation. It involves the collection, collation and analysis of data and information about a training program/course in relation to its purpose and conduct. Feedback from stakeholders, including trainees, trainers/facilitators and training administrators is used. A training review confirms the inputs and processes of training and assessment that are satisfactory, and/or makes recommendations for improving those that require development. Continuous improvement The concept of continuous improvement should pervade the Conduct phase of the DTM. Trainers/facilitators, other training staff and trainees involved in the Conduct phase should evaluate their involvement with a view to improving training delivery. Improvements are possible in course design, the administration of training and assessment; the resources, materials and equipment used; and the application of competencies by trainees, training and administration staff. The formality of staffing the evaluation activity can vary. At a basic level the self-improvement of individuals involves little or no staffing. The conduct of training reviews on completion of courses is a more formal level, while one-off, targeted training reviews are a highly structured activity. To keep track of the developments in training and assessment, and to enable reference to measures of performance, accurate and comprehensive training, and training review records must be maintained. Use of an external agency In accordance with national and Defence guidelines, the quality assurance of all training by, or on behalf of Defence (by Defence staff, contractors, professional service providers, or other Registered Training Organisations (RTO)) remains a Defence responsibility. External agencies 5 4

227 ADFP Chapter 5 may be contracted to provide training and assessment services, but where they are, the designated Conduct Authority retains responsibility for the products of the services provided. ADMINISTER TRAINING PREPARATION 5.18 Introduction The first process of conducting training and assessment is to coordinate the provision and readiness of all the required inputs; the trainees/assessees, staff, major resources, materials and equipment. This is primarily an administration function Purpose. The purpose of administering training preparation is to ensure that the implementation of training and assessment is achieved effectively and efficiently to meet the planned requirement Trigger. The administration of training preparations is triggered by a combination of the need for trained individuals and the readiness of the TMP. This does not mean that the process and phase begin on completion of the TMP. In many situations the preparations for the conduct of training must begin during the Develop phase of the DTM. Occasionally they might be initiated a period after completion of the Develop phase. Inputs, process and products The circumstances of each training situation vary. Consequently the inputs, process and products of administering training preparation summarised in table 5 1 should be considered as indicative. 5 5

228 ADFP Chapter 5 Inputs Process Products Policies and instructions, course venue, curriculum, learning and assessment resources, required materiel and financial resources, training staff and nominated trainees. Step 1: Promote the training. Step 2: Training nominations and panelling. Step 3: Prepare major resources. Step 4: Prepare materials and equipment. Step 5: Manage course sessions. Prepared inputs to training, including but not limited to training schedule, training staff, designated trainees, facilities, materials and consumable resources. Table 5 1: Inputs, process and products of administering training preparation Step 1: Promote the training Training, particularly new training involves change, which must be promoted and managed. In promoting the positive aspects of training, the need for, and the benefits of the required competencies to both the organisation and the individuals should be explained, together with the implications of the developments (new work, employment opportunities, promotion, commitment of time resources etc) The promotion of programs and courses should feature any opportunities for RPL incorporated. In situations where training is voluntary, their existence will encourage participation. It should also highlight any flexibility in the delivery options, for the same reason The promotion of training programs/courses should be started sufficiently in advance of the first programmed activity to ensure that stakeholders can plan their involvement and are motivated towards its success. Step 2: Training nominations and panelling Defence exercises close control of the career development and training of members, aiming for just-in-time training for specific job or career requirements. Posting action managed by career managers is used to achieve attendance at long courses, and direction by supervisors for attendance at shorter courses. The requirement for trained people is known, and the supply of individuals to courses is managed accordingly. The enrolment of trainees to course sessions is managed by the panelling authority in consultation with schools/units conducting the training in consultation with posting staff or supervisors, and the individuals affected. 5 6

229 ADFP Chapter In accordance with their employment conditions (eg plan on a page); the system for Defence civilian personnel is more flexible. Courses are promulgated and both individuals and their supervisors can arrange nominations The Defence personnel management system, Personnel Management Key Solutions (PMKeyS), has a course nomination/enrolment and panelling function that links with the course and course session set up data. The person operating the system must have successfully completed training in the Professional Development and Training (PD&T) functionality of PMKeyS. Joining instructions are sent to the trainees involved by the conducting authority. Step 3: Prepare major resources Preparing the major resources to conduct a course ensures they are available and ready. Resources that typically need preparation include: a. facilities and infrastructure such as schools, training rooms, workshops, accommodation and related buildings for traditional training; electronic networks and software that enables e-learning; b. people including staff for training management, administration and implementation (assessors, instructors, guest speakers/subject matter experts, mentors, workplace supervisors); and c. major Defence assets such as ships, aircraft and army units If the conduct of training follows on closely from a pilot course for which resources were prepared, preparatory work may be minimal. It can, however, involve: a. securing the facilities and infrastructure; b. acquiring and fitting furnishings and equipment; c. ensuring the long-term availability of any required domestic and social accommodation; d. advertising for and recruiting, and/or assigning staff for training, administration, the conduct of training, assessments and evaluation; e. ensuring staff have prerequisite qualifications for their work (including the registration of RTO assessors); 5 7

230 ADFP Chapter 5 f. coordinating the ongoing provision of resource inputs from other organisations (such as Services); and g. budgeting and allocating the funds required for the training. Step 4: Prepare materials and equipment The materials and equipment used for the conduct of training and assessments and for implementing Training Review and Evaluation (TR&E) are specified in, and constitute part of the TMP Before training begins, the specified resources must be obtained in sufficient numbers, pre-positioned, have their readiness assured and, possibly, tested. This work should be completed in sufficient time to enable the uninterrupted conduct of training. Step 5: Manage course sessions Training cannot be efficient unless stakeholders know when it is taking place. Off-the-job training programs are normally planned annually, although such plans may be updated during their life span. Programs give the start and finish dates of scheduled courses, together with information about the number of trainees. They are used within schools and training units to develop detailed timetables which include, on a session-by-session basis, what is to be covered, the location for instruction and the trainer who will lead the activity Course programs are centrally managed in Defence using the PMKeyS database. The software facilitates the programming of training courses, and the nomination and confirmation of trainees for them. It can also be used to provide feedback on the availability of course places, and the demand for them. The set up of course sessions in PMKeyS involves entering information about the times, duration, numbers and panelling of trainees, locations and resource requirements. This work requires the person entering the data to have successfully completed training in the PD&T functionality of PMKeyS The management of course sessions should be undertaken sufficiently in advance of implementation to allow adequate preparation time and to identify any planning problems In many situations it is appropriate to program not just the implementation of training and assessment, but also the TR&E processes and any post-course administration to be applied. 5 8

231 ADFP Chapter 5 IMPLEMENT PRE TRAINING ASSESSMENTS/TESTS 5.36 Introduction The associated TMP specifies whether pre-course testing and/or assessment are necessary or not Trigger. Pre-testing is triggered by a session being scheduled. Inputs, process and products The inputs, process and products of each of the three processes, central to the conduct of training, are summarised in table 5 2. Inputs Process Products Trainees, assessment and training staff, venue, TMP, specified major resources, equipment, materials and finances. Step 1: Assess/test trainees. Step 2: Evaluate the evidence. Step 3: Instigate appropriate action. Qualified people, those who are ready to undertake training or unsuccessful applicants. Table 5 2: Inputs, process and products for implementing pre-testing/assessments 5.39 As is true throughout the DTM, although the steps of conducting training are, to a large extent, sequential, the associated work normally overlaps. The work is managed accordingly Most courses do not require the assessment of individuals before training processes commence. Those that do may do so: a. for RPL purposes so that unnecessary training doesn t take place; b. to check that individuals enrolled on the course meet prerequisite requirements; or c. to enable training to be customised to learners needs. Their application can contribute to the avoidance of inefficiencies in the conduct of training. 5 9

232 ADFP Chapter 5 Step 1: Assess/test trainees Defence conducts a wide range of assessment and testing activities, ranging between the resource intensive selection of trainees for flying aircraft and the self-administering of multiple choice questions at the start of a short computer-based training program. Guidance on what pre-course assessment should be applied, when and how; together with the tools to be used, is contained in TMPs Arrangements for the conduct of pre-course assessment are made in preparatory administration. Further preparation at the level of the individual managing the activity may also be necessary (by an appointed assessor in most cases, but by the trainee for self-administered assessment). This can include detailed scheduling, liaison between people involved, the preparation of assessment tools and ensuring the availability of required resources The conduct of any assessment/test should follow the guidance contained in the TMP. This guidance normally requires an assessment to have: a. an introduction explaining: (1) the purpose of the activity, (2) the requirements of people involved, (3) the time allowed, (4) how performance will be assessed, and (5) what feedback will be provided; b. the activity itself, during which: (1) the set procedures are followed; (2) guidance on those procedures is available; (3) as required, the performance of the trainee is monitored; and (4) evidence of that performance is collected; c. a debrief, during which: (1) appropriate feedback is given, and (2) the trainee is invited to comment on the process. 5 10

233 ADFP Chapter 5 Step 2: Evaluate the evidence In most circumstances the evidence arising from an assessment/test should be evaluated shortly after collection, so that feedback can be provided and appropriate action instigated while the outcomes are most valid. The evaluation is based on criteria provided in the TMP. It results in three broad categories of outcome: a. trainees progress to course training (CT) (which may be tailored to meet individual needs); b. trainees have prior learning recognised (and their training program is individualised accordingly); or c. unsuccessful applicants. Step 3: Instigate appropriate action Appropriate administrative action is taken when the evidence has been evaluated. The action taken should: a. be appropriate to the purpose of the assessment/test; b. involve feedback to the individual involved; and c. involve the recording of the evidence, its evaluation and the outcome. CONDUCT TRAINING 5.46 Introduction The conduct of training describes trainers interactions with trainees for the purpose of facilitating learning. Within the DTM it describes the implementation of the activities designed and developed in previous phases of the DTM The conduct of training can have many forms. It includes classroom instruction, practical exercises, technology-based training, individual and group project work, discussions, seminars, lectures, tutorials, simulations, fieldwork and much more. It may take place in dedicated training environments, in workplaces or elsewhere Purpose. The purpose of conducting training is to develop the required skills, knowledge and attitudes in people to undertake their roles to meet the Defence mission Trigger. Training is triggered by a session being scheduled and the completion of any pre-training assessments or tests. 5 11

234 ADFP Chapter 5 Inputs, process and products The inputs, process and products of conducting training are summarised in table 5 3. Inputs Process Products Venue; TMP; required facilities, training staff, trainees, materiel and financial resources and the results of pre-training assessments or tests. Step 1: Prepare training activity. Step 2: Deliver training activity in accordance with the plan. Step 3: Monitor trainees progress and adjust training as required. Trained people. Table 5 3: Inputs, process and products of conducting training 5.51 Defence uses most types of training activity, including instruction (onand off-the-job), distance learning, e-learning, formative assessment, coaching, practical work, demonstration and group/team work. This publication neither lists them all, nor explains the details of how they are implemented. Guidance on what methods should be applied in a program/course, when, and how, together with the necessary equipment and material, is contained in the TMP. Training personnel using a TMP should be appropriately trained and qualified The following steps provide broad guidance applicable to most training/learning situations. Step 1: Prepare training activity While arrangements for the implementation of training are administered suitably before a program/course begins, more detailed preparation by the trainer(s)/facilitator(s) is necessary before each instructional activity. This preparation may include detailed scheduling, liaison between personnel involved, the preparation of training/lesson plans, ensuring the availability and readiness of the venue and other resources, and rehearsing activities. Contingency planning should also be considered, so that problems such as equipment failure, non-appearance of trainers, program changes, and any need for remedial training are not unduly disruptive In most instructional situations structure is provided through a planned sequence of activity based on appropriate learning theories. The broad structure and sequence of programs/courses is provided in TMPs. At this stage of the Conduct phase, the structuring of particular training activities such as lessons, lectures and presentations is undertaken, with the results recorded in lesson plans or lecture/presentation notes. 5 12

235 ADFP Chapter 5 Step 2: Deliver training activity in accordance with the plan The conduct of training describes the task of implementing what has been designed, developed and prepared in accordance with the TMP and the trainer s detailed plans. Plans guide efficient and effective training and learning. The human factor of training and learning (the ability to adapt to differing characteristics, personalities, learning styles and needs) involves skills and knowledge in: a. communicating, b. group facilitation, c. questioning, d. giving and receiving feedback, and e. managing relationships Whatever the method and medium of training and learning employed, successful outcome-orientated training and learning requires trainers to: a. address learner needs and expectations; b. possess the skills to implement the method/mode of delivery; c. apply principles of adult learning; d. take advantage of learning conditions; e. use exercises so that learners can apply what they learn; f. anticipate and help overcome problems with transferring learning to the job; g. motivate learners; h. plan, conduct and review assessments; i. build and sustain interest; j. stimulate many senses to aid retention; k. repeat key points to aid retention; l. provide for as much learner participation as possible; and m. explain why the information (or experience) is or will be useful to learners. 5 13

236 ADFP Chapter 5 Step 3: Monitor trainees progress and adjust training as required Good instruction/training incorporates the monitoring of trainee progress and adjusting the training to match identified needs. This type of formative assessment should be undertaken continuously by a trainer through informal questioning and observation. It is also conducted more formally, using progress tests and assessments specified in the TMP. In both circumstances progress tests/assessments are useful not only for measuring what has been learned but also to provide feedback. They also reinforce the learning process and provide information that can be used for continuous improvement. CONDUCT ASSESSMENT 5.58 Introduction Assessment in Defence is the process of collecting evidence and making judgments about trainees. Further, assessment is an integral part of training and focuses on the total proficiency of the person in the workplace. Assessment may take place in a training establishment or it may occur in the person s workplace Assessment can be used before instruction commences in order to determine the attributes of trainees on entry to training. Pre-testing can be used to modify entry lessons and to implement recognition processes. Assessment can be used as an ongoing process to provide feedback to trainees and trainers on the effectiveness of the training and the achievement of learning outcomes/competencies Purpose. Within the DTM, the major purpose of assessment is to determine job competency and/or achievement of learning outcomes. Other purposes of assessment may include: a. formative assessment during training, b. summative assessment of performance at the completion of training, c. statutory certification requirements, d. diagnostic assessment to determine training needs, e. personal feedback on performance, f. skills audit, and g. identification of suitability to perform particular tasks or roles for promotion. 5 14

237 ADFP Chapter Trigger. Triggers are completion of some training or an application for recognition. Inputs, process and products The inputs, process and products of conducting assessment are summarised table 5 4. Inputs Process Products Trainees, assessors, assessment guidelines and other assessment materials from the TMP, recognition evidence, venues and resources. Step 1: Prepare summative assessments. Step 2: Conduct the assessment in accordance with the guidelines. Step 3: Evaluate conduct of assessment in accordance with evaluation plan/strategy. Individuals recognised as job competent, course qualified or neither. Table 5 4: Inputs, process and products for conducting assessment Step 1: Prepare summative assessments Arrangements for the conduct of summative assessments should accord with the guidance provided in the TMP and any provisions required by the organisation (school, unit, RTO 2 etc). Most situations are likely to involve: a. scheduling the assessment; b. informing the individuals involved of arrangements and what will be expected of them; c. ensuring the availability and readiness of major resources, equipment, materials and assessment tools, and personnel; d. preparing/reviewing assessment tools to be applied; e. identifying and addressing equity and diversity issues; and f. planning for contingencies, such as equipment failure Assessors must be qualified to assess. This may involve training and professional development of assessors. 2 The Department of Defence RTO maintains an assessment kit for the guidance of its registered assessors. The kit is issued on successful completion of assessor training. 5 15

238 ADFP Chapter This planning applies to both programmed tests on course and to workplace assessment of competence. Step 2: Conduct summative assessments The conduct of summative assessments should follow the designed and developed guidance contained in the TMP and the administrative procedures of the organisation (school, unit, RTO etc) involved. Most situations are likely to involve: a. an assessor briefing of the assessment process; b. assessees use of assessment tools and materials to produce/collect the required evidence of performance; c. the assessors and/or designated authority following established procedures to act on any suspicions of fraud, plagiarism or other inappropriate actions; d. the receipt of evidence by assessors; e. the assessor makes a judgment comparing the evidence to the standard; f. opportunities for assessees to provide feedback on the process or appeal the assessment decision; and g. record results. Collect the evidence Of primary concern in assessment is adherence to the principles and processes involved in the collection of evidence to support a decision about the competence of the person being assessed. In a competency-based assessment system, evidence is something that supports the person s claim of having achieved competency. Evidence of competency can take many forms, such as direct evidence, primary evidence, secondary evidence, supplementary evidence, multiple sources of evidence. The forms of evidence required will depend on the purpose of the assessment. In every case, one source of evidence is probably insufficient, however, the more critical the assessment then the greater the need for more evidence and/or sources of evidence To gain RPL/recognition of current competencies, a range of evidence is usually used, including: a. examples of previous work; 5 16

239 ADFP Chapter 5 b. documentary evidence of previous achievements; c. personal references from people familiar with the learner s achievements; or d. demonstration of performance by means of practical test. Evaluate the evidence In aggregating the evidence to determine/make judgments of the assessees competence/achievement of learning outcomes, two strategies are proposed depending on the type of evidence collected: a. Objective aggregation of information by assuming that someone is not yet competent until there is evidence of competence that accords with the required assessment criteria/marking guide. b. Subjective balancing the evidence for and against competence, ie would a reasonable person in the occupation be prepared to overlook failure on a number of elements or performance criteria in the standards (eg successful performance on a particularly difficult task and one which tends to discriminate accurately those who are competent from those who are not, might outweigh evidence of failure on a minor task of lower difficulty level) Generally speaking, grading of learner performance does occur within competency based assessment systems. It is, however, a complex area with a number of issues and no singularly agreed approach. The grading of assessment in Vocational Education and Training is the practice of assessing and reporting levels of performance with the general purpose of recognising merit or excellence. Confirm the results Inform the relevant personnel (including assessee, administrator) of the outcome of the assessment. Step 3: Evaluate assessment conduct methodology in accordance with evaluation plan/strategy Assessment processes should be regularly evaluated/reviewed. The evaluation/review of the assessment processes is often most easily and cost-effectively conducted when the training itself is evaluated. 5 17

240 ADFP Chapter Australian Quality Training Framework (AQTF) audits have identified the need for accurate and complete assessment documentation. The assessment package is a critical component of the TMP. It provides the plan for the assessment as well as the documentation. Figure 5 2 summarises those questions that are critical to the evaluation of the assessment processes. Figure 5 2: Critical questions in the review of assessment processes 5.74 Why assess? The important first step in any review of the assessment processes is to establish why assessment is being conducted. The answer to this question will provide the benchmark criteria that will impact on many of the other checks that will be carried out in the review What criteria? The second step is to confirm that assessment is valid against the objective (learning outcome and assessment criteria or competency units). 5 18

241 ADFP Chapter How to assess? The third step is to examine how the evidence for the assessment is being gathered. This will include the use of procedures and assessment and the way the evidence is documented What is enough? The final step in the evaluation/review process will be to check for validity, reliability, fairness and flexibility Identified changes. The assessment methods and tools are evaluated and feedback from trainees and assessors involved in administering the assessments are used to establish appropriate amendments to the assessment methods/tools/guidelines. Improvements and changes to the assessment tools/methods/guidelines are recommended/made, where necessary, and fed back to appropriate phases of the DTM. Moderation Moderation is the process whereby instructors/assessors discuss and reach agreement about assessments in a particular assessment system in order to achieve consistent assessment judgments. In so doing, a shared understanding develops between instructors/assessors about what is being assessed and why, and how different evidence is being interpreted. AQTF standards require moderation processes. POST COURSE ADMINISTRATION 5.80 Introduction The purpose of post-course administration is to record and report the outcomes of training and assessment Trigger. Post-course administration is triggered by the completion of a training program/course and assessment. 5 19

242 ADFP Chapter 5 Inputs, process and products The inputs, process and products of post-course administration is summarised in table 5 5 and explained in the following paragraphs. Inputs Process Products Implemented program/course, TMP, assessment results. Step 1: Record and report outcomes. Step 2: Instigate trainee failure management, if required. Step 3: housekeeping (restoring, restocking etc). Step 4: Issue course attendance certificate/record of attainment (ROA). Issued course attendance certificate/roa, CT and assessment records. Table 5 5: Inputs, process and products for post-course administration Step 1: Record and report outcomes The outcomes of training and assessment activities must be recorded to meet AQTF standards for storage, access and recording requirements. The prime database for Defence is PMKeyS, which should be used to record training results. Step 2: Instigate training failure management When individuals are assessed as not yet meeting a required assessment standard they may be required to attend remedial training or to repeat a program/course/unit. Such training should be organised and administered by the Conduct Authority. Other actions could include removal from the course or discharge from the Service, as administered by the appropriate authority. Step 3: Housekeeping On completion of training and assessment, administrative action should be taken to clean, maintain and restock the resources as required. Step 4: Issue course attendance certificates/records of attainment Graduates of courses receive certificates or ROAs to recognise attendance and achievement of course outcomes. These are issued by the organisation conducting training If formal Australian Qualifications Framework qualifications, including Statements of Attainment, are an outcome of training and assessment, the relevant Defence RTO issues these under their scope of registration. 5 20

243 ADFP Chapter 5 CONDUCT TRAINING REVIEW (INTERNAL EVALUATION)5.88 Introduction Training review is a continuous improvement process of measuring and evaluating the conduct of training and assessment. Implicit in this is the evaluation of all, or a targeted selection of their inputs, processes and products. The term training review, in the context of the DTM, is also referred to as internal evaluation Purpose. The purpose of conducting a training review is to determine that the training outcomes have been achieved and to bring about incremental change. Conducting a training review will also determine that the training has been effective and efficient Trigger. Conducting a training review is triggered by the completion of a training program/course. Typically training reviews are conducted on a routine basis, however, a training review may be conducted in response to a particular concern, such as excessive failures in one assessment. Inputs, process and products The inputs, process, and products of a training review are summarised in table 5 6. Inputs Process Products Implemented program/course, TMP with training review materials, assessment results, and feedback from trainees and staff. Step 1: Plan and prepare training reviews. Step 2: Collect and collate training review information. Step 3: Report training review. Training review reports with recommendations. Table 5 6: Inputs, process and products for conducting training review 5.92 A training review involves collecting and analysing information about training and assessment with a view to their improvement. Two types of information are generally sought, equating to the first two levels of Kirkpatrick s model (reaction and learning) 3. The first of Kirkpatrick s levels, reaction, considers participants perceptions to the training. The second level, learning, describes the difference between what the participant knew and could do before and after the training. Collecting and analysing information of these types is used to improve the conduct of training. 3 Kirkpatrick s model was described in chapter 3 The Design phase. 5 21

244 ADFP Chapter Strategies for reviewing training, together with the materials and instruments for conducting a training review, are contained in TMPs. Based on the strategy that: a. an individual review is planned and prepared; b. information is collected, collated and analysed; and c. decisions for continuous improvement are made and reported A training review is usually conducted by staff from the Conduct Authority. A training review is likely to be more objective if undertaken by a section or individuals who are not associated with the conduct of training. Step 1: Plan and prepare training reviews The planning and preparation of training reviews should be based on the strategy presented in the TMP, the procedures of the organisation conducting the training and past experience of the course/program being considered The planning of a training review should address the following questions: a. What aspects of the training need consideration? b. What information/data will provide indications about these aspects? c. When will this information/data be available? d. What is the best method for gathering the information/data? e. How will the information/data be processed? f. What time is available for gathering and processing the information/data? g. Who will be involved in the gathering and processing of the information/data? h. Who must be informed of the process and its products? 5.97 The answers to these questions form the basis of a plan for a training review. Plans may vary in their rigidity/flexibility according to circumstance, noting that it is commonplace for them to be adapted when, during their implementation, circumstances suggest value in doing so. 5 22

245 ADFP Chapter Preparing for a training review may involve compiling documentation, and preparing the participants and any information/data collection instruments to be used. The participants commonly include trainees, trainers, assessors, and/or administration staff. Their cooperation must be obtained early, and a constructive and purposeful relationship maintained. To this end, it is important that they be kept fully informed of the process and their part in it. Collection instruments (eg data collection software, interview questionnaires and briefing scripts) are useful for gathering and recording information/data. All contribute to the consistency and validity of the process. They are, however, less adaptable to unusual situations wherein it may be preferable to employ a less restrictive approach to collecting information. Step 2: Collect, collate and analyse training review information The collection of information for training review can involve: a. observing training and assessment activities; b. reviewing training documentation; c. applying questionnaires and surveys (printed or electronic); d. reviewing training results and records; and e. interviewing/surveying individuals and groups During the collection process, particularly if it is unstructured, care needs be taken to: a. collect only useful information/data; b. avoid unproductive herrings ; and c. be receptive to potentially useful material not previously considered The collation of data/information involves its arrangement into a format that helps its analysis: questionnaire returns need sorting, interview notes need transcribing etc. All pertinent information should be classified into areas that are meaningful to the purpose for which it was gathered Analysis of training review information involves closely examining the collated material so that conclusions can be drawn, areas and degrees of agreement or contrast identified, and; from these, facts, probabilities and possibilities derived. These are then considered to determine whether: a. no further action is justified; 5 23

246 ADFP Chapter 5 b. conclusions should be communicated to stakeholders (eg credit for good work or results); or c. areas of potential improvement should be investigated and reported Where areas of potential for improvement are identified they are considered, together with possible solutions, in relation to the purpose(s) of the review, and decisions are made about possible course of action (use of a problem solving strategy is recommended). The areas most likely to be identified will concern the Conduct phase of the DTM, but their causes might require amendments to the products of previous phases of the Model. Step 3: Report training review The final step of a training review is the reporting of its results. A training review report should identify the successful elements of the training as well as areas of potential improvement with recommendations how this should be achieved. The report should be sent to all stakeholders and, if necessary, followed up to ensure proper administration of its contents. Annexes: A. Flow chart showing the Conduct phase of the Defence training model B. Defence training model purpose, inputs, process and products of the Conduct phase 5 24

247 Annex A to ADFP Chapter 5 FLOW CHART SHOWING THE CONDUCT PHASE OF THE DEFENCE TRAINING MODEL A 5A 1

248

249 Annex B to ADFP Chapter 5 DEFENCE TRAINING MODEL PURPOSE, INPUTS, PROCESS AND PRODUCTS OF THE CONDUCT PHASE B CONDUCT PHASE PURPOSE INPUTS PROCESS PRODUCTS To ensure the implementation of training and assessment is achieved effectively and efficiently to meet the planned requirement. Policies and instructions, course venue, curriculum, learning and assessment resources, required materiel and financial resources, training staff, and nominated trainees. To check readiness of Prospective trainees, trainees. assessment staff, major resources, assessment guidelines, associated materials and equipment. To train people. Venue, Training Management Package (TMP), required facilities, training staff, trainees, material and financial resources. Administer training preparations. Implement pre-training assessments/tests. Prepared inputs to training, including but not limited to training schedule, training staff, designated trainees, facilities, materials and consumable resources. Either qualified people, candidates with prerequisites or unsuccessful applicants. Conduct training. Trained people. 5B 1

250 ADFP PURPOSE INPUTS PROCESS PRODUCTS To determine job competency and/or achievement of learning outcomes. The recording of assessment and training outcomes and their continuous improvement. Assessees and assessment staff, assessment guidelines and other materials, venue, finances. Implemented course/program, TMP with training review materials, assessment results, feedback from trainees and staff. Conduct assessments. Successful course graduates, award of competencies. Conduct training review (internal evaluation). Issued qualifications, course training and assessment records and training review reports containing recommendations for developments. 5B 2

251 ADFP Chapter 6 CHAPTER 6 THE EVALUATE PHASE 6 Introduction The Evaluate phase of the Defence training model (DTM) provides a measure of effectiveness of training. Evaluation measures how well training has prepared trainees for the workplace and whether the original workplace performance need has been met. The activities described in this phase are derived from level three and four of the Kirkpatrick model. This phase usually follows completion of training and assessment in the Conduct phase as shown in figure 6 1, however, an evaluation may be initiated at any time in the DTM cycle if necessary. Figure 6 1: Evaluate phase of the Defence training model 6.2 Evaluation provides feedback across all phases and steps of the DTM and identifies under and over-training, the strengths and shortfalls of training and assessment, and recommends any needed improvements. These recommendations then become the inputs to the Analyse and other phases of the DTM, and represent the start of a new cycle of development. 6.3 Purpose. The purpose of evaluation is to measure the effectiveness of training in meeting workplace performance requirements. 6 1

252 ADFP Chapter 6 Key products and responsibilities The key products of evaluation are reports that recognise that the training system is either functioning satisfactory or in need of change. Changes may be recommended to: a. the related units of competency and/or Training Requirement Specification (TRS); b. the training design of the program/course; c. the major resources, materials and equipment used for training and assessment; d. the conduct of training and assessment; and/or e. a non-training contributing factor to workplace performance. 6.5 The nominated Evaluate Authority is responsible for ensuring that training is evaluated and reported. They are also responsible for maintaining communication channels for the collection of ongoing evaluation feedback, and for the systems that record and use this information. 6.6 Use of an external agency. In accordance with national and Defence guidelines, the quality assurance of all training by, or on behalf of Defence (by Defence staff, contractors, professional service providers, or other Registered Training Organisations) remains a Defence responsibility. Because of this, and the need to develop an understanding of the state of training over time, ongoing evaluation type work is not contracted to an external agency. An external agency may, however, be used to undertake a one-off, in-depth evaluation study. Steps and products of the Evaluate phase The Evaluate phase has four processes, or steps. They are illustrated in the flow diagram in annex A, together with their purposes, inputs and products, in annex B. The process steps are explained in paragraphs Whilst presented as applicable to programmed, project-type studies, the process-steps are applicable to ongoing evaluation work. 6.8 As with all phases of the DTM, the application of the steps of the Evaluate phase should be appropriate to the circumstances. In situations of high significance (eg training and assessment closely associated with combat capability) the application should be rigorous. Abbreviated processes may be appropriate in lesser circumstances. 6 2

253 ADFP Chapter 6 Step 1: Plan and prepare the evaluation The planning and preparation of evaluation studies should be based on the strategies contained in the Training Management Package, any Terms of Reference provided, and the normal practices of the organisations conducting the training and the evaluation. If the workplace/jobs being addressed have been the subject of previous evaluation work, records of that work should also be considered The planning of an evaluation study should determine: a. the aim of the study; b. the data/information required; c. the sources of this data/information; d. how and when the data/information will be gathered and processed; e. what will be done with the results; and f. in what time frame will the work be completed Evaluation study aim. The aim of an evaluation study should be precise and clear. The standard aim is to check whether the training provided by the XXX program/course meets the requirements of the YYY job. This aim does not direct attention to the original workplace performance need (for which factors other than training will be pertinent). If a study is to consider factors other than training, its aim should reflect the greater scope Data/information required. While any feedback on the performance of graduate trainees on-the-job is likely to be useful for evaluation, most studies seek answers to the following questions: a. How much has workplace performance been enhanced by training? b. What tasks are performed on-the-job for which no training has been provided? c. What tasks have been trained for which are not required on-the-job? d. Did the job holders perform at a standard below that required of the job? 6 3

254 ADFP Chapter 6 e. What tasks have performance standards, as defined in learning objectives, which are inadequate for early, effective employment on-the-job? f. What tasks have performance standards, as defined in learning objectives, which are higher than required on-the-job? g. With what frequency are competencies trained for used on-the-job? h. What procedures or drills trained on course vary with job practices (and what are the consequences of this)? i. In what tasks do job holders experience significant difficulty in adapting from the training to the job situation? j. What attitudes have job holders displayed to their job, category/employment group, and the Service/department? 6.13 Sources of evaluation data/information. Information and data that can be used for evaluation may be gained from any one or combination of the following sources: a. Course graduates and job supervisors. When employed on-the-job, course graduates can provide feedback about the requirements of their work in relation to the training received and assessment experienced. Their supervisors are the authorities on-the-job requirements and the standard of work being completed. Combined, these two workplace sources are the most focused and reliable source of feedback. When using them, the time between training and the evaluation must be considered. Too soon and all parts of the job may not have been experienced. Too late, and graduates will have forgotten the details of the training they experienced. b. New training staff and visitors from workplace. New staff members joining training organisations from the job environment are sources of evaluation feedback. So too are visitors from the workplace, including past graduates. Such feedback tends to be piecemeal, but can indicate areas of strength or weakness and, if collated over time can provide clear and detailed understanding. 6 4

255 ADFP Chapter 6 c. Reports and records. Information and data can be extracted from reports and studies prepared for purposes other than evaluation. Operational and exercise reports, defect reports and safety records are examples. Occasionally too, job owners and supervisors communicate unsolicited evaluation feedback. These sources are usually of indicative value, and in need of support through focused information/data gathering. d. Defence documents and communications. Defence instructions, publications, manuals, memorandums and the communications that promulgate amendments to them should be monitored for changes to policies, procedures, structures and organisation that may affect jobs and the training conducted for them. Such changes are frequent, and require incorporating in training documentation and courses conducted Gathering and processing data/information. The planning of evaluation work should incorporate consideration of how and when the data/information will be gathered, and how it will be processed. Data/information gathering methods include: a. Workplace measurement. Potentially the most valid feedback gathering method, workplace measurement is time consuming. b. Questionnaires (electronic or paper-based). The use of questionnaires is a quick (if electronic) and inexpensive method of collecting feedback, particularly from large numbers of dispersed people. The value of the data/information tends to reflect the effort and skill in preparing and administering the questionnaire. Even if well implemented though, a poor response in quantity and quality can result in unreliable feedback. Because of this, questionnaires are usually best used in conjunction with a workplace visit, when their administration and completion can be controlled. c. Interviews/discussions (structured and unstructured). Interviews and discussions can be undertaken in the workplace environment. They can be inexpensive and easy methods of gathering current evaluation feedback provided that a small number will suffice. Structured interviews can focus on specific issues, but can result in potentially useful feedback being missed. Less structured discussions can capture such feedback, but risk being so individualised and subjective. Feedback gained from interviews and discussions may need to be crosschecked. 6 5

256 ADFP Chapter 6 d. Document and network research. Research of Defence documents, the Internet and communications is useful when the data/information is known to exist and a focused search for it is undertaken. Unstructured searches and scanning of sources can be unproductively time consuming and wasteful in bandwidth. e. Combination of methods. The gathering of data/information for most evaluation studies involves a combination of the preceding methods, thereby providing means of crosschecking results As large quantities of data/information can be gathered, planning for evaluation work should consider how it will be collated and processed. This is especially true for the comparatively unstructured and possibly seemingly insignificant feedback gathered from ongoing evaluation work Managing the results. Evaluation work should be planned to accord with a concept of how the results will be managed towards achieving worthwhile outputs and outcomes. The concept should be shared and accepted by the main stakeholders in these outputs and outcomes the authorities responsible for training. Involving the authorities or their representatives in the planning and preparation of evaluation work is therefore invaluable Time frame. The time frame planned for evaluation work will depend on the breadth and depth of investigations, the reliability of the data/information and resulting conclusions required, and any external deadlines An evaluation plan incorporating a schedule for completing the work is developed to accord with the factors outlined in paragraphs 6.9 to Plans can vary in their rigidity/flexibility, and it is normal for them to be adapted when, during implementation, value in doing so is seen Preparing for evaluation work involves: a. gaining the needed training documentation; b. selecting and making ready the people involved (including the training and briefing of any evaluation staff); c. arranging any workplace or other visits; and d. developing any information/data collection instruments to be used (eg data collection software, interview questionnaires and briefing scripts). 6 6

257 ADFP Chapter The cooperation of people involved must be obtained early, and a constructive and purposeful relationship maintained. Keeping everyone involved in the evaluation fully informed of the process and their part in it is very important. Step 2: Collect and collate evaluation information The collection of information for evaluation purposes can involve: a. reviewing course and training development documentation; b. observing and measuring workplace performance; c. applying questionnaires and surveys (printed or electronic); d. reviewing training results and records; and e. interviewing/surveying individuals and groups During the collection process, particularly if it is unstructured, care needs be taken to: a. collect only useful information/data; b. avoid unproductive red herrings ; and c. be receptive to potentially useful material not previously considered The collation of data/information involves its arrangement into a format that helps its analysis: questionnaire returns need sorting, interview notes need transcribing etc. All pertinent information should be sorted into areas that are meaningful to the purpose for which it was gathered. Step 3: Analyse and synthesise evaluation information/data Analysis and synthesis of evaluation information involves closely examining and combining the material gathered and collated to identify areas and degrees of agreement or disagreement. From these areas facts, probabilities, possibilities and conclusions relating to the aim of the evaluation are derived. The conclusions are the outputs of the work and are of three types: a. areas of training that are satisfactory, b. areas of training that are unsatisfactory and need improvement, and c. conclusions that other factors affecting performance require attention. 6 7

258 ADFP Chapter As evaluation work centres on the appropriateness of training, the most common problem identified is a mismatch between required job performance and the TRS. Other problem areas can involve derived training documentation and the conduct of training Where areas of potential improvement are identified, courses of action for achieving the improvement should be identified and evaluated (problem solving strategies might be used). This analysis work equates to the Needs Analysis work that begins the Analyse phase of the DTM (and accounts for the DTM s circular form). Stakeholders should be involved in the process, not least as this supports any change management required. Step 4: Report evaluation The final step of the Evaluation phase is the reporting of results. An Evaluation Report should follow the presentation conventions of the Australian Defence Force Publication (ADFP) Defence Writing Standards (currently ADFP 102). It should recognise areas of training that are successful, affirm those that are satisfactory, identify areas where improvements are desirable and recommend how those improvements should be achieved. In some circumstances it might also be appropriate to recommend further investigation, particularly where non-training issues are involved As with most DTM work, stakeholders should be involved in the preparation of an Evaluation Report such that its contents are no surprise, but more a summary and formal statement of issues and actions that are understood and accepted The Evaluation Authority sends Evaluation Reports to all stakeholders, especially those responsible for implementing the recommendations. The report is also filed for use in any follow-up or new evaluation work. Annexes: A. Flow chart showing the Evaluate phase of the Defence training model B. Defence training model purpose, inputs, process and products of the Evaluate phase 6 8

259 Annex A to ADFP Chapter 6 FLOW CHART SHOWING THE EVALUATE PHASE OF THE DEFENCE TRAINING MODEL A 6A 1

260

261 Annex B to ADFP Chapter 6 DEFENCE TRAINING MODEL PURPOSE, INPUTS, PROCESS AND PRODUCTS OF THE EVALUATE PHASE B EVALUATE PHASE PURPOSE INPUTS PROCESS-STEP PRODUCTS To ensure effectiveness and efficiency in the evaluation process and inform stakeholders, gain commitment. To gather useable data and information. Evaluation strategy, terms of reference, previous Evaluation Reports, workplace, reports of workplace performance evaluation materials, Course Reports, stakeholder training and assessment feedback, occupational analysis reports, job specification, Training Requirement Specification, curriculum, resources (including financial). Detailed plan for evaluation, information collection materials, job holders, their supervisors, and other stakeholders. 1. Plan and prepare evaluation. 2. Collect and collate evaluation information/data. Evaluation plan and program, approvals for workplace access, information collection materials, trained/prepared/proficient evaluators. Evaluation data and information. 6B 1

262 ADFP PURPOSE INPUTS PROCESS-STEP PRODUCTS To generate appropriate conclusions and possible improvement strategies. To determine whether the performance need has been met and gain endorsement/commitment to improvement strategies. Evaluation data and information, subject matter experts, information on non-training factors affecting performance. Evaluation conclusions, opportunities for improvement. 3. Analyse and synthesise evaluation information/data. Conclusions about the effectiveness of training and opportunities for improvement. 4. Report the evaluation. Evaluation Report with recommendations. 6B 2

263 ADFP GLOSSARY Unless stated otherwise, Australian Defence Force (ADF) terms and definitions are used within this publication. Externally sourced terms and definitions, approved for ADF use, are designated in brackets following the definition, using the following legend: DEST Department of Education, Science and Training (previously known as the Australian National Training Authority (ANTA) accredited course A structured sequence of vocational education and training (VET) that leads to an Australian Qualifications Framework (AQF) qualification or Statement of Attainment. Accredited courses are listed on the National Training Information Service (NTIS) see (DEST) Analyse phase of the Defence training model The Analyse phase of the Defence training model (DTM) is when the need for training (new or revised) to improve human performance is analysed and specified. The two main products of the Analyse phase are job specifications that describe jobs and Training Requirement Specification (TRS) that specify Defence training requirements. assessment The process of collecting evidence and making judgments about something. assessment criteria The lowest of three levels of learning outcome from a training process. It contains a performance, the conditions under which that performance is undertaken and the standard required for successful achievement. An assessment criterion should be accompanied by the method to be used for evaluating achievement. assessment guidelines An endorsed component of a training package that underpin assessment and which set out the industry approach to valid, reliable, flexible and fair assessment. Assessment guidelines include the assessment system overview, assessor requirements, designing assessment resources, conducting assessment and the sources of information on assessment. (DEST) 1

264 ADFP assessment materials An optional component of training packages that complement endorsed industry assessment guidelines and could take the form of assessment exemplars or specific assessment tasks and instructions. (DEST) assessment strategy A broad methodology for using assessment to support training and, finally, determining whether the required performance has been achieved. An assessment strategy for training should detail the program, its activities, their purpose and likely resource requirements. Australian Qualifications Framework (AQF) A nationally consistent set of qualifications for all post-compulsory education and training in Australia. (DEST) Australian Quality Training Framework (AQTF) The nationally agreed recognition arrangements for the VET sector. The AQTF is based on a quality assured approach to the registration of training organisations seeking to deliver training, assess competency outcomes and issue AQF qualifications and/or Statements of Attainment (SOA). AQTF ensures the recognition of training providers and the AQF qualifications and SOA they issue, across Australia. (DEST) Authorities (Analysis, Design, Development, Conduct and Evaluation) In relation to the DTM are, in accordance with Defence Instruction (General) PERS Implementation of the Defence Training Model in Defence Vocational Education and Training, responsible for managing key training processes (including their quality assurance and continuous improvement), and for approving and maintaining the products of their respective phase of the model. competency The ability to perform tasks and duties to the standard expected. (also competence) competency analysis The process of identifying the knowledge, skills (mental and physical) and attitudes/attributes that combine to enable mastery of a competency, together with any significant factor that can influence its training. The results are recorded on a Competency Analysis Sheet. 2

265 ADFP competency assessment Determines whether competency has been achieved: whether an individual can perform something to the standard expected in the workplace, as expressed in the relevant endorsed industry/enterprise competency standards or the learning outcomes of an accredited course. (DEST) competency assessment strategy Describes the formal assessment strategy for assessing workplace competency. It states what is to be assessed, where and when, and who is to gather and make judgments of the evidence. It is written during the Design phase of the DTM. competency based training and assessment (CBTA) An approach to learning which places primary emphasis on what the learner can do. It is focussed on outcomes rather than on learning processes or time spent engaged in those processes. It is concerned with the attainment and demonstration through application in the workplace of knowledge and skills to a specified level of competency. CBTA is concerned with the flexible use of endorsed competency standards in the workplace. It provides for national recognition of achievement and allows skills and knowledge to be transferable and portable under the National Training Framework (NTF). (DEST) competency profile A document describing the competencies of an individual or group. The competencies listed should be either nationally endorsed or locally identified competency standards or units. competency standard An industry-determined specification of performance which sets out the skills, knowledge and attitudes required to operate effectively in employment. Competency standards are made up of units of competency, which are themselves made up of elements of competency, together with performance criteria, a range of variables, and an evidence guide. Competency standards are an endorsed component of a training package. (DEST) Conduct phase of the Defence training model The phase of the DTM when training and associated processes are implemented. 3

266 ADFP continuous improvement A planned process, which allows an organisation to systematically review and improve the quality of its products, services and associated processes. The continuous improvement processes of the DTM include training review, evaluation, and the formative evaluation of work in each of its phases. (DEST) course A structured and integrated program of education or training, usually consisting of a number of modules (subjects) or shorter programs, and leading to the award of a qualification. (DEST) course learning outcome (CLO) The highest of three levels of learning outcome from a training process. It contains a performance, the conditions under which that performance is undertaken, and the standard required for successful achievement. A course learning outcome should be accompanied by the assessment method to be used for evaluating that success. course learning outcome levels Provide a shorthand method of indicating the meaning of course learning outcomes to stakeholders, particularly the employers of recent training graduates. Four levels are used: a. trained, but not against a prescribed standard (background or familiarisation training); b. partially trained against prescribed standard (additional training and/or experience required before assessment, employment under supervision only); c. fully trained and ready for assessment against prescribed standard (assessment required, employment under supervision until assessed); and d. trained and assessed competent against the prescribed standard (ready for employment). course map A diagram showing the major modules and sessions of a course or program of training, together with alternative paths for its undertaking. Its use enables flexibility in scheduling and programming without detriment to the achievement of learning objectives. 4

267 ADFP curriculum The specifications for a course or subject (module) which describe all the learning experiences a student undergoes, generally including objectives, content, intended learning outcomes, teaching methodology, recommended or prescribed assessment tasks, assessment exemplars etc. In the DTM it is based on training and assessment strategies designed for specific learning outcomes. (DEST) Defence assessment kit A set of policies and procedures under which Defence competency assessments are conducted (the Defence Services Learning Network Recognition Kit, version 5). Defence training model (DTM) The name of the model linking the processes of the systems approach to training used in Defence. Department of Education Science and Training (DEST) A Commonwealth statutory authority providing a national focus for vocational education and training. Design phase of the Defence training model When the assessment and learning strategies for training are selected, together with the strategies for evaluating the whole process of preparing for and then conducting training. Develop phase of the Defence training model When the resources needed to conduct, assess and evaluate training are produced and/or prepared. diagnostic assessment Describes assessment that helps learners and their supervisors determine the learner s education, training and development needs. Recognition of prior learning (RPL) is a form of diagnostic assessment. If recognition of current competency leads to the award of a unit(s) of competency, then it is summative assessment. education 1. The process of imparting knowledge or developing skills, understanding, attitudes, character or behaviours. 2. The process of acquiring knowledge, skills, understanding, attitudes, etc. 3. The knowledge, skills, understanding, attitudes, etc acquired. 4. The field of study concerned with teaching and learning. (DEST) 5

268 ADFP element of competency A basic building block of a unit of competency which describe the key activities or elements of the work covered by the unit. (DEST) Evaluate phase of the Defence training model When feedback on how well training has prepared learners for the workplace, and whether the original capability requirement has been met, is gathered and processed. evaluation 1. The process or results of systematic collection and analysis of data or information to make an assessment or appraisal in relation to stated objectives, standards or criteria in education or training which may be applied to policies, organisations, organisational functions, programs, projects, courses or other activities. 2. In the context of the DTM: a. Evaluation is the process of checking how well training has prepared learners for the workplace and whether the original capability requirement has been met. b. Formative evaluation is the process that provides feedback on the progress of work contributing to achieving something. c. Summative evaluation is the process conducted on completion of work to determine how well it achieved its purpose. evaluation strategy A guide to how evaluation against the Training Requirement Statement is to be organised in workplaces. It should include information on the methods to employ; exemplar materials; and who is responsible for data collection, data analysis and completing the Evaluation Report. evidence guide The part of a competency standard which provides a guide to the interpretation and assessment of the unit of competency, including the aspects which need to be emphasised in assessment, relationships to other units, and the required evidence of competency. (DEST) feasibility analysis The process used to determine the best option, or mix of options, for ensuring the competence of jobholders. The results are presented in a Feasibility Report, which, if training is a solution, includes a Training Requirement Statement. 6

269 ADFP formative assessment Describes assessment to assist and support a process (eg learning) during the undertaking of that process (eg by advising the learner about the quality of performance and progress towards the attainment of a competency). job analysis The process used to determine/confirm the workplace performance requirement, current or foreseen, for a given employment classification (task, job, career or work group). The results are recorded on a job specification. job specification The Defence document that records the product of a job analysis. It classifies a job and details its composition and characteristics. key competencies The generic, underpinning, transferable skills or competencies essential for effective participation in work. They apply to work generally, rather than being specific to work in a particular occupation or industry. There are seven key competencies: collecting, analysing and organising information; communicating ideas and information; planning and organising activities; working with others and in teams; using mathematical ideas and techniques; solving problems and using technology. They have three performance levels: Level 1 undertakes tasks effectively using routine approaches; Level 2 manages tasks selecting routine approaches; and Level 3 evaluates and reshapes tasks to establish new approaches. Note Training packages developed from 2006 will include employability skills learner analysis The process used to determine the characteristics and qualities of the prospective group of learners. The output is a learner profile, critical to which is a list of the typical learner s competencies. learner profile A document that is developed on the basis of a Target Population Profile. It describes the existing competencies and capabilities of prospective learners, their preferred learning style, special aptitudes and motivation. This information helps ensure the suitability of the assessment and learning strategies. 7

270 ADFP learning analysis The process of identifying the desired outcomes of the training and learning process: the learning outcomes. learning environment analysis A study of the things or conditions that could surround training and which should therefore influence the design and development of strategies for its assessment and conduct. Recorded in a learning environment profile, it is used to determine the assessment and learning strategies for competencies and training respectively. learning outcomes Describe the intended results of training. They clearly and concisely describe what a learner will be able to do on satisfactorily completing a unit of training. The DTM uses two types of outcome: CLO and module learning outcome (MLO). Learning outcomes are the building blocks of training and learning courses. They are written with three parts: the performance (using an action verb), the conditions (such as use of tools, references or other support) and standards (defining the criteria of success, including any time limitations). Each learning outcome should be accompanied by the method to be used for evaluating achievement. learning strategy A non-endorsed component of a training package which provides information on how training programs may be organised in workplaces and training institutions. This may include information on learning pathways, model training programs and training materials. (DEST) module learning outcome (MLO) The middle of three levels of learning outcome from a training process. It contains a performance, the conditions under which that performance is undertaken and the standard required for successful achievement. An MLO should be accompanied by the assessment method to be used for evaluating that success. National Training Framework (NTF) The system of VET that applies nationally. It is made up of the AQTF and nationally endorsed training packages. (DEST) National Training Information Service (NTIS) Developed by DEST to provide access to current and emerging training market information and products in vocational education and training. The NTIS webpage is at It comprises a database of: a. VET accredited courses, 8

271 ADFP b. competency standards, c. training packages, and d. training providers and text information on a range of complementary issues. needs assessment A precursor to the Analyse phase of the DTM. It is the product of an analysis to establish whether applying the model is likely to achieve a worthwhile improvement in workplace performance. performance criteria The part of a competency standard specifying the required level of performance in terms of a set of outcomes which need to be achieved in order to be deemed competent. (DEST) performance gap The difference between the performance required for a job and the capabilities of the target population: the people likely to perform it. The gap has two dimensions: scope and scale. The scope of the gap describes the work that the target population cannot presently perform. The scale describes the significance of this problem. proficiencies A term defined by Defence Personnel Management Key Solutions (PMKeyS) as training, education or experiential outcome achieved by an individual which is essential to satisfy a specified workplace requirement associated with an established Defence position. There are five types of proficiencies in PMKeyS: certificate, licence, experience, stream and skill. Proficiencies are required in PMKeyS as they link training regimes to individual profiles and positions. Proficiencies are awarded to a trainee automatically upon the successful completion of a course session or attached to a member s profile manually as required. A certificate of proficiency usually consists of a set of related competency units and/or elements. range statement The part of a competency standard which specifies the range of contexts and conditions to which the performance criteria apply. (also known as range of variables) (DEST) recognition of current competence The acknowledgment of competencies currently held by a person, acquired through training, work or life experience. More commonly known as RPL. (DEST) 9

272 ADFP recognition of prior learning (RPL) The acknowledgment of a person s skills and knowledge acquired through previous training, work or life experience, which may be used to grant status or credit in a subject or module. (DEST) record of attainment (ROA) Defence certification issued to a student for partial completion of a qualification. Registered Training Organisation (RTO) A title for an organisation registered by a State or Territory recognition authority to deliver training and/or conduct assessments and issue nationally recognised qualifications in accordance with the AQTF. (DEST) skills, knowledge and attitude (SKA) analysis Describes the identification of the skills, knowledge and attitudes that together enable someone to consciously do something. statement of attainment (SOA) Certification issued to a student for partial completion of a qualification, including, where relevant, the units of competency achieved under nationally endorsed standards. Achievements recognised by statements of attainment can accumulate towards a qualification within the AQF. (DEST) Statement of Performance Need A document that records the performance gap: ie the difference between the competencies and other proficiencies required for workplace performance, and those the target population already has. summative assessment Describes assessment on completion of a process. In training it can be used to determine both achievement of a training program/course, and whether a unit of competency has been achieved. RPL is a form of summative assessment. Target Population Profile A document describing the significant characteristics of the target population for training. It may incorporate the numbers in the group, age, gender, education and experience levels, probable turnover rate, and the current competencies they can be expected to have. 10

273 ADFP to pilot training Refers to testing newly developed training and subjecting it to critical training review. The purpose of piloting training is to identify aspects of its design and development that can be improved. training Describes the development of skills, knowledge, attitudes, competencies etc, through instruction or practice. Within this instruction the term training is used as a contraction of vocational education and training, the term more commonly used in the context of Australia s NTF. Types of training within Defence include: a. collective, or team training: the training of one or more groups of people, as when working-up an ADF unit for operational readiness; b. formal training: training conducted in an organised manner, usually towards defined outcomes such as qualifications; c. individual training: the training of individuals; d. joint training: training in which elements of more than one Service participate; and e. on-the-job training: formal training that occurs at a learner s workplace. (DEST) training design The process that produces the curriculum that specifies how training is to be conducted. training environment profile A description of the situation(s) in which training is to be conducted. It is likely to include location, learning facilities and equipment, staffing, availability of external training policy and other constraints. First prepared for the Analyse phase of the DTM, a more detailed version is used in the Design phase. Training Management Package (TMP) Presents the complete set of documentation necessary for managing the conduct of a training program or course. It contains the TRS, the curriculum, the training review and evaluation strategies, and the materials developed to conduct the course/program. 11

274 ADFP training package Describes an integrated set of nationally endorsed standards, guidelines and qualifications for training, assessing and recognising people s skills, developed by industry to meet the training needs of an industry or group of industries. Training packages consist of core endorsed components of competency standards, assessment guidelines and qualifications, and optional non-endorsed components of support materials such as learning strategies, assessment resources and professional development materials. (DEST) Training Requirement Specification (TRS) The formal document that specifies the Defence training requirement in terms of the competencies to be trained, the need for people to be trained in them (how many and in what time frame), the broad type of training that is to be employed and how that training will be recorded. training review The continuous improvement process of evaluating and reporting the Conduct phase of the DTM. training strategy A broad methodology for the conduct and management of training. A training strategy should provide guidance on the methods, location, activities, media, recommended or proscribed tasks, formative assessment, and likely resource requirements that training will involve. unit of competency A component of a competency standard. A unit of competency is a statement of a key function or role in a particular job or occupation. See also element of competency, performance criteria and range of variables. (DEST) vocational education and training (VET) Encompasses post-compulsory education and training, excluding degree and higher-level programs delivered by higher education institutions, which provides people with occupational or work-related knowledge and skills. VET also includes programs that provide the basis for subsequent vocational programs. Alternative terms used internationally include technical and VET, vocational and technical education and training, technical and vocational education, vocational and technical education, and further education and training. (DEST) 12

275 ADFP ACRONYMS AND ABBREVIATIONS ADDP Australian Defence Doctrine Publication ADF Australian Defence Force AQF Australian Qualifications Framework AQTF Australian Quality Training Framework ARF Australian Recognition Framework ASLAV Australian Light Armoured Vehicle BPO Business Process Owner CAS Competency Analysis Sheets CBTA competency based training and assessment CLO course learning outcome CT course training DEST Department of Education, Science and Technology DETC Defence Education and Training Committee DI(G) Defence Instruction (General) DPE Defence Personnel Executive DTI duty task inventory DTM Defence training model F2F face to face ISC Industry Skills Council ITAB industry training advisory bodies JMAP joint military appreciation process LSA logistic support analysis MJT manager of joint training MLO module learning outcome NTF National Training Framework NTIS National Training Information Service OHS occupational health and safety OJT on-the-job training PD&T professional development and training PMKeyS Personnel Management Key Solutions PO Policy Owner RDP rapid development process ROA record of attainment RPL recognition of prior learning RTO Registered Training Organisation 13

276 ADFP SME SOA TMP TOR TR&E TRS subject matter expert statement of attainment Training Management Package Terms of Reference Training Review and Evaluation Training Requirement Specification 14