The Opportunities and Challenges Faced in Utilising e-based Assessment. Paul Howarth - Managing Director EMEA

Transcription

1 The Opportunities and Challenges Faced in Utilising e-based Assessment Paul Howarth - Managing Director EMEA

2 Pearson Overview Penguin Group Pearson Plc 5.6 billion annual sales World s # 1 education business From kindergarten to CEO Operations in 55 countries Publishing in 17 languages Exams translated in over 20 languages Over 37,000 employees Pearson VUE - The Leader in Computer Based Testing Flexible and robust content development delivery solutions Over 5000 Test centres in 165 countries Easy to deploy customer site testing A commitment to quality and outstanding service

3 Who is Pearson VUE? Pearson VUE is the world s leading assessment and testing business Established as Virtual University Enterprises (VUE) in 1994, VUE was acquired by Pearson plc in 2000 Pearson VUE acquired the former Company Promissor in 2006, thereby extending not only our test centre network, but also our leadership in the certification market Pearson VUE delivers tests globally for both the Professional and IT markets Over 5,200 test centres operating in 165 countries In the UK it owns and operates 150 test centres in high street locations Pearson VUE operate the largest CBT in the world for DSA

4 What is e-assessment? e-assessment is the use of information technology for any assessment-related activity. Due to the increase in e-learning, the term e-assessment is becoming widely used as a generic term to describe the use of computers within the assessment process. e-assessment can be used to assess cognitive and practical abilities. Cognitive abilities are assessed using e-assessment software; practical abilities are assessed using e-portfolios or simulation software. An e-assessment system comprises two components: An assessment engine An item bank. An assessment engine comprises the hardware and software required to create and deliver a actual test. Most e-assessment engines run on standard hardware so the key characteristic is the software's functionality. The software does not include the questions themselves; these are provided by an item bank. Once created, the engine uses the item bank to generate a test.

5 Different Types of Assessment Formative Assessment Undertaking the assessment constitutes a learning experience in its own right. E.g. Assessment over a period of time, such as a school term, where there is some element of project work and/or combined with essay type questioning which measures a candidate's understanding of a subject. Summative Assessment Usually undertaken at the end of a period of learning in order to generate a grade that reflects the student's performance. E.g. End of term exams, could include multiple-choice items and essay type questions. Diagnostic Assessment This form of assessment involves making judgments as to how a candidate is performing against a predetermined set of criteria. This type of assessment is widely issued identify students gaps and focus the teaching on specific topics Diagnostic Assessment Learning Summative Assessment

6 What are the different types of delivery for e-assessment? Web-Based Web-Based assessment refers to assessment activity which requires the use of the internet. In reality few high stakes assessment sessions are actually conducted online in real time but the transfer of data prior to and after the assessment session is conducted via the internet. Download This is more common method of test delivery for medium stakes or high stakes assessments, there are more protocols in place to ensure that the delivery of tests is secure and not at risk of exposure. The encrypted test is downloaded onto computers in the appropriate test centre/s at a designated date and time and then released when the candidate arrives to take the test. Where can it be deployed? Online Assessments can be deployed anywhere with an internet connection. Downloaded tests can be deployed anywhere with computer.

7 CBT Models Security and reliability Desk top assessment testing In-class testing Test centre certification testing Value/application Diagnostic Low stakes system Formative High stakes system Features and functionality Summative

8 Reasons to Implement Computer Based Testing e-assessment has many advantages over traditional (paper-based) assessment: Item trialling - Cost effective way to develop test items Easy-to-deploy no need to distribute paper exams and collect for marking Accessible for all candidates, including those with specific learning difficulties Extra time can be added to the test for candidates SLD Larger fonts and adjusted colours option available to candidates with Special Educational Needs Voiceovers can be created and delivered through headphones Secure test delivery and improved reliability over human marking Fast results processing; immediate results to the candidate Common interface and technology making the test delivery experience the same for all candidates Option to deliver the tests at different times and locations to suit the candidate's needs Enhanced question styles which incorporate interactivity and multimedia

9 Key Questions When Implementing CBT 1. Does it make psychometric sense? The Exam 2. Does it make organisational sense? The Operations 3. What benefits will accrue to my customers and stakeholders? 4. What service model is best for my programme? The Customer The Modus Operandi

10 Transitioning from Pencil & Paper Testing Factors to consider Test specifications Test design Item types User interface Scoring models Testing logistics Security procedures Testing center locations Testing windows

11 Implementing an E-assessment Key Decisions Who are we testing and why? What needs to be measured? What are the logistics for testing where and when should we test? Set up project management team Test Development Test Specification syllabus areas Recruitment of item writers Careful selection of Subject Matter Experts who are able to contribute positively to the project Day to day management of item writers and the items they create Confidentiality agreements Intellectual Property agreements Collecting, storing and version control of items QA of the items Approval by the client Creation of trial tests Analysis of trial data Psychometric analysis and final test build

12 Benefits of Computer-Based Testing Enhanced security No printed materials to lose Transmittal of encrypted data Random item presentation Candidate monitoring Increased candidate convenience Greater data capture capabilities Data forensics to identify cheating behavior Speed studies Opportunities for innovation Item formats Scoring

13 Pearson VUE Core Services Test Development The expertise to develop reliable, Fair and Valid Tests Defining your assessment needs and working with you to design, write and publish tests Registration and Scheduling We can offer a choice test registration and payments options Test Delivery A secure independent testing network to maintain the integrity of your tests Over 5,200 Test Centre in over 162 countries Elimination of complex logistics problems distribution, storage and tracking of test papers The ability to offer tests on demands at convenient times for candidates Consistent and pleasant experience for candidates Management Information Fast and reliable management information to support your testing programme Reduction in the time-consuming job of scoring tests and preparing hand written reports

14 Why Choose Pearson VUE? End to end test development and delivery services Security and integrity is our main priority International experience 24/7 support for centres and candidates Local dedicated teams Scalable, proven technology We chose Pearson VUE to deliver our 1.5 million driving theory tests each year because of their high standards of service and customer service. We are confident in their ability to meet our current and future needs, and to work in partnership with us to improve road safety in the UK. Nick Carter, Director of Strategy and Performance and Deputy CEO, Driving Standards Agency

15 Some of our EMEA clients

16 Thank you for listening