DATA- DRIVEN ADAPTIVE TRAINING

Transcription

1 Paladin:Paradigm Knowledge Solutions A BUSINESS INCORPORATED IN CANADA AND QUEBEC DEDICATED TO THE PURSUIT OF EXCELLENCE IN PROFESSIONAL SERVICES AND DATA-DRIVEN ADAPTIVE TRAINING IN AEROSPACE AND DEFENCE. DISRUPTIVE DATA- DRIVEN ADAPTIVE TRAINING In the 21st century, pilots and crew will train using adaptive learning environments tailored to their individual strengths and weaknesses. This is how.

2 CONTENT 01. INTRODUCTION ADAPTIVE TRAINING OUR TEAM PROTOTYPE DEVELOPMENT STRATEGY WORK WITH US CONTACT 13 2 Data-Driven Adaptive Training

3 INTRO How do you train the next generation of pilots and air crews, the digital natives? How do you make use of advances in e-learning technologies and media? Tablet computers, smartphones, web applications? What role will Big Data play in that future? Boeing estimates that there will be a global need for 500,000+ new pilots and as many other air crew members over the next 20 years. This amount exceeds global training capacity. In order to effectively train more pilots and crews in less time, but without sacrificing quality or safety, a new paradigm in training is required. The time has come to move away from counting the hours in a simulator and start measuring competency. It s time for training tailored to the skills of each individual. It s time for data-driven adaptive training. Paladin:Paradigm Knowledge Solutions, Inc. 3

4 WHAT IS DATA-DRIVEN ADAPTIVE TRAINING? THE BASICS DATA-DRIVEN ADAPTIVE TRAINING IS DELIVERING THE RIGHT TRAINING AT THE RIGHT TIME FOR EACH INDIVIDUAL The basic idea The goal of any training program is to deliver the right instruction at the right time for each person. The problem is that almost all of our training media are static. Almost every classroom, whether virtual through an electronic device or real and physical, treats every student the same, regardless of their background or skill level. Flight simulators do a better job at helping each student have their own experience of flight, and flight instructors can deliver targeted feedback at the right time. But even flight instructors get overwhelmed and need help keeping track of the individual progress of each trainee. And how can flight schools or training providers scale to meet demand cost-effectively without sacrificing quality or safety? We believe that there are huge unrealized opportunities buried in the data of each training program. We use sophisticated algorithms to analyze trainee interactions with learning software to quantitatively and qualitatively measure latent knowledge and skill level. We form an evolving picture of their learning progress that we can use to deliver targeted training at the right time. These advances are made possible by the advent of machine learning, a subfield of artificial intelligence research, and Big Data analytics the tools and techniques for handling gigantic data sets to mine them of patterns and intelligence. 4 Data-Driven Adaptive Training

5 How do we measure latent knowledge and skill progression? Big ideas are made up of lots of little ideas. Mastering a new skill or knowledge domain means mastering all the concepts that build up the subject. On our prototype e-learning platform, our subject matter experts have broken down traditional training media into their component ideas and skills. Each of these becomes a data point that we track when a student takes a course. This data allows us to build an evolutionary picture of the student s understanding of a subject. On each metric, we can measure their progress towards mastery. Based on these metrics, we can modify the delivery of training content so that the student only sees what they are ready for. These tools also allow us to present visual dashboards to either the student or to the instructor. The instructor also gets a bird s-eye overview of how an entire cohort is progressing. If an individual is falling behind relatively, the instructor can deliver personalized feedback. Paladin:Paradigm Knowledge Solutions, Inc. 5

6 OUR EXPERT TEAM AGILE AND GROWING WE ARE BRINGING TOGETHER EXPERTS IN ACADEMIA AND INDUSTRY TO REIMAGINE THE NEXT GENERATION IN FLIGHT TRAINING ADOLFO KLASSEN MIKHAIL KLASSEN GORDON WOOLLEY PRESIDENT & FOUNDER CHIEF DATA SCIENTIST PILOT TRAINING EXPERT SUZANNE KEARNS ADAPTIVE TRAINING EXPERT A veteran of the aerospace simulation & training industry with a record of bringing innovative solutions to the market. An astrophysics PhD with a hunger for data and experience in machine learning and artificial intelligence systems. A highly qualified and experienced pilot and examiner, now a consulting expert on flight crew training and simulation. An Associate Professor at Western University and a specialist in instructional design and competency-based training. 6 Data-Driven Adaptive Training

7 NETWORK It takes the right people to solve hard problems. That is why we ve assembled a network of experts across the aerospace industry, research universities, flight training schools, and the silicon valley and technology startup world. These include pilots, instructors, engineers, professors, technologists, and programmers. This combination of talent and expertise allows us to move faster and bring leading-edge research on artificial-intelligence-driven learning systems to market, to create the first data-driven adaptive training product. Paladin:Paradigm Knowledge Solutions, Inc. 7

8 PROTOTYPING AN ADAPTIVE E-LEARNING PLATFORM FOR AB INITIO FLIGHT TRAINING How we successfully partnered with a flight school at a Canadian university to build a prototype e-learning platform for testing adaptive training. 8 Data-Driven Adaptive Training

9 Development Steps Our company is at an early stage. We are developing technologies that combine research in educational data mining and adaptive learning, while partnering with flight schools and experts in e-learning. We are bringing the future into the present Recruiting Experts We knew that building an adaptive training platform would require bringing together the right talent. Early in the project we approached Dr. Suzanne Kearns, an associate professor at Western University in London, Canada, and a specialist in e-learning and human factors. Dr. Kearns understood how the advent of massive open online courses (MOOCs) would usher in new data mining strategies for improved learning and knowledge retention. The potential for this to be scaled to other learning platforms, such as offline e-learning and full-flight simulators seemed obvious to us. Dr. Kearns already had a modular set of flight safety and human factors courses that could be adapted and instrumented. The next step was to find a suitable test bed. Partnering with Seneca College Flight School Seneca College, based in Toronto, Canada, operates an ab initio flight training program. Through our partnership with Seneca, an applied research grant from National Science & Engineering Research Council of Canada could be deployed to further develop adaptive coursewares and build the necessary technology for adaptive training. In addition to Suzanne Kearns, we brought on an instructional designer, Anna von Hoyningen Huene, and a subject matter expert, Gordon Woolley, to advise and guide course development and define key skill metrics. Delivering an Adaptive Training Platform An early beta test provided data for us to mine and study. We were able to build predictive measures of student drop-out rates and estimate final performance from intermediate assessments. Borrowing from the Educational Data Mining literature, we then began developing algorithms to perform knowledge tracing: actively measuring what students are learning in real time through probabilistic modeling. These tools will enable us, in the near future, to be able to predict student aptitudes across a range of skill metrics, which will allow us to adaptively tune the training to each learner. Paladin:Paradigm Knowledge Solutions, Inc. 9

10 STRATEGY FUTURE TRENDS THE INDUSTRY MUST ADAPT TO THE NEXT GENERATION OF LEARNERS AND THE TECHNOLOGICAL CHALLENGES AND OPPORTUNITIES PRESENTED BY BIG DATA The aviation industry has long been forward-looking in introducing and embracing new technologies, such as full-flight simulators and computer-based training. However, much of today s training methods were not designed for the next generation of learners, which grew up surrounded by technology. Tomorrow s learners are tech-savvy multi-taskers who embrace virtual reality, gaming, and simulation. Creating learner-focused training technology means addressing these characteristics. The advantage of increased use of simulation, computer-based training, and gaming environments is the opportunity for gathering data and measuring competency in new ways. Individuals and training technology generate massive amounts of data. E-learning tracks student responses through web forms and online assessments, simulators create vast amounts of data exhaust, and individuals freely offer large amounts of personal information via social media. Wearable technology and eye-tracking present opportunities for measuring flight crew stress and other emotional and cognitive states such as focus, distraction, or boredom with the training exercise. Tools and techniques for mining these large data streams are now becoming available and data scientists are increasingly employed not just at technology companies, but across all industries for extracting business intelligence and identifying new opportunities from data. While the use of Big Data has privacy concerns, these can be handled responsibly while still taking advantage of opportunities. This is already happening in aviation: Etihad is using Big Data analytics on flight data to create performance metrics. GE has sponsored multiple data science competitions to optimize flights routes. PASSUR Aerospace created RightETA that uses machine learning to better predict aircraft arrival times at airports. Many other opportunities exist that are only beginning to be explored. A few examples of these are: Predicting the likelihood of a candidate completing a particular training exercise or programme. Identifying, through data-level performance metrics, the attributes of top-performing candidates to make better recruitment decisions. Adapting training content to the strengths and weaknesses of each individual. Most static e-learning is targeted at the average learner. Adaptive Training seeks to change this by leveraging data to deliver smarter, more targeted training, resulting in shorter times to completion. In the future, new content will be built around the adaptive training concept, but many of the advantages can be built onto existing training media right now, including psychometrics, predicting pass rates, and performing cohort analysis. 10 Data-Driven Adaptive Training

11 ADAPTIVE TRAINING FOR FULL-FLIGHT SIMULATORS In the case of full-flight simulators, adaptive training can be used to empower instructors. By collating and processing candidate data, we can build learner profiles, track progress, and suggest customizable lesson plans. In doing so, we relieve the instructors of the cognitive burdens of tracking all of this themselves. The system can also flag weaknesses and suggest revision at optimally determined intervals. This allows the instructors to focus on what they do best: teaching and mentoring. Smart information systems that make use of the data exhaust from flight simulators serve to make the training process more cost-effective and efficient while providing instructors with valuable insights. RESEARCH-PROVEN When training curricula are delivered on an adaptive schedule based on the real-time competency of the learner, performance improvements of 36% have been measured. We believe a savings of 30% in training time and/or cost can be realized through the use of adaptive technologies and analytics-based profiling. Paladin:Paradigm Knowledge Solutions, Inc. 11

12 WORK WITH US LET PPKS EMPOWER YOUR TRAINING PRODUCTS Work with your current training products We don t believe in throwing out all the excellent training hardware and software that our partners have already invested in. Only in making them smarter. Develop a data strategy Supercharge your technology Every modern technology creates data, We will then build dashboards and but making sense of it is challenging. Let analytics around these data products to us work with you to develop a data sci- suit your company s needs, whether that ence pipeline and strategy for getting the is making training curricula personalized most intelligence out of your data. and adaptive, or creating competencybased metrics to be tracked, or building predictive analytics into your current products. 12 Data-Driven Adaptive Training

13 Modern evidence-based management relies on data, and not only opinions, for achieving effectiveness and efficiency. RENEE BOUCHER FERGUSON CONTACT US WEB: PALADINPARADIGMKS.COM TEL: +1 (514) Paladin:Paradigm Knowledge Solutions, Inc. 13

14