Learning Analytics for Community Colleges. Presented By: Vicki Katz

Transcription

1 Learning Analytics for Community Colleges Presented By: Vicki Katz

2

3 Blackboard Platforms Web Based Teaching & Learning Synchronous Classrooms & Collaboration Multi-Modal Alerts & Notifications Campus Commerce & Security Education Analytics & Reporting Mobile Access & Connectivity Strategic Consulting Student Services & Support Hosting

4 Overview Blackboard s newest platform Extends the historical success of istrategy s product suite, acquired December 2010 Application provides pre-defined and customizable data models for the leading student information systems 20+ years of data warehouse and business intelligence experience, the last 10 years focused on higher ed higher education clients

5 Community Colleges Research I Public Private

6 Why Analy)cs? How can we proac+vely iden)fy at- risk students and provide appropriate assistance? As a student, how am I performing rela+ve to my peers? What are the correla)ons between course design and student success? Who are the most innova)ve instructors? Is our online course usage growing as predicted? 6

7 Why Can t You Get That Now? Select * from Source.PS_STDNT_CAR_TERM stdnt_car_term inner join Student.DimTerm dim_term on stdnt_car_term.strm = dim_term.sourcekey and dim_term.includeterminload = 1 left join Source.PS_ACAD_PROG acad_prog on stdnt_car_term.emplid = acad_prog.emplid and stdnt_car_term.acad_career = acad_prog. stdnt_car_term.stdnt_car_nbr = acad_prog and acad_prog.effdt <= dim_ter The Reporting Queue Data Genius 7

8 Blackboard Analytics Solution A Packaged Analytics Application Best Practices Defined Metrics Star Schema Business Rules OTHER Derived Information OLAP Pre-Built Data Integration Transformation Layer Data Warehouse Pre-Built Reports & Dashboards Student Management Financial Aid Finance Management Workforce Management Learn Mgmt Other Apps/Data 8

9 Blackboard Analy)cs for Learn Improve Student Success Op+mize Online Instruc+on Leverage LMS Investments Facilitate proactive intervention Increase student engagement Encourage successful behaviors Increase adoption Understand and improve course quality Measure teaching effectiveness Monitor usage over time Drive capacity planning Measure online learning initiatives 9

10 Informa)on Needs Vary But There Are Common Themes On Demand Easy to Access and Easy to Digest DATA TO HELP ME 10

11 Provide Access to Ac)onable Informa)on 11

12 Integrates and Transforms Data Enabling Robust Analyses User Activity Data Course Item Data Instructor Attributes Student Attributes Student System Grade Center Results Course Attributes Enrollment Data Performance Analyses Best Practices Analyses Trend Analyses 12

13 And Provides Mul)ple Ways to Access Data Dashboards Dynamic Analysis Reports 13

14 Enabling You to Find Answers and Take Ac)on Improve Student Success Leverage LMS Investments Optimize Online Instruction DEMONSTRATION 14

15 Field Trial Feedback One field trial enthusiast studied the cause and effect of online course interactivity on student outcomes the early results are compelling. Blackboard Analytics for Learn presented to the Dean s Council the response was favorable. There is a keen interest in utilizing the solution to improve student retention and persistence through activity and student outcomes analyses. 15

16 Blackboard Analy)cs for Learn Improve Student Success Op+mize Online Instruc+on Leverage LMS Investments Facilitate proactive intervention Increase student engagement Encourage successful behaviors Increase adoption Understand and improve course quality Measure teaching effectiveness Monitor usage over time Drive capacity planning Measure online learning initiatives 16

17 Thank you! Questions & Discussion

18 About Blackboard Analytics for Learn: Improve decision making. Improve institutional performance. 18

19 DEMO SLIDES 19

20 Analytics Starts with Questions Improve Student Success How can we proactively identify at-risk students and provide appropriate assistance? Are students engaged in their courses? Facilitate proactive intervention How does level of engagement or activity influence grades? Increase student engagement Encourage successful behaviors How can we identify patterns of student success? Can we motivate students through comparative analytics? 20

21 Motivate Students with Comparisons 21

22 Proactively Find At-Risk Students 22

23 Does Engagement Affect Grades? 23

24 Does Engagement Affect Grades? 24

25 Measure Aggregate Learning Outcome Performance PROTOTYPE: SUBJECT TO CHANGE 25

26 Analytics Starts with Questions Op+mize Online Instruc+on Are minimum standards of design being followed? What are the correlations between use of certain LMS tools and student success? Who are the most active instructors? What impact are they having on student success? Increase adoption Understand and improve course quality Measure teaching effectiveness How does usage differ by department, tenure, etc.? How are our programs and departments performing on learning outcomes over time? Is our training working? What areas might need new training or additional training? 26

27 Instructor Self-Comparison 27

28 Course Design Trends by Unit 28

29 Faculty Effort is it Effective? 29

30 Analytics Starts with Questions Leverage LMS Investments How has the use of Learn in the classroom changed over time? Is our online course usage growing as predicted? Monitor usage over time Drive capacity planning Measure online learning initiatives Do we understand our usage enough to do capacity planning or plan help desk staffing? What role should online learning play in our broader institutional strategy? What metrics will help determine this? 30

31 Activity Analysis & Capacity Planning 31

32 How can I monitor performance against goals? 32

33 APPENDIX 33

34 Offers A Suite of Products Blackboard Analytics for New in April 2012 Enrollment Management o Student o Financial Aid Learn Advancement Finance Human Resources 34

35 For Access By Mul)ple Audiences Academic Leadership Researchers and Analysts Blackboard Administrators Students Instructors 35

36 How can we use timely, actionable information from our online learning environment to improve student success? 36

37 Learning Analy)cs Metrics/Repor)ng Academic Leadership Instruc)onal Technology Course Repor)ng Overall usage per date range User statistics per month Course statistics per month For Learn At-risk analysis Submission activity Engagement vs. grades correlation Aggregate learning outcomes Top 50 courses Course design summary & trends Usage, trends & best practices by: instructor, department, subject, term LMS usage and trends Capacity planning Course open after term end Assessment Coordinators Course, program, and gen edlevel goal & rubric-based reporting Goal coverage reporting against curriculum Artifact repository Aggregate, student, cross-course, crossterm goal performance (LMS scores) Instructors By Course: Student performance against goals Rubric-based reporting User activity per day Early warning system Course dashboard - comparison of instructor s course design to other courses Correlation of grades to activity Students By Course: Student performance against goals Student dashboard - performance across courses and compared to peers 37

38 Course Reporting One course, one semester view Instructors are primary users 38

39 Cross-course rubric Assessment Coordinators are primary users 39

40 for Learn Cross-department, cross-college view Executives are primary users 40

41 Technical Requirements Requires the use of Blackboard Learn TM, Release 9.1, Service Pack 4 or later Pre-built SIS integration with: Oracle (PeopleSoft) Datatel (Colleague) SunGard Higher Education An Integration method is provided for other SISs Requires a locally hosted SQL Server environment Minimum hardware & software requirements available on request 41

42 Metrics & Dimensions for Analysis Learn Metrics Key SIS Elements Gradebook Possible Score Score Weight Grade Pct Weighted Element Count Highest Score Lowest Score Improvement Average Score Attempts Days To Grade Goals TBD Course Design Item Count Files Attached VTBE Objects VTBE Characters File Size Folder Depth Links Folders User Activity Sessions Logins Session Minutes Courses Content Items Tools Modules Clicks Pages Course Activity Courses Accessed Course Access Minutes Session Course Sequence Content Items Pages Content Activity Content Items Accessed Content Item Hits Content Access Minutes Session Sequence Folder (or Content) Depth Submissions TBD Course Section Subject Department College Class Number Term Course Career/Level Instruction Method Students Birthdate Gender Ethnicity Citizenship Current Term Primary Major Career/Level Academic Level Current Academic Standing Primary College Instructors Gender Ethnicity Highest Education Level Faculty Rank Tenure Status Primary Department 42

43 Blackboard Analytics for Learn TM Example Pre-Built Reports Login exception report The number of days since last login for students who meet the specified criteria Gradebook exception report The grade for course(s) for students who meet the specified criteria Student at-a-glance Logins, Interactions, Grade Center Score, compared to averages (there will be more soon) Instructor at-a-glance Logins, Interactions, Grade Center Score, compared to averages (there will be more soon) Most active instructors Active Sections, Interactions, Avg Interactions Per Section, Sections Total Logins by Schools Courses, Sections, Students, Total Logins, Avg Logins Per Section, Avg Logins Per User User Activity Compared to Others Over Time Sections, Avg Logins Per Section, Difference in Logins Compared to Peers 43