A National Online Essay Evaluation Service: Valid, Reliable, & Cost Effective Writing Assessment Leslie C. Perelman <perelman@mit.edu> Program in Writing and Humanistic Studies Massachusetts Institute of Technology
imoat Enhancing the best way of assessing writing Enthusiastic reception from both students & faculty Cost effective Reliable scoring and delivery
Motivations Existing modes of placement inadequate Standardized tests Timed impromptu Portfolio Students write on computers not paper
The Common Practice Student walks into large room and writes with pen and paper on small desk Student reads, at most, a two or three paragraph text for each essay Student writes one or two essays on specific questions Handwritten essays are scored (usually by two readers) Score is sent on paper to student and / or advisor imoat Student works at home Student views readings of any length online Student accesses questions online Student writes one or two essays at home within a specified period using a computer. Essays are scored on either printed copy or on screen by readers in real or virtual grading sessions Student and advisor access both score and text of essays.
A Better Way to Assess Writing
Architectural Overview Student Administrator Reader Internet Internet Web Server MS Windows 2000 / IIS MS ASP.NET C# Database MS SQL/2000
Create Class imoat High-Level Workflow Administrator Student Reader Create Test Upload Students Register for Exam Assign Readers View Readings & Questions Finalize Scores Write & Submit Essay Publish Results Reporting View Results [& Comments] Read & Score Essays [Write Comments]
Create Class imoat High-Level Workflow Administrator Student Reader
Admin - Create Class
Admin - Create Class
Admin - Create Test Group
Admin - Create Test Group
Create Class imoat High-Level Workflow Administrator Student Reader Create Test
Admin - Create Test - Define
Admin - Create Test Set Dates
Admin - Create Test Enter Readings
Admin - Create Test Enter Readings
Admin - Create Test Enter Readings
Admin - Create Test Enter Questions
Create Class TestWrite High-Level Workflow Administrator Student Reader Create Test Upload Students
Admin - Upload Students
Admin - Upload Students
Admin - Upload Students
Admin - Upload Students
Create Class imoat High-Level Workflow Administrator Student Reader Create Test Upload Students Register for Exam
Student - Login
Student - Registration
Student - Registration
Student - Registration
Student - Registration
Create Class imoat High-Level Workflow Administrator Student Reader Create Test Upload Students Register for Exam View Readings & Questions
Student View Readings
Student View Readings
Student View Assignment
Student View Assignment
Create Class imoat High-Level Workflow Administrator Student Reader Create Test Upload Students Register for Exam View Readings & Questions Write & Submit Essay
Student Submit Essay
Student Submit Essay
Student - Confirmation Email
Create Class imoat High-Level Workflow Administrator Student Reader Create Test Upload Students Register for Exam Assign Readers View Readings & Questions Write & Submit Essay
Admin Assign Readers
Admin Assign Readers
Create Class imoat High-Level Workflow Administrator Student Reader Create Test Upload Students Register for Exam Assign Readers View Readings & Questions Write & Submit Essay Read & Score Essays
Reader Get Essay to Read
Reader Get Essay to Read
Reader Read Essay
Reader Grade Essay
Reader Second Reader, Second Read
Create Class imoat High-Level Workflow Administrator Student Reader Create Test Upload Students Register for Exam Assign Readers View Readings & Questions Finalize Scores Write & Submit Essay Read & Score Essays
Admin Resolve Split Score
Admin Resolve Split Score
Admin Resolve Split Score
Admin Real-time Reader Reporting
Admin Real-time Reader Reporting
Admin Real-time Reader Reporting
Admin Real-time Reader Reporting
Create Class imoat High-Level Workflow Administrator Student Reader Create Test Upload Students Register for Exam Assign Readers View Readings & Questions Finalize Scores Write & Submit Essay Publish Results Read & Score Essays
Admin Define Result Categories
Admin Define Result Categories
Admin Define Result Categories
Admin Set Test Cuts
Admin Send to Results
Admin Finalize and Publish
Create Class imoat High-Level Workflow Administrator Student Reader Create Test Upload Students Register for Exam Assign Readers View Readings & Questions Finalize Scores Write & Submit Essay Publish Results Read & Score Essays View Results
Student View Results (not yet available)
Student View Results
Student View Results
Student View Results
Create Class imoat High-Level Workflow Administrator Student Reader Create Test Upload Students Register for Exam Assign Readers View Readings & Questions Finalize Scores Write & Submit Essay Publish Results Read & Score Essays Reporting View Results
Reporting Download Results
Reporting Download Results
Reporting Download Results
Reporting Download Results
Reporting Download Results (Excel)
Create Class imoat High-Level Workflow Administrator Student Reader Create Test Upload Students Register for Exam Assign Readers View Readings & Questions Finalize Scores Write & Submit Essay Publish Results Read & Score Essays Reporting View Results