Evaluation of Adaptive Web Systems

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Evaluation of Adaptive Web Systems"

Transcription

1 INFSCI 2480: Adaptive Web Systems Evaluation of Adaptive Web Systems Peter Brusilovsky School of Information Sciences University of Pittsburgh, USA With slides of Michael Yudelson, CMU Three Types of Evaluation Formal Look at prediction of performance, complexity and learnability Task Analysis, GOMS Analytical Without users Heuristic Evaluation, Cognitive Walkthrough Empirical

2 Empirical Evaluation of Adaptive Systems Empirical evaluation assessing by observing in experiments, rather than appraising theoretical validity Adaptive systems created for practical use, hence empirical methods should be used for their evaluation. Empirical Evaluation Best for objective assessment of design Best for broadest range of usability and performance problems Requirements to do empirical evaluation 1. Careful planning 2. Careful execution 3. Users must represent actual user population 4. Must have expense account pay users

3 Empirical Methods Focus groups Interviews Questionnaires Observations Passive Instrumented Think-aloud Contextual enquiry Expert reviews Cooperative evaluation Participative evaluation Usability testing Controlled experiments Evaluation in a Broader Context Given the abundance of existing techniques and their implementations, the evaluation of the already suggested approaches is as important as coming up with novel ones It is important to know whether a particular adaptive technique works, what is the context of its most potent application, and what is the scale of the adaptation effect it produces Lack of evaluation data and/or inability to generalize from it, is one of the major barriers on the way of adaptive methods to become mainstream technology Evaluation is an essential part of research and practical application of the adaptive technologies

4 Quick Example Evaluation example System A non-adaptive [web] system System B adaptive version of system A Assess (evaluate) user satisfaction with working in system A or B (we assume users like B better) System interface (A/B) is controlled factor, independent variable User satisfaction measured, observable, dependant variable Agenda Variables Factors controlled (interface, time, etc) Quantitative vs. qualitative Subjective vs. objective User-related vs. system-related Study Designs Useful Tips Examples

5 Quantitative vs. Qualitative Variables Quantitative measured on numeric scale (ordinal, interval, rational, etc) Speed, number of problems solved, number of pages read Qualitative (categorical) no natural sense of ordering (nominal scale) Navigational pattern (sequential, cyclic, etc) Learning style Subjective vs. Objective Variables Subjective involve personal opinion explicitly/implicitly expressed User satisfaction, perceived responsiveness of the system Objective involve unambiguous manifestation of phenomena Time spent, no. errors found, prediction quality Objectively measured variable may express subjective theory navigational pattern

6 User and System Variables User-related describe user characteristics Demographics (gender, age, SES, etc.) Observed performance (success rate, number of pages viewed, etc.) Typically collected using questionnaires System-related describe system s operation Average response time, no. concurrent queries, perceived response delay (etc.) Typically collected using system logs Quick Test Give example of Quantitative variable, qualitative variable Subjective variable, objective variable User-related variable, system-related variable

7 Design Single condition vs. multiple conditions Between-subject, within-subject, and crossed Time-constrained vs. longitudinal Holistic vs. layered Single vs. Multiple Conditions Condition setup of the environment/system Multiple conditions Non-adaptive vs. adaptive system comparison (2) Non-adaptive vs. adaptation 1 and adaptation 2 (3) Single condition Correlation of system use with success rate Compare time spent reading to understanding

8 Between- or within-subject Between-subject Each subject is only exposed to one condition Comparison is drawn between condition groups Control condition system without any studied enhancements (adaptation) control group Within-subject Subjects exposed to several (all) conditions E.g. both adaptive and non-adaptive system are available at the same time, user preference is studied Crossed design Subjects are exposed to all withinconditions Ordering is varied to compensate for practice effect. E.g. adaptive vs. non adaptive Group 1: Non-Adaptive, Non-Adaptive (control) Group 2: Non-Adaptive, Adaptive Group 3: Adaptive, Non-Adaptive Group 4: Adaptive, Adaptive

9 Time-constrained vs. Longitudinal Time-constrained limited time for interacting with system Best for limiting uncontrolled factors Subject might have a bad day and not perform up to his true ability. Longitudinal time is not limited and/or regulated Less probe to the bad day effect Prone to uncontrolled nuisance factors Nuisance factor Intensive reading in addition to using an AHS Holistic vs. Layered Holistic (black-box) System is studied as a whole w/o attention to internal implementation Layered Adaptive systems are often multi-component, components are quite independent Study of the whole with special attention to each component (or layer) E.g. UMS and adaptation layers in the AS

10 Performance Evaluation Benchmark test Consistency / reproducibility of results Capacity planning test How far the system will scale under certain conditions Soak test Longitudinal (with control condition), for performance degradation, memory leaks over time Peak-rest test Load ramp-up and decrement Additional References Brusilovsky, P., Karagiannidis, C., & Sampson, D. (2004). Layered evaluation of adaptive learning systems. International Journal of Continuing Engineering Education and Lifelong Learning, 14(4/5), Carmichael, D. J., Kay, J., & Kummerfeld, B. (2005). Consistent modelling of users, devices and sensors in a ubiquitous computing environment. User Modeling and User-Adapted Interaction, 15(3-4), Chin, D. N. (2001). Empirical Evaluation of User Models and User-Adapted Systems. User Modeling and User-Adapted Interaction, 11(1-2), Kobsa, A., & Fink, J. (2006). An LDAP-based user modeling server and its evaluation. User Modeling and User-Adapted Interaction, 16(2), Yudelson, M., & Brusilovsky, P. (2008). Adaptive Link Annotation in Distributed Hypermedia Systems: The Evaluation of a Service-Based Approach. In: W. Nejdl, J. Kay, P. Pu & E. Herder (Eds.), 5th International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH 2008) (pp ), Hannover, Germany, July 29 - August 1, 2008: Springer, Berlin/Heidelberg Yudelson, M., Brusilovsky, P., & Zadorozhny, V. (2007). A User Modeling Server for Contemporary Adaptive Hypermedia: An Evaluation of the Push Approach to Evidence Propagation. In: C. Conati, K. McCoy & G. Paliouras (Eds.), 11th International Conference on User Modeling (UM 2007) (pp ), Corfu, Greece, June 25-29, 2007: Springer, Berlin/Heidelberg

A Database of Empirical Evaluations of Adaptive Systems

A Database of Empirical Evaluations of Adaptive Systems A Database of Empirical Evaluations of Adaptive Systems Stephan Weibelzahl Gerhard Weber Pedagogical University Freiburg Kunzenweg 21 79117 Freiburg, Germany [weibelza,weber]@ph-freiburg.de The development

More information

Conference Navigator 2.0: Community-Based Recommendation for Academic Conferences

Conference Navigator 2.0: Community-Based Recommendation for Academic Conferences Conference Navigator 2.0: Community-Based Recommendation for Academic Conferences Chirayu Wongchokprasitti chw20@pitt.edu Peter Brusilovsky peterb@pitt.edu Denis Para dap89@pitt.edu ABSTRACT As the sheer

More information

Rosa Arruabarrena, Tomás A. Pérez, J. López-Cuadrado, J. Gutiérrez, and J.A. Vadillo

Rosa Arruabarrena, Tomás A. Pérez, J. López-Cuadrado, J. Gutiérrez, and J.A. Vadillo On Evaluating Adaptive Systems for Education Rosa Arruabarrena, Tomás A. Pérez, J. López-Cuadrado, J. Gutiérrez, and J.A. Vadillo Dpto. de Lenguajes y Sistemas Informáticos, Universidad del País Vasco

More information

Appendix B Checklist for the Empirical Cycle

Appendix B Checklist for the Empirical Cycle Appendix B Checklist for the Empirical Cycle This checklist can be used to design your research, write a report about it (internal report, published paper, or thesis), and read a research report written

More information

Adaptive Navigation for Self-Assessment Quizzes

Adaptive Navigation for Self-Assessment Quizzes Adaptive Navigation for Self-Assessment Quizzes Sergey Sosnovsky 1 University of Pittsburgh, School of Information Sciences, 135, North Bellefield Avenue, Pittsburgh, PA, 15260, USA sas15@pitt.edu Abstract.

More information

A Web-based Framework for User-centred Evaluation of End-User Experience in Adaptive and Personalized E-Learning Systems

A Web-based Framework for User-centred Evaluation of End-User Experience in Adaptive and Personalized E-Learning Systems 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology A Web-based Framework for User-centred Evaluation of End-User Experience in Adaptive and Personalized E-Learning

More information

Performance Evaluation of a Privacy-Enhancing Framework for Personalized Websites

Performance Evaluation of a Privacy-Enhancing Framework for Personalized Websites Performance Evaluation of a Privacy-Enhancing Framework for Personalized Websites Yang Wang and Alfred Kobsa Donald Bren School of Information and Computer Sciences University of California, Irvine, U.S.A.

More information

INFSCI 2955 Adaptive Web Systems. Systems

INFSCI 2955 Adaptive Web Systems. Systems INFSCI 2955 Adaptive Web Systems Session 1-2: Adaptive E-Learning Systems Peter Brusilovsky School of Information Sciences University of Pittsburgh, USA http://www.sis.pitt.edu/~peterb/2955-092/ Overview

More information

Non-Invasive Adaptation Service for Web-based Content Management Systems

Non-Invasive Adaptation Service for Web-based Content Management Systems Non-Invasive Adaptation Service for Web-based Content Management Systems Kevin Koidl, Owen Conlan, Vincent Wade Centre for Next Generation Localisation and Knowledge and Data Engineering Group, Trinity

More information

Research Methods & Experimental Design

Research Methods & Experimental Design Research Methods & Experimental Design 16.422 Human Supervisory Control April 2004 Research Methods Qualitative vs. quantitative Understanding the relationship between objectives (research question) and

More information

Predicting Customer Models Using Behavior-Based Features in Shops

Predicting Customer Models Using Behavior-Based Features in Shops Predicting Customer Models Using Behavior-Based Features in Shops Junichiro Mori 1, Yutaka Matsuo 1, Hitoshi Koshiba 2, Kenro Aihara 2, and Hideaki Takeda 2 1 The University of Tokyo, Tokyo, Japan jmori@ipr-ctr.t.u-tokyo.ac.jp,

More information

Qualitative data acquisition methods (e.g. Interviews and observations) -.

Qualitative data acquisition methods (e.g. Interviews and observations) -. Qualitative data acquisition methods (e.g. Interviews and observations) -. Qualitative data acquisition methods (e.g. Interviews and observations) ( version 0.9, 1/4/05 ) Code: data-quali Daniel K. Schneider,

More information

Single and Multiple-Case Study Designs IS493

Single and Multiple-Case Study Designs IS493 1 2 Research Strategies Basic oppositions Survey research versus Case study quantitative versus qualitative The whole gamut Experiment Survey Archival analysis Historical research Case study 3 Basic Conditions

More information

Direct Integration: Training Software Developers and Designers to Conduct Usability Evaluations

Direct Integration: Training Software Developers and Designers to Conduct Usability Evaluations Direct Integration: Training Software Developers and Designers to Conduct Usability Evaluations Mikael B. Skov Jan Stage Presented by: Jan Stage Aalborg University Department of Computer Science jans@cs.aau.dk

More information

UI Design and Information Visualization. - Chaitra Chandrasekhar

UI Design and Information Visualization. - Chaitra Chandrasekhar UI Design and Information Visualization - Chaitra Chandrasekhar Presentation 1 Friday, 25 th February 2005 Agenda 1. 1. Motivation 2. 2. Role of of User 3. 3. User Capabilities 4. 4. Data Perspective 5.

More information

Adaptive Web Interfaces

Adaptive Web Interfaces Adaptive Web Interfaces Presented by : Anwesha Deb Spring 2010 CS6125- WHIM ad2773@columbia.edu Introduction Webpages have evolved from just plain text to hypertext with inclusion of media i.e. has become

More information

User Interface Design

User Interface Design User Interface Design Winter term 2005/2006 Thursdays, 14-16 c.t., Raum 228 Prof. Dr. Antonio Krüger Institut für Geoinformatik Universität Münster 20. Februar 06 IfGi Universität Münster User Interface

More information

A CONCEPT-MAP MODELING APPROACH TO BUSINESS PLANNING IN A COMPUTER ENVIRONMENT

A CONCEPT-MAP MODELING APPROACH TO BUSINESS PLANNING IN A COMPUTER ENVIRONMENT Concept Mapping: Connecting Educators Proc. of the Third Int. Conference on Concept Mapping Tallinn, Estonia & Helsinki, Finland 2008 A CONCEPT-MAP MODELING APPROACH TO BUSINESS PLANNING IN A COMPUTER

More information

B. 3 essay questions. Samples of potential questions are available in part IV. This list is not exhaustive it is just a sample.

B. 3 essay questions. Samples of potential questions are available in part IV. This list is not exhaustive it is just a sample. IS482/682 Information for First Test I. What is the structure of the test? A. 20-25 multiple-choice questions. B. 3 essay questions. Samples of potential questions are available in part IV. This list is

More information

Evaluation of Adaptive Course Construction Toolkit (ACCT)

Evaluation of Adaptive Course Construction Toolkit (ACCT) Evaluation of Adaptive Course Construction Toolkit (ACCT) Declan Dagger, Vincent P. Wade Knowledge and Data Engineering Group, Department of Computer Science, Trinity College Dublin, Ireland {Declan.Dagger,

More information

Usability Testing and Evaluation of Questionnaire Re-design Work

Usability Testing and Evaluation of Questionnaire Re-design Work Usability Testing and Evaluation of Questionnaire Re-design Work Petri Godenhjelm 1, Jussi Rouhunkoski 1 Statistics Finland, Työpajankatu 13, 00022, Tilastokeskus,Finland Abstract Statistics Finland had

More information

An Approach to Adaptivity and Collaboration Support in a Web-Based Learning Environment

An Approach to Adaptivity and Collaboration Support in a Web-Based Learning Environment An Approach to Adaptivity and Collaboration Support in a Web-Based Learning Environment Martina Holenko Dlab, Nataša Hoić-Božić University of Rijeka, Department of Informatics Key words: collaborative

More information

Course Catalog. www.airweb.org/academy

Course Catalog. www.airweb.org/academy www.airweb.org/academy Course Catalog 2015 Hosted by the Association for Institutional Research, Data and Decisions Academy courses provide self-paced, online professional development for institutional

More information

Evolution of Interests in the Learning Context Data Model

Evolution of Interests in the Learning Context Data Model Evolution of Interests in the Learning Context Data Model Hendrik Thüs, Mohamed Amine Chatti, Roman Brandt, Ulrik Schroeder Informatik 9 (Learning Technologies), RWTH Aachen University, Aachen, Germany

More information

Conceptualization And Measurement

Conceptualization And Measurement 1 Research Methods Chapter 4 Conceptualization And Measurement Prof. Kathrin Zippel 2 Agenda What do we need concepts for? What does measurement mean in social sciences? What do we mean by levels of measurement?

More information

Web User Search Pattern Analysis for Modeling Query Topic Changes

Web User Search Pattern Analysis for Modeling Query Topic Changes Web User Search Pattern Analysis for Modeling Query Topic Changes Sheng Han, Ayse Goker, and Daqing He School of Computer and Mathematical Sciences, The Robert Gordon University Aberdeen AB25 1HG, Scotland

More information

Adaptive Hypermedia Services for E-Learning

Adaptive Hypermedia Services for E-Learning Adaptive Hypermedia Services for E-Learning Peter Brusilovsky, Sergey Sosnovsky, Michael Yudelson School of Information Sciences University of Pittsburgh, Pittsburgh PA 15260, USA {peterb, sas15, mvy3}@pitt.edu

More information

Dr Roy Ramphal (UNISA)

Dr Roy Ramphal (UNISA) Dr Roy Ramphal (UNISA) It is a process that involves obtaining scientific knowledge by means of various objective methods and procedures. Objective means that these methods and procedures do not rely on

More information

Federated Query Processing over Linked Data

Federated Query Processing over Linked Data An Evaluation of Approaches to Federated Query Processing over Linked Data Peter Haase, Tobias Mathäß, Michael Ziller fluid Operations AG, Walldorf, Germany i-semantics, Graz, Austria September 1, 2010

More information

evaluation techniques

evaluation techniques Evaluation Techniques chapter 9 evaluation techniques Evaluation tests usability and functionality of system occurs in laboratory, field and/or in collaboration with users evaluates both design and implementation

More information

Usability Measurement of Malaysia Online News Websites

Usability Measurement of Malaysia Online News Websites IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.5, May 2008 159 Usability Measurement of Malaysia Online News Websites RUSLI ABDULLAH Faculty of Computer Science and Information

More information

Usability Evaluation with Users CMPT 281

Usability Evaluation with Users CMPT 281 Usability Evaluation with Users CMPT 281 Outline Usability review Observational methods Interview methods Questionnaire methods Usability ISO 9241-11: The extent to which a product can be used by specified

More information

Learning SQL Programming with Interactive Tools: From Integration to Personalization

Learning SQL Programming with Interactive Tools: From Integration to Personalization Learning SQL Programming with Interactive Tools: From Integration to Personalization P. BRUSILOVSKY, S. SOSNOVSKY, M. YUDELSON, D.H. LEE, V. ZADOROZHNY, and X. ZHOU University of Pittsburgh, Pittsburgh,

More information

Complementary Research Methods

Complementary Research Methods Complementary Research Methods Qualitative and Quantitative Methodologies http://www.gla.ac.uk/t4/education/files/scre/research.pdf Research Methodologies A continuum rather than either/or Qualitative

More information

Introduction to Statistics

Introduction to Statistics 1 Introduction to Statistics LEARNING OBJECTIVES After reading this chapter, you should be able to: 1. Distinguish between descriptive and inferential statistics. 2. Explain how samples and populations,

More information

Usability Test Plan Docutek ERes v. 5.0.05 University of Texas Libraries

Usability Test Plan Docutek ERes v. 5.0.05 University of Texas Libraries Usability Test Plan Docutek ERes v. 5.0.05 University of Texas Libraries Prepared for: INF 385p Usability (Dr. Randolph Bias) Prepared by: Laura Bright, Rachael Gilg and Heather Nodler Date: March 24,

More information

Methodology for Creating Adaptive Online Courses Using Business Intelligence

Methodology for Creating Adaptive Online Courses Using Business Intelligence Methodology for Creating Adaptive Online Courses Using Business Intelligence Despotović, S., Marijana; Bogdanović, M., Zorica; and Barać, M., Dušan Abstract This paper describes methodology for creating

More information

Techniques for Analyzing Empirical Visualization Experiments Through Visual Methods

Techniques for Analyzing Empirical Visualization Experiments Through Visual Methods Techniques for Analyzing Empirical Visualization Experiments Through Visual Methods Tanja Blascheck and Thomas Ertl Institut for Visualization and Interactive Systems, University of Stuttgart Universitätsstraße

More information

Collective Information Practice: Exploring Privacy and Security as Social and Cultural Phenomena

Collective Information Practice: Exploring Privacy and Security as Social and Cultural Phenomena Collective Information Practice: Exploring Privacy and Security as Social and Cultural Phenomena Paul Dourish and Ken Anderson Presented by: Monika Akbar Overview Privacy & Security in HCI 3 parallel approach

More information

THE UNIVERSITY OF EDINBURGH. PROGRAMME SPECIFICATION M.A. Honours in Psychology and Business Studies1

THE UNIVERSITY OF EDINBURGH. PROGRAMME SPECIFICATION M.A. Honours in Psychology and Business Studies1 THE UNIVERSITY OF EDINBURGH PROGRAMME SPECIFICATION M.A. Honours in Psychology and Business Studies1 1) Awarding Institution: University of Edinburgh 2) Teaching Institution: University of Edinburgh 3)

More information

Evaluating Usability of E-Learning Systems in Universities

Evaluating Usability of E-Learning Systems in Universities Evaluating Usability of E-Learning Systems in Universities Nicholas Kipkurui Kiget Department of Computer Science Masinde Muliro University of Science and Technology Kakamega, Kenya Professor G. Wanyembi,

More information

Cloud Computing: A Comparison Between Educational Technology Experts' and Information Professionals' Perspectives

Cloud Computing: A Comparison Between Educational Technology Experts' and Information Professionals' Perspectives Noa Aharony 1 Cloud Computing: A Comparison Between Educational Technology Experts' and Information Professionals' Perspectives Noa Aharony Department of Information Science, Bar-Ilan University Noa.aharony@biu.ac.il

More information

User Profiling and Privacy Protection for a Web Service oriented Semantic Web

User Profiling and Privacy Protection for a Web Service oriented Semantic Web User Profiling and Privacy Protection for a Web Service oriented Semantic Web Nicola Henze and Daniel Krause Distributed Systems Institute, Semantic Web Group, University of Hannover Appelstraße 4, 30167

More information

Usability Testing Methods An Overview

Usability Testing Methods An Overview Usability Testing Methods An Overview Usability Examples of user interfaces that you like or dislike Are you the only one? What about other audiences? Have your preferences changed over time? Are there

More information

Considering Learning Styles in Learning Management Systems: Investigating the Behavior of Students in an Online Course*

Considering Learning Styles in Learning Management Systems: Investigating the Behavior of Students in an Online Course* Considering Learning Styles in Learning Management Systems: Investigating the Behavior of Students in an Online Course* Sabine Graf Vienna University of Technology Women's Postgraduate College for Internet

More information

Usability Studies on Interactive Health Information Systems; Where Do We Stand?

Usability Studies on Interactive Health Information Systems; Where Do We Stand? ehealth Beyond the Horizon Get IT There S.K. Andersen et al. (Eds.) IOS Press, 2008 2008 Organizing Committee of MIE 2008. All rights reserved. 327 Usability Studies on Interactive Health Information Systems;

More information

Second Edition. Measuring. the User Experience. Collecting, Analyzing, and Presenting Usability Metrics TOM TULLIS BILL ALBERT

Second Edition. Measuring. the User Experience. Collecting, Analyzing, and Presenting Usability Metrics TOM TULLIS BILL ALBERT Second Edition Measuring the User Experience Collecting, Analyzing, and Presenting Usability Metrics TOM TULLIS BILL ALBERT \ \ PREFACE TO THE SECOND EDITION ACKNOWLEDGMENTS BIOGRAPHIES xili xv xvli CHAPTER

More information

PhD Qualifying Examination: Human-Computer Interaction

PhD Qualifying Examination: Human-Computer Interaction PhD Qualifying Examination: Human-Computer Interaction University of Wisconsin Madison, Department of Computer Sciences Spring 2014 Monday, February 3, 2014 General Instructions This exam has 7 numbered

More information

METHODOLOGIES FOR STUDIES OF PROGRAM VISUALIZATION

METHODOLOGIES FOR STUDIES OF PROGRAM VISUALIZATION Full paper ABSTRACT METHODOLOGIES FOR STUDIES OF PROGRAM VISUALIZATION Niko Myller & Roman Bednarik Department of Computer Science University of Joensuu PO Box 111, FI-80101 firstname.surname@cs.joensuu.fi

More information

An Iterative Usability Evaluation Procedure for Interactive Online Courses

An Iterative Usability Evaluation Procedure for Interactive Online Courses An Iterative Usability Evaluation Procedure for Interactive Online Courses by Laurie P. Dringus ABSTRACT The Internet and World Wide Web (W3) have afforded distance learners simple links to access information.

More information

Process of Strategic Marketing Decision Making in Small Business. Graham Jocumsen and Michael Gardiner, University of Southern Queensland.

Process of Strategic Marketing Decision Making in Small Business. Graham Jocumsen and Michael Gardiner, University of Southern Queensland. Process of Strategic Marketing Decision Making in Small Business Graham Jocumsen and Michael Gardiner, University of Southern Queensland Abstract The current inadequate level of understanding of how small

More information

Privacy Support for Users Beyond Transparency and Control Alfred Kobsa

Privacy Support for Users Beyond Transparency and Control Alfred Kobsa Privacy Support for Users Beyond Transparency and Control Alfred Kobsa Donald Bren School of Information and Computer Sciences University of California, Irvine 2012 White Paper on U.S. Consumer Privacy

More information

Research Statement Chaiwoo Lee As I study topics around user-centered product design and development, I always try to keep the broader implications of my research in mind. Research on user-centered design

More information

NORTHERN VIRGINIA COMMUNITY COLLEGE PSYCHOLOGY 211 - RESEARCH METHODOLOGY FOR THE BEHAVIORAL SCIENCES Dr. Rosalyn M.

NORTHERN VIRGINIA COMMUNITY COLLEGE PSYCHOLOGY 211 - RESEARCH METHODOLOGY FOR THE BEHAVIORAL SCIENCES Dr. Rosalyn M. NORTHERN VIRGINIA COMMUNITY COLLEGE PSYCHOLOGY 211 - RESEARCH METHODOLOGY FOR THE BEHAVIORAL SCIENCES Dr. Rosalyn M. King, Professor DETAILED TOPICAL OVERVIEW AND WORKING SYLLABUS CLASS 1: INTRODUCTIONS

More information

Integrating adaptive functionality in a LMS

Integrating adaptive functionality in a LMS Integrating adaptive functionality in a LMS Kees van der Sluijs 1 and Kai Michael Höver 2 1 Technische Universiteit Eindhoven, Eindhoven, The Netherlands 2 IMC Information Multimedia Communication AG,

More information

E-tailing: Analysis of Customer Preferences towards Online Shopping in Pune Region

E-tailing: Analysis of Customer Preferences towards Online Shopping in Pune Region E-tailing: Analysis of Customer Preferences towards Online Shopping in Pune Region *Prof. Vishal Raut Associate Professor-Trinity Institute of Management & Research Pune, Maharashtra, India **Prof. Dr.

More information

Testing Websites with Users

Testing Websites with Users 3 Testing Websites with Users 3 TESTING WEBSITES WITH USERS Better Practice Checklist Practical guides for effective use of new technologies in Government www.agimo.gov.au/checklists version 3, 2004 Introduction

More information

Basics of Causal Loop Diagrams: Structure & Behavior

Basics of Causal Loop Diagrams: Structure & Behavior Basics of Causal Loop Diagrams: Structure & Behavior Nathaniel Osgood (osgood@cs.usask.ca) Institute for System Science & Health Causal Loop Diagram Food Ingested Hunger Focuses on capturing causality

More information

Marketing Research Core Body Knowledge (MRCBOK ) Learning Objectives

Marketing Research Core Body Knowledge (MRCBOK ) Learning Objectives Fulfilling the core market research educational needs of individuals and companies worldwide Presented through a unique partnership between How to Contact Us: Phone: +1-706-542-3537 or 1-800-811-6640 (USA

More information

Developing Adaptive Internet Based Courses with the Authoring System NetCoach

Developing Adaptive Internet Based Courses with the Authoring System NetCoach Developing Adaptive Internet Based Courses with the Authoring System NetCoach Gerhard Weber, Hans-Christian Kuhl, and Stephan Weibelzahl Pedagogical University Freiburg, Germany [weber,kuhl,weibelza]@ph-freiburg.de

More information

Model-Based Design and Evaluation of Interactive Applications

Model-Based Design and Evaluation of Interactive Applications Model-Based Design and Evaluation of Fabio Paternò CNUCE-C.N.R Pisa, Italy Structure of the Tutorial n Introduction to Model-based Approaches n State of art in Task Models n ConcurTaskTrees n Task Models

More information

Running head: USABILITY ENGINEERING, COGNITIVE SCIENCE, AND HEALTHCARE INFORMATION SYSTEMS

Running head: USABILITY ENGINEERING, COGNITIVE SCIENCE, AND HEALTHCARE INFORMATION SYSTEMS Usability Engineering 1 Running head: USABILITY ENGINEERING, COGNITIVE SCIENCE, AND HEALTHCARE INFORMATION SYSTEMS Usability Engineering, Cognitive Science and Healthcare Information Systems Yong IL Choi

More information

Enterprise 2.0 in Action: Potentials for Improvement of Awareness Support in Enterprises

Enterprise 2.0 in Action: Potentials for Improvement of Awareness Support in Enterprises Enterprise 2.0 in Action: Potentials for Improvement of Awareness Support in Enterprises Hilda Tellioglu & Simon Diesenreiter Institute of Design and Assessment of Technology Vienna University of Technology,

More information

Web services to allow access for all in dotlrn

Web services to allow access for all in dotlrn Web services to allow access for all in dotlrn German Moreno 1, Olga C. Santos 2, Jesus G. Boticario 2, Ramon Fabregat 1 1 Institute of Informatics and Aplications (IIiA), Universidad de Girona, Spain,

More information

A Framework for Integrating Software Usability into Software Development Process

A Framework for Integrating Software Usability into Software Development Process A Framework for Integrating Software Usability into Software Development Process Hayat Dino AFRICOM Technologies, Addis Ababa, Ethiopia hayudb@gmail.com Rahel Bekele School of Information Science, Addis

More information

UserZoom s Unmoderated Online Remote Usability Testing

UserZoom s Unmoderated Online Remote Usability Testing Do you need help selling UserZoom's UX research solution to your Management Team? We're here to help! The following Business Case was created by one of our current customers in the Financial Industry 1.

More information

IKI80050T: Metodologi Penelitian

IKI80050T: Metodologi Penelitian IKI80050T: Metodologi Penelitian Research Methods & Design Zainal Hasibuan & Ruli Manurung Magister Teknologi Informasi Fasilkom UI Zainal Hasibuan & Ruli Manurung MTI Fasilkom UI IKI80050T 2009/2010 Ganjil

More information

College of Business & Economics Courses MANAGEMENT COURSES (250)

College of Business & Economics Courses MANAGEMENT COURSES (250) College of Business & Economics Courses MANAGEMENT COURSES (250) Field trips may be required for any of the courses. Students should check with the Associate Dean, College of Business and Economics, to

More information

Designing an Adaptive Virtual Guide for Web Applications

Designing an Adaptive Virtual Guide for Web Applications 6th ERCIM Workshop "User Interfaces for All" Long Paper Designing an Adaptive Virtual Guide for Web Applications Luisa Marucci, Fabio Paternò CNUCE-C.N.R. Via V.Alfieri 1, 56010 Ghezzano - Pisa, Italy

More information

What you will study on the MPH Master of Public Health (online)

What you will study on the MPH Master of Public Health (online) Public Health Foundations Core This module builds upon the nine core competencies of Public Health Practice set out by the Faculty of Public Health UK. The English NHS will form the central empirical case

More information

UX RESEARCH & TESTING TECHNIQUES

UX RESEARCH & TESTING TECHNIQUES UX RESEARCH & TESTING TECHNIQUES Find the right technique for the job Fatima Kanji December 2015 WWW.AKENDI.COM WWW.AKENDI.CO.UK 30 Duncan St, Suite 203 Toronto, ON M5V 2C3 Canada +1 416.855.3367 375 Richmond

More information

The paradigm of navigation in virtual reality covers a broad spectrum of research topics related to spatial memory, presence and narrative!

The paradigm of navigation in virtual reality covers a broad spectrum of research topics related to spatial memory, presence and narrative! Introduction! The paradigm of navigation in virtual reality covers a broad spectrum of research topics related to spatial memory, presence and narrative! Navigation mode in a virtual reality environment

More information

LBPerf: An Open Toolkit to Empirically Evaluate the Quality of Service of Middleware Load Balancing Services

LBPerf: An Open Toolkit to Empirically Evaluate the Quality of Service of Middleware Load Balancing Services LBPerf: An Open Toolkit to Empirically Evaluate the Quality of Service of Middleware Load Balancing Services Ossama Othman Jaiganesh Balasubramanian Dr. Douglas C. Schmidt {jai, ossama, schmidt}@dre.vanderbilt.edu

More information

A Human Cognitive Processing Perspective in Designing E-Commerce Checkout Processes

A Human Cognitive Processing Perspective in Designing E-Commerce Checkout Processes A Human Cognitive Processing Perspective in Designing E-Commerce Checkout Processes Marios Belk 1, Panagiotis Germanakos 1,2, Argyris Constantinides 3, George Samaras 1 1 Department of Computer Science,

More information

Assignment # 4 Summing up the Course

Assignment # 4 Summing up the Course Assignment # 4 Summing up the Course Usability form an Industrial Prospective (DV1301) Ch. M Nadeem Faisal Muhammad Asim Farrukh Sahar Rana Asif Sattar 790102-P373 811115-P373 800102-P373 840715-P177 mnmi07@student.bth.se

More information

Understanding and Supporting Intersubjective Meaning Making in Socio-Technical Systems: A Cognitive Psychology Perspective

Understanding and Supporting Intersubjective Meaning Making in Socio-Technical Systems: A Cognitive Psychology Perspective Understanding and Supporting Intersubjective Meaning Making in Socio-Technical Systems: A Cognitive Psychology Perspective Sebastian Dennerlein Institute for Psychology, University of Graz, Universitätsplatz

More information

Introduction into Sparks of the Learning Analytics Future

Introduction into Sparks of the Learning Analytics Future Introduction into Sparks of the Learning Analytics Future Mykola Pechenizkiy Department of Computer Science Eindhoven University of Technology (TU/e) The Netherlands m.pechenizkiy@tue.nl Dragan Gašević

More information

Lessons Learned from Scaling Up a Web-Based Intelligent Tutoring System

Lessons Learned from Scaling Up a Web-Based Intelligent Tutoring System Lessons Learned from Scaling Up a Web-Based Intelligent Tutoring System Jozsef Patvarczki, Shane F. Almeida, Joseph E. Beck, and Neil T. Heffernan Worcester Polytechnic Institute Department of Computer

More information

Institute for Culture and Society Student Research Program 2015 Project Lists

Institute for Culture and Society Student Research Program 2015 Project Lists Institute for Culture and Society Student Research Program 2015 Project Lists Project 12: Exploring visual tools for visualizing and communicating data in innovative ways: Safe and Well Online... 2 Project

More information

Fan Fu. Usability Testing of Cloud File Storage Systems. A Master s Paper for the M.S. in I.S. degree. April, 2013. 70 pages. Advisor: Robert Capra

Fan Fu. Usability Testing of Cloud File Storage Systems. A Master s Paper for the M.S. in I.S. degree. April, 2013. 70 pages. Advisor: Robert Capra Fan Fu. Usability Testing of Cloud File Storage Systems. A Master s Paper for the M.S. in I.S. degree. April, 2013. 70 pages. Advisor: Robert Capra This paper presents the results of a usability test involving

More information

A Framework of Context-Sensitive Visualization for User-Centered Interactive Systems

A Framework of Context-Sensitive Visualization for User-Centered Interactive Systems Proceedings of 10 th International Conference on User Modeling, pp423-427 Edinburgh, UK, July 24-29, 2005. Springer-Verlag Berlin Heidelberg 2005 A Framework of Context-Sensitive Visualization for User-Centered

More information

Considering the Cultural Issues of Web Design in Implementing Web-Based E-Commerce for International Customers

Considering the Cultural Issues of Web Design in Implementing Web-Based E-Commerce for International Customers Considering the Cultural Issues of Web Design in Implementing Web-Based E-Commerce for International Customers Kyeong. S. Kang The First International Conference on Electronic Business, Faculty of Information

More information

Providing Adaptive Courses in Learning Management Systems with Respect to Learning Styles *

Providing Adaptive Courses in Learning Management Systems with Respect to Learning Styles * Providing Adaptive Courses in Learning Management Systems with Respect to Learning Styles * Sabine Graf Vienna University of Technology Women's Postgraduate College for Internet Technologies Vienna, Austria

More information

Levels of Software Testing. Functional Testing

Levels of Software Testing. Functional Testing Levels of Software Testing There are different levels during the process of Testing. In this chapter a brief description is provided about these levels. Levels of testing include the different methodologies

More information

DOCTOR OF PHILOSOPHY DEGREE. Educational Leadership Doctor of Philosophy Degree Major Course Requirements. EDU721 (3.

DOCTOR OF PHILOSOPHY DEGREE. Educational Leadership Doctor of Philosophy Degree Major Course Requirements. EDU721 (3. DOCTOR OF PHILOSOPHY DEGREE Educational Leadership Doctor of Philosophy Degree Major Course Requirements EDU710 (3.0 credit hours) Ethical and Legal Issues in Education/Leadership This course is an intensive

More information

USING INTERNET-BASED SURVEYS FOR ACADEMIC RESEARCH: OPPORTUNITIES AND PROBLEMS

USING INTERNET-BASED SURVEYS FOR ACADEMIC RESEARCH: OPPORTUNITIES AND PROBLEMS USING INTERNET-BASED SURVEYS FOR ACADEMIC RESEARCH: OPPORTUNITIES AND PROBLEMS Narcyz Roztocki, State University of New York at New Paltz Abstract This paper presents a preliminary investigation of the

More information

EVALUATING THE INTELLIGENT FEATURES OF A WEB-BASED INTELLIGENT COMPUTER ASSISTED LANGUAGE LEARNING SYSTEM

EVALUATING THE INTELLIGENT FEATURES OF A WEB-BASED INTELLIGENT COMPUTER ASSISTED LANGUAGE LEARNING SYSTEM EVALUATING THE INTELLIGENT FEATURES OF A WEB-BASED INTELLIGENT COMPUTER ASSISTED LANGUAGE LEARNING SYSTEM VICTORIA TSIRIGA, and MARIA VIRVOU Department of Informatics, University of Piraeus, 80, Karaoli

More information

Webpage Usability Evaluation and Optimization Design: A case study

Webpage Usability Evaluation and Optimization Design: A case study Webpage Usability Evaluation and Optimization Design: A case study Yanmin Xue, Yefei Shi School of Art and Design, Xi an University of Technology, Xi an, Shaanxi,CHINA Abstract: Webpage usability research

More information

HCI & web usability testing

HCI & web usability testing Four main threads in the program Ethical, legal, social issues in computing IS Application areas: information architecture human-computer interaction CS Application areas: Usability Testing artificial

More information

DATA ANALYSIS, INTERPRETATION AND PRESENTATION

DATA ANALYSIS, INTERPRETATION AND PRESENTATION DATA ANALYSIS, INTERPRETATION AND PRESENTATION OVERVIEW Qualitative and quantitative Simple quantitative analysis Simple qualitative analysis Tools to support data analysis Theoretical frameworks: grounded

More information

Strategies for Evaluating Software Usability

Strategies for Evaluating Software Usability Strategies for Evaluating Software Usability Ronan Fitzpatrick Department of Mathematics, Statistics and Computer Science, Dublin Institute of Technology, Kevin Street, Dublin 8, Ireland. Tel: +353 (1)

More information

GLOSSARY OF EVALUATION TERMS

GLOSSARY OF EVALUATION TERMS Planning and Performance Management Unit Office of the Director of U.S. Foreign Assistance Final Version: March 25, 2009 INTRODUCTION This Glossary of Evaluation and Related Terms was jointly prepared

More information

Defining an agenda for learning analytics

Defining an agenda for learning analytics Defining an agenda for learning analytics Associate Professor Cathy Gunn Faculty of Education The University of Auckland Learning analytics is an emergent field of research in educational technology where

More information

Merging learner performance with browsing behavior in video lectures

Merging learner performance with browsing behavior in video lectures Merging learner performance with browsing behavior in video lectures Konstantinos Chorianopoulos Department of Informatics Ionian University Corfu, GR-49100 Greece choko@ionio.gr Michail N. Giannakos Department

More information

Chapter 14: Usability testing and field studies. The aims: Usability testing

Chapter 14: Usability testing and field studies. The aims: Usability testing Chapter 14: Usability testing and field studies The aims: Explain how to do usability testing through examples Outline the basics of experimental design Discuss the methods used in usability testing Discuss

More information

The aims. Testing & modeling users. Experiments, user testing & usability testing. User testing is not research

The aims. Testing & modeling users. Experiments, user testing & usability testing. User testing is not research The aims Testing & modeling users Describe how to do user testing. Discuss the differences between user testing, usability testing and research experiments. Discuss the role of user testing in usability

More information

RAVEN, Network Security and Health for the Enterprise

RAVEN, Network Security and Health for the Enterprise RAVEN, Network Security and Health for the Enterprise The Promia RAVEN is a hardened Security Information and Event Management (SIEM) solution further providing network health, and interactive visualizations

More information

Performance Monitoring of Web-Based IT Systems

Performance Monitoring of Web-Based IT Systems Performance Monitoring of Web-Based IT Systems Master of Science Thesis in Software Engineering and Management NAVEED QAMAR University of Gothenburg Chalmers University of Technology Department of Computer

More information

Announcements. Project status demo in class

Announcements. Project status demo in class Web Design cs465 Announcements Project status demo in class Why? You will likely be involved in Web design You have many of the skills necessary Understand similarities and differences between GUI design

More information

Open-EMR Usability Evaluation Report Clinical Reporting and Patient Portal

Open-EMR Usability Evaluation Report Clinical Reporting and Patient Portal Open-EMR Usability Evaluation Report Clinical Reporting and Patient Portal By Kollu Ravi And Michael Owino Spring 2013 Introduction Open-EMR is a freely available Electronic Medical Records software application

More information