Software evaluation criteria and terminology REPORT # : R07/0203 AUTHOR(S) : Diane Belyk & David Feist DATE : March 2002

Transcription

1 Athabasca University CENTRE FOR DISTANCE EDUCATION Online Software Evaluation Report TITLE : Software evaluation criteria and terminology REPORT # : R07/0203 AUTHOR(S) : Diane Belyk & David Feist DATE : March 2002 Abstract This report discusses issues of software selection and terminology, and it highlights the particular terms that are useful in evaluation studies such as covered by the current series. A framework for the evaluation of educational software is presented. Glossaries of terminology relating to online collaborative methods are appended, as defined by the American Society for Training & Development (ASTD), and CNET Networks. Product selection criteria Building effective online learning environments requires a change in the perception of knowledge and the roles and tools of learners, teachers, support staff and administrators. The task of selecting appropriate tools to facilitate online learning is complicated by the sheer volume of product choices, and funding constraints. Ideally, product selection should be based on rigorous investigation and evaluation of the ability of each product to meet criteria specific to the learning needs and objectives of its users. A series of categories and criteria is suggested for the evaluation of online collaborative tools in the development, delivery, and administration online learning (ASTD, 2001). 1) Cost (institutional and user): 1.1. System requirements (Open platform; platform-specific; server purchased vs. hosted) Bandwidth (modem, cable, ADSL, T-1, etc.) License fees (scaled per user) User software/hardware (webcam, etc.) Download/ installation. 2) Complexity (user focus): 2.1 Technical support (user manual; frequently asked questions; online and offline help). 2.2 Collaborative tools (Asynchronous , conferencing; Synchronous chat, audio- conferencing, whiteboard, virtual networking; Isosychronous desktop videoconferencing). 2.3 Usability (seamless technology; intuitivity; ease of use; navigation; consistency; stability; functionality). 3) Control: 3.1 Secured access (password protection; encryption; firewall). 3.2 Personalization. 3.3 Privacy (protection by parental/ teacher controls; banner/ pop-up ads).

2 2 4) Clarity: resolution, sound, size, layout, etc. 5) Common Technical Framework (CTF) - an ASTD classification scheme: 5.1 Interoperability. 5.2 Advanced Distributed Learning (ADL) partnership (protocol or standards: IEEE, ISO, SCORM, etc.). 5.3 Scalability 5.4 Platform. 5.5 Integration. 5.6 File-sharing. 6) Features 6.1 Learner tools (search; references/url links; glossary; spell-checking; formatting; attachments; upload/download files; self-assessment; grades). 6.2 Teacher tools (content development; student assessment; student management; student tracking) 6.3 Administrator tools (registration; report generation) Weighing product selection factors Before selecting specific online educational products, educators and institutions should consider each of the above factors, balancing as far as possible the merits of specific products against the general features of the educational programme and system. The selection of a specific product requires attention to: software reliability; availability of technical support by the institution to the users; availability of support by the software supplier to the institution and the users; cost to the institution e.g. is full ownership (i.e. local server support) available for the software? cost and/or burden to the learner. Specific questions arising in product selection are as follows. 1) If the success of the course or programme depends heavily on the software tool, can the institution guarantee the software s longevity? In the current state of development of online collaborative software, this is possibly the most difficult variable for an institution to control. For all of its efforts to identify appropriate and reliable software sources, the institution should identify back-up products through evaluation studies such as those reported in this series. 2) Is the institution prepared to provide the necessary support for the selected tools? [N.B. Athabasca University s Centre for Distance Education (source of the current report series) provides online support mechanisms for its selected collaborative tools, in addition to the more general support provided by the University s main helpdesk (Baggaley, 2002).] 3) Will the learners derive sufficient learning benefits despite the costs imposed by the collaborative technologies? During the current, relatively early state of development of these technologies, evaluation studies such as reported in this series generate valuable discussion among teachers and students about the technologies merits and demerits in specific situations.

3 3 4) Will instructors use the technology to transform learning or direct learning? (Garrison, Anderson, & Archer, 2000; Sherry & Wilson, 1997; Shotsberger, 1997). The instructors will and ability to use the technologies is at least as essential to their successful adoption in online learning as the selection of the product itself. References BAGGALEY, J.P. (2002) Personal communication (Athabasca University, Alberta). GARRISON, D. R., ANDERSON, T., & ARCHER, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. Internet and Higher Education, 11(2), SHERRY, L., & WILSON, B. (1997). Transformative communication as a stimulus to Web innovation. In B. H. Khan (Ed.) Web-based instruction. Englewood Cliffs, NJ: Educational Technology Publications (pp ). SHOTSBERGER, P. G. (1997). Emerging roles for instructors and learners in the Web-based instruction classroom. In B. H. Khan (Ed.), Web-based instruction. Englewood Cliffs, NJ: Educational Technology Publications (pp ). APPENDIX I: Glossary of ASTD terminology (from the American Society for Training and Development s web site) ADL (Advanced Distributed Learning): initiative by the US Department of Defense to achieve interoperability across computer and Internet-based learning courseware through the development of a common technical framework. Applet: a small application. Application: software a user activates to work on a computer. Asynchronous learning: learning in which interaction between teachers and students occurs intermittently with a time delay. Audioconferencing: voice-only connection of more than two users. Backbone: a primary communication path connecting multiple users. Bandwidth: information carrying capacity of a communication channel. Bps (Bits per second): measurement of data transmission speed in a communications system. Browser: software that allows users to find and view information on the Internet.

4 4 Chat: real-time or relay text communication between online users. Collaborative tools: allow learner to work with others via , threaded discussions or chats. Convergence: the combination of digital information such as text, voice and video with their delivery mechanisms such as WebTV. Desktop videoconferencing: video-conferencing on a personal computer. Download: the electronic transferring or copying of a file from one computer to another. Echo cancellation: the process of eliminating the acoustic echo in a videoconferencing room. E-learning: covers a wide set of applications and processes such as Web-based learning, computerbased learning, virtual classrooms and digital collaboration. (electronic mail): messages sent from one computer user to another. End-to-end solution: a marketing term used by large e-learning suppliers. Meant to imply that their products and services will handle all aspects of e-learning. FAQ (Frequently asked questions): a file established for public discussion groups containing questions and answers that new users often ask. File server: a computer with a large storage device on a network used for storing files and software that can be shared by users on the network. Firewall: a method to give users access to the Internet while retaining internal network security. Fully interactive video (two-way interactive video): two sites interacting with audio and video as if they were located in the same place (co-located). Host: a network computer that can receive information from other computers. Hosting: outsourcing of the technology and commerce parts of a company's Internet-based learning system to an outside organization. Interoperability: the ability of hardware or software components to work together effectively. Java: an object-oriented programming language developed by Sun Microsystems. Java isn't dependent on specific hardware and can be launched within an HTML document or stand-alone. Java applet: a small Java program launched through a browser. JavaScript: a scripting language that is simpler than Java and can add interactivity to Web pages.

5 5 Learning platforms: internal or external sites often organized around tightly focused topics, which contain technologies (ranging from chat rooms to groupware) that enable users to submit and retrieve data. Navigation: finding one s way from page to page on the World Wide Web. Online community: a meeting place for learners on the Internet, designed to facilitate interaction and collaboration among people who share common interests and needs. Open platform: a computer and network design concept that dictates that all users will have the ability to access, create and publish information, as well as understand each other's information. Packet: a bundle of data transmitted over a network. Personalization: tailoring Web content to an individual user. Protocol: a formal set of standards, rules or formats for exchanging data that assures uniformity between computers and applications. Real-time communication: communication in which information is received at (or nearly at) the same instant as it is sent. Resolution: the clarity of the image on the video display screen. Scalability: the degree to which a computer application or component can be expanded in size, volume, or number of users served and continue to function properly. SCORM (Sharable Content Object Reference Model): a set of standards that, when applied to course content, produces small reusable learning objects. SCORM-compliant courseware elements can be easily merged with other compliant elements to produce a modular repository of training materials. Seamless technology: technology that is easy to use, intuitive in nature and is not the focus of the learning experience. Synchronous learning: a real-time, instructor led online learning event in which all participants are logged in at the same time and communicate directly with each other. Synergy: the dynamic energetic atmosphere created in an online class when participants interact and productively communicate with each other. System requirements: the technological conditions required to run a software application. Includes the operating system, programming language, database, hardware configuration, bandwidth, processing power, etc. T-1: high speed digital data channel that is a high-volume carrier of voice and/or other data. Teleconferencing: two-way electronic communication between two or more groups in separate locations using audio, video, and/or computer systems.

6 6 Usability: the measure of how effectively, efficiently and easily a person can navigate an interface, find information on it, and achieve specific goals. Videoconferencing: use of video and audio signals to link participants at different and remote locations. VoIP (voice over IP): digital voice transmission using the Internet Protocol, and avoiding the fees charged by telephone companies. Whiteboard: an electronic version of a dry-erase board that enables learners in a virtual classroom to view what an instructor, presenter, or fellow learner writes or draws. APPENDIX II Additional glossary terms (from web site of CNET Networks Inc.) ISDN (Integrated Services Digital Network): wide bandwidth digital transmission using the public switched telephone network capable of transferring 64 kilobits of digital data per second. POTS ( plain old telephone service ): logging on to the Internet using a regular modem. Resolution: a measure of graphics that is used to describe what a printer can print, a scanner can scan and a monitor can display. Monitor resolution refers to the number of pixels in the whole image. Common resolutions include 640 by 480 (VGA appropriate for a 14-inch monitor), 800 by 600 (appropriate for a 15-inch monitor) and 1024 by 768 (appropriate for a 17-inch monitor) and 1280 by The next report in this series will compare the merits of fully-featured and stand-alone conferencing products. N.B. Owing to the speed with which web addresses become outdated, online references are not cited in these summary reports. They are available, together with updates to the current report, at the Athabasca University software evaluation site: Italicised product names in this report can be assumed to be registered trademarks. JPB (Ed., Evaluation Reports).