Software Engineering Issues for Ubiquitous Computing

Transcription

1 Software Engineering Issues for Ubiquitous Computing Gregory D. Abowd College of Computing & GVU Center Georgia Institute of Technology Atlanta, GA USA abowdcc.gatech.edu ABSTRACT In the last decade, we have experienced the advent of the paradigm of ubiquitous computing, with the goal of making computational services so pervasive throughout an environment that they become transparent to the human user. Research in ubiquitous computing raises many challenging issues for computer science in general, but successful research in ubiquitous computing requires the deployment of applications that can survive everyday use, and this in itself presents a great software engineering challenge. In our experience, we have found three features common across many ubiquitous computing applications -transparent interfaces that provide appropriate alternatives to the desktopbound traditional graphical user interface, the ability to modify behavior of a application based on knowledge of its context of use, and the ability to capture live experiences for later recall. Building ubiquitous computing applications with these features raises software engineering problems in toolkit design, software structuring for separation of concerns and component integration. We will clarify these problems and discuss our approaches towards their solution. Keywords Ubiquitous computing, transparent interfaces, contextaware computing, automated capture, toolkit design, software structure, component integration 1 INTRODUCTION The history of computing is peppered with paradigm shifts in how the relationship between humans and computers is perceived. The most recent paradigm shift is ubiquitous computing, or ubicomp for short [27, 28, lo]. The ubicomp vision pushes computational services out of conventional desktop interfaces and into the environment in increasingly transparent forms. In this paper, we discuss the software engineering problems that Permission to make digital or hard copies of all or part of this work fol personal or classroom use is granted without fee provided that copies are oat made or distributed for profit or commercial advaotage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute lo lists. requires prior specific permission and/or a fee. ICSE 99 Los Angclcs CA Copyright ACM 1999 l /99/05...$5.00 arise in conducting computer-enhanced research toward this vision of future environments. Ubicomp research is experimental in nature [28]. Good research in the field should satisfy the following criteria: There should be a motivating application. In the words of Mark Weiser, applications are the whole point of ubiquitous computing [28]. The Gystem built should address some notion of scale. Scale in this case can refer to the physical spa+ covered, the number of people involved, the number and variety of devices supported or the amount of time over which an application is run. The system should be subjected to real and everyday use. This characteristic is especially important because of the next criteria. The use of the system should be evaluated to determine its impact on the user community. These last two criteria are very important, because together they present a serious robustness challenge in an uncommon computing domain. Ubiquitous computing applications attempt to invade our everyday lives. Significant evaluation can only occur when the novelty of some new service wears off and the new service is expected by its user population. This means the application must be robust enough to run continuously for long stretches of time. Whereas many business and safety critical systems also face this 24 by 7 constraint, it is not common for everyday computing tasks, and even less common for systems which use a variety of different devices. Ubicomp applications evolve organically. Even though they begin with a motivating application, it is often not clear up front the best way for the application to serve its intended user community. The best approach is often to prototype a solution rapidly, put it into place and solicit feedback from the user population. Constant evaluation of this sort often results in the need to modify the application, often without the luxury of 75

2 much downtime. This rapid prototyping model is not a new one to software engineering, but it brings particular challenges in ubicomp research that we will discuss later. Adhering to the experimental model of research, and taking an applications focus, we have built several ubicamp applications at Georgia Tech over the past four years.l We will discuss two projects in some detail in this paper, for they will help to illustrate the major contributions of this paper: l There are three general features that are shared across a wide variety of ubicomp applications. These features are the ability to provide transparent interfaces, the ability to automatically adapt the behavior of a program based on knowledge of the context of its use, and the ability to automate the capture of live experiences for later recall. l Rapid prototyping of ubicomp applications with the above three features requires advances in the toolkits or libraries made available for ubicomp programmers, specific software structuring to support the correct separation of concerns, and lightweight componentintegration techniques that support the lowest common denominator among a wide variety of devices and operating systems. These questions arise out of a number of years designing, implementing and evaluating a variety of ubiquitous computing applications at Georgia Tech. Overview of paper The thesis of this paper is that software engineering advances are required to support the research endeavors in ubiquitous computing. In Section 2 we will present two ubiquitous computing projects conducted at Georgia Tech -the Classroom 2000 project and the Cyberguide project. These projects will be used in Section 3 to help define the three common features of ubicomp applications -transparent interfaces, context-awareness and automated capture. We then categorize the important software engineering issues that must be met in order to support continued ubicomp research. These issues are toolkit design (Section 4), software structuring for separation of concerns (Section 5), and component integration(section 6). 2 TWO UBICOMP PROJECTS Much of our insight has been gained from practical experience building ubicomp applications. Since the concrete details of particular systems will help in expressing challenges raised later on, we will describe here two Specific details on our projects can be obtained at projects. The first project, Classroom 2000, is an experiment to determine the impact of ubiquitous computing in education and involves the instrumentation of a room. The second project, Cyberguide, is a suite of flexible mobile tour guide applications that cover an area ranging in size from a building to a campus. Classroom 2000 Classroom 2000 is an experiment to determine the impact of ubiquitous computing technology in education through instrumented classrooms [2, 6, 1, 41. In a classroom setting, there are many streams of information presented to students that are supplemented by discussions and visual demonstrations. The teacher typically writes on some public display, such as a whiteboard or overhead projector, to present the day s lesson. It is now common to present supplemental information in class via the World Wide Web. Additional dynamic teaching aids such as videos, physical demonstrations or computer-baaed simulations can also be used. Taken in combination, all of these information streams provide for a very information-intensive experience that is easily beyond a student s capabilities to record accurately, especially if the student is trying to pay attention and understand the lesson at the same time! Unfortunately, once the class has ended, the record from the lecture experience is usually a small reflection of what actually happened. One potential advantage of electronic media used in the classroom is the ability to preserve the record of class activity. In the Classroom 2000 project, we are exploring ways in which the preservation of class activity enhances the teaching and learning experience. The instrumented classroom, shown in Figure 1 makes it easy to capture what is going on during a normal lecture. In essence, the room is able to take notes on behalf of the student and the entire lecture experience is turned into a multimedia authoring session. Figure 2 shows a sample of the kind of notes that are provided automatically to the students within minutes of the conclusion of class. This particular class was an introductory software engineering course. Web pages and presented slides are all available and presented in a timeline. The timeline and lecturer annotations on slides can,be used to index into an audio or video recording of the lecture itself. A necessary pre-condition for the evaluation of Classroom 2000 is that it be perceived by a large user population as providing a reliable service over an extended period of time, minimally a 10 week quarter. The project began in July 1995 and the first quarter-long classroom experiment was in January Within one year, a purpose-built classroom was opened and has been operational for the past 18 months, supporting over 40 graduate and undergraduate classes in the College of Computing, Electrical and Computer Engineering and Mathematics. By the end of 1998, variations of the 76

3 initial prototype classroom environment will have been installed in 5 locations on Georgia Tech s campus and at single locations at three other universities in Georgia. This project has satisfied the difficult criteria for ubiquitous computing set out in Section 1, largely due to engineering successes. This level of success simply would not have been possible without engineering a software system to automate much of the preparation, live recording and post-production of class notes. The software system supporting Classroom 2000 is called Zen*, and we will discuss what features of Zen* lead to the current success of the project and which features need further research. Cyberguide When a traveller visits an unfamiliar location, it is useful to have some sort of information service to provide background about the location. A very effective information service is the human tour guide, who provides some organized overview of a area, but is often able to answer spontaneous questions outside of the prepared overview. The Cyberguide project was an attempt to replicate this human tour guide service through use of mobile and hand-held technology and ubiquitous positioning and communication services [3, 14, 13]. Over the course of two years, we built a suite of tour guide systems for various indoor and outdoor needs within and around the Georgia Tech campus. One example supports a community of users in pursuit of refreshment at neighborhood establishments around the Georgia Tech campus. This prototype, called CyBARguide, used a Newton MessagePad and a GPS receiver to cover approximately 12 square miles of midtown Atlanta, using multiple maps at varying levels of detail. The interface on the Newton is shown in Figures 3 and 4. Users driving around Atlanta can find out the location of establishments that satisfy certain requirements (special offers, free parking, good ambience). Deciding on a particular location results in an interactive map that provides directions to the establishment. In addition, after visiting an establishment, the user can leave comments that are then available to future users. The various Cyberguide prototypes varied over whether they supported indoor or outdoor (or mixed) tours and whether they provided support for individuals or groups (synchronous and asynchronous). Positioning information was used to inform the interface where the user was located. That information could then be used to provide information automatically about the surrounding environment. Different modes of interaction were also made possible, ranging from hand-held, pen-based interactive books to hands-free interaction with heads-up displays or audio-only interfaces. Positioning information ranged from commercially available services, such as GPS, or home-grown beacon systems for indoor use. The history of where a tourist had travelled over time was used to provide an automated compilation of a travel diary and to make suggestions on places of interest to visit based on past preferences. Communication was provided either through an asynchronous docking metaphor 77

4 Highlktder 2000 Monroe Dr. Midtown Atlanta, GA Hours + Parking Lot: ye$ Parking Information Rating Price XCoord [Dolt- Y Coord da 1 Food&Drink 1 Ambiance 1 I Recreation I Specialities I Personal Comment5 Figure 3: The interactive map of the CyBARguide prototype indicating the user s location (the triangle) and the location of establishments previously visited (the beer mugs). Figure 4: CyBARguide s user-modifiable database supports the long-term collection of group comments for a location. or wireless/cellular connections. Ultimately, this project was not as successful for ubicamp research as it could have been. Though successive prototypes were similar in functionality, very little was preserved from version to version. User evaluation indicated the need for major modifications and these modifications were not easy to do quickly, so no single prototype experienced extended use. As a proof of concept, Cyberguide was very useful. As a research prototype that allowed for investigation of how ubicomp affects our everyday lives, it was not successful. Unfortunately, too many ubicomp research projects fall into this category. 3 COMMON FEATURES OF UBICOMP AP- PLICATIONS Having briefly outlined two ubicomp research projects, we can now define some common features that stretch across almost any ubicomp application. We will define each theme, and demonstrate its relevance in terms of the above projects and some other related work. Understanding these features of ubicomp applications will seed discussion of the challenges of providing these features in the rapid prototyping model of research. Transparent interaction It has long been the objective of interface design to re- move the physical interface as a barrier between a user and the work she wishes to accomplish via the computer. While many of the advances in usability have come from the development of techniques that allow a designer to build a system based on modeiling of the user s tasks and mental models, we are still largely stuck with the same input devices (keyboard, mouse and tiny display monitor) that have been commercial standards for 15 years; This physical interface is anything but transparent to the user and it violates ubicomp vision of pervasive computation without intrusion. As the vision of ubicomp is fulfilled and computational services are spread throughout an environment, advances are needed to provide alternative interaction techniques. For example, in Classroom 2000, it is.important that the electronic whiteboard look and feel like a whiteboard and not a computer. In a traditional classroom, the lecturer comes to class and only has to pick up a marker or chalk to initiate writing on the board. Interaction with the electronic whiteboard needs to be that simple. Our initial Zen* system required a lecturer to launch a browser, point to the ZenPad Web Ipage, select the class, give a lecture title, set up the audio and video encoding machines and authenticate themselves to the system before class could begin. As class begun, three people were needed to start the system, synchronizing the beginning of three different streams of information. Hardly a transparent process! In fact, had we not pro- 78

5 vided human assistance to set up the class, we would not have had so many adopters early on. What we lacked in an automated transparent entry to the system we had to make up in human support. Today, the situation is still not ideal, but the start up procedure is limited to starting a single program, entering the title for the lecture and pressing a button to begin lecture. The goal is to make the system vanish even further, ultimately returning to the situation in which the lecturer simply enters the room, picks up the pen and begins to lecture. As another example, the Cyberguide prototypes varied quite a bit in the interface provided to the end user. The goal was to reproduce the flexibility of a human tour guide, which meant allowing for questions at arbitrary times with constant knowledge of where the traveller was located. This knowledge can be used to provide an apparently flexible and intelligent interface. Limited speech interaction is augmented by knowing what a person is attending to and preparing the system to recognize utterances with related keywords. In addition to providing end-user flexibility, there is a development challenge to provide similar functionality across varied interfaces and interaction techniques. Transparent interaction techniques is a very active area of research which includes handwriting and gesture recognition, freeform pen interaction, speech, computational perception, tangible user interfaces (using physical objects to manipulate electronic information) and manipulation interfaces (embedding sensors on computational devices to allow for additional modes of interaction). Context-awareness An application like Cyberguide can take advantage of user mobility by adapting behavior based on knowledge of the user s current location. This location can refer to the position and orientation of a single person, many people, or even of the application itself. Location is a simple example of conte~:t, that is, information about the environment associated with an application. Context-aware computing involves application development that allows for collection of context and dynamic program behavior dictated by knowledge of this environment. Researchers are increasing our ability to sense the environment and to process speech and video and turn those signals into information that expresses some understanding of a real-life situation. In addition to dealing with raw context information such as position, a context-aware application is able to assign meaning to the events in the outside world and use that information effectively. Context-awareness is not unique to ubiquitous computing. For example, explicit user models used to predict the level of user expertise or mechanisms to pro- vide context-sensitive help are good examples used in many desktop systems. With increased user mobility, and with increased sensing and signal processing capabilities, there is a wider variety of context available to tailor program behavior. Moreover, context-awareness is a critical feature for a ubiquitous computing system because important context changes are more frequent. In a ubiquitous computing environment it is likely that the physical interfaces will not be owned by any one user. When a user owns the interface -as is usually the case with personal digital assistant or a laptop computer- over time this interface can be personalized to the user. Context can be useful in these situations, as has been demonstrated by location-aware computing applications. When the user does not own the interface, it is likely that the same physical interface will be used by many people. Any single user (or group of users) will prefer to have the interface personalized for the duration of the interaction. Context-awareness will allow for this rapid personalization of computing services. The restricted context-awareness based on position was the focus of Cyberguide and many other research efforts have focussed on location-aware computing [25, 21, 26, 221. In Classroom 2000, a different kind of context was necessary. We used information about the location of the electronic whiteboard and the schedule of classes to automatically predict what class was beginning and drastically streamline start-up activities. It was also important to determine focus of class discussion (Web page or electronic whiteboard slide) in order to decorate the timeline of the captured notes (shown in Figure 2) with the relevant item. In variations of the Zen* system that we have built to support informal and unscheduled meetings, we use a number of different sensing techniques to determine when a recorded session should begin and who is in attendance, all of which is contextual information. Automated capture One of the potential features of a ubiquitous computing environment is that it could be used to capture our everyday experiences and make that record available for later use. Indeed, we spend much time listening to and recording, more or less accurately, the events that surround us, only to have that one important piece of information elude us when we most need it. We can view many of the interactive experiences of our lives as generators of rich multimedia content. A general challenge in ubiquitous computing is to provide automated tools to support the capture, integration and access of this multimedia record. The purpose of this automated support is to have computers do what they do best, record an event, in order to free humans to do what they do best, attend to, synthesize, and understand what is happening around them, all with full confidence that the 79

6 specific details will be available for later perusal. Automated capture is a paradigm for simplified multimedia authoring. In Classroom 2000, the Zen* system was built with this capture problem foremost in th.e minds of the designers. The classroom experience was viewed as producing a number of streams of relevant :information -prepared information presented as a sequence of slides, annotations on the electronic whiteboard, a series of Web pages viewed, what is said by the lecturer and what is seen by the students. The objective of the system was to facilitate the capture of all of these streams and the preparation of visualizations that allow a student to effectively relive the classroom experience. Many other researchers have developed similar note-taking or meeting capture applications [18, 12, 29, 24, 30, 9, 20, 71. Cyberguide can also be seen as a capture problem. As a tourist travels from location to location, a record of what was visited, what was seen there, and even personal comments, results in a captured trail that can be revisited later. Various prototypes created a historian component with the responsibility of capturing significant events and preparing summaries that were then made available to the user in a variety of formats. 4 TOOLKIT DESIGN ISSUES Having established three important functional features of ubicomp applications, we can now further discuss the software engineering challenges that they present. Language requirements The landmark work of Douglas Engelbart and his team of researchers at SRI in the 60 s demonstrated the power of building toolkits to bootstrap the development of increasingly sophisticated interactive systems. Each of the functional themes discussed above provide opportunities for developing toolkits which augment the programming capabilities to implement applications. For the development of more applications that support transparent interaction, we must be able to treat other forms of input as easily as we treat keyboard and mouse input. A good example justifying the need for transparent interaction toolkits is freeform, pen-based interaction. Much of the interest in pen-based computing has focussed on recognition techniques to convert pen input to text. But many applications, such as the note-taking in Classroom 2000, do not require conversion from pen to text. Relatively little effort has been put into standardizing support for freeform, pen input. Some formats for exchanging pen input between platforms exist, such as JOT, but no support for using pen input effectively within an application. For example, Tivoli provides basic support for creating ink data, distinguishing between uninterpreted ink data and special gestures [15, 171. A particularly important feature of these other data types is the ability to cluster them. In producing Web-based notes in Classroom 2000, we want annotations done with a pen to link to audio or video. The annotations are timestamped, but it is not all that useful to associate an individual penstroke to the exact time it was written in class. We used a temporal and spatial heuristic to statically cluster penstrokes together and assign them some more meaningful, word-level time. Chiu and Wilcox have produced a more general and dynamic clustering algorithm to link audio and ink [8]. Clustering of freeform ink is also useful in real-time to allow for a variety of useful whiteboard interactions (e.g., insert a blank line between consecutive lines) and implicit structuring has been used to do this [16]. These clustering techniques need to become standard and available to all applications developers who wish to create freeform, pen-based interfaces. Designing context-aware applications is difficult for a number of reasons. One example comes from our experience with location-awareness in Cyberguide. We used many different positioning systems throughout the project, both indoor and outdoor. Each prototype had its own positioning-system specific code to handle the idiosyncracies of the particular positioning system used. A location-aware application should be written to a single location-aware API, but this does not exist. An analogy to GUI programming is appropriate here. GUI programming is simplified by having toolkits with predefined interactors or widgets that can be easily used. In theory, a toolkit of commonly used context objects would be useful, but what would be the building blocks of such a toolkit? The context that has been most useful is that which provides location information, identification, timing and an association of sensors to entities in the physical world. While desktop computing exists with a WIMP (windows, icons, menus and pointers) interface, we are suggesting that context-aware computing exists with a TILE (time, identity, location, entities) interface. In the next section, we will return to the issue of a context-aware infrastructure that separates the concerns of the environment from those.of tbe application. Our collected experience building a variety of capture applications has lead to the development of a capture toolkit to enable rapid development. The primitives of this toolkit are captured objects (elements created at some time that are aggregated to create a stream of information), capture surfaces (the logical container for a variety of streams of captured objects), service providers (self-contained systems that produce streams of recorded information) capture clients (interactive windows that control one or more capture surfaces and service providers), capture servers (mul- 80

7 tithreaded servers that maintain relationships between capture clients and service providers and handles storage and retrieval to a multimedia database), and access clients (programs that aggregate captured material for visualization and manipulation). Perhaps the biggest open challenge for toolkit design is the scalable interface problem. We deal with a variety of physical devices, and they differ greatly in their size and interaction techniques. This variability in the physical interface currently requires an application programmer to essentially rewrite an application for each of the different devices. The application, in other words, does not scale to meet the requirements of radically different physical interfaces. One approach to scaling interfaces is through automated transformation of the interaction tree representing the user interface. There has been some initial research on ways to transform an interface written for one device into an equivalent application on a different device. For example, the Mercator project automatically converts an X-based graphical interface into an auditory interface for blind users [19]. Similar transformation techniques were employed by Smith to automatically port a GUI interface to a Pilot [23]. The focus of this prior work has been transformation of an existing application. Another avenue to be pursued would be more abstract interface toolkits that can be bound to appropriate physical interfaces for different devices. Just as a windowing system promotes a model of an abstract terminal to support a variety of keyboard and pointing/selection devices, so too must we look for the appropriate interface abstractions that will free the programmer from the specifics of desktop, hand-held and hands-free devices. 5 SOFTWARE STRUCTURING ISSUES In Section 1, we explained why rapid prototyping and frequent iteration with minimal downtime were necessary in ubicomp applications development. In Brooks terms, ubicomp applications need to grow organically [5]. One barrier that stands in the way of this goal is the difficulty of modifying an existing system made up of a complex collection of interacting parts. If the designer is not careful, modifications to one part of the system can easily have undesirable ramifications on other parts of the system. In Classroom 2000, the initial system contained a single electronic whiteboard surface with audio recording. Over time, and due to requests from users (both teachers and students) we included an extended whiteboard, an instrumented Web browser (initially built by one of the users and integrated into the system in a matter of days), video, and improved HTML notes. While this growing of the system was not without pain, we never experienced a downtime during a quarter s classes and frequently introduced new functionality in the middle of a quarter. This evolution was possible because very early on we adopted a structure to the capture problem that separated different concerns cleanly. This structuring divides the capture problem into four separate phases [2]: pre-production to prepare materials for a captured lecture; live recording to synchronize and capture all relevant streams; post-production to gather and integrate all captured streams; and access to allow end-users to view the captured information Clear boundaries between the phases allowed for a succession of evolved prototypes with improved capabilities and minimal down-time. The lesson here is an old one: structuring a problem so as to separate concerns determines long-term survivability. We did not have the same experience of organic evolution with Cyberguide. Nor did we have this success with one other major context-aware project, CyberDesk [ll]. The main reason for this is that we have not yet succeeded in separating issues of context gathering and interpretation from the triggering of application behavior. The analogy with GUI programming mentioned in Section 4 is again appropriate here. One of the main advantages of a GUI toolkits and user interface management systems is that they are successful in separating concerns of the application from concerns of the user interface. This allows flexibility in changing one (usually the presentation) without affecting the other. For context-aware computing, we need to be able to effectively separate the concerns of the application from the concerns of the environment. That is why we are currently developing a generic context server which holds pieces of contextual information (time, identity, location, and entities associated with sensors) organized in a way that allows an application to register interest in parts of the environment s context and be automatically informed of relevant changes to that context. It is the responsibility of the context server to gather and interpret raw contextual information. The requirements on the context server are more complex, however. Context is dynamic. The source of a piece of context may change while an application is running. For example, as a tracked individual walks outside, a GPS receiver may broadcast updates on his location. Stepping inside would mean a switch to any number of positioning systems. The context server must 81

8 hide these details from the application, which is only interested in where the person is, not how that location is determined. Context sensing devices differ in the accuracy with which they record information. GPS might be accurate to within 100 feet, but a building location system might have a finer resolution. Different positioning systems can deliver the same information but in different forms. GPS delivers latitude/longitude, whereas an Active Badge [25] indoor system delivers position via cell identities that are separately mapped to buildingrelevant locations. The context server needs to insulate the application from these worries. Finally, different forms of context can become available at different times. In essence, an application may not know what kinds of context are available to it until run-time. The context server needs to be able to discover context-generating resources dynamically and inform running applications appropriately. An example of this resource discovery problem occurs in Classroom Different instrumented classrooms provide different recording capabilities and those recording capabilities do not always work. A context server that serves the Zen* system would inform particular capture clients of the functionality of recording streams in its environment, allowing the client to customize its interface appropriately. 6 COMPONENT INTEGRATION Ubicomp presents a rather arduous requirement of robustness, reliability and availability to the end user. No single researcher has the time or desire to develop all portions of a ubicomp application and be confident that it will satisfy that requirement on the variety of devices that a user will want. The only hope is to rely on third-party software, which leaves the challenge of gluing together pieces of a system, or component integration. Though there is a wealth of academic research and commercial products that provide very general component integration technologies, they are not appropriate for ubicomp application development. This is because those integration technologies are not universally available on the range of devices and operating systems that are required. Our requirement for component integration is a some standard method for communication and control that can be realized across the wide array of devices and operating systems. One strategy would be to adopt TCP, as it provides a lowest common denominator for communications for a large array of devices. At a higher level, a lightweight communications protocol as HTTP is desirable. HTTP sergers are available in any language you desire and are simple enough to run on the limited resources of most embedded devices. Most HTTP servers now also support the extensible standard markup language XML. X,ML is a text-based protocol, so it will again be supportable for a wide variety of devices. While there may ultimately be limitations to this lightweight approach to component integration, it is sufficient at present to support the ad hoc integration that is required for rapid prototyping of ubicomp applications. 7 CONCLUSIONS Ubiquitous computing provides a tantalizing vision of the future and hard problems that engage researchers from many subdisciplines of computer science. To conduct ubicomp research effectively, there are significant software engineering challenges that in themselves provide direction for that research community. Ubicomp research is inherently empirical and relies on a rapid prototyping development cycle that can rea.ct to feedback from user evaluation. Development under these conditions requires technical advances. Ubicomp applications share several functional features. Specifically, they strive to produce transparent interaction techniques, they adapt their behavior in accordance with changes to the context of their use, and they provide automated services to capture live experiences for later access. Engineering software solutions that provide these functional features requires advances in the design of toolkits used to build ubicomp applications, the structuring of software systems to separate concerns of environmental context and phases.of capture, and the adoption of lightweight component integration techniques. Acknowledgements The author would like to thank all of the researchers in the Future Computing Environments (FCE) Group in the College of Computing and GVU Center at Georgia Tech. Their abundant energy, enthusiasm and sweat have fed the ideas expressed in this paper as they have germinated over the past few years. Further information on the research of the FCE Group can be found at http: //waw. cc. gatech. edu/f ce. This work is sponsored by the National Science Foundation, through a Faculty CAREER grant IRI arid by DARPA, through the EDCS project MORALE. REFERENCES [l] G. D. Abowd, C. G. Atkeson, J. Brotherton, T. Enqvist, P. Gulley, and J. LeMon. Investigating the capture, integration and access problem Iof ubiquitous computing in an educational setting. In Proceedings of the 1998 conference on Human Factors in Computing Systems - CHI 98, pages , May [2] G. D. Abowd, C. G. Atkeson, A. IFeinstein, 82

9 [31 G. D. Abowd, C. G. Atkeson, J. Hong, S. Long, R. Kooper, and M. Pinkerton. Cyberguide: A mobile context-aware tour guide. ACM Wireless Networks, 3: , PI [51 PI 171 P. Brown. The stick-e document: a framework for creating context-aware applications. In Proceedings of Electronic Publishing 96, pages University of Kent, Canterbury, September PI PI PO1 PI WI P31 C. Hmelo, R. Kooper, S. Long, N. Sawhney, and M. Tan. Teaching and learning as multimedia authoring: The classroom 2000 project. In Proceedings of the ACM Conference on Multimedia - Multimedia 96, pages , G. D. Abowd, J. Brotherton, and J. Bhalodia. Automated capture, integration, and visualization of multiple media streams. In Proceedings of the 1998 IEEE conference on Multimedia and Computing Systems, F. Brooks. Mythical Man-Month. Addison Wesley, 20th anniversary edition edition, J. A. Brotherton and G. D. Abowd. Rooms take note: Room takes notes! In Proceedings of the i998 Spring AAAI Symposium on Intelligent Environments, pages 23-30, Published as AAAI Technical Report SS P. Chiu and L. Wilcox. A dynamic grouping technique for ink and audio. In Proceedings of UIST 98, page???, L. Degen, R. Mander, and G. Salomon. Working with audio: Integrating personal tape recorders and desktop computers. In Proceedings of ACM CHI 92 Conference, pages , May A. J. Demers. Research issues in ubiquitous computing. In Proceedings of the Thirteenth Annual ACM Symposium on Principles of Distributed Computing, pages 2-8, A. Dey, G. D. Abowd, and A. Wood. Cyberdesk: A framework for dynamic integration of desktop and network-based applications. Iinowldege-Based Systems, 11:3-13, M. Lamming and M. Flynn. Forget-me-not: Intimate computing in support of human memory. In Proceedings of FRIEND2I: International Symposium on Next Generation Human Interfaces, pages RANK XEROX, S. Long, D. Aust, G. D. Abowd, and C. G. Atkeson. Rapid prototyping of mobile context-aware applications: The cyberguide case study. In Proceedings of the 1996 conference on Human Factors P41 S. Long, R. Kooper, G. D. Abowd, and C. G. Atkeson. Rapid prototyping of mobile context-aware applications: The cyberguide case study. In Proceedings of the 2nd Annual International Conference on Mobile Computing and Networking, November P51 ii61 P71 T. P. Moran, P. Chiu, and W. van Melle. Pen-based interaction techniques for organizing material on an electronic whiteboard. In Proceedings of the ACM Symposium on User Interface Software and Technology - UIST 97, pages ACM Press, October W31 WI WI WI P21 P31 in Computing Systems - CHI 96, pages , Short paper in Conference Companion. S. Minneman, S. Harrison, B. Janseen, G. Kurtenbath, T. Moran, I. Smith, and B. van Melle. A confederation of tools for capturing and accessing collaborative activity. In Proceedings of the ACM Conference on Multimedia - Multimedia 95, November T. Moran, P. Chiu, and B. van Melle. Finding and using implicit structure in human-organized spatial layouts of information. In Proceedings of CHI 96, pages , T. P. Moran, L. Palen, S. Harrison, P. Chiu, D. Kimber, S. Minneman, W. van Melle, and P. Zelweger. I ll Get That Of the Audio : A case study of salvaging multimedia meeting records. In Proceedings of ACM CHI 97 Conference, pages , March E. D. Mynatt. Transforming graphical interfaces into auditory interfaces for blind users. Human Computer Interaction, 12(1&a), H. Richter, P. Schuchhard, and G. D. Abowd. Automated capture and retrieval of architectural rationale. In The First Working IFIP Conference on Software Architecture (WICSAI), February To appear as technical note. B. N. Schilit, N. I. Adams, and R. Want. Contextaware computing applications. In Proceedings of 1st International Workshop on Mobile Computing Systems and Applications, pages XEROX PARC, December W. N. Schilit. System architecture for contextaware mobile computing. PhD thesis, Columbia University, I. Smith. Support For Multi-Viewed Interfaces. PhD thesis, Georgia Tech College of Computing,

10 [24] L. Stifelman. The Audio Notebook. PhD thesis, MIT Media Laboratory, [25] R. Want, A. Hopper, V. Fa.lcao, and J. Gibbons. The active badge location system. ACM Transactions on Information Systems, 10(1):91-102, January [26] R. Want, B. Schilit, N. Adams, R. Gold, K. Petersen, J. Ellis, D. Goldberg, and M. Weiser. The PARCTAB ubiquitous computing experiment. Technical Report CSL-95-1, Xerox Palo Alto Research Center, March [27] M. Weiser. The computer of the 21st century. Scientific American, 265(3):66-75, September [28] M. Weiser. Some computer science issues in ubiquitous computing. Communications of the ACM, 36(7):75-84, July [29] S. Whittaker, P. Hyland, and M. Wiley. Filochat: Handwritten notes provide access to recorded conversations. In Proceedings of ACM CHI 94 Conference, pages , April [30] L. D. Wilcox, B. N. Schilit, and N. Sawhney. Dynomite: A dynamically organized ink and audio notebook. In Proceedings of CHI 97, pages ,