Sensing Meets Mobile Social Networks: The Design, Implementation and Evaluation of the CenceMe Application

Transcription

1 Sensing Meets Mobie Socia Networks: The Design, Impementation and Evauation of the CenceMe Appication Emiiano Miuzzo, Nichoas D. Lane, Kristóf Fodor, Ronad Peterson, Hong Lu, Mirco Musoesi, Shane B. Eisenman, Xiao Zheng, Andrew T. Campbe Computer Science, Dartmouth Coege, Hanover, NH 3755, USA Eectrica Engineering, Coumbia University, New York, NY 127, USA ABSTRACT We present the design, impementation, evauation, and user experiences of the CenceMe appication, which represents the first system that combines the inference of the presence of individuas using off-the-shef, sensor-enabed mobie phones with sharing of this information through socia networking appications such as Facebook and MySpace. We discuss the system chaenges for the deveopment of software on the Nokia N95 mobie phone. We present the design and tradeoffs of spit-eve cassification, whereby persona sensing presence (e.g., waking, in conversation, at the gym) is derived from cassifiers which execute in part on the phones and in part on the backend servers to achieve scaabe inference. We report performance measurements that characterize the computationa requirements of the software and the energy consumption of the CenceMe phone cient. We vaidate the system through a user study where twenty two peope, incuding undergraduates, graduates and facuty, used CenceMe continuousy over a three week period in a campus town. From this user study we earn how the system performs in a production environment and what uses peope find for a persona sensing system. Categories and Subject Descriptors: C.2.1 [Network Architecture and Design]: Wireess Communications; J.4 [Socia and Behaviora Sciences]: Socioogy. Genera Terms: Design, Experimentation, Performance. Keywords: Appications, Socia Networks, Mobie Phones. 1. INTRODUCTION One of the most common text messages peope send each other today is where r u? foowed by what u doing?. With the advent of powerfu and programmabe mobie phones, most of which incude a variety of sensing components (e.g., acceerometers, GPS, proximity sensors, microphone, cam- Permission to make digita or hard copies of a or part of this work for persona or cassroom use is granted without fee provided that copies are not made or distributed for profit or commercia advantage and that copies bear this notice and the fu citation on the first page. To copy otherwise, to repubish, to post on servers or to redistribute to ists, requires prior specific permission and/or a fee. SenSys 8, November 5 7, 28, Raeigh, North Caroina, USA. Copyright 28 ACM /8/11...$5.. era, etc.) there is a new way to answer these questions. In essence, mobie phones can create mobie sensor networks capabe of sensing information that is important to peope, namey, where are peope and what are they doing? The sensing of peope is driving a new appication domain that goes beyond the sensor networks community s existing focus on environmenta and infrastructure monitoring, where peope are now the carriers of sensing devices, and the sources and consumers of sensed events. The expanding sensing capabiities of mobie phones (e.g., Nokia N95 and Appe iphone) combined with the recent interest by the mobie phone vendors and ceuar industry in open programming environments and patforms, typified by the recent reease of the Android patform [2] and the Appe iphone SDK [1], is acceerating the deveopment of new peope-centric sensing appications and systems [3]. In this paper, we present the design, impementation, evauation, and user experiences of the CenceMe appication [4], a new peope-centric sensing appication. CenceMe expoits off-the-shef sensor-enabed mobie phones to automaticay infer peope s sensing presence (e.g., dancing at a party with friends) and then shares this presence through socia network portas such as Facebook. We evauate a number of important system performance issues and present the resuts from a user study based on an experiment conducted over a three week period in a campus town. The user study incuded twenty two users consisting of undergraduates, graduates and facuty at Dartmouth Coege. We discuss resuts, experiences, and essons earnt from the depoyment of CenceMe on off-the-shef mobie phones. These phones, whie fairy powerfu computers, present a number of imitations in supporting the demands of a continuous persona sensing appication such as CenceMe. We impement CenceMe on the Nokia N95 phones. Athough the N95 is a top-end device with a great dea of computation capabiity, the Symbian operating system and Java Micro Edition (JME) virtua machine which runs on top of the N95 are rather imiting due to the fact that they have both been designed to use sma amounts of memory and computationa resources. Additiona impementation chaenges arise from the fact that manufacturers and operators imit the programmabiity of mobie phones to preserve the cosed nature of their devices and operationa networks. For this reason appropriate certificates purchased from a Certificate Authority are needed, yet are not sufficient for fu depoyment of an appication such as CenceMe. We show 337

2 the tradeoffs and discuss the difficuties in impementing an aways-on sensing appication on the Symbian/JME patform which more generay is designed to accommodate simpe appications such as gaming and caendar pugins. The contribution of this paper is as foows: The design, impementation, and evauation of a fuy functiona persona mobie sensor system using an unmodified mobie phone patform. The design of ightweight cassifiers, running on mobie phones, which reaize a spit-eve cassification paradigm. We show they have a imited impact on the phone s functionaity. Measurements of the RAM, CPU, and energy performance of the cassifiers and the CenceMe software suite as a whoe, showing the tradeoff between the time fideity of the data and the atency in sharing that data. A vaidation of the CenceMe appication through a user study. This is one of the first user studies that invoves a arge group of peope using a persona sensing appication running on off-the-shef mobie phones for a continuous period of time. The study provides usefu insights into how peope understand and reate to persona sensing technoogy. The study offers some suggestions on the further deveopment of peope-centric sensing appications. The structure of the paper is as foows. In Section 2, we present a number of design considerations when buiding an aways-on sensing appication such as CenceMe on mobie phones. The CenceMe impementation is discussed in Section 3, whie in Section 4 the phone and backend cassifier agorithms are presented. In Section 5, we show the performance of the CenceMe cassification agorithms as we as detaied power, RAM, and CPU measurements. In Section 6, we present the resuts of our user study and then in Section 7 discuss reated work. We concude in Section 8 with some fina remarks. 2. DESIGN CONSIDERATIONS Before describing the impementation of the CenceMe appication on the phone and backend servers, we first discuss the system deveopment chaenges encountered when impementing an appication such as CenceMe on the phone. These impact severa aspects of the architectura design. 2.1 Mobie Phone Limitations OS Limitations. Athough top-end mobie phones have good computationa capabiity, often incuding mutipe processors, they are imited in terms of the programmabiity and resource usage contro offered to the deveoper. For exampe, the Nokia N95 is equipped with a 33 MHz ARM processor, 22 MHz DSP, and 128 MB RAM. However, when deveoping a non-trivia appication on mobie phones a number of chaenges arise. This is due in part because mobie phones are primariy designed for handing phone cas in a robust and resiient manner. As a resut, third party appications running on the phone may be denied resource requests and must be designed to aow interruption at any time so as not to disrupt reguar operation of the phone. This paces a heavy burden on appication exception handing and recovery software. Whie programmers may expect exception handers to be caed rarey, in Symbian they are caed often and are critica to keeping an appication and the phone operationa. At the same time, testing exception handers is difficut because a voice ca can interrupt appication code at any point in its execution; OS induced exceptions are outside the contro of the programmer. API and Operationa Limitations. Additiona imitations arise from the APIs provided by the phone manufacturers. JME impements a reduced set of the Java Standard Edition APIs for use on mobie phones. Because each phone mode is different even from the same manufacturer, the Symbian OS and JME must be ported to each phone which typicay resuts in missing or mafunctioning APIs for important new or existing components, such as an acceerometer or GPS. These API imitations may not be resoved by the manufacturer because new modes repace od modes in quick succession. As a resut, the programmer is forced to come up with creative soutions to API imitations. Exampes of such API imitations and operationa probems encountered with the N95 incude a missing JME API to access the N95 interna acceerometer and JME audio API that exhibits a memory eak, respectivey. Security Limitations. To preserve the phone s integrity and protect the ceuar network from maicious attacks, phone manufacturers and ceuar network operators contro access to critica components, incuding the APIs for access to the fie system, mutimedia features, Buetooth, GPS, and communications via GPRS or WiFi, through a rights management system. Propery signed keys from a Certificate Authority are needed to remove a restrictions on using these APIs. Energy Management Limitations. An important driver for appication designers on mobie phone patforms is power conservation, in particuar, when radio interfaces such as Buetooth, GPS, and GPRS are used by the appication. As we show in Section 5, the phone s Buetooth, GPS, and GPRS radios are responsibe for draining most of the battery power when CenceMe is running. As appication deveopers, we want to buid appications that offer good fideity and user experience without significanty atering the operationa ifetime of the standard mobie phone. Therefore, designing efficient duty cyces for the appication and its use of power hungry radios such Buetooth and GPS radio is necessary to extend the phone s battery ife. In addition to the power consumed by Buetooth and GPS, data upoad from the phone via GPRS can aso draw a arge amount of power, particuary when the phone is far from a ce base station. A chaenge is therefore to reduce the use of these radios without significanty impacting the appication experience. Currenty, the Symbian version of JME does not provide APIs to power cyce (i.e., togge on and off) the Buetooth and GPS radios to impement an efficient radio duty-cyce strategy. The sensing and cassification agorithms that run on the phone can aso consume a considerabe amount of energy if eft unchecked. As discussed in Section 5, samping the phone s microphone and running a discrete Fourier transform on the sound sampe uses more power than samping the acceerometer and cassifying the acceerometer data. Given this, the ony way to reduce energy at the appication ayer is to design a sensing duty-cyce that sampes sensors ess frequenty and avoids the use of the radios for communications or acquisition of sateite signas for ocation coordinates. 338

3 2.2 Architectura Design Issues In response to the observations discussed above we design the CenceMe appication using spit-eve cassification and power aware duty-cycing. We aso deveop the appication with software portabiity in mind. Spit-Leve Cassification. The task of cassifying streams of sensor data from a arge number of mobie phones is computationay intensive, potentiay imiting the scaabiity of the system. With this in mind we propose the idea of pushing some cassification to the phone and some to the backend servers. However, some cassifiers require data that is ony avaiabe at the server (e.g., for mutipe users in the case of the socia context cassification discussed in Section 4.2). We ca the output of the cassification process on the phone primitives. When primitives arrive at the backend they are stored in a database and are ready to be retrieved for a second eve of more compex cassification. The cassification operation on the backend returns facts, which are stored in a database from where they can be retrieved and pubished. With the spit-eve cassification approach some of the cassification can be done on the phone with the support of the backend, or under certain circumstances done entirey on the phone. CenceMe s spit-eve design offers a number of important advantages: i) supports customized tags. A customized tag is any form of activity, gesture, or cassified audio primitive that the user can bind to a persona meaning. For exampe, a customized tag coud be created by a user by associating a certain movement or gesture of the phone (e.g., the phone being moved aong an imaginary circe) with a user suppied meaning or action, e.g., going to unch. After associating the tag unch with the action, the next time the user repeats the action the user s presence state unch is recognized, upoaded, and shared with their socia network. This technique gives the user the freedom to buid her own cassified state beyond a set of defauts offered by CenceMe, hence, it provides extensibiity of the appication; ii) provides resiiency to ceuar/wifi radio dropouts. By pushing the cassification of primitives to the phone, the primitives are computed and buffered when there is no or intermittent radio coverage. Primitives are stored and upoaded in batches when the radio coverage becomes avaiabe; iii) minimizes the sensor data the phone sends to the backend servers improving the system efficiency by ony upoading cassification derived-primitives rather than higher bandwidth raw sensed data; iv) reduces the energy consumed by the phone and therefore monetary cost for the data ceuar connection by merging consecutive upoads of primitives; and finay v) negates the need to send raw sensor data to the backend, enhancing the user s privacy and data integrity. As discussed in Section 4, we design the cassifiers that produce the primitives to be ightweight to match the capabiities of the phone. Power Aware Duty-Cyce. To extend the battery ifetime of the phone when running the CenceMe appication we appy schedued seep techniques to both the data upoad and the sensing components. This eads to the foowing question: how ong can the sensors, Buetooth, GPS, and communications upoad be in a seep mode given that the arger the seep interva the ower the cassification responsiveness of the system? Typicay, a rea time sensing system woud suppy sensor data using a high rate duty cyce. However, such an approach woud confict with energy conservation needs. Our approach is based on a duty cyce design point that minimizes samping whie maintaining the appication s responsiveness, as judged by users. This design strategy aows CenceMe to operate as near to reatime as is possibe; that is, some system deay is introduced before a person s sensing presence is updated on the backend servers. In the case of the current impementation the introduced deay varies according to the type of presence being inferred. The introduction of deay to improve the overa energy efficiency of the system makes good sense given the goa of CenceMe to aow buddies in socia networks to casuay view each other s sensing presence. For exampe, knowing that a buddy is in a conversation one minute after the actua conversation began seems reasonabe. Other activities may aow even greater introduced atency; for exampe, peope remain at parties for periods typicay greater than five minutes or more, therefore, the deay introduced by the cassifier in this case has itte effect on the accuracy of the system status reports. In Section 5.2 we present the CenceMe system performance evauation under varying upoad and sensing duty cyces to best understand these tradeoffs. In Section 6 we discuss resuts from the user study that indicate that even though users view their buddies status via the CenceMe porta infrequenty they expect current information when viewed to be accurate and timey. This ends itsef to a design that senses at an even ower duty cyce on average but temporariy increases the sensing rate when a buddy s page is accessed. This resuts in bandwidth and storage capacity improvements. Software Portabiity. To design for better software portabiity we push as much as we can to JME. We foow this design goa to maximize software re-usabiity given that the majority of modern mobie phones use a Java virtua machine to support JME programs. However, because of the API imitations discussed earier, a number of components need to be impemented directy using native Symbian APIs to support the necessary features offered by the phone but not avaiabe through JME. 3. CENCEME IMPLEMENTATION In this section, we present the CenceMe impementation detais. The CenceMe appication and system support consists of a software suite running on Nokia N95 mobie phones and backend infrastructure hosted on server machines. The software instaed on the phones performs the foowing operations: sensing, cassification of the raw sensed data to produce primitives, presentation of peope s presence directy on the phone, and the upoad of the primitives to the backend servers. Primitives are the resut of: i) the cassification of sound sampes from the phone s microphone using a discrete Fourier transform (DFT) technique and a machine earning agorithm to cassify the nature of the sound; ii) the cassification of on board acceerometer data to determine the activity, (e.g., sitting, standing, waking, running); iii) scanned Buetooth MAC addresses in the phone s vicinity; iv) GPS readings; and finay, v) random photos, where a picture is taken randomy when a phone keypad key is pressed or a ca is received. Cassification agorithms that infer more compex forms of sensing presence (i.e., facts) run on backend machines, as discussed in Section Phone Software Figure 1 shows the CenceMe software architecture for the 339

4 GUI Audio Cassifier Storage Launcher/ Controer Primitives Acceerometer Cassifier Raw Sensor Data Upoader Manager Sensing Controer Acceometer Cient Audio Cient Random Photo GPS Sensor Buetooth Daemon JME Symbian C++ Acceerometer Sensor Audio Sensor Event Detector Figure 1: Architecture of the CenceMe phone software. Figure 2: CickStatus on the Nokia N95. Nokia N95 phone. The phone architecture comprises the foowing software components: Symbian Servers. The acceerometer sensor, audio sensor, and event detector sensor are Symbian C++ modues that act as daemons producing data for corresponding JME cient methods. Their function is, respectivey: poing the on board acceerometer sensor, samping the phone s microphone, and detecting incoming/outgoing cas and keypad key presses. The sensed data is sent to the JME methods through a socket. Events detected by the event detector daemon are used by the random photo modue at the JME eve to generate random pictures, to trigger a photo upon an incoming phone ca or to signa the appication that it has to restart after a phone ca for reiabiity reasons. Buetooth Daemon. This component resides at the JME eve and is used to perform an inquiry over the Buetooth radio to retrieve the MAC addresses of any neighboring Buetooth nodes. The MAC addresses of the neighboring nodes are used to determine if there are CenceMe phones in the area at the time of the inquiry. Acceerometer Cient. This component is written in JME and connects through a socket to the acceerometer sensor to retrieve the acceerometer data byte stream. The byte stream is stored in oca storage and retrieved by the activity cassifier to compute the activity primitive, as discussed in Section 4.1. Audio Cient. This JME cient component connects through a socket to the Symbian audio server to retrieve the audio byte stream that carries the PCM encoded representation of the sound sampe. The byte stream is stored in oca storage and retrieved by the audio cassifier to compute the audio primitive, as discussed in Section 4.1. Random Photo. This JME modue is designed to trigger the capture of a photo upon detection of incoming cas or pressed keypad keys. The events are received through a socket from the event detector daemon. When the picture is taken it is stored ocay unti the next upoad session. GPS. The JME GPS impementation suppies a caback method that is periodicay caed by the Nokia GPS daemon to provide the geographica ocation of the phone. The GPS coordinates are stored ocay and then upoaded to the backend servers. Sensing Controer. This component is responsibe for orchestrating the underying JME sensing components. The sensing controer starts, stops, and monitors the sensor cients and the Buetooth manager and GPS daemon to guarantee the proper operation of the system. Loca Storage. This component stores the raw sensed data records to be processed by the phone cassifiers. As the cassification of raw data records is performed, the data records are discarded, hence none of the samped data persists on the phone. This is particuary important to address the integrity of the data and the privacy of the person carrying the phone since none of the raw sensed data is ever transferred to the backend. Primitives, GPS coordinates, and Buetooth scanned MAC addresses are stored in oca storage as we, waiting for an upoad session to start. Upoad Manager. This component is responsibe for estabishing connections to the backend servers in an opportunistic way, depending on radio ink avaiabiity which can be either ceuar or WiFi. It aso upoads the primitives from oca storage and tears down the connection after the data is transferred. Detais about how the upoad manager interacts with the backend are discussed in Section 3.2. Privacy Settings GUI. The privacy settings GUI aows the user to enabe and disabe the five sensing modaities supported on the phone, (viz. audio, acceerometer, Buetooth, random photo, and GPS). Users can contro the privacy poicy settings from the phone and the CenceMe porta. By doing so users determine what parts of their presence to share and who they are wiing to share sensing presence with or not as the case may be. CickStatus. To compement the fu visuaization of current and historica sensing presence avaiabe via the CenceMe porta (a screenshot of the porta is shown in [12]), we deveoped CickStatus, a visuaization cient that runs on the mobie phone. The sensing presence is rendered as both icons and text on the phone GUI, as shown in Figure 2. The presence rendered by CickStatus is subject to the same privacy poicies settings as when viewed using the CenceMe porta. After a user ogs in with their CenceMe credentias, they are presented with a ist of their CenceMe buddies downoaded from the CenceMe server. CenceMe buddies are Facebook friends running CenceMe on their N95. Whie this is aways done at start up, a user has the abiity to refresh their buddy ist at any time via a menu command option. By highighting and seecting a buddy from buddy ist, a user triggers CickStatus to fetch via GPRS or WiFi the atest known sensing presence for the seected buddy from the CenceMe server. This presence is dispayed on a separate resut screen; from there a user can either exit to return to 34

5 3rd Party Appications Apache/ Tomcat Web Porta Event-driven Cassifiers XML-RPC handers Web services API Cick Status 3rd Party Appications Periodic Cassifiers MYSQL Push Connectors Facebook Figure 3: Software architecture of the CenceMe backend. their buddy ist or refresh the currenty dispayed buddy s presence. WatchTasks. The purpose of WatchTasks is to restart any process that fais. WatchTasks aso serves severa other anciary purposes incuding: i) aunching CenceMe when the phone is turned on; ii) starting the CenceMe appication software components in the correct order; iii) restarting the CenceMe midet after a phone ca is compete. This is detected when the event detector daemon exits, signaing the end of a ca; iv) restarting a support daemons when CenceMe fais. Such action is necessary when we cannot reconnect to specific daemons under certain faiure conditions; and finay v) restarting a the CenceMe software components at a preset interva to cear any mafunctioning threads. The CenceMe phone suite uses a threaded architecture where each JME component shown in Figure 1 is designed to be a singe thread. This ensures that component faiure does not compromise or bock other components. 3.2 Backend Software The CenceMe backend software architecture is shown in Figure 3. A software components are written in Java and use Apache 2.2 and Tomcat 5.5 to service primitives from phones and the appication requests from the CenceMe porta, CickStatus, and Facebook. Communications between the phone and the backend uses remote procedure cas impemented by the Apache XML-RPC ibrary on the server. Requests are handed by Java servets in combination with a MySQL database for storage. Phone Backend Communications. Data exchange between the phone and the backend is initiated by the phone at timed intervas whenever the phone has primitives to upoad. Primitives are upoaded through XML-RPC requests. Once primitives are received at the backend they are inserted into the MySQL database. Backend-to-phone communications such as in the significant paces service described in Section 4.2 are piggybacked on both: i) the return message from XML-RPC requests initiated by the phone for primitive upoad or periodic ping messages that the phone sends with an ad-hoc XML-RPC contro message; and ii) the XML-RPC acknowedgment sent to the phone in response to a primitive upoad. Presence Representation and Pubishing. CenceMe presence is represented through a set of icons that capture the actua presence of a person in an intuitive way. For exampe, if a person is driving a car they are represented by the car icon; if a person is engaged in a conversation, an icon of two peope taking represents the state. CenceMe pubishes presence by means of either a pu or push approach. Popuar appications such as Facebook and MySpace require a push approach. This aows content to be inserted via some variant of a HTTP transported markup anguage (e.g., FBML, XML). Other appications such as Skype, Pidgin, and igooge require a pu mechanism to make content avaiabe. The CenceMe backend supports pu-based data pubishing by exposing a standard web service based API. This API is aso used to support the data needs of CenceMe components such as CickStatus and the CenceMe porta. Push-based pubishing is supported by the PushConnector component shown in Figure 3. This component handes the generic operation of pushing CenceMe presence based on user preferences to a number of appications. For the Facebook impementation, three Facebook widgets are offered to expose a subset of the functionaity avaiabe on the porta, namey, BuddySP, Sensor Status, and Sensor Presence. Buddy SP is a buddy ist repacement widget that ists CenceMe friends for user navigation. It is the same as the standard widget that ists friends within Facebook but augments this ist with a mini-sensor presence icon view. Sensor Status provides automated textua status message updates such as Joe is at work, in a conversation, standing. Finay, Sensor Presence provides a simpified version of the user s current status through an iconized representation of the user s presence. 4. CENCEME CLASSIFIERS In this section, we discuss the agorithms used by the CenceMe cassifiers running on the phone and the backend according to the spit-eve cassification design discussed earier. 4.1 Phone Cassifiers Audio cassifier. The audio cassifier retrieves the PCM sound byte stream from the phone s oca storage and outputs the audio primitive resuting from the cassification. The primitive is stored back in oca storage (see Figure 1). This audio primitive indicates whether the audio sampe represents human voice and is used by backend cassifiers such as the conversation cassifier, as discussed in Section 4.2. The audio cassification on the phone invoves two steps: feature extraction from the audio sampe and cassification. The feature extraction is performed by running a 496 bin size discrete Fourier transform (DFT) agorithm. A fast Fourier transform (FFT) agorithm is under deveopment. An extensive a-priori anaysis of severa sound sampes from different peope speaking indicated that Nokia N95 sound streams associated with human voice present most of their energy within a narrow portion of the -4 KHz spectrum. Figures 4(a) and 4(b) show the DFT output from two sound sampes coected using the Nokia N95. The pots show the capture of a human voice, and the sound of an environment where there is not any active conversation ongoing, respectivey. It is evident that in the voice case most of the power concentrates in the portion of spectrum between 25 Hz and 6 Hz. This observation enabes us to optimize the DFT agorithm to be efficient and ightweight 341

6 Power Frequency (KHz) (a) DFT of a human voice sampe registered by a Nokia N95 phone microphone. Power Frequency (KHz) (b) DFT of an audio sampe from a noisy environment registered by a Nokia N95 phone microphone. Figure 4 by operating in the 25 Hz to 6 Hz frequency range. Cassification foows feature extraction based on a machine earning agorithm using the supervised earning technique of discriminant anaysis. As part of the training set for the earning agorithm we coected a arge set of human voice sampes from over twenty peope, and a set of audio sampes for various environmenta conditions incuding quiet and noisy settings. The cassifier s feature vector is composed of the mean and standard deviation of the DFT power. The mean is used because the absence of taking shifts the mean ower. The standard deviation is used because the variation of the power in the spectrum under anaysis is arger when taking is present, as shown in Figure 4. Figure 5 shows the custering that resuts from the discriminant anaysis agorithm using the mean and standard deviation of the DFT power of the sound sampes coected during the training phase. The equation of the dashed ine in Figure 5 is used by the audio cassifier running on the phone to discern whether the sound sampes comes from human voice or a noisy/quite environment with 22% mis-cassification rate. Audio sampes miscassified as voice are fitered out by a roing window technique used by the conversation cassifier that runs on the backend, as discussed in Section 4.2. This boosts the performance fideity of the system for conversation recognition. Activity cassifier. The activity cassifier fetches the raw acceerometer data from the phone s oca storage (see Figure 1), and cassifies this data in order to return the current activity, namey, sitting, standing, waking, and running. The activity cassifier consists of two components: the preprocessor and the cassifier itsef. The preprocessor fetches the raw data from the oca storage component and extracts features (i.e., attributes). Given the computationa and memory constraints of mobie phones, we use a simpe features extraction technique which prove 4 4 Standard Deviation Taking No taking Mean Figure 5: Discriminant anaysis custering. The dashed ine is determined by the discriminant anaysis agorithm and represents the threshod between taking and not taking. to be sufficienty effective, rather than more computationay demanding operations such as FFT. The preprocessor cacuates the mean, standard deviation, and number of peaks of the acceerometer readings aong the three axes of the acceerometer. Figure 6 shows the raw N95 acceerometer readings aong the three axes for sitting, standing, waking, and running for one person carrying the phone. As expected, the sitting and standing traces are fatter than when the person is waking and running. When standing, the deviation from the mean is sighty arger because typicay peope tend to rock a bit whie standing. The peaks in the waking and running traces are a good indicator of footstep frequency. When the person runs a arger number of peaks per second is registered than when peope wak. The standard deviation is arger for the running case than waking. Given these observations, we find that the mean, standard deviation, and the number of peaks per unit time are accurate feature vector components, providing high cassification accuracy. Because of ack of space, we do not report simiar resuts to those shown in Figure 6 for other peope. However, we observe strong simiarities in the behavior of the mean, standard deviation, and the number of peaks for the acceerometer data across different individuas. Our cassification agorithm is based on a decision tree technique [32][33]. The training process of the cassifier is run off-ine on desktop machines because it is computationay costy. In order to maximize the positive inference of an individua s activity, prior work suggests that the best pace on the body to carry a phone is the hip [34]. After interviewing the participants in our user study, we conjecture that most of peope carry their phones in their pants pockets, cipped to a bet or in a bag. We coected training data from ten peope that randomy paced the mobie phone inside the front and back pockets of their pants for severa days. We pan to consider other usage cases in future work. At the end of the training phase, we feed the training set to the J48 decision tree agorithm, which is part of the WEKA workbench [28]. The output of the decision tree agorithm is a sma tree with depth three. Such an agorithm 342

7 ADC ADC ADC ADC Time 4 2 (a) Sitting (b) Standing Time 1 5 (c) Waking Time (d) Running Time Figure 6: Acceerometer data coected by the N95 on board acceerometer when the person carrying the phone performs different activities: sitting, standing, waking, and running. is ightweight and efficient. The time needed by the preprocessor and the cassifier to compete the cassification process is ess than 1 second on average running on the Nokia N Backend Cassifiers Backend cassifiers foow the spit-eve cassification design and generate facts based on primitives provided by the phone or facts produced by other backend cassifiers. Facts represent higher eve forms of cassification incuding socia context (meeting, partying, dancing), socia neighborhood, significant paces, and statistics over a arge group of data (e.g., does a person party more than others, or, go to the gym more than others?). However, some of the cassifiers (e.g., conversation and CenceMe neighborhood) wi eventuay be pushed down to the phone to increase the system cassification responsiveness. In this case, the primitives woud sti be upoaded to the backend in order to make them avaiabe to other backend cassifiers. Backend cassifier processing is invoked in two ways: either event triggered or periodic. An exampe of an event triggered cassifier is the party cassifier: it receives as input the primitives from the phone that contain the voume of an audio sampe and the activity of the user and returns whether the person is at a party and dancing. Aong with trigger based cassifiers there is a coection of periodicay executed cassifiers. An exampe of such cassifiers is the Am I Hot cassifier that runs periodicay according to the avaiabiity of data in a window of time, (i.e., day ong data chunk sizes). In what foows, we describe the backend cassifiers and their impementation in more detai. Conversation Cassifier. This cassifier s purpose is to determine whether a person is in a conversation or not, taking as input the audio primitives from the phone. However, given the nature of a conversation, which represents a combination of speech and siences, and the timing of samping, the audio primitive on the phone coud represent a sience during a conversation. Thus, the phone s audio primitives x y z are not accurate enough to determine if a person is in the midde of a conversation. To address this the backend conversation cassifier uses a roing window of N phone audio primitives. The current impementation uses N=5 to achieve cassification responsiveness, as discussed in Section 5.1. The roing window fiters out pauses during a conversation to remain atched in the conversation state. The cassifier triggers the conversation state if two out of five audio primitives indicate voice. The no conversation state is returned if four out of five audio primitives indicate a no voice. We determined experimentay that fewer sampes are needed to trigger the conversation state than no conversation state. We therefore design the conversation cassifier foowing an asymmetric strategy that quicky atches into the conversation state but moves more conservativey out of that state. We made this choice because if the conversation cassifier can be used as a hint to determine if a person can be interrupted (for instance with a phone ca), then we ony want to drop out of conversation state when the conversation has definitey ended. The accuracy of the conversation cassifier is discussed in Section 5.1. Socia Context. The output of this cassifier is the socia context fact, which is derived from mutipe primitives and facts provided by the phone and other backend cassifiers, respectivey. The socia context of a person consists of: i) neighborhood conditions, which determines if there are any CenceMe buddies in a person s surrounding area or not. The cassifier checks whether the Buetooth MAC addresses scanned by the phone, and transmitted to the backend as a primitive are from devices beonging to CenceMe buddies (i.e., the system stores the Buetooth MAC addresses of the phones when CenceMe is instaed); ii) socia status, which buids on the output of the conversation and activity cassifiers, and detected neighboring CenceMe buddies to determine if a person is gathered with CenceMe buddies, taking (for exampe at a meeting or restaurant), aone, or at a party. For exampe, by combining the output of the conversation cassifier, the activity primitive, and neighboring Buetooth MAC addresses a person might be cassified as sitting in conversation with CenceMe friends. Socia status aso incudes the cassification of partying and dancing. In this case a combination of sound voume and activity is used. We use a simpe approach that uses an audio voume threshod to infer that a person is at a party or not. Training for this is based on a few hours of sound cips from ive parties using the N95 microphone. We aso take a simpe approach to the cassification of dancing. We determine a person is dancing if the person is in the party state and the activity eve is cose to running, given that the acceerometer data trace for running is cose to dancing. Athough we reaize the definition of socia context is somewhat simpistic and coud be improved, this is a first step toward the representation of peope s status and surroundings in an automated way. Mobiity Mode Detector. We empoy GPS ocation estimates as input to a mobiity mode cassifier [22][23]. This cassification is currenty ony binary in its output, cassifying the mobiity pattern as being either traveing in a vehice or not (i.e., being stationary, waking, running). We use a simpe feature vector based on mutipe measures of speed; that is, using mutipe distance/time measurements for variabe sizes of windowed GPS sampes and the buit-in 343

8 speed estimation of the GPS device itsef. The cassifier is buit with the JRIP rue earning agorithm, as impemented in WEKA [28], based upon manuay abeed traces of GPS sampes. We compensate for any inaccuracy in GPS sampes by fitering based on the quaity measures (i.e., horizonta diution of precision and sateite counts) and outier rejection reative to the estimates of previous and subsequent GPS sampes. Location Cassifier. The function of this component is to cassify the ocation estimates of users for use by other backend cassifiers. GPS sampes are fitered based on quaity (as discussed above) to produce a fina ocation estimate. Cassification is driven based on bindings maintained between a physica ocation and a tupe containing: i) a short textua description; ii) an appropriate icon representation; and iii) a generic cass of ocation type (i.e., restaurant, ibrary, etc.). Bindings are sourced from GIS databases and CenceMe users. We use the Wikimapia [27] for GIS data in our impementation. Reying soey on GIS information imits the richness of shared presence. Typicay, peope tend to spend a arger proportion of their time in reativey few ocations. This motivates the idea of user-created bindings. CenceMe aows users to insert their own bindings via either the porta or the phone. Using the phone, users can manuay bind a ocation when they visit it. Simiary, users can use the porta to aso add, edit or deete bindings manuay. CenceMe aso provides the abiity to earn significant paces in an automated manner in contrast to the manua bindings discussed above. New bindings earned by the system are based on the mobiity pattern of the user. This aspect of CenceMe directy buids on the existing work in ocation trace anaysis referred to as significant paces [25] [31]. In CenceMe we perform k-means custering using WEKA [28] where the parameters of the custering agorithm are determined experimentay. Once a potentia significant pace is discovered the next time the person enters that ocation the phone prompts the person s mobie phone to confirm or edit the detais of the ocation. Defaut abes and icons are initiay based upon the most popuar nearest known existing binding defined by the user or CenceMe buddies. To reduce the burden on the users to train the cassifier with their own bindings we structure the cassifier to initiay borrow existing bindings from their CenceMe buddies [24]. Am I Hot. Making the arge voumes of data coected by CenceMe easiy digestibe to users is a chaenge. We address this chaenge using a series of simpe and meaningfu metrics that reate historica trends in user data to either recognizabe socia stereotypes or desirabe behaviora patterns. These metrics are cacuated on a daiy basis and users view patterns in their own data and compare themseves with their buddies. The metrics incude the foowing: i) nerdy, which is based on individuas with behaviora trends such as being aone (from the Buetooth activity registered by the person s phone), spending arge fractions of time in certain ocations (e.g., ibraries) and ony infrequenty engaging in conversation; ii) party anima, which is based on the frequency and duration with which peope attend parties and aso takes into account the eve of socia interaction; iii) cutured, which is argey ocation based, being driven by the frequency and duration of visits to ocations such as theaters and museums; iv) heathy, which is based upon physica activities of the user (e.g., waking, jogging, cycing, going to the gym); and finay, v) greeny, which identifies users having Tabe 1: Activity cassifier confusion matrix Sitting Standing Waking Running Sitting Standing Waking Running Tabe 2: Conversation cassifier confusion matrix Conversation Non-Conversation Conversation Non-Conversation ow environmenta impact, penaizing those who drive their cars reguary whie rewarding those who reguary wak, cyce or run. 5. SYSTEM PERFORMANCE In this section, we present an evauation of the CenceMe appication and system support. We start by discussing the performance of the CenceMe cassifiers and then present a set of detaied power, memory, and CPU benchmarks. Finay, we present the resuts from a detaied user study. 5.1 Cassifiers Performance We examine the cassifiers performance based on a smascae supervised experiments. We discuss cassifier accuracy, and the impact of mobie phone pacement on the body, environmenta conditions, and sensing duty cyce. The resuts are based on eight users who annotate their actions over a one week period at intervas of approximatey 15 to 3 minutes, uness otherwise stated. Annotations act as the ground truth for comparison with cassifier outputs. The ground truth data is correated to the inference made by the CenceMe cassifiers. This data is coected at different ocations and by carrying the mobie phone in various positions on the body. Tabes 1, 2 and 3 show the confusion matrices for the activity, conversation, and mobiity cassifiers, respectivey, over a one week period. These reported vaues represent good approximations; the human annotations may be inaccurate or incompete at times Genera Resuts Whie the activity inference accuracy reported in Tabe 1 is up to 2% ower than that reported using custom hardware [14], we achieve our resuts using ony the acceerometer on a Nokia N95 and engineering the system to be power efficient and work around the resource imitations discussed earier. We find that our cassifier has difficuty differentiating sitting and standing given the simiarity in the raw acceerometer traces, as shown in Figure 6. We observe that variations in ocae (e.g., office, restaurant) and peope (e.g., body type, weight) do not significanty impact the activity cassification performance. The conversation cassification accuracy reported in Tabe 2 is high, but the cassifier aso suffers from a reativey high rate of fase positives. This is due to a combination of cassifier design and mis-annotation by participants. The cassifier reports conversation even if the person carrying the phone is sient but someone is taking nearby. Naturay, participants often did not account for this fact. Furthermore, due to the asymmetric state atching for the conversation 344

9 1 1 1 conversation non-conversation.8.8 Accuracy Accuracy.6.4 True positive.6.4 Operating point.2 indoor quiet indoor noisy outdoor Location (a) Conversation cassifier in different ocations Audio sensor duty cyce (sec) (b) Conversation cassifier accuracy with a variabe duty cyce Fase positive window size =5 window size =1 window size = 3 (c) ROC curves for the conversation cassifier. Figure 8 Tabe 3: Mobiity mode cassifier confusion matrix Vehice No Vehice Vehice No Vehice Accuracy pocket hip Body position sitting standing waking running neckace Figure 7: Activity cassification vs. body position. cassifier discussed in Section 4.2, the cassifier remains in the conversation state for a onger time than the rea conversation duration, generating fase positives Impact of Phone Pacement on the Body Whie mobie phone pacement on the body is a persona choice, prior work has shown body pacement to affect the accuracy of activity inference [34]. We assess the impact on cassification when the Nokia N95 is paced at different paces on the body, namey, in a pocket, on a anyard, and cipped to a bet. Cassification accuracy derived from the ground truth annotated data is shown in Figure 7. The pocket and bet positions produce simiar resuts for a cassified activities, whie the anyard position yieds poor accuracy when cassifying sitting, and a reativey ower accuracy for running. In foow-up aboratory experiments, we find that the ength of the anyard cord and the type of anyard we provided to participants affect the resuts. If the anyard is ong the phone rests frequenty on the body, particuary whie waking and standing, aowing for accurate cassification. However, even when seated a anyard-mounted phone may swing from side to side with incidenta torso movements, causing a mis-cassification as standing or waking. Furthermore, running is sometimes cassified as waking because the anyard damps the acceerometer signatures that indicate running, compared to other body positions (e.g., bet, pocket) where the phone is more rigidy affixed to the body. We find that conversation cassification accuracy is much ess sensitive to the body pacement of the phone. When the phone is worn as a anyard, conversation and no conversation are detected with 88% and 72% accuracy, respectivey. The same test repeated with the phone in a pocket yieds a cassification accuracy of 82% for conversation and 71% for no conversation, despite the muffing effect of cothing Impact of Environment We find activity cassification accuracy to be independent of environment. Mobiity cassification is inherenty not tied to a particuar ocation but rather on transitions between ocations. However, we do see an impact from the environment on conversation cassification accuracy. Figure 8(a) shows the cassification accuracy categorized by ocation, where the different ocations are: outdoors, indoor noisy (i.e., an indoor ocation with background noise such as in a cafe or restaurant), and indoor quiet (i.e., with very ow background noise such as at the ibrary or office). The cassifier detects conversation with more than an 85% success rate when in an indoor noisy environment. In outdoor scenarios there is an increase in fase positives but the accuracy of detection of conversation, a design focus, remains high. Lower conversation detection accuracy in very quiet indoor environments occurs because the cassifier is trained with the average case background noise. In a noisy environment there is an increase in power across a of the frequencies so a threshod set for this environment in mind wi be arger than if a very quiet environment is assumed. As a resut, in very quiet environments fewer conversations are detected since the contribution of background noise is ower. These performance characteristics are a direct resut of the audio cassifier design, which attempts to reduce the use of the phone s resources Impact of Duty Cyce Appying a seep scheduing strategy to the sensing routine is needed in order to increase the battery ifetime of the phone. Note that in Section 5.2 we discuss ifetime gains 345

10 with a ten minute inter-sampe time. However, this has a negative impact on the performance of the cassifiers, particuary in detecting short-term (i.e., duration) events that occur between sampes. For exampe, in Tabe 3, the vehice state is ony correcty detected 68% of the time. This ower accuracy is a product of shorter car journeys around town for durations ess than the inter-samping rate. This probem is aggravated by other factors such as the deay in acquiring good GPS-based positioning data. To investigate the impact of duty cycing on conversation cassification, we set up an experiment with eight users that periodicay reprogrammed their phones with different duty cyces whie keeping a diary. Figure 8(b) shows the performance of the phone s conversation cassifier as the microphone sensing duty cyce varies. Each vaue represents the average of five trias. We see that there is itte benefit in adopting a seeping time smaer than 1 seconds. However, onger duty cyces impact performance. We observe ony a 4% accuracy using the conversation cassification for a 6 second duty-cyce, which is the ongest duty-cyce we considered experimentay. A onger sensing duty cyce aso impies a reduction of the conversation cassifier roing window size to maintain the high responsiveness of the cassifier. A smaer conversation cassifier roing window size eads to a higher miscassification rate. This becomes apparent if we ook at the Receiver Operating Characteristic (ROC) curves of the conversation cassifier as shown in Figure 8(c). The ROC curves show the impact of the window size and threshod that triggers conversation (refected in the curve shape) on the cassifiers true positive and fase positive rates. We use offine anaysis to determine the output of the conversation cassifier as we ater the window size and threshod vaue. We observe that the arger the window (i.e., N=1,3), the arger the true positives to fase positives ratio becomes. In our current impementation, we adopt N=5 and an audio sensing rate of 3 seconds (our defaut operating point is abeed in the figure). With these parameters the worst-case conversation cassification deay omitting communication deays is 1.5 minutes. On the other hand, if we used a window where N=3, which woud give higher accuracy, we woud get a deay of 9 minutes on average. This iustrates the trade off between samping rate and cassification speed. However, we choose to operate at a point in the design space that increases the true positive rate at the expense of being ess accurate in the detection of non-conversation because the cost, from a user s perspective, of being wrong when detecting a conversation is arger than the cost of being wrong when detecting non-conversation. 5.2 Power Benchmarks Power measurements of CenceMe are made using the Nokia Energy Profier, a standard software too provided by Nokia specificay for measuring energy use of appications running on Nokia hardware. The profier measures battery votage, current, and temperature approximatey every third of a second, storing the resuts in RAM. Figure 9 shows the typica contribution of various sensors and cassifiers to the overa energy budget during a ten minute sensing cyce. Buetooth proximity detection requires a 12 second scan period to capture neighboring MAC addresses due to the cache fushing imitations of the Buetooth API in JME. GPS ocation detection is inherenty power hungry and takes time to acquire a ock when turned Figure 9: Detais of the power consumption during a samping/upoad interva. Figure 1: The tradeoff between energy consumption and data atency in CenceMe. on. CenceMe aows 12 seconds for a ock to be acquired and then the N95 keeps the GPS activated for another 3 seconds (which is out of our contro). The highest spikes shown on the pot are due to the upoad of data which uses the ceuar radio. The next highest spikes are due to samping of audio data. The period of severa seconds foowing the audio sampe is where the audio cassifier runs, using a reativey high amount of energy to compute a DFT. The acceerometer samping and activity cassification are fast and use itte power. Whie this is a typica pattern of energy consumption there are other factors which can cause variations, incuding: distance to ce tower, environmenta radio characteristics, the amount of data to upoad, the number of Buetooth neighbors, denia of resources due to the phone being in use for other purposes, network disconnections, sensor sampe intervas, sampe durations, upoad interva, GPS ock time, and temperature. Figure 1 shows the energy consumption measured with the profier for samping intervas ranging from 1 seconds to 6 seconds with power in Watts on the vertica axis. The second ine and axis in the graph shows the atency in getting the facts to the backend as a function of the sampe interva incuding the sampe interva itsef, cassifier atency, and network deay. The audio cassifier atency is actuay a mutipe of three times the vaues on this ine since the cassifier needs at east three facts from the phone in order to detect conversation and socia setting. The horizonta axis shows the samping interva for the acceerometer and audio. The 346

11 proximity and GPS sensors are samped at ten times the x- axis vaue (e.g., a 6 second interva means Buetooth and GPS are samped at 6 seconds, or ten minute intervas). The combination of the two ines show the tradeoff between energy use and data atency for any particuar samping interva. There is no optima samping interva since users wi have different requirements at different times. For exampe, users may want a short sampe interva when they are active, a sow interva when they are inactive, and a very sow interva when their phone is running out of energy. We are currenty considering severa methods of automatic adaptation of the sampe rate based on sensor input and battery state, combined with a user preference seector that ets the user shift the emphasis between ong battery ife and greater data fideity. Overa battery ifetime running the entire CenceMe software suite on a fuy charged N95 is measured five times by running the battery to depetion under norma use conditions whie using no other appications on the phone. This resuts in /-.59 hours of usage. The reason for the arge standard deviation is that there are many factors impacting battery ife such as temperature, the number of cas and duration, the number of CickStatus queries, range from ce towers when used, and the environmenta and atmospheric conditions. Without the CenceMe software running, and the phone in a competey ide state, ow power state power consumption is.8 +/-.1 Watt-Hours per hour. The CenceMe suite consumes.9 +/-.3 Watt-Hours per hour when running with no user interaction. The conversation and socia setting cassifier consumes.8 +/-.3 Watt- Hours per hour with a other parts of the CenceMe system ide. The activity cassifier consumes.16 +/-.4 Watt- Hours per hour with a other parts of the CenceMe system ide. Any use of the phone to make cas, pay videos or isten to music wi reduce the runtime. Whie the approximatey 6 hour ifetime is far beow the ide ifetime of the Nokia N95, we have identified severa areas where we beieve we can significanty reduce power usage whie aso decreasing data atency, as discussed in Section Memory and CPU Benchmarks We aso carried out benchmark experiments to quantify the RAM and CPU usage of the CenceMe software running on the N95 using the Nokia Energy Profier too. For a measurements we enabe the screen saver to decoupe the resource occupation due to the CenceMe modues from that needed to power up the N95 LCD. We start by measuring the amount of RAM and CPU usage when the phone is ide with none of the CenceMe components running. We then repeat the measurement when either the acceerometer samping and activity cassification or audio samping and cassification are active. Then we add each of the remaining CenceMe modues unti the whoe software suite is running. The resuts are shown in Tabe 4. As expected, audio samping and feature vector extraction require more computation than the other components. This is in ine with the power measurements resut shown in Figure 9 where audio samping and processing are shown to use a reativey high amount of energy. We aso note that the memory foot print does not grow much as components are added. Together CenceMe and CickStatus occupy 5.48MB of RAM. Tabe 4: RAM and CPU usage CPU RAM (MB) Phone ide 2% (+/-.5%) 34.8 Acce. and activity cassif. 33% (+/- 3%) Audio samping and cassif. 6% (+/- 5%) Activity, audio, Buetooth 6% (+/- 5%) 36.1 CenceMe 6% (+/- 5%) 36.9 CenceMe and CickStatus 6% (+/- 5%) USER STUDY Because CenceMe is designed to be a socia network we need to go beyond simpe measures of system performance to best understand the utiity of peope-centric sensing appications such as CenceMe. Our goa is to bring CenceMe to the attention of potentia users, ask them to use CenceMe and provide detaied feedback about their user experience by means of a survey. For this reason we conducted an operationa experiment. The experiment conducted over a three week period invoved 22 peope. Participants were each given a Nokia N95 with the CenceMe software (incuding CickStatus) and a free voice/data pan. Users had server side accounts and access to the CenceMe porta. Whie some of the users were friends we paced a users in the same buddy ist as a means to create some community. The poo of candidates picked within the popuation of students and staff at our university was composed of 12 undergraduate students, 1 research assistant, 1 staff engineer, 7 graduate students, and 1 professor. The research assistant and four undergraduates have itte computer science background. Sixteen participants are active Facebook users. Before discussing the detaied experience of users, we summarize some resuts from the user study: Amost a of the participants iked using CenceMe and its features. One user wrote: it s a new way to be part of a socia network. Facebook users are particuary active in terms of wiingness to share detaied status and presence information with their friends. Privacy coud be a concern but users are fine with sharing their presence status as ong as they have the means to easiy and efficienty contro their privacy settings. CenceMe stimuates curiosity among users. Users want to know what other peope are doing whie on the move. CenceMe can aid peope in earning their own activity patterns and socia status. A new way to connect peope. Amost a the participants find the idea of providing and viewing detaied information about peope they are cose to compeing, usefu, and fun. In particuar, ocation, activity/conversation, the historica og of the person s presence, random images, and socia context are the features that peope ike the most. This pattern is confirmed in Figure 11(a), where the cumuative participants feature utiization for different hours of the day derived from the anaysis of system ogs on the backend is shown. It is evident that ocation information which reveas where friends are is the feature most used by the 347

12 Feature utiization GPS ocation Random photo Feature Am I hot Favourite hang-outs (a) Utiization distribution across the different features. Number of operations Insertion Deetion Hour of day (b) Comparison between the number of random pictures inserted into the database versus the number of pictures deeted. Number of queries Porta CickStatus Hour of day (c) Comparison between the CenceMe porta and CickStatus usage. Figure 11 participants. The random photos was aso found to be of interest because it can be used as a way to tag the person s day as in a diary: oh yeah... that chair... I was in cassroom 112 at 2PM. The photos are often burred, since they are taken outside the contro of the person, but they sti serve the diary tagging purpose. Some of the participants did not particuary ike the fact that the system takes pictures outside their contro, so they opted to turn that feature off by customizing their privacy poicy on the phone. What is the potentia CenceMe demographic? We beieve that peope-centric sensing appications such as CenceMe coud become popuar among socia networking appication users, for whom sharing context and information is popuar. For many of these users, privacy is ess of a concern than for others, as shown by their interest in pubicy pubishing persona history in detai in bogs and on socia networks. This tendency is aso highighted in Figure 11(b) which shows a comparison between the cumuative number of random photos inserted into the database versus the tota number of photos deeted for different hours of the day. Once photos are upoaded users are given the opportunity to seectivey deete them from the system. Few participants (4 out of 22) disabed the random photo for the entire duration of the experiment and others disabed it at different times of the day to meet their privacy needs or the needs of the peope around them. In genera, as shown in Figure 11(b), the number of non-deeted photos avaiabe for sharing is much greater than the number of deeted photos. Most participants did not mind having pictures taken at any time of the day and in random settings and then being shared with a the other participants. Many of them were excited by the idea of guessing what their friends were doing through the hint provided by random photos. Moreover, no CenceMe presence sharing restriction was appied by the participants, who aowed their sensing presence to be accessibe by everyone in the group. Athough some users stated that they coud foresee wanting to appy a presence sharing restriction poicy under certain conditions (e.g., if their parents had access), they fet comfortabe with the idea of others seeing their presence most of the time. Learn about yoursef and your friends. CenceMe made me reaize I m azier than I thought and encouraged me to exercise a bit more. This quote is taken from one participant s survey. Other users expressed simiar thoughts. Users view CenceMe as an appication that potentiay coud te them things that might be intuitivey obvious, but are often invisibe in their ives due to famiiarity and repetition. Some exampes are ack of physica activity and spending a ot of time in front of a computer. Near-rea time presence sharing and historica presence representation are ways to capture peopes ifestye and trends about activity, socia context (am I often aone? do I party too much?), and ocation. My friends aways with me. The study highights that the participants enjoyed retrieving their friends presence on the mobie phone with CickStatus in addition to checking the porta. The average number of times per day they checked presence was 4 ± 3 times, where 3 is the standard deviation. Figure 11(c) shows a comparison between the tota number of times presence is accessed through the porta or via CickStatus distributed throughout the day. Athough the number of times the participants access the porta is arger than their use of CickStatus on the N95, CickStatus is activey used. This is cear from Figure 11(c), where the use of CickStatus rises during the time of day when peope are presumaby most ikey on the move because they are going to cass (between noon and 6PM) or hanging out with friends (between 8PM and 11PM). Overa, the user experience is positive. Because many of them enjoyed using CenceMe, they kept the CenceMe phone for a whie after the end of the experiment. We are currenty working on revising some of the components and improving a few architectura eements in order to refect some of the vauabe feedback from the participants. Specificay, future revisions of the CenceMe system wi incude: An improved CenceMe software modue on the phone that proongs the battery ife. Our goa is to achieve a 48 hour duration without recharging the device. An enhanced version of the porta to provide finer grained privacy poicy settings as we as an enhanced CickStatus user interface to provide the user with more powerfu ways to browse their friend s presence. A shorter cassification time for primitives and facts because many of the participants beieve that rea time access to buddies sensing presence shoud be one of the features of the system. System architectura revisions are currenty under consideration to meet this 348

13 requirement. A burst mode for sensing may prove to be usefu. 7. RELATED WORK There is growing interest in the use of sensor-enabed mobie phones for peope-centric sensing [2][21][15][26][29]. A number of diverse appications are emerging. In [5], the authors describe an appication that determines poution exposure indexes for peope carrying mobie devices. A microbogging service is discussed in [8] that uses mobie devices to record mutimedia content in-situ and shares this content in a rea-time. In [9], we discuss the integration of the CenceMe appication with Second Life [1]. The use of persona sensor streams in virtua words is a new and interesting area of research. The work presented in this paper significanty extends our initia work on CenceMe [4], where we discussed the basic idea and the resuts of some isoated experiments. Cephones have been used to earn about socia connections [17][18] and provide context-aware communications using ocation information from ceuar towers and manuay configured preferences in the icams system [11]. The icams system aows users to pick the preferred method of communication according to a person s status and ocation (e.g., in person, emai, home/work phone). WatchMe [13] is a simiar system that aims at choosing the best way to communicate with buddies. WatchMe reies on GPS trace anaysis to determine whether a person is waking or driving, and uses the phone s microphone to infer taking and sient states. CenceMe differs from icams and WatchMe because of the rich context it provides about a person in an automated and transparent way. In the same way CenceMe aso differs from Twitter [19], an appication to pubish textbased status messages generated by users. There is a arge body of work on activity inference and modeing using customized sensors worn by peope [35][7] [36][6][37]. CenceMe differs from this work because it impements the activity inference agorithms on commercia mobie phones. As discussed in this paper there are a number of important design tradeoffs that need to be taken into account when impementing aways-on peope-centric sensing appications ike CenceMe on off-the-shef mobie phones. Systems such as SATIRE [16] aso assume sensing devices with great capabiities being embedded into smart cothing. An interactive dancing project [3] requires peope to wear customized sensors mounted on shoes to track dancing activity. In [7] the authors discuss their experience buiding efficient cassification techniques on the Inte Mobie Sensing Patform (MSP), a sma form factor wearabe device for embedded activity recognition. The MSP patform is quite powerfu compared to many ceuar devices. The CenceMe cassifiers have been taiored to operate on ess capabe devices than the MSP whie remaining effective. 8. CONCLUSION We presented the impementation, evauation, and user experiences of the CenceMe appication, which represents one of the first appications to automaticay retrieve and pubish sensing presence to socia networks using Nokia N95 mobie phones. We described a fu system impementation of CenceMe with its performance evauation. We discussed a number of important design decisions needed to resove various imitations that are present when trying to depoy an aways-on sensing appication on a commercia mobie phone. We aso presented the resuts from a ong-ived experiment where CenceMe was used by 22 users for a three week period. We discussed the user study and essons earnt from the depoyment of the appication and highighted how we coud improve the appication moving forward. Acknowedgments This work is supported in part by Inte Corp., Nokia, NSF NCS , and the Institute for Security Technoogy Studies (ISTS) at Dartmouth Coege. We woud ike to thank Peter Boda and Chieh-Yih Wan for their support with this work. ISTS support is provided by the U.S. Department of Homeand Security under award 26-CS-1-1, and by award 6NANB6D613 from the U.S. Department of Commerce. The views and concusions contained in this document are those of the authors and shoud not be interpreted as necessariy representing the officia poicies, either expressed or impied, of any funding body. A specia thanks goes to our shepherd Mark Corner. 9. REFERENCES [1] Appe iphone SDK. [2] Android. [3] A. T. Campbe, S. B. Eisenman, N. D. Lane, E. Miuzzo, R. A. Peterson, H. Lu, X. Zheng, M. Musoesi, K. Fodor, G. Ahn. The Rise of Peope-Centric Sensing, In IEEE Internet Computing, vo. 12, no. 4, pp , Ju/Aug, 28. [4] E. Miuzzo, N. D. Lane, S. B. Eisenman, A. T. Campbe. CenceMe - Injecting Sensing Presence into Socia Networking Appications In Proc. of EuroSSC 7, Lake District, UK, October 23-25, 27. [5] D. Estrin, et a. Seeing Our Signas: Combining ocation traces and web-based modes for persona discovery. In Proc. of HotMobie 8, Napa Vaey, CA, USA, Feb , 28. [6] P. Zappi, et a. Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Seection, In Proc. of EWSN 8, Boogna, Itay, Jan. 3/31 - Feb.1, 28. [7] T. Choudhury, et a. The Mobie Sensing Patform: an Embedded System for Capturing and Recognizing Human Activities, In IEEE Pervasive Magazine, Spec. Issue on Activity-Based Computing, Apri-June 28. [8] S. Gaonkar, et a. Micro-Bog: Sharing and Querying Content Through Mobie Phones and Socia Participation, In Proc. of MobiSys 8, Breckenridge, CO, USA, June 17-2, 28. [9] M. Musoesi, E. Miuzzo, N. D. Lane, S. B. Eisenman, T. Choudhury, A. T. Campbe,. The Second Life of a Sensor: Integrating Rea-word Experience in Virtua Words using Mobie Phones, In Proc. of HotEmNets 8, Charottesvie, Virginia, USA, June 28. [1] Second Life. [11] Y. Nakanishi, K. Takahashi, T. Tsuji, and K. Hakozaki. icams: A Mobie Communication Too Using Location and Schedue Information. In IEEE Pervasive Computing, Vo 3, Iss 1, pp , Jan-Mar

14 [12] A. T. Campbe, et a. Transforming the Socia Networking Experience with Sensing Presence from Mobie Phones. (Demo abstract) In Proc. of ACM SenSys 8, November 5-7, 28, Raeigh, North Caroina, USA. [13] N. Marmasse, C. Schmandt, and D. Spectre. WatchMe: Communication and Awareness Between Members of a Cosey-knit Group. In Proc. of 6th Int Conf. on Ubiq. Comp., pp , Nottingham, Sep 24. [14] J. Lester, T. Choudhury, G. Borrieo. A Practica Approach to Recognizing Physica Activities. In Proc. of 4th Int Conf. on Perv. Comp., pp. 1-16, Dubin, May 26. [15] T. Abdezaher, et a. Mobiscopes for Human Spaces. In IEEE Perv. Comp., vo. 6, no. 2, pp. 2-29, Apr-Jun, 27. [16] R. Ganti, P. Jayachandran, T. Abdezaher, J. Stankovic. SATIRE: A Software Architecture for Smart AtTIRE. In Proc. of 4th Int Conf. on Mobie Systems, Appications, and Services, Uppsaa, Jun 26. [17] N. Eage, A. Pentand. Eigenbehaviors: Identifying Structure in Routine. Behaviora Ecoogy and Sociobioogy (in submission), 27. [18] N. Eage, A. Pentand. Reaity Mining: Sensing Compex Socia Systems. In Persona Ubiq. Comp., pp , May, 26. [19] Twitter. [2] J. Burke, et a. Participatory sensing. In ACM Sensys Word Sensor Web Workshop, Bouder, CO, USA, Oct 26. [21] Sensorpanet. [22] D. J. Patterson et a. Inferring High-Leve Behavior from Low-Leve Sensors, In Proc. of Ubicomp 3, Seatte, USA, Oct. 23. [23] P. Mohan, V. N. Padmanabhan, and R. Ramjee, Nerice: Rich Monitoring of Road and Traffic Conditions using Mobie Smartphones, In Proc. of ACM SenSys 8, Raeigh, NC, USA, Nov 28. [24] N. D. Lane, H. Lu, S. B. Eisenman, A. T. Campbe. Cooperative Techniques Supporting Sensor-based Peope-centric Inferencing, In Proc. of the Sixth Internationa Conference on Pervasive Computing, Sydney, Austraia, May 28. [25] D. Ashbrook, T. Starner. Using GPS to earn significant ocations and predict movement across mutipe users, In Persona and Ubiquitous Computing Journa, vo. 7, no. 5, pp , 23. [26] A. Kansa, M. Goraczko, and F. Zhao. Buiding a Sensor Network of Mobie Phones. In Proc of IEEE 6th Int IPSN Conf., Cambridge, MA,USA, Apr 27. [27] Wikimapia. [28] I. H. Witten and E. Frank. Data Mining: Practica machine earning toos and techniques, 2nd Edition, Morgan Kaufmann, San Francisco, 25. [29] A. T. Campbe, et a. Peope-Centric Urban Sensing (Invited Paper). In Proc. of WICON 6, Boston, MA, USA, Aug 26. [3] J. Paradiso, K.-Y. Hsiao, E. Hu Interactive Music for Instrumented Dancing Shoes In Proc. of ICMC 99, Bejing, China, Oct [31] C. Zhou, et a. Discovering Personay Meaningfu Paces: An Interactive Custering Approach. In ACM Trans. on Information Systems, vo. 25, num 3, 27. [32] E. M. Tapia, et a., Rea-time recognition of physica activities and their intensities using wireess acceerometers and a heart rate monitor, In Proc. of Internationa Symposium on Wearabe Computers, IEEE Press, 27. [33] R. O. Duda, P. E. Hart, D. G. Sork, Pattern Cassification, Second Edition, Wiey Interscience. [34] L. Bao, S. S. Intie. Activity Recognition from User-Annotated Acceeration Data, In 2nd Internationa Conference, PERVASIVE 4, Vienna, Austria, Apri 21-23, 24. [35] E. Webourne, J. Lester, A. LaMarca and G. Borrieo. Mobie Context Inference Using Low-Cost Sensors. In Proc. of LoCA 25, Germany, May 25. [36] J. Lester, et a. Sensing and Modeing Activities to Support Physica Fitness. In Proc. of Ubicomp Workshop: Monitoring, Measuring, and Motivating Exercise: Ubiquitous Computing to Support Fitness), Tokyo, Japan, Sep 25. [37] R. DeVau, M. Sung, J. Gips and A. Pentand. MIThri 23: Appications and Architecture. In Proc. of 7th Int Symp. on Wearabe Computers, White Pains, Oct