Personalisation in a System Combining Pervasiveness and Social Networking

Transcription

1 Personalisation in a System Combining Pervasiveness and Social Networking Sarah Gallacher, Elizabeth Papadopoulou, Nick K. Taylor, Fraser R. Blackmun, M. Howard Williams School of Maths and Computer Sciences, Heriot-Watt University, Edinburgh, UK {S.Gallacher, E.Papadopoulou, N.K.Taylor, F.R.Blackmun, M.H.Williams}@hw.ac.uk Ioanna Roussaki, Nikos Kalatzis, Nicolas Liampotis, School of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece {ioanna.roussaki, nikosk, nicolas.liampotis}@cn.ntua.gr Daqing Zhang Institut TELECOM SudParis Paris, France daqing.zhang@it-sudparis.eu Abstract One of the key objectives of a pervasive computing system is to provide appropriate support to enable the user to manage the increasingly complex environment surrounding her. This includes managing the ever-increasing number of devices which can be accessed wirelessly as well as the vast range of services at her disposal. The aim of the Persist project was to develop a pervasive system that would bridge the gap between fixed smart spaces (e.g. smart homes) and systems created for mobile users. Using the concept of Personal Smart Spaces the Persist project has built a prototype system to demonstrate some of the capabilities that this can provide. The Societies project is currently building on these ideas to develop a new type of system that combines pervasive with social networking functionality. Personalisation is an essential feature of any pervasive system and plays a key role in the prototype implemented in Persist. This will also play a key role in the new platform being developed in the Societies project. This paper describes how personalisation is handled within the Persist system and some ideas for the new platform. Keywords- pervasive systems; smart spaces; social networking; personalisation I. INTRODUCTION The main challenge underlying the creation of pervasive systems is to develop mechanisms to help the user in controlling and managing the growing number of devices (including sensors, computers and general appliances) in her environment. This stems from the original vision [1] of an environment in which the user is surrounded by devices to aid him/her in everyday life. As this forecast growth rapidly becomes reality, so the need for pervasive systems becomes increasingly important. Besides the growth in the number of devices, the rapid expansion in the number of services available to the user has been phenomenal witness the number of applications which This work was supported in part by the European Commission under the FP7 programme (PERSIST and SOCIETIES projects). have been developed for the Apple iphone, ipad Touch and ipad. Overall the number of services across all devices and domains is becoming overwhelming and unmanageable, providing further motivation for the need for pervasive systems. In response to this need, this active area of research has produced and is continuing to produce a range of prototypes exploring different approaches to solving this problem (e.g. Adaptive House [2], MavHome [3], GAIA [4], Synapse [5], Ubisec [6], etc.). In addition to addressing the main challenge, the creation of these prototype systems has also led to the development of new functionality that can be used to improve the user experience. The prototypes that have been developed thus far test different subsets of ideas in this area. Two particular types of system that have emerged from this are the fixed smart spaces (e.g. smart homes) and the systems created for mobile users. The Persist project [7] is a European research project being carried out by ten partner organisations, the aim of which was to integrate these two different types of system and provide a basis for implementing general pervasive computer systems. In order to achieve this, the notion of a Personal Smart Space (PSS) was introduced. A PSS consists of a collection of devices linked together by an ad hoc network, which may interact with other PSSs when these are encountered. Because of the peer-topeer nature of these interactions, such smart spaces can be deployed without the need for fixed infrastructures. At the same time a PSS provides an adaptive environment that can respond to whatever devices and services are available to the user in any particular context. The Societies project is a larger European research project with sixteen partners, from Ireland and Portugal in the West to Israel in the East. Our aim is to leverage the recent technical developments in pervasive computing and social networking to create an innovative service platform that supports

2 device/service management, communication and interaction as well as resource sharing in the future community context (both static and dynamic, physical space-based and virtual spacebased). This will combine the strengths of pervasive systems with those of social networks to meet the needs of a wide range of different applications and users. In order to achieve this, one of the major challenges lies in the development of approaches that will alleviate the user from some of the detailed interaction and decision making that is needed. To do this in a way that is acceptable to the user, it is essential that her needs and preferences are taken into account. Personalisation is the set of processes that adapt the behaviour of a system to suit the needs and preferences of individual users, thereby providing different user experiences for different users as their preferences dictate. In the case of pervasive systems it adapts the user s view of the environment and the services in it to suit the user s needs in accordance with the user information held by the personalisation system. Hence a major problem lies in developing approaches to personalisation that will help to ensure that the different needs of individual users are adequately reflected in the decisions taken. This is the focus of this paper. The main part of this paper is concerned with the approach taken to personalisation in systems based on the concept of Personal Smart Spaces (PSSs). These ideas have been realised in the pervasive system architecture developed in the Persist project [8], and the resulting prototype that has been developed has been used to demonstrate the functionality that can be provided using this approach. The latter part of the paper describes some aspects of the Societies project ( which will create a system that combines pervasive systems with social networking capabilities and evaluate it in a set of three separate real user trials. The next section provides a brief background to Personal Smart Spaces and personalisation. Section 3 outlines the Persist platform while section 4 gives a brief introduction to personalisation in Persist. Section 5 provides an insight into the Societies project which aims to combine pervasiveness with social networking. Section 6 concludes. II. PERSONAL SMART SPACES AND PERSONALISATION A. Personal Smart Spaces In general the notion of a smart space can be defined as a multi-user, multi-device, dynamic interaction environment that enhances a physical space by virtual services [9, 10]. The services are the means of interaction between participants, objects and the smart spaces. Alternatively, Singh et al [11] define smart spaces as ordinary environments equipped with visual and audio sensing systems that can perceive and react to people without requiring them to wear any special equipment. These definitions have been the basis of the vision of a smart space in a number of different projects and prototypes. However, both of these (and most approaches to smart spaces) rely on the availability of rooms equipped with a range of sensors and appropriate infrastructure. On the other hand developments in the area of mobile systems have sought to include some forms of pervasive system behaviour too. Projects such as Daidalos [12] have aimed at providing the user with the ability to access and manage the devices surrounding her as well as giving appropriate support to handle the bewildering array of services that might be available at any time and location. This must all be done within the context of an adaptive system that provides adequate protection for the user s privacy. The notion of a Personal Smart Space [8] was introduced to integrate these two different approaches (fixed smart spaces and mobile systems) in a clean and consistent fashion to provide the user with a degree of pervasive support at all times, which is enhanced by additional functionalities provided by other PSSs whenever these are possible, without the need for a large investment in infrastructure. A Personal Smart Space may be defined as the set of services that are running or available within a dynamic space of connectable devices where the set of services and devices are owned, controlled, or administered by a single user or organisation. More specifically, a PSS must have the following three essential properties: (1) It must have an owner. From the above a PSS consists of a collection of devices and services that are owned, controlled or administered by a single user or organisation and that work together on behalf of the person or legal entity that owns it. As such it forms a pervasive subsystem on behalf of its owner. From the point of view of personalisation it maintains a set of preferences of the owner that are used to personalise the behaviour of the PSS and its services, and, by extension, services from another visited PSS, subject to any conflict resolution on those preferences (such as the temperature of a room), both proactively and by reacting to changes in the environment. (2) It may be mobile or stationary. In the case where a PSS is owned by a person, its physical boundary will move around with the user. On the other hand, a PSS associated with a building (e.g. office) will be stationary. Both types of PSS are identical in terms of their functionality. They have exactly the same architecture but differ in the devices and third party services that they offer. (3) It must be able to identify and interact with other PSSs. By using an ad hoc network, a mobile PSS can interact with other mobile PSSs or with a stationary PSS to exchange information or access services. Thus a PSS can be realised as an ad hoc network which may interact with the networks of other PSSs when these are encountered. This has the advantage of not requiring any fixed infrastructure to be provided by Internet Service Providers or Mobile Network Operators, although it is able to take advantage of infrastructure when it is available. Thus users can deploy their own personal smart spaces, populating them with their mobile and fixed devices. B. Personalisation In the case of pervasive systems the idea of personalisation is quite wide ranging and the definition used here is as follows:

3 Personalisation is the process of adapting the behaviour of a system to match the needs and preferences of the user. Thus personalisation is responsible for making the system behave or appear differently for different users or for the same user in different contexts. Note that this does not include the situation where a service produces different results for different values of input from the user. Personalisation in pervasive systems affects decision making at a number of different points in the system. For example, where the user requests a service and different options are possible, personalisation may be responsible for the selection of an appropriate instance. Where the user would normally select and execute a particular service, personalisation might be responsible for proactively doing this on behalf of the user. When a service is executed, personalisation may tailor it according to the user s needs and preferences. A useful distinction to make in relation to personalisation of a PSS is between user preferences and proactivity. We regard user preferences as defining the needs and preferences of the user in relation to PSS services as well as to third party services depending on the user s current context. On the other hand proactivity should take decisions on behalf of the user in order to personalise services and environments. For example, the user preferences of a user PSS may capture the preferred ambient lighting and temperature of the user under normal working conditions. When the user is in an office controlled by a PSS, her PSS will pass this information to the office PSS, which in turn will pass these preferences to the appropriate environmental control devices. However, if there is no user PSS present in the office, the preferences of the office PSS may be to save energy by switching off all lighting and heating. By contrast proactivity is defined by the set of rules that determine when to take an action on behalf of the user and what action to take when this is required. An example in the case of a user PSS might be the initiation of some service (e.g. switch on heating when user arrives home). In the case of a static PSS a proactive rule might be used to alert security staff when a particular suspicious situation arises. Thus in order to be acceptable to the end user, it is essential that pervasive systems are adaptive to the needs of the individual user and personalise their behaviour according to the needs and preferences of different users and the particular context which the user is in. For this purpose some form of knowledge must be held about the user s preferences and behaviour patterns and must be applied when the appropriate decisions are taken. This may be done proactively by identifying what actions the user might wish to take and performing these actions on the user s behalf. The simplest approach to handling such knowledge is through the use of user preferences based on rules. Initial systems using this approach made the assumption that such preferences would be entered manually by the user. However, building up a realistic set of preferences in this way is very time consuming and experience has shown that the user soon loses interest and the resulting preference sets are incomplete and not very useful. As a result systems sought alternative approaches such as monitoring of the user s behaviour followed by some form of learning (e.g. the fixed smart space MavHome [3] project, or mobile applications, e.g. Specter). Another alternative is to use other forms of knowledge representation such as Bayesian networks or Hidden Markov Models to capture user behaviour and represent user needs (e.g. Daidalos project [12]). The recent emergence of mobile sensing and online community activities, along with growing interest in the Internet of Things, has led to a new research area called social and community intelligence (SCI). It aims at revealing the behaviour of individuals and groups, their social interactions as well as community context by mining the digital traces left by people while interacting with cyber-physical spaces [14]. With this vision, the user preferences, the community preferences, the proactivity for individual and group can all be derived if the related historical digital traces can be collected, which will greatly enhance the personalization processes. III. STRUCTURE OF A PERSONAL SMART SPACE IN PERSIST The architecture of a PSS [13] that has been implemented in the Persist system consists of five layers, each containing functionality essential to the operation of the PSS. This architecture is illustrated in Fig. 1. Each layer addresses a well defined part of the PSS functionality, and is described briefly below. Figure 1. The high level architecture of a Personal Smart Space (1) Device Layer. The lowest level of the architecture comprises the set of devices making up the PSS. (2) System Run-Time Environment Layer. The purpose of this layer is to provide platform independence as far as possible by acting as an abstraction layer between the operating system of the device and the higher level PSS software components. (3) Overlay Network Management Layer. This layer is responsible for managing Peer-to-Peer communication, both within the PSS itself and between PSSs.

4 (4) Service Run-Time Environment Layer. This layer provides the support services needed to underpin the PSS Framework (layer 5). (5) PSS Framework Layer. This is the core of the PSS architecture. It provides the intelligence required to adapt the behaviour of the PSS to suit the needs of its owner through personalisation, context awareness, learning and management of user preferences, extraction of user intent, reference to recommender systems, etc. In addition to these five layers, there is a vertical block that intersects four of the layers. This block is concerned with Security and Privacy Management, supporting features such as access control, identity management, privacy and trust management, and policy management. IV. PERSONALISATION IN PERSIST In order to personalise any system or service one must have some representation of the user s needs and preferences and the proactive actions to be taken. These may be stored in a variety of different forms, e.g.: Rule-based format Neural net Bayesian network List of attribute-value pairs Different applications have different requirements and may prefer different forms of user model. In the case of pervasive systems the requirements are fairly complex particularly since many of the decisions required (e.g. on selection, execution or adaptation of services) are context-dependent. As a result within the Persist prototype the first three different representations are all used. The advantage of the first format is that users can interact with the rules, checking them and updating them as necessary. The other two forms are well suited to situations where the user is not likely to inspect or change these. This paper will focus on the rule-based approach. The main functions of personalisation within Persist are as follows: (1) Maintenance of a set of user preferences through monitoring user actions and applying learning techniques. (2) Customising services by using user preferences to determine what parameters to pass to the service. (3) Monitoring context conditions that might affect the way in which services have been personalised and re-personalising them if necessary. (4) Determining when to take an action proactively on the user s behalf, and carrying out this action. (5) Using user preferences to negotiate the access that services may make of user data. These functions are realized in the subsystem architecture shown in Fig. 2. Figure 2. High level view of Personalisation components A. User Preferences Within Persist user preferences are generally stored in a recursive rule format of form: if <condition> then <action> or if <condition> then <action> else <action> where the actions may in turn be conditional. The User Preference subsystem is responsible for creating and maintaining the set of user preferences for a user (via monitoring and machine learning or by direct user input). Each preference might be reactive in that it responds to a user action (e.g. whenever the user requests a news service, if the user is at home the system selects the television set). Alternatively it may be proactive, and initiate an action on the user s behalf when a particular context arises. For example, suppose that whenever the user arrives home after work in the winter, she switches up the temperature on the central heating. If the learning system identifies this, a preference rule could be set up to initiate this automatically for the user. B. User Intent In addition to the user preferences, another mechanism that is used by Personalisation is the User Intent subsystem. In this case the aim is to look ahead and predict future actions based on past patterns of actions. The distinction between the two subsystems lies in the immediacy of the actions. User preferences are an immediate mechanism in that when a particular context condition arises, a corresponding action is taken. User intent is concerned with sequences of actions leading to a consequence at some later time in the future. C. Proactive Behaviour The way in which proactive behaviour is handled is as follows. The Proactivity subsystem receives input from both the User Preference subsystem and the User Intent subsystem regarding the proactive actions that each of them recommends to be taken by the Proactivity subsystem. These inputs are passed to the Decision Maker component. Every input received

5 by the Decision Maker has a confidence level field associated with it and each input provider has a similar confidence level which is calculated on the basis of the number of times the input it has provided has been successfully implemented. These two confidence levels are used to decide which input should be implemented if the inputs suggest conflicting actions. As long as there is no conflict between the actions, the Decision Maker will trigger the Decision Implementer to initiate the recommended action. In doing so it also notifies the user and provides a means for the user to intervene if necessary and stop the action from proceeding. On the other hand, if there is a conflict between the actions suggested by the two (User Intent and User Preference Management components) which cannot be resolved by this simple filtering, this is referred to a separate component in the Proactivity subsystem responsible for Conflict Resolution. In such a case, the latter component uses a conflict resolution algorithm to determine what, if any, action should be taken. The conflict resolution algorithm uses the confidence levels and the Quality of Context of the context attributes that led the input components to suggest these actions. If the conflict resolution algorithm is unable to resolve the conflict, the user is prompted to provide an answer using the Feedback Manager. The user s input is then implemented and the User Intent and the User Preference Management components are informed of the success or failure of their predictions. This feedback is used to adjust the confidence levels of the user preferences and user intent predictions as well as the confidence level of each input provider. D. User Preferences for Sharing When one PSS encounters another, a set of user preferences is used to determine what sharing is to be permitted with this new PSS. In particular, a grouping facility is available and whenever the user creates a new group (via a GUI designed for this purpose), the user is asked whether or not he/she wishes to be alerted when another PSS from this group is detected. If the user decides that this is what is wanted, a preference rule is set up to invoke a third party service and pass it an appropriate set of parameters whenever a PSS from this group is detected. At any point after this, if the Grouping component identifies a PSS from a group, the Preference Condition Monitor (PCM) component will be triggered, which in turn will evaluate its preferences. If the condition part succeeds, the PCM will pass it to the Proactivity component to apply. V. SOCIETIES The idea of identifying groups of PSSs in Persist provides an essential first step towards managing the more complex interactions between individuals and communities of users using more general systems. In the Societies project this is taken to a whole new level as social networking is adopted as a cornerstone of a new type of system. To illustrate the power of such a system, consider the following extract from the set of scenarios currently being used to drive the development of the Societies system: Scene 1: Harry is a new student who has just arrived at Heriot-Watt University (HWU). He is alerted to important communities that he is strongly encouraged to join, particularly the "Freshers" community that all new students can join. On joining the Freshers community Harry inherits several community preferences. One such preference is the preferred venue to buy lunch on campus. He is also automatically added to a "Computer Science" community for the degree course he is taking. Scene 2: That evening Harry attends a Freshers' event called the "Proactive Disco". It is a community based disco that takes into account the music preferences of all the people currently dancing on the dance floor (identified using sensor technology) and decides what music tracks to play. Scene 3: Since starting at HWU Harry has joined a number of other available communities including the "Dorm 1" community, related to the dorm he stays in. One useful feature that it provides is the "Student Cooking" service. This service compares the cooking ingredients provided by members and suggests that community members with compatible ingredients get together to make a meal between them. It also factors in group food preferences and learns who prefers to dine with whom over time. Different incentives and awards are given to community members for various reasons, e.g. the most active members or members who are considered the best cooks. Scene 4: Harry is leaving his dorm room to attend his first lecture. His CSS identifies his intent to attend the lecture so the navigation service is automatically started to direct Harry to the lecture room. The directions are presented to Harry as superimposed images on the lenses of his glasses (that support augmented reality). This is his preferred method (over audio directions or visual directions on his smart phone). On his way Harry's CSS flags another person nearby who also has the intent of attending the same lecture. Since they share intents and since the other person s mood is happy, Harry's CSS suggests an introduction which he accepts. Harry's CSS shows him a picture of the other person and tells Harry his name is Tom. They begin to chat and Harry walks the rest of the way to the lecture with his new acquaintance. The set of requirements that is emerging for this type of system is much more extensive than that for the Persist system and a new type of architecture is called for that can deal with this combination of functionalities. For example, the issue of scalability takes on new significance when communities can end up with thousands, tens of thousands or even greater numbers of users. For personalisation the basic problem remains that of providing the ability to adapt the behaviour of the system to meet the needs and preferences of the user. However, we now have the problem of discovering and managing both individual user preferences and community preferences (e.g. Scene 1) and dealing with conflicts between these. Moreover, by recording the interactions among people in the physical spaces as well as interactions between people and environments using pervasive systems, by collecting and extracting user profiles and relationships from the social networks, we can get a rich set of digital traces ( digital footprints [14]) about people that can be used to recognize and predict user s preferences, intent, etc.

6 More specifically, in order to extend the PSS beyond the individual to dynamic communities of users, the Personalisation system described in the previous section needs to address additional requirements, which entail new research challenges. These requirements include the following: Community preference modeling and management Dynamic management of user behaviour models of individual users and communities of users Learning of both individual user and community preferences and habits Inheritance of preferences and behaviours through community hierarchies Dynamic community creation and membership based on similar preferences and behaviours All of these requirements introduce promising new research fields that are currently being investigated by the authors. VI. SUMMARY AND CONCLUSION Both pervasive and social networking systems are central to the development of systems for the future. While much work has been done on the development of pervasive systems the challenges are great and such systems are still in their infancy. Social networking on the other hand has taken off and is well established throughout the world. Integrating the two concepts is challenging but has potential for significant benefits for all. However, personalisation and context awareness are key features that are needed for both types of system, and a major problem lies in how these should be handled. As we have seen, following a user preference rule-based approach, each user will have her own set of user preferences and intent that can be used by the system to take decisions proactively to support her. These may be learnt by monitoring the user s behaviour and extracting patterns from it. However, in addition to individual preferences one now also has community preferences, goals, etc., derived from monitoring the actions taken by members of a community (e.g. the selection of a place to eat in scene 1) which may complement or conflict with the user preferences. This paper addresses this problem. It first describes how personalisation is dealt with in Personal Smart Spaces, which form the basis of the prototype pervasive system implemented in Persist. It then goes on to describe briefly the problem of personalisation in the context of a system that combines pervasive and social networking functionality. It provides an outline of the scope of such a system through some scenarios that are being used to drive the development process. There are several major challenges in developing this prototype, especially in the area of learning. Not only will this be used with data from monitoring the actions of an individual to extract user preferences at an individual level, but it will also seek to identify community preferences at the level of a community of users. We would also like to be able to learn how to identify when communities might be formed and propose this to a user. And so on. Finally, these approaches will be evaluated in user trials in three separate domains in the Societies project to assess their strengths and weaknesses and their acceptability to real end users. ACKNOWLEDGMENT Besides thanking the European Commission for their support for the PERSIST and SOCIETIES projects, the authors also wish to thank colleagues in the two projects without whom this paper would not have been possible. Apart from funding these two projects, the European Commission has no responsibility for the content of this paper. REFERENCES [1] M. Weiser, The computer for the 21 st century, Scientific American 265(3), pp , [2] M. C. Mozer, Lessons from an Adaptive House, in D. Cook & R. Das (Eds.), Smart Environments: Technologies, protocols and applications, 2004, pp [3] M. G. Youngblood, L. B. Holder and D. J. Cook, Managing Adaptive Versatile Environments, in Proc. 3 rd IEEE Int. Conf. on Pervasive Computing and Communications (PerCom 05), 2005, pp [4] T. Kindberg and J. Barton, A web-based nomadic computing system, Computer Networks 35, pp , [5] H. K. Y. Si, A Stochastic Approach for Creating Context-Aware Services on Context Histories in Smart Home, in Proc. ECHISE2005, Pervasive 05, 2005, pp [6] J. Groppe and W. Mueller, Profile Management Technology for Smart Customizations in Private Home Applications, in Proc. 16 th Int. Workshop on Database and Expert Systems Applications (DEXA 05), 2005, pp [7] M.Crotty, N.Taylor, H.Williams, K.Frank, I.Roussaki and M.Roddy, A Pervasive Environment Based on Personal Self-Improving Smart Spaces, in Proc Workshop on Constructing Ambient Intelligence, AmI 2008, Springer Verlag CCIS 32, 2010, pp [8] E. Papadopoulou, S. Gallacher, N. K. Taylor and M. H. Williams, Personal Smart Spaces as a Basis for Identifying Users in Pervasive Systems, in Proc. Int. Workshop on Ubiquitous Service Systems and Technologies (USST 2010), in 2010 Symposia and Workshops on Ubiquitous, Autonomic and Trusted Computing, IEEE CS Press, 2010, pp [9] T. Kindberg et al, People, places, things: Web presence for real world, in Proc. 3rd IEEE Workshop on Mobile Computing Systems and Applications (WMCSA 2000), 2000, pp [10] C. Prehofer, J. van Gurp and C. di Flora, Towards the Web as a Platform for Ubiquitous Applications in Smart Spaces, in Proc. 2nd Workshop on Requirements and Solutions for Pervasive Software Infrastructures (RSPSI), [11] R. Singh, P. Bhargava and S. Kain, State of the art smart spaces: application models and software infrastructure, ACM Ubiquity 7(37), pp. 2 9, [12] Williams, M.H., Taylor, N.K., Roussaki, I. Robertson, P., Farshchian, B., Doolin, K.: Developing a Pervasive System for a Mobile Environment. In: echallenges 2006 Exploiting the Knowledge Economy, IOS Press, 2006, [13] I. Roussaki, N. Kalatzis, K. Doolin, N. K. Taylor, G. Spadotto, N. Liampotis, M. H. Williams, Self improving personal smart spaces for pervasive service provision, Towards the Future Internet, IOS Press, 2010, pp [14] D.Q. Zhang, B. Guo, B. Li, Z.W. Yu, Extracting Social and Community Intelligence from Digital Footprints: An Emerging Research Area, Proceedings of the 7th International Conference on Ubiquitous Intelligence and Computing (UIC 2010), LNCS 6406, pp. 4-18, 2010