ROBOTIKY V BUDÚCNOSTI

Transcription

1 INTERNATIONAL SCIENTIFIC CONFERENCE IN LOOK DAYS OCTOBER 2011 AULA MAXIMA, TECHNICAL UNIVERSITY OF KOŠICE SLOVAKIA ZÁKLADNÝ KONCEPT IT A ZNALOSTNÉHO MANAŽMENTU PRE OBLASŤ ROBOTIKY V BUDÚCNOSTI BASIC CONCEPT OF IT AND KNOWLEDGE MANAGEMENT FOR FUTURE ROBOTICS MÁRIA VIRČÍKOVÁ 1, PETER SINČÁK 2 1 ING. MÁRIA VIRČÍKOVÁ, KATEDRA KYBERNETIKY A UMELEJ INTELIGENCIE, FAKULTA ELEKTROTECHNIKY A INFORMATIKY, TECHNICKÁ UNIVERZITA V KOŠICIACH, SLOVENSKÁ REPUBLIKA, MARIA.VIRCIKOVA@TUKE.SK 2 PROF. ING. PETER SINČÁK, CSC., KATEDRA KYBERNETIKY A UMELEJ INTELIGENCIE, FAKULTA ELEKTROTECHNIKY A INFORMATIKY, TECHNICKÁ UNIVERZITA V KOŠICIACH, SLOVENSKÁ REPUBLIKA, PETER.SINCAK@TUKE.SK MÁRIA VIRČÍKOVÁ 1, PETER SINČÁK 2 (EN) 1 ING. MÁRIA VIRČÍKOVÁ, DEPARTMENT OF CYBERNETICS AND ARTIFICIAL INTELLIGENCE, FACULTY OF ELECTRICAL ENGINEERING AND INFORMATICS, TECHNICAL UNIVERSITY OF KOSICE, SLOVAKIA, MARIA.VIRCIKOVA@TUKE.SK 2 PROF. ING. PETER SINČÁK, CSC., DEPARTMENT OF CYBERNETICS AND ARTIFICIAL INTELLIGENCE, FACULTY OF ELECTRICAL ENGINEERING AND INFORMATICS, TECHNICAL UNIVERSITY OF KOSICE, SLOVAKIA PETER.SINCAK@TUKE.SK Abstrakt (MATERINSKÝ JAZYK): Článok popisuje multi-agentový system pre vytvorenie siete pre roboty. Hlavná výhoda spočíva v tom, že ak sa určitú znalosť naučí jeden robot zo skupiny (siete) robotov, zdieľa túto znalosť s ostatnými robotmi v sieti. Úvod vysvetľuje koncept sieťových robotov a ich aplikácie. Predstavujeme aplikácie vyvinuté v Centre pre inteligentné technológie na Katedre kybernetiky a umelej inteligencie TUKE rozpoznávanie obrazu a povelov. Detailnejšie popisujeme náš prístup učenia robotov pozorovaním človeka, v ktorom sa robot učí od človeka pohyby alebo gestá. Klúčové slová (materinský jazyk): kolaborácia človek-robot, multi-agentový system, sieťové robotické systémy Abstract (EN) This paper describes a multi-agent system for creating a network for robots. The main advantage is that if some knowledge is learned by one robot from the group (network) of robots, it shares this knowledge with other robots in the network. The introduction explains the concept of networked robots and their applications. We introduce applications developed at the Center for Intelligent Technologies at the Department of Cybernetics and Artificial Intelligence TUKE- object and command recognition. In more detail we describe our approach to robot learning by observing humans, in which the robot learns from human movements or gestures. Key words (EN): human-robot collaboration, multi-agent system, networked robot systems Strana 1

2 1 Introduction Robots are functioning in environments performing tasks that require them to coordinate with other robots, cooperate with humans and act on information derived from multiple sensors. The specific objective of the field called Human-robot interaction is developing a series of tools to have a robust communication interface between robots and human users. As described in our previous work [1], in many cases, these human users, robots and sensors are not collocated and the coordination and communication happens through a network. It a relatively young research field, still quite dynamic and open to ideas and concepts coming from other research areas. As reported in European Robotics Research Network [2], Europe holds a strong position in this field, and it could become a world leader in the near future if enough support is provided. They mention these possible application fields, regarded as viable in the next ten years: Elderly care, where a network robotic system (NRS) can assist elderly people in their personal homes or retirement homes, with the goal to provide physical and cognitive support, to facilitate communication with and monitoring by remote relatives and care givers, and to detect and respond to emergencies. Security applications and intelligent buildings, where a NRS can track and classify the behavior of people, hazardous situations or threatening acts can be detected, and actions can be decided to maintain a safe status. Networked service robot systems, for applications such as trash collection, delivery and logistics, both in public spaces (e.g., city streets) and in private workplaces (e.g., factory floors). Flexible automation and collaborative manufacturing, where heterogeneous, cooperative robotic systems are expected to set forth the future of networked control in industrial settings. Wide-scale environmental monitoring, by deploying autonomous sensor networks with the ability to self-deploy, self-reconfigure, and self-repair, capable of monitoring in a largely unsupervised way large environments for pollutions, environmental threats, and other hazardous situations. Cooperative search and rescue, where NRS actively search for people or objects, and eventually support rescue personnel in dealing with emergencies. Networked robots allow multiple robots and auxiliary entities to perform tasks that are well beyond the abilities of a single robot. Cooperation entails more than one entity working toward a common goal while coordination implies a relationship between entities that ensures efficiency or harmony. Communication between entities is fundamental to both cooperation and coordination and hence the central role of the network. 2 Networked robots This new technology has being denominated Network Robot Systems and, as said in [3], includes the following elements: 1. Physical embodiment: Any networked robot system has to have at least a physical robot which incorporates hardware and software capabilities 2. Autonomous capabilities: A physical robot must have autonomous capabilities to be considered as a basic element of a networked robot system. 3. Network-based cooperation: The robots, environment sensors and humans must communicate and cooperate through a network. 4. Environment sensors and actuators: Besides the sensors of the robots, the environment must include other sensors, such as vision cameras and laser range finders, and other actuators, such as speakers and flickers Strana 2

3 5. Human-robot interaction: In order to consider a system as networked robot system, the system must have a human-robot related activity. FIG. 1 HIERARCHICAL SYSTEM ARCHITECTURE: THE INFORMATION FLOW BETWEEN SEVERAL HUMAN TELEOPERATOR REMOTE ROBOTS AND A HUMAN (SOURCE: [4]) It seems that the basic concept of creating Intelligent Systems will be based on the following principles: 1. Networked entities with defined hierarchy, fast mobile connection will be absolutely needed and for majority of applications in business and industry. 2. Knowledge sharing defined policy - domain oriented knowledge will be great value. 3. Integration of approaches into standard services provided for clients. 4. Appropriate division of processes for client (Agent) and server WILKI is based on application domain. 5. World Incremental Knowledge Integrator and its implementation is under discussion using Cloud Technology or hybrid of Public and Private Cloud Approach. With the ideas discussed above, we created a Multi-agent Serving System (MASS). MASS system is written in C# programming language. It can be used for different tasks, not only from the field of artificial intelligence. The system is based on client-server architecture, where clients ask the server and the server responds. Basic building blocks of the system are called plugins. Individual plugins can be connected to each other, and thereby build a functional algorithm (ActualSystem-AS) which consists of these building blocks. The system provides a graphical representation of individual system components, and thus the easier visual control. The server can run several AS simultaneously, each with a different functionality. Every AS can be connected to several clients at the same time. Strana 3

4 FIG. 2 MASS SYSTEM BASED ON CLIENT-SERVER ARCHITECTURE We can say that client-server architecture is a multiagent system. In the case that various plug-ins are programmed for command recognition and a client learns to recognize some particular word, all other clients are able to recognize it also. If one client learns to recognize some object, all other clients recognize it. 3 Cooperation between network robots and human beings Much research has been done on humans interacting with robots but especially the interaction between human beings and a group of connected robots provides interesting aspects. The general objective of our work is the development of new ways of cooperation between networked robots and human beings, which to date has been seldom explored in the field of Human-Robot Interaction. The IEEE Society of Robotics and Automation s Technical Committee on Networked Robots made the following definition of Networked Robots: A networked robot is a robotic device connected to a communications network such as the Internet or LAN. The network could be wired or wireless, and based on any of a variety of protocols such as TCP, UDP, or Many new applications are now being developed ranging from automation to exploration. There are two subclasses of Networked Robots: 1. Tele-operated, where human supervisors send commands and receive feedback via the network. Such systems support research, education, and public awareness by making valuable resources accessible to broad audiences; 2. Autonomous, where robots and sensors Exchange data via the network. In such systems, the sensor network extends the effective sensing range of the robots, allowing them to communicate with each other over long distances to coordinate their activity. The robots in turn can deploy, repair, and maintain the sensor network to increase its longevity, and utility. A broad challenge is to develop a science base that couples communication to control to enable such new capabilities. Strana 4

5 Human interaction with a multiple number of robots increases complexity, but can also be a source of benefits when considering the exploitation of the complementarities of the robots for perception and actuation. A team of agents should be able to learn collective behaviors, such as strategies to pursue their goals in the environment, in the face of competitors. Learning in multirobot systems is affected by specific challenges like multiple goals, noisy perception and actions, and inconsistencies in the internal states and in environment models between the individual robots. The first application and also the guiding application of the Multi-Agent Serving System MASS was the object recognition. It was developed by T. Reiff and can be better studied in [5]. The second step was a command recognition - at the date it contains a database of verbal commands in different languages. The command recognition was developed by Z. Fedor more information can be found in [6]. The key advantage of our system is if a user teaches a robot some word in some language /an object, all other robots which are part of the multi-robotic system are able to recognize this word /the object. Object recognition and command recognition are important components of the Human-Robot Interactive System in general, where a human user can teach one robot to recognize an object visually and all the robots within the networked group learn it or can give verbal commands to a single robot, or to a multi-robotic system. Also, in human-human teamwork, sharing information through verbal and nonverbal channels plays an important role in coordinating joint activity and we think this will be the case for human-robot teams as well. ` Our present work consists in teaching robots new gestures and movements. We are developing a tele-robotic approach, in which human shows to robot how it should perform a motion or a gesture. Our challenge was to create a system in which an inexperienced human teacher could develop new motion tasks via interaction process. Robots acquire knowledge of what to do and how to do it from observing human demonstrations. The algorithm calculates positions of each joint of human and maps it to the robot s joint. FIG. 3 IN OUR TELE-ROBOTIC APPROACH WE MAP USER S MOVEMENTS/GESTURES TO ROBOT Teaching through demonstrations requires bidirectional communication between a human user and a robotic student. The key question is how to map a teacher action to an agent action. These agent actions in our experiments are the motor capabilities of the robots. We mimic some user movements using Microsoft Kinect sensor device, a horizontal bar connected to a small base with a motorized pivot and is designed to be positioned lengthwise above or below the video display. The device features an RGB camera and depth sensor among other capabilities. Kinect can recognize 48 points of the human body and in we extract this information while he is moving in time. We obtain a vector containing information about the position of each point in space and map it to the related joint of the humanoid robot Nao (Fig. 3). Kinect provides between others a full body 3D motion capture capability. The hardware emits and infrared light which rebounds off object in the room. After processing the rebounding light, the system differentiates between people and other objects. The system gives the body a virtual skeleton with moving joints so the machine can follow user movements. From programmer s point of view, rather than having access to the pixels of a flat image, or even the depth information from the raw Kinect data, we have a description of users as a series of joints in space. For many applications, especially gesture-based interfaces and motion capture, this is exactly the data which we need to get started. Strana 5

6 FIG. 4 HUMANOID ROBOT NAO We work with humanoid robot Nao which has a human like appearance and various sensors for interacting with humans. It is nowadays commonly used humanoid platform for education environment. It is a medium-sized robot available mainly for universities and laboratories for research and education purposes. The creator is a French company, Aldebaran Robotics ( In community of artificial intelligence, by humanoid we mean not only a robot that has the physical appearance of a human, but also has a fairly advanced artificial intelligence, allowing it to reasonably approximate human behavior and interactions. As our robot increasingly resemble human in shape, we map his servos to the points of human body. This way we model different robot s motor skills via its interaction with several human users we personalize robot s behaviors according to the interaction of the specific human user. The algorithms used in our approach are discussed in the paper [7]. FIG. 5 HOW THE SENSOR DEVICE KINECT DETECTS A PERSON (BLUE) AND THE NAO ROBOT (GREEN) MASS is not trying to upload the functionality on the robot but it is using the robot as an input/output device while the processing is on a normal computer. With this approach a supported robot is only a plugin, i.e. small library with functions, which has a functionality to get video, audio, sensor data, etc. from the robot and the functionality to control a robot movement. Strana 6

7 4 Conclusion This work presented a multi-agent system, in which we implemented object recognition and command recognition. Our present work consists in tele-robotics approach and teaching robots gestures or any kinds of movements. A major challenge is to create robot networks that are proactive and anticipate our needs and commands rather than reacting (with delays) to human commands. We think it is very important for a middleware to be free and open source. Therefore anyone can try the web interface of the system, read the wiki pages, download the installation package or find the MASS repository address at There are also links to some of our demonstration videos with MASS. Acknowledgement This work is the result of the project implementation: Development of the Center of Information and Communication Technologies for Knowledge Systems (ITMS project code: ) supported by the Research & Development Operational Program funded by the ERDF. References [1] SINČÁK, P., REIFF, T., VIRCIKOVA, M.: Towards Intelligent Systems With Inceremental Ability. In: Quo Vadis Intelligent Systems. ISBN , 2011, pp [2] SAFFIOTTI, A., LIM, P.: European Robotics Research Network: Two Hot Issues in Cooperative Robotics: Network Robot Systems, and Formal Models and Methods for Cooperation. A white paper from the EURON Special Interest Group on Cooperative Robotics. (April 30, 2008). [3] SANFELIU, N., HAGITA., A. SAFFIOTTI.: Network Robot Systems. In: Special Issue: Network Robot Systems, Robotics and Autonomous Systems, [4] SCHILLING, K.: Networked Robots: Research challenges. In: ETSI Networked Mobile Wirelles Robotics Workshop. Hannover, [5] REIFF, T., SINČÁK, P.: Multi-Agent Serving System For Intelligent Technologies. In: ICCC, [6] FEDOR, Z., SINČÁK, P.: AIBO talking procedure in multi languages based on incremental learning approach. In: SAMI 2009 : 7th International Symposium on Applied Machine Intelligence and Informatics : January 30-31, 2009, Herľany, Slovakia. - S.l. : IEEE, [7] VIRCIKOVA, M., SINČÁK, P.: Integration of Verbal and Non Verbal Human-Robot Interactive System. In: Humanoids 2011 IEEE Conference. October 26-28, Slovenia. (In Press) Strana 7