Adding Web 2.0 features to a Fleet Monitoring Dashboard

Transcription

1 SpaceOps 2010 Conference<br><b><i>Delivering on the Dream</b></i><br><i>Hosted by NASA Mars April 2010, Huntsville, Alabama AIAA Adding Web 2.0 features to a Fleet Monitoring Dashboard Robert A. Foweraker 1 and Christian Kumpf 2 MakaluMedia GmbH, Robert-Bosch Strasse 7, Darmstadt, Germany MakaluMedia GmbH have developed a Fleet Monitoring web application for EUTELSAT S.A. called TeleViews. The satellite control engineers use TeleViews to access operational data, generated by the Mission Control System, from their own offices. In this paper we examine the reasons for creating systems like TeleViews and the reason why we chose to implement the system as a web application. We have been able to integrate several web 2.0 technologies and concepts into TeleViews in order to improve the usability of the system. We attempt to explain these concepts and describe how these web 2.0 technologies have been used in the development of TeleViews. We look briefly at the overall system architecture and discuss aspects of data visualisation, user collaboration and interoperability. I. Introduction Firstly, what do we mean by Web 2.0? This is a much-overused term, but a quick look at WikiPedia gives a definition that serves very well for the purpose of this paper. The term "Web 2.0" (2004 present) is commonly associated with web applications that facilitate interactive information sharing, interoperability, user-centred design and collaboration on the World Wide Web. Secondly, what do we mean by a Fleet Monitoring Dashboard? The web application described in this paper is the TeleViews system that we have developed for EUTELSAT S.A. This system acquires decoded Telemetry, Telecommand and Event data, together with ancillary data from the Mission Control System (MCS). This data is then transferred outside of the secure Control Centre network and stored in a long-term archive. The archived data is made available to satellite engineers on the office network through a web browser interface. Systems like this are increasingly common, although there are many differences in implementations and features. What are the motivations for moving certain support functions out of the Mission Control System? Satellite operators would very much like to take advantage of new technologies, however there are good reasons not to add them directly into the MCS. They need to ensure that the control chain is protected from risk and one way to reduce risk is to avoid the complexity that comes from adding too much new technology into the core MCS. The amount of data that a legacy MCS has to deal with is increasing all the time. They now have to handle a greater number of more complex satellites, which have more telemetry parameters than ever before. There is always a retrieval speed problem when extracting data from the MCS. By transferring the data out of the MCS, we can safely employ new technologies to handle the needs of many users retrieving lots of data simultaneously without risk to the core MCS. Improving the user experience is another driver. For the average user, there is an increasing contrast between the usability of the core MCS and their appreciation of what is possible with other office applications or web 2.0 applications. Their expectations have increased and web-based support tools can help bridge the gap and keep pace with future developments. Security considerations are also a driving factor. It is important to make the MCS as secure as possible by isolating its network as far as possible from the outside world. When taken to the logical extreme, this makes it hard for the engineers to obtain the raw data they need to do their jobs. By transferring all relevant data outside the MCS automatically in a secure fashion, we can empower the users by giving them access to the raw data so that they can process it themselves on their office workstations. Other support systems, for example an automatic report generation application, can then also access this raw data securely. 1 Satellite Engineering and Business Development Manager 2 Lead Software Engineer 1 Copyright 2010 by MakaluMedia GmbH. Published by the, Inc., with permission.

2 Therefore, the current trend is to move many of the support functions out of the core MCS. We believe that web applications are ideal for implementing these support functions. II. TeleViews System Overview In this section we provide a basic overview of the TeleViews system. Separate papers could be written on each of the components of the system, and we hope to do this in the future, however our aim in this section is to put the main subject of the paper into context and mention some interesting features, rather than provide a detailed description of the system. The overall design principal adopted during the development was to be as open as possible. The whole system is written in scripting languages and we make use of open source components where possible. All interfaces are well documented and hopefully easy to use. The first problem is getting the data out of the MCS. Acquiring the telemetry data is by far the greatest challenge, simply because of the volume of data and the accuracy required. A high-level requirement of TeleViews is to always let the MCS handle the data decoding (TM, TC, Events). This requirement ensures that we see the same decoded values within the MCS and within TeleViews. This approach also ensures that TeleViews has access to all derived parameters calculated by the MCS. A large amount of data must be acquired (several hundred samples per second, per satellite, for the telemetry alone) without affecting the performance of the MCS. The system must also handle downtime of the MCS or TeleViews itself, which can result in gaps in the decoded data. Once the data has been acquired it must be transferred out of the MCS. This involves transfer across a network firewall in a secure and robust manner. The system must also cope with network interruptions without losing of data. TeleViews stores all satellite data in a MySQL Relational Database. Storing the telemetry data is very challenging, though storing the system events is also not trivial. There are many telemetry parameters to store the latest satellites can have 60,000 separate telemetry parameters defined in the database. By storing only the changes it is possible to reduce the total storage requirements by at least a factor of 5. This is because in practice, approximately 80% of the parameters never change value, while 20% change value more or less constantly. Even when storing change-only telemetry samples we can still have over 400 telemetry parameters changing value per second for a recent telecommunications satellite. Inserting this amount of data into the database while maintaining good performance in data retrieval required an innovative schema. Ensuring that the database retrieval speed does not degrade as the archive size increases was a major design driver. In our experience the engineers don t like change-only telemetry data for the following reason. Is the absence of the data in the archive because the value didn t change, or because there was a TM drop or a gap in the data? To improve the user acceptance of a change-only data archive we need to be very careful to record the position of the gaps in the data; In other words, the engineers will accept a few gaps in the data, as long as they know where they are. EUTELSAT currently have 24 satellites and their fleet is increasing by several satellites per year. Scalability of the system was a very important requirement. This led us to separate the data acquisition & transfer function, the storage & retrieval function and the user interface function. The component under the highest load will always be the data storage & retrieval function the Archive Server. In TeleViews, the data storage function can be split over separate hardware if this is required. Currently we store the data for around 10 satellites on a single physical Archive Server. It is possible to move a data archive from one physical server to another without interruption of service. The data acquisition & transfer function is also scalable, although the current system only requires a single server. 2 Figure 1. TeleViews System Architecture

3 The user interface function is the web server that serves the web application. Ruby on Rails is used as the web application framework for TeleViews. The load on this web server depends on the number of users, and this is quite low in a satellite operation environment compared to a website with public access. Typically there may be 50 engineers using the system on a regular basis, and they will never all be using the system at the same time. Therefore a single physical server can easily handle this function, especially when other servers are doing the heavy lifting involved with the data retrieval. The User Interface (UI) server stores all the data related to the display definitions imported from the MCS. We detect changes in the satellite database(s) and automatically copy these files to the UI server and re-import the definitions into TeleViews. By splitting the UI server from the Archive server we have effectively separated the data from the presentation layer, which is always good in web applications. Now that we have given a brief overview of the system architecture we can move onto the main topic of this paper: how we can improve the usability of the system by introducing web 2.0 concepts. The following three sections look at this in detail. III. Data Visualisation The data as displayed on the MCS is constantly changing and the same is true of the TeleViews displays. Therefore static web pages are not suitable, the page contents need to refresh in the background without reloading the page. Fortunately this is very easy to do using AJAX, a major web 2.0 innovation. The displays are refreshed with data retrieved directly from the Archive servers. We also wanted to make the displays as interactive as possible as we share the satellite information with the users. Initially we tried to think of ways to rethink the standard display types, like the AlphaNumeric Displays (ANDs), charts and command stacks. There was much resistance to this idea from the users who wanted to maintain familiarity of presentation. Maintaining the same basic displays also reduces training costs. One key web metaphor is the hyperlink and we used this to the maximum possible effect. Everything that seemed like it should be clickable was converted into a hyperlink. We also wanted to make it as easy as possible for the engineers to find the data that they want as quickly as possible. One technology that is seen increasingly often on web 2.0 applications is the so-called mashup. This is where data from more than one source is combined in the browser. Good examples are web sites that overlay map images provided by another site with their data provided by another, for example the ground track of the ISS overlaid on a map of the world. The key concept is that the data can come from more than one source and that the data is combined in the users client - web browser (more or less). Browsers actually enforce a single site policy to protect against cross site scripting attacks, so in effect it is not as straightforward as it could be to combine data directly in the browser using JavaScript, but there are ways around this limitation. To solve this we set up a proxy in the UI server so that all requests for data residing on the Archive server are routed through it. This also has other advantages in practice: there is a single point of access to all data and therefore a single place to enforce user authentication. We can more easily restrict who has access to the data. Firewall rules can be set so that the Archive server is not directly available to the office users. One example of this combining of data, are the viewers for satellite telemetry, commands and events. The information regarding the display definitions resides on the UI server, this is essentially the data imported from the satellite database - the AND and chart definitions, information regarding the limit sets and calibration curves. The actual TM samples, commands and events are stored separately in the satellite specific Archive server. The UI server supplies the information to build the basic displays. The data is then fetched from the Archive server (via the proxy). We have plans to take this concept further in the future to extend the displays with data from both the operations schedule and the electronic logbook. The schedule contains information on planned events - for example a manoeuvre or an eclipse. The electronic logbook is a separate web application developed for the EUTELSAT Control Centre. The satellite controllers can use the logbook to log important operational events - for example problems with the ground stations, maintenance work on the MCS servers or new database distributions. All of this is potentially interesting information to display alongside an AND so that information on what was happening at that time is to hand. An engineer may be looking at an AND for a certain satellite at a certain time and information regarding the scheduled events for that satellite can be retrieved and displayed automatically alongside. Being able to combine all relevant information on a single display, or at least in separate browser tabs, is especially useful when an engineer needs to investigate a problem from home in the middle of the night. 3

4 IV. Collaborative Features and User Centred Design The classic Mission Control System requires a user login to restrict who has access to the system, they have not been extended, in our knowledge, with collaborative features. We will try to describe, by way of example, the kind of features we are thinking about. A large telecom satellite operator will usually have satellites from several manufacturers and generations. They are typically buying a few satellites each year and the satellites can have lifetime greater than 15-years. The satellite engineers tend to specialize on one platform or family of satellites - they form an engineering group. They are typically not interested in the other satellites. Within this group, the engineers are also often specialised in a particular subsystem and therefore the displays that they are interested in will be different. How can we help them here? For a start we can look at the web applications access rules. We group the engineers according to the family of satellites that they are working on, and restrict access to the satellite data accordingly. Secondly we can help the engineers find the displays that they are interested in more easily. We log each time an engineer; lets call him Bob, visits to a particular display. Now we can present Bob with a list of the displays he has visited recently or most often and because we will mainly be interested in this subset of displays, this list will become more useful with time. We can also give Bob the possibility to save his favourite displays in a separate list and access them more quickly. We can also store the times at which displays are viewed - engineers often want to look at a set of different displays at a certain time, and don't want to keep navigating back to that time on each display that they open. Bob may also be interested in creating a new display that is not really required on the main MCS, but is more useful for certain tasks he performs from his office rather than the MCS. Therefore we can allow him to define and save these extra displays. A second engineer, Alice, who works on the same satellite family as Bob, would also like access these extra displays, so we allow Bob the possibility of sharing his new display. He can share the display with another user, his whole engineering group, or with all engineers who have access to that satellite. Now Alice is investigating at an anomaly and finds something interesting at a certain time on a certain display, for example a chart with a spike in the data. Now Alice could simply all her colleagues about this, but the system already knows the engineers in her group, so instead she could tag this 'event', which is simply a display, a time period and comment and share it with her group - or other groups or individuals. Now when the other engineers log into the system, they could also see a list of recent events that have been tagged and shared with them on their own dashboard. They can then simply click on the displayed event and be taken straight to the relevant display at the relevant time. The comment(s) made by the other users would be visible alongside the data. The other users would be able to add their own comments. This becomes more powerful than sending s - the observations and comments are kept together with the data and can be kept for long periods of time. This is all a bit like some aspects of social networking websites. If we assume that all the satellite engineers are friends, hopefully a reasonable assumption, then each user s dashboard becomes enriched with content added through their own activity and that of their engineering group or other groups. This can be shared displays or tagged events or even general announcements regarding certain satellites, recovered from the electronic logbook. One of the aims of TeleViews is to facilitate this interactive information sharing and collaboration, in a way that is as painless as possible for the users. This is also the reason for describing the system as a dashboard each separate user will have a slightly different view of the system and we try to capture this view through a personalised dashboard. At this time, TeleViews is in a pre-collaborative stage, although the system has been designed with collaboration in mind. We are working on allowing users to create their own displays and share them. After this we will allow the users to tag events and share these too. Because we are outside the core MCS and working with a centralised web server, these kinds of changes are much easier to introduce without any risk. These collaborative features will add great value to the system and the value will increase over time as more displays and events are shared. V. Interoperability In TeleViews we have tried to provide different output formats for viewing the same data. For example an AND can be viewed as a web page, rendered in HTML, making extensive use of AJAX to update the data in the background at regular intervals. The same AND can be made available to other applications, therefore we can also provide a simpler output format, for example XML or JSON, containing the basic data at an instant in time. We have implemented easy to remember links for the different displays, such as the following: 4

5 This is a link to the AND XYZ123 of satellite SAT at time :00:00. We could also ask for this in XML format: Less obvious, but equally useful, is using the same easy to remember links to build new ANDs or charts in a simple manner: For chart data exactly the same ideas apply. We can have an HTML view showing the chart and updating in real time, and also one or more graphical outputs showing a static image of the chart at the requested time. Backend applications like gnuplot allow easy generation of charts with different output formats, like PNG, PDF or SVG. A vector format like SVG or PDF provides very high quality and allows for zooming without losing detail. However, when trying to plot many thousands of samples the file size of these formats can become very large. In this case an output format like PNG is more suitable. The UI server can make the decision on which format to produce based on the number of samples to plot. Figure 2 Different ways to view the same data XML or JSON output formats are also possible; where instead of an image of the chart we get the raw data that generated it. The charting feature can be used by other web applications. We previously developed another web application that is used to monitor the payload state of each satellite. This application models the communications payload and knows about the downlink channels and which transponder the channel is currently using. It also knows which telemetry parameters report the gain step settings for a particular transponder. Therefore, it can ask for TeleViews to produce a chart of the gain step for a particular channel and open this in a separate browser window. This avoids having to implement the charting functionality in two separate places. In TeleViews, we use link rewriting to make the URLs user friendly. The three different views of the chart data are actually handled by three completely separate controllers in the web application. The raw CSV data is actually generated by the Archive server and not the user interface server. This makes the links friendlier to the user, however there is another reason for taking this approach. If we remember the discussion in the collaboration section above, it would be interesting to extend the system so that users can create there own displays and save them. All we 5

6 need to do is save a list of parameters and the display type, and we can save the new display. If we want to let the users tag events, then we just need to save the display name and the event time along with a command. Because we have a well-defined API to fetch data from the Archive server, other applications can also retrieve the archived data and work with it. Web application APIs and data interchange formats are usually extremely easy to work with. This is because those interfaces that are widely used tend to be the ones that are easiest to use. To retrieve archived from TeleViews we have implemented an HTTP API, in fact this is exactly the same interface that the display viewers use to retrieve data. Most languages and programming environments support this kind of interface, for example Python, Java, C or Visual Basic. We use JSON as the main data interchange format, although we also support a Comma Separated Values (CSV) format to allow easy import of data into a spreadsheet. The data format is specified in the HTTP request. For example, the following URL would retrieve CSV formatted values: We are currently developing a separate automated report generation application that will be able to retrieve data from TeleViews using this interface. The users will be able to edit Python scripts within a web browser and run them in a restricted environment on the web server. These scripts can be scheduled to run daily checks or generate certain reports automatically. For example searching a complete days of telemetry for certain events or producing the daily list of all out of limit events for a particular satellite. This approach avoids installation of a programming environment on a users workstation and provides version tracking. VI. Some Other Advantages Another major advantage offered by web applications is that the only required software on the users workstation is a compatible web browser. New features can be rolled out by updating the software on the web server rather than having to update all the clients. This can save significant time in software support, both in deployment and getting permission for installing new software. Because TeleViews is only available to a restricted set of users, we do not have to support all web browsers and can concentrate on the most open, easily available ones, such as Mozilla Firefox. The TeleViews web application is built with scripting languages. Therefore it is usually possible to install a new version of the web application without any interruption in the service. Problems can usually be diagnosed and patches deployed without physical access to the hardware this can significantly reduce maintenance costs. VII. Conclusion Our experience of developing the TeleViews system has proved that web applications are an efficient and effective solution to many of the existing problems faced by fleet operators. Web applications are open by design and provide a solid foundation upon which to build more advanced features. The usability of the TeleViews system has been improved by reusing ideas and technologies developed for the World Wide Web and broadly considered to be web 2.0 features. We are currently improving TeleViews to add more collaborative features and the system architecture has been developed with this in mind. We look forward to being able to integrate data from several other sources into the TeleViews displays. Table 1. The TeleViews System in figures Number of Satellites available through TeleViews 24 Average number of telemetry parameters per satellite Telemetry samples per month for fleet (change only) ~2 Billion Average displays per satellite (ANDs and Charts) 1500 Average telemetry retrieval speed (formatted) ~100,000 samples per second 6

7 Appendix A Acronym List AJAX AND API CSV HTML HTTP JSON MCS PDF PNG SVG TC TM UI URL XML Asynchronous JavaScript and XML AlphaNumeric Display Application Programming Interface Comma Separated Values HyperText Markup Language HyperText Transport Protocol JavaScript Object Notation a lightweight data-interchange format (compared to XML) Mission Control System Portable Document Format Portable Network Graphics Scalable Vector Graphics TeleCommand TeleMetry User Interface Universal Resource Locator extensible Markup Language 7

8 Appendix B Glossary AJAX Mashup Asynchronous Javascript and XML This is a technique for refreshing the content of a web page without reloading it. The data is transferred asynchronously in the background and the page content is refreshed by JavaScript code manipulating the nodes of the Document Object Model. Although the word XML appears in the title, the data-interchange format can be any text. We use JSON widely in our web applications because it is much easier and faster to parse with JavaScript. In web development, a mashup is a web page or application that uses or combines data or functionality from two or many more external sources to create a new service. (from WikiPedia) 8