Commotion Network Dashboard Application for Commotion Wireless Request for Development Proposals

Transcription

1 Commotion Network Dashboard Application for Commotion Wireless Request for Development Proposals Date: Prepared by: Jernej Kos Institute for Development of Advanced Applied Systems 1. Overview In this proposal we present our vision for the Commotion Network Dashboard, based on our open source platform for building network dashboards. Our proposal combines efficiency, security and platform independence with modularity and extensibility that will enable the dashboard to be easily extended also in the future. We have several years of experience in developing network dashboards suitable for supporting operations within community wireless networks during which we have successfully developed, deployed and maintained our own dashboard within a country-wide community wireless network called wlan slovenia. It currently monitors more than 800 nodes. This is the reason why we will be able to leverage real-world experience to provide the best possible solution within the proposed budget. 2. License The core nodewatcher platform and all developed modules are going to be licensed under GNU Affero General Public License version 3. The documentation and other non-code elements are going to be licensed under Creative Commons Attribution-ShareAlike license version Architecture The proposed Commotion Network Dashboard is going to be built using our open source platform called nodewatcher, that is being developed specifically for building different kinds of network dashboards, focusing on community wireless networks. Using the nodewatcher platform as a basis for development of the Commotion Network Dashboard enables us to build an extensible and modular network dashboard, a feat that would otherwise not be possible with this budget. Description of the architecture is split into multiple sections. First we describe the core nodewatcher platform and its components that will be used for developing the Commotion Network Dashboard. We further show how the platform enables modular design and quickly describe the core components that enable handling node/network monitoring, time series data collection, storage and visualization all with future extensibility in mind. 3.1 Platform The nodewatcher platform's server part is written in Python and uses the Django web framework. All functionality is split into different modules some modules represent core functionality while other provide dashboard-specific extensions. These modules are actually standard Django applications using the core infrastructure that the platform provides. Such structure is already known to other developers using the Django framework, which makes code maintenance easier. Core functionality entails components that are normally used when building a network dashboard. Not all components will be used by the Commotion Network Dashboard as the RfP specifications don't mention some functionality this will be indicated (by a *) together with the component description as

2 a possible later dashboard extension 1. We present a quick overview of components here: Database Model Registry To enable an extensible schema for node models and to enable automatic generation of configuration forms in the frontend, the platform provides a thin abstraction layer over the standard Django ORM. This abstraction layer is currently used for storing two different kinds of per-node data one is node configuration and the other are current values reported by the monitoring system from live network feeds. In both cases, the registry abstraction serves as a way to a) access models related to a node and b) for modules to simply add new configuration or monitoring models and have them integrated with the rest of the platform and other modules. At the same time, we are also developing a common community wireless network core schema that can describe nodes in all community networks using such an abstraction enables easier exchange of data between different community networks. For the Commotion Network Dashboard we are going to develop additional configuration and monitoring models that are required by the specification (detailed in further sections) and when possible (when the modules are generally useful) we will also consider including them into the core schema. Resource Allocation from Pools* Used when some resources must be allocated to nodes from a common pool. This includes IPv4 addresses for configuring interfaces on a node when addresses are not automatically generated by a node (as this is hard to do in IPv4 environments when we want to avoid collisions). This component will not be directly used in Commotion Network Dashboard at this time as the Commotion-OpenWRT nodes use auto-configuration. Monitoring, Data Storage and Visualization In order to enable the dashboard to monitor live network operation, the monitoring component is provided. It enables any module to a) implement a new data collection method that fetches data from nodes and provides it for other modules to process, b) process this data and possibly generate events when detecting anomalies, and c) access current and past data for performing further analysis and generating a visualization. The current snapshot of the data collected is stored into the standard relational database via the registry abstraction. For historical viewing of time series data and later visualization, we have developed a component called datastream. This component is independent from the base nodewatcher platform and can also be used elsewhere. Its currently implemented backend enables the storage of time series data into a non-relational MongoDB database, taking care of proper aggregation and computation of derived streams such as counter-based derivatives. This data can then be directly used by the frontend to support an interactive graph visualization using HTML5 and Javascript libraries which are also going to be used in the case of Commotion Network Dashboard. Firmware Image Generator* The platform supports generation of firmware images that are already configured for specific nodes. Any platform module can register so called CGM (Configuration Generating Module) functions that use a node's high-level configuration stored via the model registry abstraction and outputs configuration for a specific platform (currently, only OpenWRT is supported, but the whole system is extensible and can support any platform). Also, the image generator includes a database of device descriptors, containing hardware characteristics of individual router devices enabling the generated configuration to be generated for any hardware that the platform supports. This component enables simple node provisioning with no configuration required at the nodes themselves and all configuration being centrally stored and validated in real time via the monitoring component. Commotion Network Dashboard does not currently specify a need for such a way of node configuration and provisioning, but it is possible to add as a future extension. 1 Please note that any features marked as future extension possibilities are not included in the proposed budget calculation.

3 OpenWRT Package Feed This component is the client part of the nodewatcher platform. It is a package feed for OpenWRT containing packages that enable monitoring and management of remote nodes. These packages will be upgraded with Commotion-related functionality and since Commotion platform is also based on OpenWRT, most of the packages can be reused and new ones added. We have designed all custom packages to be fully configurable via UCI (the Unified Configuration Interface that OpenWRT uses for all of its configuration). We will follow the same pattern for all packages developed for Commotion Network Dashboard as this will enable interoperation with other packages. 3.2 Modules In this section we describe the proposed modules that will be developed in addition to the nodewatcher platform. For each module we specify all the components that will be developed, based on the platform description above (for example, what configuration models are going to be developed, what monitoring processors are going to be developed, what client functionality will be developed etc.) Module: Login, Account Registration and Management General: This module enables registration of new user accounts and management of per-user settings. The module will support standard ways of confirming registration (confirmation via a link sent to a user's address, approval by an administrator), registration form will include a recaptcha Module: Networks General: The networks module enables the support and management of different networks within the same Commotion Network Dashboard deployment. Each node is configured to be part of a specific network. Configuration Schema: A model linking a node with the network instance needs to be developed. This functionality is similar to different projects support that is present in the core schema Module: Network Statistics General: This module enables the collection and display of various per-network statistics. Such statistics include things like the number of nodes in different states (up, down, pending, etc.), number of nodes grouped by different maintainers, by different hardware, the number and length of links, etc. Monitoring: Per-network statistics will be computed and updated by a processor using the monitoring component of the nodewatcher framework Module: Maintainer Contact Information General: This module will provide the ability for node maintainers/network administrators to store private maintainer contact information inside the dashboard. Visibility of this information will be configurable, but limited to maintainers/administrators by default. Configuration Schema: A new configuration model containing fields for specifying contact information will be developed Module: Node Security Associations General: Each node needs to establish a two-way security association with the dashboard server component. We propose to use X.509 certificates and HTTPS for this purpuse. These certificates can be

4 self-signed and generated on the nodes (the only problem here is the lack of entropy for generating the cryptographic keys when nodes are booted up) or generated by the dashboard and integrated into the firmware (this can be achieved by leveraging the Firmware Image Generator component of the nodewatcher platform). The module is responsible for verifying and recording the node's fingerprint into the node's configuration as a security association item. A mode of operation where an administrator is required to confirm the fingerprint's validity can also be implemented. Once a fingerprint is confirmed it is validated on each monitoring run and the resulting secure channel can be used for communication between the node and the dashboard. Security associations also include SSH key fingerprints which can be used for verification when performing remote upgrades. Configuration Schema: A security association model needs to be designed. It will include fields for the type of the association (X.509 certificate fingerprint for HTTPS, SSH fingerprint, etc.), the association data (cryptographic fingerprint value), time of establishment and (optional) whether it has been confirmed by an administrator. Monitoring: The monitoring module will check whether a valid security association exists for the node that is being checked. If it does, it will use any valid association (in theory, we can support multiple) and if one is not found, the channel will not be established and a problem with the node will be reported. In this case, no monitoring data will be fetched from the node. When no valid association exists, the public key fingerprint will be recorded and stored into the configuration registry as an unconfirmed association (depends on policy if it is required that an administrator confirms the association or not). Client: On the client a HTTPS capable web server will be used we recommend the uhttpd daemon with libcyassl support (this is a small library implementing, among other things, TLS). This will serve as a secure channel between the dashboard and the nodes Module: Firmware Images Without Configuration General: For supporting the Commotion Network Dashboard model of provisioning nodes where all nodes with the same hardware get the same firmware image without any per-node configuration being generated. This module will enable the user with maintainer role to obtain a firmware image for his node(s), where the system will select the proper image automatically based on the selected hardware type. Functionality provided by this module will be built using the Firmware Image Generator component provided by nodewatcher platform Module: Node Hardware Type Discovery General: In order to enable automatic node registration into the dashboard, it should be possible to automatically discover the hardware that is running on a node. To perform this, existing nodewatcher device descriptors will be extended to support reverse device resolution. Monitoring: Module will register a processor that will interpret hardware type data provided by the node client and set an appropriate hardware type. An option to automatically update node's existing configuration will be provided. Client: Addition to nodewatcher-base OpenWRT package so that the node's hardware type is reported based on information available from the node's Linux kernel Module: Dashboard Administration General: The dashboard administration module will enable global management functions. This

5 includes managing user accounts, setting up user roles and permissions, managing networks. The administration module will be built as a container where different modules can register their additional functionalities so for example the Networks module can add management of different networks to the dashboard administration. Core Platform: Some changes in the core platform regarding permissions handling might be necessary in order to support roles (sets of permissions). The core platform permissions are based on the Django Guardian framework which is an extensible framework that uses standard Django APIs Module: Node Information Display and Modification General: A frontend module that is able to display current node information to the user. This information includes the node's geographical location (when available) using a third-party mapping solution (OpenStreetMap or Google Maps). Also included are any data items obtained via the monitoring system and stored into the monitoring schema by other modules including topological position of the node. This module will also include visualizations of statistical data obtained via the monitoring system via the datastream interface. All visualizations will be rendered client-side and will be interactive the user will be able to select timespans, zoom in/out of the chart, pan the chart etc. 3.3 Deployment Requirements Commotion Network Dashboard server component is going to be deployed in the same way as any Django-based web application. All third party dependencies will be installable either via standard Python package repositories (eg. pypi) or as standard Debian/Ubuntu packages. Full installation documentation based on a standard Debian/Ubuntu Linux distribution is going to be included. The client component will consist of standard OpenWRT installable packages that will support configuration via UCI. 4. Deliverables The following items will be delivered: 1. Source code for nodewatcher platform and Commotion Network Dashboard, if needed in the form of a Git repository. All development will be done in the open on GitHub, unless something different is required by the contract. 2. Documentation for both, nodewatcher platform and the Commotion Network Dashboard. This includes code reference, development documentation with examples of API use, installation instructions and user manual. This deliverable will be provided in the form of Sphinx compatible restructuredtext documents (source) and HTML/PDF files (built documentation). 5. Development Timeline Duration of this development project is 12 months. We envision the following milestones during Commotion Network Dashboard development cycle: M1. [month 2-3] Necessary updates to nodewatcher core platform developed these are mostly backend changes with test cases and with basic frontend code. M2. [month 7] Most of Commotion Network Dashboard modules developed and integrated together with the corresponding frontend parts. At this stage the dashboard is functional

6 feature-wise and in a deployable state, but might have some rough edges and lacks documentation. M3. [month 11] Further polishing of the dashboard, full documentation ready. M4. [month 12] Complete Commotion Network Dashboard with full documentation. 6. Budget Budget breakdown is as follows: Price Taxes Final Coordination with OTI $ $ $1, Updates to nodewatcher platform $2, $ $3, Development of Commotion Network Dashboard modules $8, $3, $12, Documentation $2, $1, $4, Budget $14, $5, $20, Team Jernej Kos (jernej@kos.mx, is a computer science researcher, software developer and network engineer with over nine years of experience. He enjoys working on interesting projects, specifically with backend architecture and low-level details. He has experience with scalable web application development, development of software for embedded devices, routing protocol internals and more. In 2005 he co-founded a software development and consultancy company specializing in the above fields and also freelances and consults on various interesting projects. He is also involved with open source projects, the most prominent being wlan slovenia, where he has developed a modular platform for network monitoring and provisioning. His current research interests include secure, privacy-aware decentralized services and their uses. Mitar Milutinović is a 2nd year Computer Science PhD student at UC Berkeley researching collaboration tools on the Internet. He is managing various open source projects in this area, most notably wlan slovenija, an open wireless network of Slovenia. He is interested in how networked societies work and how tools can help people improve ways they work together. Luka Mustafa is a masters MEng Electronic Engineering with Communications Engineering MEng student at University College London (UCL) on gap-year working full-time on KORUZA ( development. In 2013 he received BEng degree at UCL with the project VALET. He is actively involved in wlan slovenija community wireless network and the construction, deployment and management of national and international wireless backbones. He actively pursuits the development of new and efficient systems by repurposing mass-produced components and equipment and contributes to several electronics projects worldwide, with intern experience in particle accelerator control systems in Cosylab.