Jamie Bird. Remotely Monitoring Network Quality of Service with Low Cost Hardware. B.Sc. (Hons) Computer Science

Transcription

1 Jamie Bird Remotely Monitoring Network Quality of Service with Low Cost Hardware B.Sc. (Hons) Computer Science 20/03/2015

2 I certify that the material contained in this dissertation is my own work and does not contain unreferenced or unacknowledged material. I also warrant that the above statement applies to the implementation of the project and all associated documentation. Regarding the electronically submitted version of this submitted work, I consent to this being stored electronically and copied for assessment purposes, including the Department s use of plagiarism detection systems in order to check the integrity of assessed work. I agree to my dissertation being placed in the public domain, with my name explicitly included as the author of the work. Date: 20/03/1015 Signed: 2

3 Abstract With the increase of network capabilities, networks are becoming more depended on by both people and other services. Network reliability can be increased through the use of accurate network monitoring. However the monitoring tools currently used by administrators and service providers only provide a subset of the available statistics due the lack of monitoring devices. Also mostly being passive measurements, an idea of the networks capabilities is not provided. This project aimed to produce a low cost network sensor that can easily be deployed around the network and provide active measurements. A system was design using the Raspberry Pi as a sensor, implemented along with a central server and deployed. Throughout this project The Networking People (TNP), a local ISP, provided support though resources, their input for requirements from a companies point of view and feedback from testing. 3

4 Table of Contents 1 Introduction Aims Chapter overview Background Existing systems Related work Hardware Software Programming Languages Console Network features Throughput/Bandwidth Round trip times Jitter DNS server status Packet routes Packet loss Conclusion Design Stakeholders Requirements Network provider Network Administrator Network user Use Case Diagram Design Choices Hardware and Software Programming language Console style Architecture Overall architecture Server Application Server Application Architecture Sensor Application Architecture Requests and Results Protocol design Request Response Database design Connection storage Throughput storage RTT storage Jitter storage DNS status storage

5 7 Evaluation Impact on network load Fault detection Accuracy Benefits for the company Reliability Requirement analysis Network provider Network administrator Network user Packet loss storage Route storage Interface design Visualisations Frameworks and libraries Implementation Development process Implementation overview Sensor implementation Sensor application Plug and Play functionality Server implementation Server Application Protocol Web interface Logging Configuration files System Operation Installation Server Sensors Deployment User interface Testing Evaluation criteria Impact on network load Fault detection Accuracy Reliability Benefits for the company Methodology Impact on network load Fault detection Accuracy Reliability Benefits for the company... 48

6 8 Conclusion Review of the aims Lessons learned Future work Overall conclusion Working Documents: 6

7 1 Introduction As network capabilities are improving more and more people and services are becoming reliant on the constant network connectivity and more generally the Internet. The consequences of if and when a connectivity issue arises are becoming increasingly more severe, therefore in turn increasing the desire for high availability. A common objective for service providers is five nines reliability, assuring % up time, equalling only 5.26 minutes of down time per year. The increasing pressure on service providers due to these demanding expectations and sever consequences has surged an increase in the use of network monitoring technologies within the past decade. The monitoring of networks by network administrators gives a live view of statistics, which can then be used to analyse the health. These statistics can aid in the high availability by allowing modifications, and upgrades to the configuration before any interrupts happen. These precautions ensure that the resources are available to cater for the trends that have been noticed through monitoring. Although network monitoring is currently highly utilised, there are some areas where it could be improved. Currently network administrators understanding of the performance of their network is limited, with the edges where end hosts are located often missed. The scope of their view only includes networking devices, such as routers and switches that have been enabled to monitor network features as daytoday communication occurs. This scope is normally only a subset of the larger network under their control, they continue after these devices eventually to reach end hosts. Leaving the performance of these areas to be assumptions. Another downfall of theses measurements is that they are passively measured, which is more suited towards network troubleshooting rather than checking the quality of service. Passive measurements cannot be accurate representations of network service performance whereas active can. Active measurements can be used to emulate scenarios and provide stats that a true representation of how the scenario would actually perform. The combination of using sensors to monitor remote network locations and providing active measurements would improve the field of network monitoring for organisations that provide networked services. The most obvious and broad example being Internet service providers, but more specific providers such as storage and VoIP would still benefit. Benefits would arise from the organisation trustfully knowing the performance of the service from the customer s point of view, resulting in quicker fault detection and more accurate fault location isolation. This project proposes the idea of using the increasing popular single board computers as network sensors for network administrators to gain a better understanding of the quality of service at remote end hosts. The sensors shall be deployed to a remote location and connect to centralised server, which manages the sensors and organises the monitored data. 7

8 A local ISP, The Networking People (TNP) have expressed their interest in this project, highlighting the issues they have in the area of quality of service monitoring and their desire for a solution. An example given was that their customer service often receives calls from customers alerting them about Internet connectivity issues, that weren t evident from their existing monitoring software. Again this is a result of assumptions of quality of service from passively monitored data. In the simplest instance a DNS server was down and therefore customers could not access websites via their domain name and assumed all connectivity was lost. They have shown support for this project by providing resources such as hardware and allowing access to their production network for the testing and analysis phase. Also they have guided the requirements and specification by providing their insight from within a company to whom the project aims to be beneficial to. 1.1 Aims The main aims of this project are as follows: Design a sensor application that will run on low cost hardware, that actively and remotely monitors network features. Design a central orchestration application for the many sensors. Implement the two applications to create a prototype system. Conduct extensive experiments on the implemented prototype in operation and analyse the results. Judging the successfulness of this projects prototype implementation. Perform a user evaluation from a perspective client, network administrators, utilising the connection with TNP and their resources to further test the system and get their feedback. Assess the feasibility of the idea being produced and deployed onto live production networks, for the benefit of service providers, and what further requirements the system would need compared to the prototype. 1.2 Chapter overview. This report is dived into eigth chapters. The second chapter explains the different methods of network monitoring, explores the systems that are in use today, examines related work for similar motives and methods, looks at the possible hardware and software components useful to the project, and defines the list of network features. The design process for the system is documented in chapter 4. Chapter 5 follows the implementation process and chapter 6 describes it in operation. Testing and evaluation in chapter 6 and 7 descript the evaluation criteria, testing process, methodology and finally analyse the system, assessing if it meets the original specification. Finally the successfulness of the system is judged in chapter 8 along with future development and improvements. 8

9 2 Background The need for network monitoring is evident by the current employment of existing monitoring system by service providers and network administrators. The main drive for having statistics of the performance is troubleshooting when a fatality occurs, so therefore there is a high interest in a high granularity of monitoring points. The current subset of network statistics is mainly passive from dedicated devices, but in the event of failure it can be desirable to troubleshoot with more accurate active measurements from more remote locations. The methods that network measurements can be grouped into are passive and active. Taking measurements from devices whilst normal traffic is passing is classed as passive. While active measurements requires injection of traffic into the network, following and taking measurements from the traffics behaviour or from devices as a result of the traffic. Simply, active creates extra traffic whilst passive does not. The passive approach is used extensively in network management and trouble- shooting, however they are limitations to emulate error scenarios. Active enables emulation of scenarios and therefore analysis of Quality of Service. However there is a reason why active has not been accepted as much as passive, being that it causes disruptions to the network from the additional traffic. As one cause for network analysis is troubleshooting, adding additional traffic onto the network whilst not fully stable can cause more trouble. 2.1 Existing systems Network monitoring tools for administrators are not a new concept. Used by ISP s and even the smallest of networks to monitor many network features. Standards and protocols have been designed, implement and published precisely for the purpose of network monitoring. Simple Network Management Protocol (SNMP)[1], defined by IETF has been widely adopted and accepted as network management specification. SNMP provides administrations with the means to monitor, test, poll, configure, analyze, evaluate, and control networks under their supervision by being supported by most network devices. In the area of this project (network monitoring rather than management) SNMP works on a device level basis i.e. switches, routers and hosts. While gathering data per device tells us the current status of the device, it does not give us the performance of an overall service, which runs across many devices. A method that s more focused on monitoring is Remote Network Monitoring (RMON)[2], specified as an extension of SNMP. This is designed to be flow based rather than device based management, which would give a better indication of specific services. These two methods provide live passive measurements, which are highly insightful to network administrates by providing statistics on how the network 9

10 usage. Although these measurements cannot be fully reliable indicators of how the network will manage with additional flows created by a service. The RIPE Atlas [3] is a project in which anyone can volunteer to host a probe which primary purpose is to collect and relay real time active network measurements back to a controller. Analysis of this data provides statistics such as connectivity, DNS resolution performance and round trip times to fixed locations. As the aims for this project are to provide a real time state of the Internet, there has only been a single deployment of the system globally. Along with this being a closed source project, preventing an ISP from implementing its own instance, therefore being an unfeasible solution for the monitoring of a single network. 2.2 Related work The deployment of selfprocessing probes across a network for the purpose of monitoring has been employed before. In the patent [4] probes are deployed to collect data about their surrounding network and relay it back to a centralised network manager, the same architecture suggested for this project. Though the sensors objective does not match that of this project (to have the probe collect data of its surrounding network rather than to provide its own performance within the network) this approach proves that a central sever with deployed probes is a successful means of network monitoring. A system that does use sensors to monitor links (among other features) is the Network Weather Service (NWS) described in [5], used in could computing. The aim is to generate a forecast model so performance can be predicted at any time and compute selection based on the best prediction for the situation. Though standalone sensors are not actually used, rather included into the compute software, periodic active measurement such as latency and bandwidth are taken, measurements that can provide insight to service performance. It is also mentioned in the communication method used for (NWS) between the sensors and its central (logically, though physically distributed) server, using Unix sockets with TCP/IP. There is no mention of the network sensors performing any computation other than echoing what is received, leaving the central management to perform the calculations. Leaving no indication of the sensors performance that could have been impaired by other factors. The methods that are used at the sensors that will be used actually obtain the results have to be considered. Alternatives to the ad hoc methods for gaining network features have been explored as active measurements can contribute to the exact problem they are trying to solve. With network performance changing throughout the day due to factors such as working hours and scheduled service, the probing results are going to vary which is expected behavior. Therefore to detect an actual issue some intelligence is going to be needed. In [8] two anomaly/fault detection algorithms are proposed that are self- evolving (operating iteratively with one piece of data at a time). Having a low computation algorithm like these would allow network fault to be detected without 10

11 some complex analysis of the larger dataset. The complexity of the storage of the network data has been mention to be a concern for network monitoring. In [6] the issue of the speed of collection exceeding the performance of collection methods such as a standard SQL database. Recording all features of packets going via a specific network interface card does this monitoring, recording data much faster than a sensor system will, therefore more simplistic approaches would be suitable for this project but similar approaches to the structure could be used. 2.3 Hardware As the sensors would need to perform similar to that of regular hosts (only sending and receiving computation) to replicate the services transmission, the sensors hardware need to be of a similar architecture. Modern single board computers [9] appear to be the bestsuited solution due to their discrete form factor, low power consumption, standalone reliability and low cost. Analysing several contenders there are aspects that will be beneficial and crucial for the board to be the basis of the sensors. Ø Having an Ethernet NIC is crucial to allow for network connectivity. The specification of the NIC would also have to high enough so not to be a bottleneck. Ø Available operating systems have the ability to recognise and run programming languages capable of accessing socket API s at a suitable layer. Ø Power consumption needs to be low enough to be powered of a USB port and can stay stable for long periods of time. Ø Small enough not to cause a disruption. Ø Minimal cost, including accessories. The Raspberry Pi [10] series has been the most popular of this style of computer. Including a 100Mbit/s NIC would provides connectivity and will recognise speeds up to that provided to large organisations. Primarily using Linux kernel based operating system on an ARM11 chip; running OS s like recompiled Debain to run on the ARM v6 instruction set, with support for programming languages like Python, C and Java. Similarly to the Raspberry Pi there is an evolution with performance improvements called The Banana Pi [11]. The aspect of performance increase that would benefit this system would be the improved NIC, which now has a speed of 1Gbit/s. This would allow speeds beyond 100Mbit/s to be monitored accurately. They system could be in the situation where it needs to recognise 100Mbit/s, for example large organisations may have a direct link to their ISP and therefore receiving a higher speed. A higher performing solution would be The Beagle bone [12]. Although the better performance could allow the sensors be truer representations of host it come at the price of falling into a higher price range. A simpler and cheaper board is the Adruino [13] because of its less powerful 8bit microcontroller, whilst still providing low level programming with C and C++. 11

12 However this does not come with an Ethernet port but a 100Mbit/s adapter is available as an accessory. The Adruino s lack of operating system disqualifies it from the selection, as it would leave too much functionality up to the sensor application that would otherwise be provided by an OS. 2.4 Software Programming Languages As the system is going to require applications at both the central management and on the sensors, the possible programming languages need to be explored. Although almost every popular programming languages can be used for networking with the correct libraries, with some becoming prominent for generalised tasks with different libraries having a wide range of complexity abstraction. For two applications to exchange data with each other they need the use of sockets, which allow access to the data plane of the operating system and network hardware. Sockets are accessed by applications via libraries and application programming interfaces (API s) with the most popular being the Berkeley Socket Library (evolved into POSIX sockets) for Unix OS s and the Windows socket (winsock) for Windows OS s. These API s are written in the C programming language with other languages libraries being a wrapperbased library. This would be to take into consideration, as it is possible for overhead and loss of functionality to occur when moving to higher- level languages. Therefore C and C++ in combination with a Unix operating system have stood out to be the choice for high performance networking applications Console With the objective to present the measurements of the sensors to an administrator, different presentation methods need to be considered. In existing system and the related work three different approaches were used; a web console, a desktop client and a SQL interface. With a more interactive being more feasible by using a web console or a desktop client the SQL interface seems inappropriate for this project. A web console is the favourable interface for existing network monitoring tools used by ISP today. It gives the benefits of accessibility from any web browser, extensive visualisation libraries and ease of interaction with a database. The three main web technologies HTML, JavaScript and PHP have been used and documented countless time to implement the front to back end interaction with central data, that standards are set and readily available for quick deployment. A desktop client can provide some benefits over the web for the analysis of data. As the computation of a standalone client more effective than that of a within a web browser, indepth with expensive algorithms is better suited to this platform. 12

13 2.5 Network features There are many network features that would be of interest to the admins of ISPs. With the idea of a sensors spread out across the network, running tests on the sensors can produce the figures for these features. Using know methods, practises and algorithms to execute the tests, a representation of the networks performance can be gathered, either periodically or on demand. The following subset of features and methods are what would be most beneficial to an ISP and plausible to implement, but the system is in no way limited to these Throughput/Bandwidth Often considered the true measure of network performance, the speed of which quantity of data can be sent between two points is a vital piece of information. These results are simple to obtain by sending a fixed size piece of data and timing how long it is take to receive i.e. size/duration = speed. A familiar bandwidth- measuring tool is Iperf [14] this requires the application to be running on both the source and destination, becoming connection based. Estimates of the bandwidth can be made without being connection based using a call and response protocol such as ICMP, which most devices reply to. Sending a variation in packet sizes and calculating the time difference in replies. Many bandwidth estimation tools have been developed [15]. For example [7] investigates if active throughput measurements can be estimated accurately using a variation of small sized packets relative to the traditional approach that consumes the maximum available bandwidth. The findings from this study show that it is possible to estimate the bandwidth to a certain precision, a method that should be taken into consideration for this project as many sensors conducting experiments with large amounts of data could easily cause network congestion Round trip times Rather than the time taken to transfer a substantial amount of data, RTT is the time taken to simply communicate and receive a reply, import for timely delivery of information. Often referred to as ping it is available in a common command line tool in most operating systems, implemented using ICMP ECHO_REQESTs and ECHO_REPLYs, and timing the differences. Most devices will reply to these requests without any authentication, meaning that the RTT between the tester and any device could be tested Jitter The variation in delivery time of packets with a similar size is called jitter. This becomes evident in real time applications such as VoIP where there is a small timeout on delivery for reasons such as a set play out rate. Jitter often appears in congested network where queues within forwarding devices can change length suddenly. There are many parameters to measuring jitter such as duration and 13

14 payload size, with throughput effect the latter. An option for the previously mentioned iperf tool allows you to measure just by sending a stream of UDP datagrams. The jitter in round trip times could be measured by simple calculating the variance in a series of ping tests DNS server status When the primary DNS is down it can be misleading to think that all connectivity is lost. When users can t view a website, because the cant resolve the hostname, the first response is to think that Internet connectivity is lost. Being able to check the status of DNS servers could be beneficial by saving time while troubleshooting. To test the status of theses server either a simple ping to the server or actually perform a DNS query. A Unix tool, nslookup [16], is available for performing DNS queries Packet routes As packets can take different routes over a network, keeping track of what routes are being taken can be a helpful to diagnose anomalies in behaviour. The most used method for determining a packets route to a specified destination is the well know traceroute [17] tool, a command available for most modern operating systems. The implementation for this can vary but most of the time it is based around the ICMP protocol, incrementally adjusting the TTL to get an ICMP_TIMEOUT response from the different hops Packet loss The reliability of a connection can be compromised during congested periods. Using a protocol without assurance such as UDP, used by real time applications, can result in significant corrupt data. Acknowledgements of packets being received would need to be returned to calculate the packet loss and therefore either a TCP connection or reporting the number of received packets is requiring. Leaving iperf as the most viable option being connection based. 2.6 Conclusion From available literature it can be concluded that there is no system with that monitor quality of service within a network at remote locations, using the more reliable and accurate active measurements. The active method has some concern because of possible impact and therefore redundancy to the system it could produce, so extensive effort needs to be made to reduce the risk of active measurements and if successful, would produce a more accurate representation than existing methods and projects so far. All the tools required for the proposed idea are available including hardware and existing networking troubleshooting applications, providing this project with good starting resources, so to focus the interests onto the orchestration of these in a way 14

15 to save manual troubleshooting. How these resources will be used together to achieve the proposed prototype is explored in the following design section. 15

16 3 Design This section of the report aims to produce some design specifications aiding in the implementation of the remote quality of service monitoring prototype. Throughout this section a software engineering approach will be taken and standard UML notation will be used to create diagrams such as use cases and architecture, derived form the system requirements, before leading into producing more project specific design documents. 3.1 Stakeholders Identifying a list of stakeholders for the system can aid in requirement engineering. The stakeholders identified for this project are: Network provider the owner of the network the will be monitored and of which the sensors will be deployed. In most cases this will be a collective and generally an ISP. Network administrator an individual or collective responsible for the maintenance of the network and therefore most like to whom the monitored data will be of interest to. Network user individuals that consume the services on the monitored network. In most cases customers to the ISP. 3.2 Requirements A list of requirements that the final system should meet organised by stakeholder. In a discussing with TNP they have provided their insight into specifications from their point of view as a network provider and a network administrator. This requirement list has influences taken from appendix A, an from TNP. This list is at a highlevel and does not go into any detail about what services should be provided and technical details about the services Network provider ID Requirement Type 1.1 The system shall not reduce the performance of the current Functional network 1.2 The system shall not compromise network security Nonfunctional 1.3 The system should benefit troubleshooting of networking Nonfunctional issues 1.4 The system should benefit customer service Nonfunctional Network Administrator ID Requirement Type 2.1 The system shall provide monitoring data of service Functional emulation running at remote locations 16

17 2.2 The system should provide a true representation of a service Functional being ran on a end host, near to the sensors deployment 2.3 The system shall provide an admin panel Functional 2.4 The system shall allow for admin initiated service emulation Functional to be ran at a remote location and monitored data returned 2.5 The system shall alert network administrators of a anomaly in Functional quality of service at a remote location 2.6 The system shall be stable enough to not need reconfigured Nonfunctional once deployed 2.7 The system should provide a wide range of services Nonfunctional 2.8 The system should provide monitoring data in a visual form Nonfunctional Network user ID Requirement Type 3.1 The system shall not impact the quality of service for a Functional network user 3.2 The system shall not require any configuration from the Functional network user 3.3 The system should improve customer service Nonfunctional 17

18 3.3 Use Case Diagram A use case diagram was used to identify the interaction between the users and the system. Shown in figure 3.1 is the toplevel use case diagram for the system as a whole. Figure 3.1. Use Case Diagram 3.4 Design Choices Hardware and Software For the hardware choice for the sensors it has bee decided that a combination of the Raspberry Pi and the similar Banana Pi would be used. The reasoning for choosing two boards is the faster NIC on the Banana Pi, which would allow for higher network speeds to be measured accurately. In the case of TNP, they do provide services faster than 100Mbit/s to customers. The choice comes from the operating systems available being Unix based, with these two being the cheapest with the largest support community. A Unix based operating system is important because of the similarities to an actual end host, availability of networking tools discussed in the network features section (2.5) of the background and the tight integration with socket library in C, which is reasoned in the next section Programming language It was decided to use C as the choice of programming language for the server and sensor components. Although many languages can program networking 18

19 application, C has the most advanced and renowned API, which would provide more finetuning communications for more options in while emulating services Console style For the user interaction of the system, a web interface was chosen. This was preferable in comparison to a desktop client because of the objective being only to show and visualise data. None of the beneficial processing that would come with a desktop client would be necessary. The ease of development and simplistic libraries will also benefit with the time scope of this project. 19

20 3.5 Architecture Overall architecture The software design for the system will consist of two main components, a central server and sensors, with a onetomany relationship, in a client server pattern. At an overall highlevel view, within the server there are subcomponents server application, database, web interface and updater, while the simpler sensor only consist of a sensor application (broken down in the next section) and also an updater. The components database, web interface and server application follow a model- viewcontroller architectural pattern respectively. This approach is popular for web applications interacting with centralised data. The overall architecture is shown in figure 3.2. Figure 3.2. Overall Architecture Server Application The server application will be the point of which all communication (except updates discussed later) will occur form within the server component. This component will be broken down in a later section (3.5.3) but its main roles are: Listening and accepting connections connection to sensors. Organising and sending automated requests to sensors. Parsing user initiated requests from the user interface to sensors. Receiving responses from the sensors. Parsing responses from sensors and passing to storage. Closing connections from disconnected sensors Storage The storage of the data collected by the server application. This will be an installation of a MySQL database and therefore providing the standard SQL interface for insertion and manipulation of data. 20

21 Interface Will be the only user interaction point. Allowing them to control connected sensors, send request to sensors and visualise results via a web browser. Implemented using the main web technologies trio, HTML, JavaScript and PHP along with some frameworks and libraries. The web server will be an installation of the Apache, the most widely used web server. User authentication for the interface will be implemented as it will be globally accessible and being able to see statistics and control services on what could be a private network could be dangerous Sensor Application The sensor application will be the majority of operation within a sensor. This component will be broken down in a later section (3.5.4), its main roles are: Connect and remain connected to the server whilst powered. Receive requests from the server. Parse requests and initiate the respective tests. Parse results from tests and forward back to the server Updater As the sensors once deployed are supposed to be standalone units, configurations will also have to be deployed. With an updater subcomponent in both the server and sensors, acting in a server/client pattern, configuration files can be sent to individual clients Intercomponent communication Communication between components on the server will be made up of libraries providing interfaces from one language to another. Manipulation from the server application to the database will be the MySQL C library [18]. For manipulating and querying between the web interface and the database will be the PHP MySQLi library [19]. It is made possible for the user to initiate tests from server application via the web interface because of PHP being server side scripting, allowing PHP access to the machine the web server is running on. This would mean that the server application and the web server will have to be on the same machine Server/Sensor communication The server/client communication between the server and sensors has three forms: Requests and Response communication used for communicating the requests with parameters from the server to the sensors and the results from the sensors back to the server. This will be one connection per connected sensor that will remain open for the duration of the connection. The format 21

22 of these messages needs to be agreed on between the server and sensors therefore a protocol is designed in a later section. Testing traffic traffic that is actively injected onto the network to take measurements from. Communication can occur between sensor/server and sensors/sensor, depending on the parameters within the request message. A varying amount connections and traffic. Configurations configuration files sent from the server to the client. A connection is opened when a file is sent and closed afterwards Server Application Architecture Breaking down the server application component sees a semi layered approach, with sub components; automation, connection manager, database parser and communication. The sever application architecture is shown in figure Automation Figure 3.3. Server Application Architecture This subcomponent is in charge of scheduling the tests/data collection from the connected sensors. The collection will be automated but allowing user initiated tests from the web console. The automation of tests shall be intelligent so not too many are happening at the same time to prevent network congestion. Minimum interval variables will be used for the scheduling Connection manager Receiving a request from the automation module to send a test request, the connection manager is in charge of the storage and look up of connections as there will be a communication connection for each sensor that is connected Communication Used to send all of the requests, parameters and receive all of the results from the sensors Database parser 22

23 Passed the results received by the communication component, the database parser understands, breaks up and inserts them into the database. Also from the connection manager component there can be modifications to add or remove connected sensors from the database when they connect or disconnect respectively Sensor Application Architecture The decomposed sensor application also takes the form of a layered design pattern. The architecture for the sensor applications is shown in figure 3.4. Figure 3.4. Sensor Application Architecture Communication Component to receive, coordinate and reply to requests from the server via the designed protocol. Communication is through a single connection that is open for the duration of the sensors deployment Testing instance Upon the communication component receiving a request, a testing instance for the request is created. The testing instances are dynamically created and killed, each under its own execution in a new process Tools API As there are already tools that available for all the required features. It would be beneficial to utilise these as they have already been tried and tested, and have already been through the validation stages. Although they are readily available it is not as straight forward to implement them within an application developed separately. Several factors prevent them from being executed from either simple system calls or implementing the source code. Such as each having its own distinct output and parameters and some not being open source. 23

24 A way to gather all of these separate tools and make them abide to a certain standard would be to wrap them all into a single API to be called from the application. 24

25 3.6 Requests and Results Protocol design As the communication between the server and sensors will contain many parameters, values and options, the format needs to be agreed on across all devices. This would be achieved with a messaging passing application layer protocol. A request, figure 3.5, and response, figure 3.6, protocol is proposed and descried below Request ID Type Destination IP Parameter Parameter Length Value Value 32 bits Figure 3.5 Request Protocol ID a 32bit integer to match up with the response. As the number of request could become very large over time, 16bits resulting in a max of 65,536 could be considered low, so opting for 32bits would be the safer option. Type 16bits indicating the type of request. As the list of request type is already defined, having them equate to a 16bits integer constant would reduce the size required in the protocol header and ensure future development does not conflict with them. Constants are shown in table 3.1 Request types Constant Type 1 Heartbeat 2 Bandwidth 3 RTT 4 Jitter 5 DNS status 6 Traceroute 7 Packet loss Table 3.1. Request Types Length 16 bit integer for the variable number of parameter/value pairs. Destination IP 32 bit IPv4 address of the destination of the test. Only required by measurements that are not sensor specific. Repeating parameters/value pairs a list of 16bit parameter type and 16bit value pairs. A variable length list, as there can be many parameters per test. With the parameter type already defined a constant list will be used, as shown in table 3.2. Repeating parameter types Constant Parameter Value Request types Description 25

26 valid for: 1 Size Bytes Bandwidth Jitter Packet loss 2 Iterations Integer RTT Jitter Packet loss 3 Protocol Either UDP or TCP Table 3.2. Parameter Types Bandwidth Jitter Packet loss The amount of data used for the measurement. Iterations taken to complete the measurement. Protocol used for the measurement Response ID Success code Result Result Length Value Value 32 bits Figure 3.6. Response Protocol ID a 32 bits integer to match with the request. Success code 16 bits indicating the successfulness of the measurement. As many of the networking tools are ran for many iterations, an operation could be partly successful. Constants for the success codes are defined in table 3.3. Success codes Constant Successfulness 1 Successful 2 Partlysuccessful 3 Failed Table 3.3. Success codes Length a 16bits integer for the variable number of result/value pairs. Repeating result/value pairs 32 bits, 16bit result type and 16bit value. A variable length list as the networking tools can return many results for one test. With the parameter type already defined a constant list will be used, as shown in table 3.4. Repeating result types Constant Result Value Request types valid for: 1 RTT_avg Float, MS RTT 2 RTT_max Float, MS RTT 3 RTT_min Float, MS RTT 4 RTT_var Float, MS RTT 5 Throughput Float, Bytes/s Throughput 26

27 6 Jitter Float, MS Jitter RTT 7 Packet loss Int, Percentage Jitter RTT Throughput 8 DNS status Boolean DNS 9 DNS resolution Flaot, MS DNS 9 Hop IP Traceroute Table 3.4. Result types 27

28 3.7 Database design A design for the central database, consisting of the tables and detailed attributes for every entity. The database design was considered before implementation to keep it at the highest normal for data redundancy Connection storage As the IP allocation shall be done via DHCP, addresses may be reused and therefore could be given to more than one sensor over time. Connections therefore will have to have another unique identifier. sensor_id IP Active Start End Description Auto int, Primary Key Varchar(15) Boolean Date Time Date Time Varchar(30) Throughput storage bw_id Src Dst Transferred Time Proto Speed DateTime Auto Foreign Foreign Int, Float, Multi, Float, DateTime int, key to key to Bytes Seconds Protocol Bytes/s Primary ID of ID of (either key sensors sensors UDP or TCP) RTT storage rtt_id Source Destination min max avg dev DateTime Auto int, Int, Foreign Int, Foreign key Int, MS Int, MS Int, MS Int, MS DateTime Primary key to to Key sensor_id of sensors sensor_id of sensors Jitter storage jitter_id Source Destination Packet_size Packets_sent Jitter DateTime Auto int, Primary Int, Foreign Int, Foreign key Int, Bytes Int Float, MS DateTime Key key to to sensor_id of sensor_id of sensors sensors 28

29 3.7.5 DNS status storage dns_id sensor_id dns_server resolution resolution_time DateTime Auto int, Int, Varchar(15), Varchar(30), Int, DateTime Primary Foreign DNS server Address MS Key key to used for resolved sensor_id resolution of sensors Packet loss storage ploss_id Source Destination packet_size packet_loss DateTime Auto int, Int, Int, DateTime Primary Key Bytes Percentage Int, Foreign key to sensor_id of sensors Int, Foreign key to sensor_id of sensors Route storage As outputs from traceroutes are multi valued, multiple tables are needed to keep within the normal form. Routes route_id Source Destination DateTIme Auto int, Int, Int, DateTime Primary Key Foreign key to Foreign key to sensor_id of sensor_id of sensors sensors Hops hop_id route_id From To Time Auto int, Int, Foreign Varchar(15), Varchar(15), Int, Primary Key key to IP address IP address MS route_id of routes. 29

30 3.8 Interface design Speaking to TNP about their opinion on user interfaces of admin panels, to get insight of an actual network administrator, their needs of the UI are very relaxed. Having worked with and being comfortable with complex UI s in existing network tools, it does not need to have a high usability rating. As some form interface is require a single initial design was made, but focus is not essential on this design section. The initial interface design that will show the sensors that are connected to the system is show in figure 3.6. This will be the first screen once the network administrator has been authenticated. This consists of a graphical web of the sensors and the central server, interconnecting existing when data is available for a connection. The sensors are recognisable by their IP address. The latest sensor and link features can be attached to the nodes with multi choice check boxes. Clicking on one of the sensors you can see a more detailed interface for an individual sensor. Figure 3.6. Connected sensors interface Visualisations On an individual sensors page the past data of its features is available, either plotted onto a graph or in a table view. The different visualisations are listed below. Round trip time (server sensor) a line chart with the round trip time between the server and the sensor plotted against time. Round trip time (sensor sensor) a line chart with the round trip time between two sensors plotted against time. The second sensor can be selected from a drop down list. Throughput (server sensor) a line chart with the measured TCP throughput between the server and the sensor plotted against time. Throughput (sensor sensor) a line chart with the measured TCP throughput between two sensors plotted against time. The second sensor can be selected from a drop down list. 30

31 Jitter & Packet loss (server sensor) a table containing the results of user- initiated UDP test between the server and sensor, as it returns both jitter and packet loss. Jitter & Packet loss (server sensor) a table containing the results of user- initiated UDP test between the two sensors. The second sensor can be selected from a drop down list. DNS status a simple Boolean stating if the last DNS resolution was successful or not. DNS resolution time a line chart with the measure successful DNS resolution times for the sensor plotted against time. Also on the individual sensor page will be options for userinitiated tests. The options for these tests will be presented with HTML forms for user input, using tags such as text boxes, check boxes, drop down menus etc., and initiating the test on a submit. There will be pre set options that will allow the userinitiated test to simulate specific services that can be loaded in via a dropdown menu. For example a VoIP service could be emulated using UDP, with small but frequent packets and using the voiceadmit DiffServ flag Frameworks and libraries Bootstrap [20] is an increasingly popular frontend web framework providing HTML and CSS templates. Using this for the will ensure consistency in the console across devices without extensive work on style sheets, allowing for a quick implementation. D3 (Data Driven Documents) [21] is a JavaScript library for the creation of interactive and dynamic data visualisation. This will be used for the organisation of the connected sensors into a topology like layout and adding the relevant information, also plotting the results onto graphs. This library will allow quicker implementation. 31

32 4 Implementation This section of the report discusses how the network sensor system was implemented. Separating the two main components, server and sensor, and describing the tools, methods and practises used within each. Some extra components only included during the development phase, such as logging and configuration files, are explained. 4.1 Development process This project adopted an iterative development process throughout. It was decided that features should be implemented in turn because of the range and amount of features, and this would prevent partial implementations. It was realised that the entirety of the project may not be completed in the time frame given. In combination with version control, using Git and GitHub, it was possibly to allow falling back to previous iterations in the case of nonworking components. 4.2 Implementation overview A technical implementation overview diagram is shown in figure 4.1, being an updated version for the overall architecture diagram in the previous section to include modifications and technologies used. The updater models have been removed as it was discovered with it being quite a thin client, all the configurations that sensor needs are never going to change or are provided to it any way. Some extra modules needed to be incorporated to enable the communication that was assumed during the architectural design. How each module was implemented and why new ones are needed is discussed in the rest of this section. Figure 4.1. Implementation Diagram 32

33 4.3 Sensor implementation Sensor application The sensor applications running on the Raspberry Pi s initiates a connection to the server. This is a single connection per sensor, using a POIX socket with the options: Address family AF_INET Internet protocol v4 Needed to communicate over the Internet. Socket type SOCK_STREAM Connection based stream. Used over a datagram/connectionless based for the reliable two communications. Protocol 0 TCP. The only supported for the stream based socket type. Once successful connected the application is essentially a receive loop, making use of the Unix select() system call that monitors the socket and waits until it has data to be received. The select function uses a timeout parameter that if exceeded will close the connection, this was needed for secure closing of the connection. As the only method of physically disconnecting a sensor from the system is to unplug its power cord, the sensor does not have a chance to initiate the 4way hand shake that is a TCP connection tear down and therefore has to be initiated from the server, as a result of lack of replies from the sensor. Regular communication between devices is ensured with heartbeat requests and replies periodically being exchanged between the server and client. The heartbeat interval is set at 1 second. In relation to the timeout variable (variations explored in the analysis section) defines how long it takes to recognise a disconnected sensor. Within the receive loop, the request is analysed and parsed to get the request type and then networking tool (selected in the background section) selected for the network feature is ran. Although parameters can be sent with the request, they are optional, so defaults are used if they are not provided. The results are returned using the same single socket. The networking tools are ran from with the application using a pipe stream to and from the application to the tool. This is done in C using popen() when then treats the process as a file descriptor. Dynamic forking is used to create new processes for requests that will in turn run commands that take a substantial amount of time and then respond with the results. Dealing with requests in separate processes avoids interferences in the situation where two tests could occur at the same time. As the returned results for many tests include a time value, a blocking command such a popen() would cause inaccuracies. The actual receiving of requests could also be blocked, resulting in a buffer overflow in the receive buffer. The creation and termination of these processes needed to be handled with care, as the variation in execution time of the testing processes there is a lack of communication to and from the parent process, including waiting, creating zombie processes. As zombie processes cause memory leaks, if repeatedly occurring over 33