The SCAMPI Scaleable Monitoring Platform for the Internet. Baiba Kaskina TERENA

Transcription

1 The SCAMPI Scaleable Monitoring Platform for the Internet Baiba Kaskina TERENA

2 Agenda Project overview Project objectives Project partners Work packages Technical information SCAMPI architecture Hardware adapters MAPI middleware Applications

3 SCAMPI goals SCAMPI Goals: Development of high-performance monitoring infrastructure up to 10 Gbps. Utilising standard PC architecture and providing simple installation Development of open and extensible monitoring architecture. Balance performance and flexibility, provide API for developing applications Development of monitoring tools. DoS detection, SLS auditing, billing and accounting etc.. Develop strategies for network monitoring up to 100 Gbps. Dissemination of project results.

4 Project Overview 30 months duration Started April 2002, scheduled to finish September 2004 Total budget = EUR 5.52 m, EC Contribution = EUR 2.88 m 10 partners: TERENA, The Netherlands (Coordinator) IMEC, Belgium FORTH, Greece Leiden Institute of Advance Computer Science, The Netherlands NETikos, Italy Uninett, Norway CESNET, Czech Republic FORTHnet, Greece Siemens, Germany Masaryk University, Czech Republic 5 Work Packages

5 Work packages (1) WP0: Requirements Analysis This work package will review existing technology and identify the future requirements of network monitoring. Lead Partner: Uninett WP1: Monitoring and Architecture Design This work package will study the various engineering options for designing an open and extensible architecture from the hardware level to the application programming interface. This includes the middleware for resource control, resource sharing and security. The architecture will be refined as the implementation progresses and more experience is gained from the experiments. This will culminate in a broader set of recommendations for next-generation environments. Lead Partner: FORTH

6 Work packages (2) WP2: System Integration This work package will implement the hardware and software components of the SCAMPI platform and undertake integration and testing. It will also develop monitoring tools such as denial-of-service detection, traffic engineering, monitoring and traffic analysis. Lead Partner: IMEC WP3: Experimental Evaluation This work package will evaluate the architecture, components and monitoring tools in laboratory, testbed and operational environments. It will provide feedback for architectural refinement and produce an overall assessment of the SCAMPI platform. Lead Partner: CESNET WP4: Project Management and Dissemination This work package is responsible for the administrative and financial aspects of the project. It will also maintain the SCAMPI web server, organise two workshops, and promote collaboration between project partners. Lead Partner: TERENA

7 A typical SCAMPI system PC - SCAMPI monitoring system Host processor Memory BUS Hardware Monitor Regular Network I/F To the Internet Monitored link Splitter

8 SCAMPI hardware Commodity Interface e.g. Intel pro/1000 Special-purpose adapter DAG card Network-processor-based card IXP1200 card Intelligent Switch From Juniper SCAMPI adapter Combo6 card

9 SCAMPI Software Architecture novel applications legacy applications lipcap pcap2mapi MAPI (Monitoring API) Remote access for monitoring apps Internet module constructor SNMP access job control system Filter module filters Adapter API MAPI2OS functions protection (OKE) module optimizer module mapper protection (OKE) user-level kernel-level Special-purpose Adapter Commodity Adapter Intelligent Router embedded functions embedded control in promiscuous mode filtering tools

10 MAPI Most applications will interact with SCAMPI through its Monitoring API (MAPI). The MAPI provides applications with the following functionality: Passive monitoring: Access to network packets Ability to install filters and code to process incoming packets Active monitoring: Ability to initiate active measurements Ability to receive results / packets of active measurement Job control: ability to submit jobs to be executed MAPI will support IPFIX MAPI provide abstraction the network flow sequence of packets that satisfy a given condition.

11 Applications (1) Intrusion Detection (FORTH) Will write an open-source Intrusion Detection application using MAPI interface; Will capitalize on the experience with similar publicly available applications, i.e. SNORT; Will detect intrusions based on a well-known set of rules; The application will not depend on the underlying hardware (any monitoring system that supports MAPI will be suitable for SCAMPI Intrusion Detection application); In the framework of the SCAMPI project the application will run on top the combo6 hardware.

12 Applications (2) Denial-of-Service (DoS) Detection (FORTH) DoS attacks flooding, TCP SYN attack, UDP flood, fragmentation attacks, ICMP attacks they led to changes in measurable characteristics of a network traffic flow; Application will be based on anomaly detection models, where the behaviour of a measurable network characteristic is compared to its normal behaviour; Application will be based on time series models, which take into account the time correlations of network traffic measurements; Later two specific algorithms, which require measurements of particular network variable, will be implemented: Generalized Likelihood Ratio algorithm Cumulative Sum algorithm The application will not depend on the underlying hardware (any monitoring system that supports MAPI will be suitable for SCAMPI Intrusion Detection application);

13 Applications (3) Billing System (FORTHnet) Will provide to an ISP ability to accurately measure various characteristics of network traffic, improve their services and provide advanced billing mechanisms and policies; Will be able to gather input data from many SCAMPI probes in order to provide aggregated statistics; Will provide the information to find the customers whose traffic volume results in a degradation of service quality or customers who experience such service quality degradation; The application will be based on the SCAMPI platform and the input will be the response from MAPI on specific requests e.g. Traffic volume that corresponds to a range of IPs Jitter on packets with high priority between a source and destination host The existing FORTnet s billing system (FOBIA) with the appropriate parameterization will be integrated with SCAMPI accounting application.

14 Applications (4) Quality of Service (QoS) Monitoring (IMEC) QoS monitoring analyses the behaviour of a specified or random stream throughout a system under observation; Two-layered architecture for QoS monitoring, i.e. QoS monitoring layer Application layer Monitoring layer will provide statistics of the observed network (packet loss, network throughput, application goodput, delay); Any application in the upper layer can request these end-to-end QoS statistics from the monitoring layer; All nodes will be equipped with a MAPI; The focus of this application an example of ease-of-use of the programmable SCAMPI architecture;

15 Applications (5) Flow-based reporting (Uninett) Flow-based reports uses the information available in flow record to produce a broad range of reports showing various information about the network traffic; The application will generate flow based reports based on the input from NetFlow/IPFIX records. It will take advantage of the SCAMPI flow record exporter which can export extra information not normally found in flow records; The components will be developed as part of the SCAMPI project, some of them will be based on already existing software like FlowTools; The web interface will allow users to browse the reports; Web interface will have access control support and mechanism for easy adding of new types of reports.

16 Summary Outcome: Open source software available to the community A common platform for development of traffic measurement tools A fast and advanced monitoring card

17 Questions? More information available at SCAMPI website: Questions?