Frysk The Systems Monitoring and Debugging Tool. Andrew Cagney

Transcription

1 Frysk The Systems Monitoring and Debugging Tool Andrew Cagney

2 Agenda Two Use Cases Motivation Comparison with Existing Free Technologies The Frysk Architecture and GUI Command Line Utilities Current Status and Future Road map

3 What is Frysk Always on system monitoring and debugging tool Lets Users: Observe running threads, processes, and hosts Follow locking and other more complex primitives Debug complete systems Front row seats to system s execution

4 Use Case #1, K. K. is a compiler developer Spends a lot of time running the test suite, constantly monitoring for compiler crashes For each crash, K. has to go back and reproduce the failing scenario under the debugger so that it is possible to diagnose what is going wrong

5 K. Uses Frysk K. starts Frysk Sets up a monitored environment Establishes observers for a terminating program

6 K. runs tests under Frysk K. runs the test suite Entire test run monitored Frysk detects crashing programs K. notified

7 K. examines crash K. can examine the crashing code

8 Use Case #2, S. S. is a System Adminstrator Having trouble with the web server Occasionally, but always at 2am, the shopping cart daemon dies S. is not getting much sleep

9 S. uses Frysk Implements a custom observer to take the following actions when the shopping cart dies: Sends a page to S. Saves the log information Disables the shopping cart

10 Motivation Customer Driven Need to Leap Frogs 20 year old technology Red Hat Initiative

11 Project Goals Graphical, fast startup, low overhead Always On Non stop C++ just works Also native and interpreted Java Scalable: O(1) or O(log(N)) algorithms

12 Project Goals, continued Powerful and complete Thread Model Thread groups, OpenMP and MPI Programmable Core Java, C++ Scriptable Python/Jython Integration with user level SystemTAP

13 Underlying Technologies Linux 2.6 Kernel Native POSIX Thread Library (NPTL) ELF (libelf) and DWARF (libdwfl) libunwind Implement in C++, Native Java GUI driven design Java GNOME

14 System Analysis Tools Oprofile SystemTap Frysk Statistical System Sampling Access to Full Model of System

15 Frysk vs Traditional Debugger Non stop All threads, processes, and hosts Auto attach Assume optimized Debug info on demand; on disk Implement in C++ and Java; Object Oriented; Event Driven; Observer Based Stop Start Single Thread and Process Explicit Attach Assume O0 Debug info loaded immediately; in memory Implemented in C; Procedural; Blocking; Polling

16 Frysk GUI vs Eclipse CDT Light Weight Integrated Wide band direct memory Java GNOME Code Display based on Text and Debug info Can be built as Eclipse Plug in Heavy Weight Framework Separate debug process Narrow band socket SWT Code Display based on Text files

17 The Frysk Architecture Common ABI GUI fstack ftrace Eclipse Frysk's Core CLI Code Model Lang Model Process Model Kernel

18 Frysk's Core Provides Object Models, or Views of System Process Model: Task, Process, and Host Language Model: Method, Stack, Variable Code Model: File, Line, Source code Strictly Event Driven: never blocks on kernel Asynchronous: separate dedicated thread CLI library based on HPDF Specification

19 Process Model Host tower.redhat.com Proc firefox Proc httpd Proc gnome terminal Task PR_Select Task PR_Select Task main Proc bash Proc bash Proc grep

20 GUI View

21 Monitoring, Multiple Observers

22 Opening Source Window...

23 ... Source View

24 Language Model Frame Method Frame Method void b() Method void do_something() Method int c() Frame Method Variables

25 Code Model Implemented with CDTParser Provides Static Information: File and Line Source Code, including variables and functions Combined with Language Model by GUI

26 Example: In line Code 1

27 Example: In line Code 3

28 Example: In line Code 2

29 Stand Alone Utilities Implemented as Wrappers around Fryk's Core Fstack Works on one or more processes Symbolic back trace produced off line Ftrace Similar to strace Correctly tracks clone and fork

30 Current Status Tracks threads, and processes Monitors System Calls Command Line Interpreter C++ parser; thread group parser; tab completion Terminal Window Minimal Program Control stop on event, resume

31 Current Status, continued Running on FC 5, and RHEL4 U4 Using GCC 4.1 Tech preview in RHEL4 U4 Using GNOME 2.8 back ported to RHEL4 RPMs available for: X86 on RHEL4; X86 and x86 64 on FC 5

32 Road Map PowerPC 64 support Xen Integration Debugging Optimized Code Address Map and Heap view Ongoing C++ Corefile Debugging Watching Variables Distributed Debugging Interaction View Eclipse Integration

33 Road Map, example

34 Further Information Project Page: IRC: irc://gimp.net/frysk Mailing list:

35 Questions?