Getting Started with RRDtool
|
|
|
- Dinah Flowers
- 10 years ago
- Views:
Transcription
1 Cuddletech TekRef Series Ben Rockwood, Cuddletech Revision History Revision v1.0 April 27th 2004 benr Initial Document Revision v1.1 May 10th 2004 benr Final sections added RRDtool is a powerful replacement for MRTG, but it's complicated and extremely confusing the first time you use it. This paper hopes to make your first experience a little less mind numbing. Table of Contents Introduction... 1 Creating an initial RRD... 2 Adding Timed Data... 5 Generating Graphs... 6 Another look, start to finish... 8 Viewing Historical Data Conclusions Introduction RRDtool was written by Tobi Oetiker, the author of MRTG. It is, effectively, the next generation of MRTG, with a complete reimplementation of MRTGs graphing and logging features. MRTG works great for simple network monitoring, but thats really all it was originally intended to do. These days people are using MRTG to monitor everything from router bandwidth, to memory and disk statistics, to the number of times the dog goes in and out of the house. There are a number of drawbacks to MRTG however. On the graphing front, MRTG graphs always use a Y axis starting at 0, if you only want to see the relevant values in a range (for temps you might only want to graph from 50F to 90F) your out of luck. You are significantly limited to the number of different values that can be graphed, if you want to see the network throughput of 10 different servers your probably going to be 1
2 forced to use 10 different graphs. The list goes on and on. The point is that RRDtool fills in the gaps that MRTG leaves wanting, and provides for open customization that was difficult if not impossible before. But, MRTG has one thing that RRDtool doesn't... simplicity. Many Network Admins who know more about air purifiers than UNIX systems are using MRTG on a regular basis, thanks to tools like cfgmaker and indexmaker, coupled with a simple and basically straightforward config syntax. RRDtool isn't quite so simple though, at least at first. Almost all functions are provided by a single tool: rrdtool. The same program is used to create databases, modify, tune and update them, generate graphs, and even make backup dumps. Lets look at the basic flow you follow when using RRDtool: Overview of steps to use RRDtool for data graphing 1. Create an empty RRD database using rrdtool create. 2. Utilize a script and/or the cron to add data to the database using rrdtool update. 3. Generate, usually via script, custom output graphs using rrdtool graph. In this tutorial we'll look quickly at each step, so that you can get off and generating your own output. Creating an initial RRD RRD stands for Round Robin Database, and it is what it sounds like. There are a fixed number of records in the database and once the last record has been written in the database the next update goes to the first record, and around and around it goes. In this way, your databases will never grow out of control. The only downside to this is that, obviously, you've got to know how much data you'll want to look at historically ahead of time so that when you generate your database you have enough data. You may want a days worth of info, or even months. When creating an RRD database we'll need to specify a couple things, namely one or more Data Sources and one or more Round Robin Archives. The data source (DS) defines what type of data is accepted and some boundaries on what constitutes good data. The round robin archives (RRA) can almost be thought of as views, it defines the different ways we can store and retrieve data. The following is an example of database creation. In this example I am setting up an RRD to monitor TempTrax temperatures. 2
3 TempTrax-RRD]$ rrdtool create temptrax.rrd \ --start N --step 300 \ DS:probe1-temp:GAUGE:600:55:95 \ DS:probe2-temp:GAUGE:600:55:95 \ DS:probe3-temp:GAUGE:600:55:95 \ DS:probe4-temp:GAUGE:600:55:95 \ RRA:MIN:0.5:12:1440 \ RRA:MAX:0.5:12:1440 \ RRA:AVERAGE:0.5:1:1440 Figure 1. TempTrax RRD Creation Command The tool rrdtool is called with the argument create followed by the RRD filename of the database to be created. The next two arguments specify the time at which the database starts, measured in seconds since the Epoch, and the step time measured in seconds which is the interval between database updates. In this case I'm using "N" as the start time which tells rrdtool to use "now" as the time, and my step interval is 300 seconds (5 minutes). The "DS" lines specify our different data sources. In this case I want to monitor 4 different temperature probes which I've named sequentially. The following field ("GUAGE") species the Data Source Type (DST) which is one of GUAGE, COUNTER, DERIVE or ABSOLUTE. The Guage DST works like you'd expect and is generally the best choice. Counters continuously increase, Derives store a derivative of the last and the current value, and Absolutes store values which reset after each reading. The last 3 fields of the Data Source lines specify the minimal heartbeat and both min and max values. The minimal heartbeat is a value measured in seconds after which the values is said to be unknown (think of it as a timeout). The min and max specify a range for "good" values; if a value is outside of this range it is said to be unknown. The "RRA" lines specify different round robin archives. These are likes views, by which the data are stored. Inside the RRD database file each RRA is stored separately, with a predefined number of records. Each time we "update" our database we are adding a Primary Data Point (PDP), which are then combined together and put into our RRA based on a Consolidation Function (CF) that determines what the actual value thats written is. In the above example the first field specifies that we're defining an RRA. The second field specifies the CF that is used, one of AVERAGE, MIN, MAX or LAST. The third field specifies the XFiles Factor (XFF), this is the percentage of PDPs that can be unknown without making the recorded value unknown. The fourth field is the number of PDPs that will make up the recorded value. The final field specifies the number of records this RRA has. Looking again at the example above, I'm creating a new RRD named temptrax.rrd that starts "Now" and is updated (stepped) every 300 seconds (5 minutes). The RRD contains 3
4 4 different Data Sources, 1 per probe of the type GAUGE. If the Data Source isn't updated every 600 seconds (10 minutes) or if the value is not between 55 and 95, then the value is considered to be in error and is written as unknown. Then, 3 different RRA's are specified. Two of the RRA's record the MIN and MAX values using 12 PDP's allowing 50% of them to be unknown. We're storing 1440 records in these RRAs which means that because we're updating every 5 minutes (step) and we're using 12 PDPs (each update is a PDP), which means we're adding a record every hour (5 mins * 12) and we're storing 1440 records meaning that we will have 60 days (1440hrs/24hrs) worth of data in our RRA. For each of these the minimum value and the maximum value from the collected PDPs will be used as the recorded value. In the last RRA defined we're using the average of our collected PDPs, still allowing for 50% unknowns, but we're using only 1 PDP per record, which means we're storing every update. We're allowing for 1440 records which means this RRA is storing (1440/12updatesperhouse/24hrs) 5 days worth of data. In this later case because we're using a single PDP the CF isn't really important and we probably should have used LAST just for cleanliness. Here's another breakdown of the DS and RRA arguments: Data Source Fields: DS:DS-Name:DST:HeartBeat:Min:Max DS DS-Name DST HeartBeat Min Max Defines a Data Source Field. The name of this Data Source. Defines the Data Source Type. Can be GAUGE, COUNTER, DERIVE or ABSOLUTE. Defines the minimum heartbeat, the maximum number of seconds that can go by before a DS value is considered unknown. The minimum acceptable value. Values less than this number are considered unknown. This is optional. Specify "U" (unknown) to not set a min The maximum acceptable value. Values exceeding this number are considered unknown. This is optional. Specify "U" (unknown) to not set a max Round Robin Archives: RRA:CF:XFF:Steps:Rows RRA Defines a Round Robin Archive. 4
5 CF XFF Steps Rows Consolidation Function. Can be AVERAGE, MIN, MAX, or LAST. Defines XFiles Factor, the number of data points that can be anally probed by martians before RRD gives a crap. Defines how many Primary Data Points (PDPs) are consolidated using the Consolidation Function (CF) to create the stored value. Defines the number of Rows (records) stored in this RRA. Adding Timed Data Adding data points (rrdtool update) can be the easiest or hardest part of configuring RRD. This is where you actually insert the data you want to plot into the RRD. The overall idea, however, is extremely simple. Look at the form: rrdtool update <file.rrd> timestamp:val1[:val2:...] Typically this command is placed at the end of a script that nabs all your values via SN- MPget or whatever method you are using. The form above is simple, rrdtool update (it should be noted you can also just use rrdupdate) followed by the RRD filename and an update string that starts with the timestamp of these values (as with create, you can use "N" to mean "Now"), and is followed by a list of colon separated values. You should be adding 1 value per Data Source, as defined in your RRD. The also must be added in order. You can actually use the rrdtool update --template ds2:ds1:ds3 command form to change the default order of the data sources, but otherwise they are added in the order they were created in your RRD. Here is an example update script which I run from cron every 5 minutes. It uses a Nagios plugin to get temps from a TempTrax E, does some string manipulation to cut off the output I don't want, and then updates RRD with the values. #!/usr/local/bin/perl # RRD Update Script: update_rrd_temps.pl $HOST = " "; $PATH = "/home/benr/rrd/temptrax-rrd"; $NumProbes = 4; for($i=1; $i <= $NumProbes; $i++) { $x = `${PATH}/check_temptraxe -H ${HOST} -p ${i}`; $x =~ s/^.*([0-9.]{4}).*$/$1/; chomp($x); push(@temps,$x); } 5
6 `/usr/local/rrdtool /bin/rrdtool update ${PATH}/temptrax.rrd "N:$TEMPS[0]:$TEMPS[1]:$TEMPS[2]:$TEMPS[3]"`; Figure 2. TempTrax RRD Update Script Obviously, the more complex you method of getting values is the more complex your update script will be. Typically this script is type of script is run from cron, however you could just use sleeps and a loop to keep the monitoring script running all the time. This is one of the nice things about RRDtools flexibility, you can get much more creative (for good or bad, you decide). Generating Graphs Generating graphs is a learned ability. It takes some time and experimentation, but isn't too tricky once you've done it once or twice. Graphs are created using rrdtool graph. There are tons of options and ways to configure graph output, but we'll just look at the most commonly used ones. Here is an example to go with the temp monitoring stuff we did earlier: rrdtool graph mygraph.png -a PNG --title="temptrax" \ --vertical-label "Deg F" \ 'DEF:probe1=temptrax.rrd:probe1-temp:AVERAGE' \ 'DEF:probe2=temptrax.rrd:probe2-temp:AVERAGE' \ 'DEF:probe3=temptrax.rrd:probe3-temp:AVERAGE' \ 'DEF:probe4=temptrax.rrd:probe4-temp:AVERAGE' \ 'LINE1:probe1#ff0000:Switch Probe' \ 'LINE1:probe2#0400ff:Server Probe' \ 'AREA:probe3#cccccc:HVAC' \ 'LINE1:probe4#35b73d:QA Lab Probe' \ 'GPRINT:probe1:LAST:Switch Side Last Temp\: %2.1lf F' \ 'GPRINT:probe3:LAST:HVAC Output Last Temp\: %2.1lf F\j' \ 'GPRINT:probe2:LAST:Server Side Last Temp\: %2.1lf F' \ 'GPRINT:probe4:LAST:QA Lab Last Temp\: %2.1lf F\j' Figure 3. TempTrax Graph Creation Command This looks worse than it is. The DEF lines define a Data Source (DS) to use. The LINE and AREA lines define the actual plotting methods. And the GPRINT lines define text that I want rendered onto the graph. The default image output format is GIF, so using the -a allows you to choose between GIF, PNG and GD formats. --title is the string that is printed onto the top of your graph. - 6
7 -verital-label is an optional string that is printed on the Y axis, typically to denote the metric. The DEF line DEFines a virtual name (vname) for an RRD Data Source (DS). It might seem odd that you need to define a virtualname, instead of just using the original DS name, but because you define the RRD filename here, you can define multiple DS's from different RRDs, which could be a problem if multiple RRDs have DS's by the same name; this method gets us around that. So the 2nd argument in the DEF line is vname=rrdfilename, followed by the actual original DS name, followed by the Consolidation Function (CF) type specified in your RRA. In this way, we are telling the graphing tool exactly what we want, from what file, from which RRA/view. The next set of lines are LINE1 and AREA. These are the actual lines on the graph. Lines come in 3 different varieties, LINE1, LINE2, and LINE3. These are actually just Small, Medium, Large. LINE1 draws a 1px line, LINE2 draws a thicker line, and Line3 the thickest, otherwise they all work the same way. AREA is used to define a line that is "filled in". The big green area from 0 to your value in MRTG are AREAs. The form for all these types is the type, followed by the vname and a color specified in hex RGB value. Do note that if you do not specify a color the line won't show up! The final argument is the name of the line itself, this is printed at the bottom of the graph with the color key. Finally, the GPRINT line allow us to write other stuff on the outputted graph. Generally it's nice to see the current values printed, so we'll draw those on. The first argument obviously is GPRINT, the second is the vname. The third argument is the CF type for data printed, here I'm using LAST because I want the most current value. And the final argument is the line thats actually printed. You will need to escape out colons here if you them so they don't chop off your line. Also, \j can be used to justify a line (push it to the right side of the graph). Inserting numeric values are done like in many programming languages: %1.1lf would give you a single precision floating point number, like 5.2 or 0.6, and %2.4lf would be a number like Whole number can be rounded with %2.0lf. 7
8 Figure 4. Generated TempTrax Graph I used two separate scripts (PERL for the update, and Borne for the graphing) but ultimately it is probably cleaner to combine the two so that you are only running one script from cron. Another look, start to finish If your the type of reader who skip most of the text in an article and only reads the figures and examples, then this is the section to actually read. The Network Admin at my company currently is using MRTG to monitor 4 interconnecting T1's we use to provide cooperate connectivity. Currently there are 6 different MRTG graphs (4 links, plus an aggregate and the faster ether interface on the router). I'd like to give him all this information on a single graph. I want to grab the IF-MIB::ifInOctets and IF-MIB::ifOutOctets OIDs from the router, and I need to do it for the 4 T1 interfaces, the 1 FastEther and the 1 virtual multilink. So when I set up my RRD I'm going to need Data Sources for each in and out value (so thats 12 total). As for RRA, I want to hold the average per hour for 6 months and the last value every step interval going back a week. This way I can provide a graph for the last month or more and a current graph. That means we want 1 PDP for the LAST with 4032 records to hold 2 weeks (12 steps/hour * 24hrs * 14days) and 4320 records to hold our 6 months worth of hourly averages (24hrs * 180days). I'll setup the RRD in the following way: rrdtool create wan-thruput.rrd \ --start N --step 300 \ DS:Serial00_In:COUNTER:600:U:U \ DS:Serial01_In:COUNTER:600:U:U \ DS:Serial10_In:COUNTER:600:U:U \ DS:Serial11_In:COUNTER:600:U:U \ DS:FastEth20_In:COUNTER:600:U:U \ DS:MutliLink_In:COUNTER:600:U:U \ DS:Serial00_Out:COUNTER:600:U:U \ DS:Serial01_Out:COUNTER:600:U:U \ DS:Serial10_Out:COUNTER:600:U:U \ DS:Serial11_Out:COUNTER:600:U:U \ DS:FastEth20_Out:COUNTER:600:U:U \ DS:MutliLink_Out:COUNTER:600:U:U \ RRA:LAST:0.5:1:4032 \ RRA:AVERAGE:0.5:12:4320 Figure 5. Bandwidth RRD Creation Command 8
9 Now, time to start dropping data into it. I hacked around a bit and built the following script: #!/usr/local/bin/perl ### WAN RRD Script $RRDTOOL = "rrdtool"; $TARGET_RRD = "wan-thruput.rrd"; $SNMPGET = "snmpget -v2c -Ovq -c public "; for($i=1; $i <= 11; $i++){ my $in = `${SNMPGET} IF-MIB::ifInOctets.${i}`; my $out = `${SNMPGET} IF-MIB::ifOutOctets.${i}`; } chomp($in) && chomp($out); push(@t1_in,$in); push(@t1_out,$out); `${RRDTOOL} update $TARGET_RRD N:${T1_IN[0]}:${T1_IN[1]}:${T1_IN[2]} :${T1_IN[3]}:${T1_IN[6]}:${T1_IN[10]}:${T1_OUT[0]}:${T1_OUT[1]} :${T1_OUT[2]}:${T1_OUT[3]}:${T1_OUT[6]}:${T1_OUT[10]}`; Figure 6. Bandwidth RRD Update Script I've taken the full pathnames out of the above script to make it fit on printed pages, but this script will grab all the input and output octet counters (an octet is a group of 8 bits) and put them in our RRD. Now to graph it. First we'll create the current graph using the latest numbers. We'll invoke RRD using the rrdtool graph tool. I'm using a shell script that can be executed from cron, that looks like this: #!/bin/bash rrdtool graph wanoutput.png -a PNG \ --title="hs T1 Links" --vertical-label "Bytes" \ --height 150 \ 'DEF:s00in=wan-thruput.rrd:Serial00_In:LAST' \ 'DEF:s01in=wan-thruput.rrd:Serial01_In:LAST' \ 'DEF:s10in=wan-thruput.rrd:Serial10_In:LAST' \ 'DEF:s11in=wan-thruput.rrd:Serial11_In:LAST' \ 'DEF:fe20in=wan-thruput.rrd:FastEth20_In:LAST' \ 'DEF:mlin=wan-thruput.rrd:MutliLink_In:LAST' \ 'DEF:s00out=wan-thruput.rrd:Serial00_Out:LAST' \ 'DEF:s01out=wan-thruput.rrd:Serial01_Out:LAST' \ 'DEF:s10out=wan-thruput.rrd:Serial10_Out:LAST' \ 'DEF:s11out=wan-thruput.rrd:Serial11_Out:LAST' \ 'DEF:fe20out=wan-thruput.rrd:FastEth20_Out:LAST' \ 9
10 'DEF:mlout=wan-thruput.rrd:MutliLink_Out:LAST' \ 'HRULE:193000#0000ff' \ 'HRULE:386000#0000ff' \ 'HRULE:579000#0000ff' \ 'HRULE:772000#0000ff' \ 'AREA:mlin#66FF99:MultiLink In' \ 'AREA:mlout#FFFF33:MultiLink Out' \ 'LINE1:fe20in#000000:FastEther In' \ 'LINE1:fe20out#ff0000:FastEther Out' \ 'GPRINT:s00in:LAST:XO T1 A In\: %6.0lf bytes ' \ 'GPRINT:s01in:LAST:XO T1 B In\: %6.0lf bytes\j' \ 'GPRINT:s10in:LAST:SBC T1 A In\: %6.0lf bytes' \ 'GPRINT:s11in:LAST:SBC T1 B In\: %6.0lf bytes\j' Figure 7. Bandwidth Graph Creation Script Again, for formating reasons I've removed full paths from the above script, I highly suggest you use fully qualified paths for all files in your script. So, in the above script we're using PNG as the output format for the graph, and naming it "wanoutput.png". The graph is labeled "HS T1 Links" and the output image is 150px high. We're also labeling the vertical axis as "Bytes" for clarity. The DEF lines define each of our RRD values and name them appropriately for later use in this graph, and use the LAST value for each so that we get the most current info. The HRULE directive is one we haven't looked at before, it simply draws horizontal lines on the output graph at specified values. Here we're drawing HRULES at 192KB intervals (the thruput limit of a T1 in K) and coloring them blue. The idea of the HRULES is to easily and quickly see exactly how much of the T1 total bandwidth is being consumed by the multilink traffic. Then we define the AREAs and LINEs. I'm only graphing out the Multilink and FastEthernet traffic, because if things are working properly the thruput of the FastEther should always match the thruput of the multilink, therefore we use an AREA for the MultiLink and a LINE1 for the FastEther, and if things are working properly you should always just see the FastEther lines outlining the top of the MultiLink area. It makes for quick reading. Please note that the order in which I've specified them is very important because each line/area is drawn in the order that you specified it. In this way, if the MultiLink Out area is larger than the MultiLink In, you'll never see the In values, but you will see the lines on top. You'll notice this in the output graph. Finally, I use some GPRINT statements to put the thruput values of each individual T1 on the output graph. I choose not to actually graph these values because the graph just got way too crowded. In this case, we really don't care what the thruput of each T1 really is anyway, we just want to know that they are all roughly passing the same amount of traffic which means the MultiLink is working properly. 10
11 Now, I drop both the script for gathering the data and graphing the data into cron: 0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/benr/rrd/wan-rrd/wan-update.pl 0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/benr/rrd/wan-rrd/make_graph.sh Figure 8. Crontab Invocation of Updating and Graphing After letting it run for awhile, we look at the graph and the following is the result. Figure 9. 1 Day Bandwidth Utilization Graph In the above output graph you can see clearly everything we want to know about our MultiLinked T1's. This the output as seen on a Monday morning so you notice that over the weekend there was very little traffic coming into our network across the T1's, but there was a fairly steady amount of traffic going out (apparently this is Microsoft Actve Directory traffic). Because of the order in which we specified our AREA statements you don't see the green AREA (MultiLnk In) at all, but you do see the FastEtherOut which is drawn ontop of both AREAs. We could fix this by reordering the AREAs and LINEs but, as you can see, during the week the traffic going out far exceeds the traffic coming in, making for better graphing. Viewing Historical Data 11
12 Continuing from our last example, lets create some other graphs for historical purposes. You'll recall that when we created the RRD we specified two different RRA's: LAST (for the most current data) and AVERAGE (for hourly averages). Now that I've got acouple days worth of data in my RRD I want to look at the historical graphs. We'll do this by creating a duplicate graphing script as we did above, but we'll add an extra argument to the rrdtool graph command: --start and optionally --end. Lets just look at the example. The following graph generation script is exactly the same as the one we created in the last section, but I've added the --start argument and changed the RRA's specified in both the DEFs and GPRINTs from LAST to AVERAGE to ensure we get the right values. #!/bin/bash rrdtool graph wanoutput-hist.png -a PNG \ --title="hs T1 Links Hourly Averages" --vertical-label "Bytes" \ --height 150 \ --start "-1month" \ 'DEF:s00in=wan-thruput.rrd:Serial00_In:AVERAGE' \ 'DEF:s01in=wan-thruput.rrd:Serial01_In:AVERAGE' \ 'DEF:s10in=wan-thruput.rrd:Serial10_In:AVERAGE' \ 'DEF:s11in=wan-thruput.rrd:Serial11_In:AVERAGE' \ 'DEF:fe20in=wan-thruput.rrd:FastEth20_In:AVERAGE' \ 'DEF:mlin=wan-thruput.rrd:MutliLink_In:AVERAGE' \ 'DEF:s00out=wan-thruput.rrd:Serial00_Out:AVERAGE' \ 'DEF:s01out=wan-thruput.rrd:Serial01_Out:AVERAGE' \ 'DEF:s10out=wan-thruput.rrd:Serial10_Out:AVERAGE' \ 'DEF:s11out=wan-thruput.rrd:Serial11_Out:AVERAGE' \ 'DEF:fe20out=wan-thruput.rrd:FastEth20_Out:AVERAGE' \ 'DEF:mlout=wan-thruput.rrd:MutliLink_Out:AVERAGE' \ 'HRULE:193000#0000ff' \ 'HRULE:386000#0000ff' \ 'HRULE:579000#0000ff' \ 'HRULE:772000#0000ff' \ 'AREA:mlin#66FF99:MultiLink In' \ 'AREA:mlout#FFFF33:MultiLink Out' \ 'LINE1:fe20in#000000:FastEther In' \ 'LINE1:fe20out#ff0000:FastEther Out' \ 'GPRINT:s00in:AVERAGE:XO T1 A In\: %6.0lf bytes ' \ 'GPRINT:s01in:AVERAGE:XO T1 B In\: %6.0lf bytes\j' \ 'GPRINT:s10in:AVERAGE:SBC T1 A In\: %6.0lf bytes' \ 'GPRINT:s11in:AVERAGE:SBC T1 B In\: %6.0lf bytes\j' Figure Month Historical Graph Creation Script Using the graph arguments --start and --end we can specify the window of time the graph displays. By default (no start/end arguments) the graph starts 24hours ago, and ends "now", so that we always see one full day of values. If we want to see a larger period of time we can change the start time to something more useful. Generally you won't need to use the end argument because you want it to end now which is the default, but it 12
13 works just like the start argument does in case you wanted to, for instance, create a historical graph just for the month of December. Start and End times are specified in forms accepted by the UNIX at command, therefore you can specify absolute seconds since the Epoch or "Jan 10" as a specific date. You can also use more useful relative times such as "yesterday", "-1month" (One Month Ago), "-2weeks", "-1year", etc. Check out the rrdfetch man page for more details. In our graph above, I've decided to give 1 month of historical data. Below is the output. Figure 11. Generated 1 Month Bandwidth Graph Using a script like the one above can be easily duplicated or modified in order to show almost any range of statistical data. Depending how many RRAs you created you can get some very granular details. Conclusions At first glance RRDtool is extremely frustrating and needlessly complicated, but most of this sentiment comes from a long time familiarization with simpler tools such as MRTG and a personal struggle with RRDtools unique syntax. However, once these hurdles can be overcome the benefits of RRDtool over almost any other tool available become immediately apparent. RRDtool embraces a "work smarter, not harder" approach once it's understood, even though it seems a lot harder than smarter in the end. Being able to have absolute control over your graph output, being able to consolidate 10+ MRTG graphs into 1 RRDtool graph, and having unlimited ability to get exactly the values you want and even manipulate them before being recorded (thanks to scripting) make RRDtool the power monitoring tool of choice. 13
14 Hopefully this tutorial has helped you get over the basic hurdles that RRDtool throws in your way and provides a gateway for better understanding of the tool. For more information, including other tutorials, man pages, examples, updates, and to get the source yourself, please visit the RRDtool website [ And if you like RRDtool, remember to show Tobi some love [ 14
Tobias Oetiker. OSCON 2005 Portland, Oregon, August 3, 2005
ISG.EE - ETH Zürich OSCON 2005 Portland, Oregon, August 3, 2005 Netflow Data presented with Fluxscope Content The world today About data collection A graph says more than 1 000 words The world today About
RRDtool. Tobi Oetiker <[email protected]> Sponsors: AboveNet www.above.net CAIDA www.caida.org ETH Zurich www.ethz.ch. RRDtool NANOG Nr.
RRDtool RRDtool \ahr-ahr- dee-t:ul\ n [E, fr. round robin database tool] :a system to store and display time-series data (i.e. utilized bandwidth, machine-room temperature, server load average). R. stores
MRTG / RRDTool. Network Management Workshop. June 2009 Papeete, French Polynesia
MRTG / RRDTool Network Management Workshop June 2009 Papeete, French Polynesia MRTG The Multi Router Traffic Grapher (MRTG) is a tool to monitor the traffic load on network-links. MRTG generates HTML pages
May 2002 16PZ-0502A-WWEN Prepared by: Internet & E-Commerce Solutions
May 2002 Prepared by: Internet & E-Commerce Solutions Contents Introduction... 3 Solution Overview... 3 Obtaining Compaq Management Agents (CMA) for Linux... 3 Integrating Compaq Management Agents MIBs
RRDtool Tips & Tricks
RRDtool Tips & Tricks Tobias Oetiker OETIKER+PARTNER AG Recipe for Success Resolve the problems before anyone else finds them. ... and talk about it Being able is only half the story,
My Secure Backup: How to reduce your backup size
My Secure Backup: How to reduce your backup size As time passes, we find our backups getting bigger and bigger, causing increased space charges. This paper takes a few Newsletter and other articles I've
Cacti 0.8 Network Monitoring
Cacti 0.8 Network Monitoring Dinangkur Kundu S. M. Ibrahim Lavlu Chapter No. 4 "Creating and Using Templates" In this package, you will find: A Biography of the authors of the book A preview chapter from
MRTG used for Basic Server Monitoring
MRTG used for Basic Server Monitoring SANS Institute Masters Presentation by T. Brian MRTG used for Basic Server Monitoring This presentation covers how-to instructions to establish basic server monitoring
Network performance overview. TEIN2 Bangkok September 2005
Network performance overview TEIN2 Bangkok September 2005 Agenda Network performance RRDtool Cacti (RRDtool Front-end) Network performance monitoring MRTG The first, open-source, easy to use performance
How To Use Multi Router Traffic Grapher (Mrtg)
The following paper was originally published in the Proceedings of the Twelfth Systems Administration Conference (LISA 98) Boston, Massachusetts, December 6-11, 1998 MRTG The Multi Router Traffic Grapher
Wait, How Many Metrics? Monitoring at Quantcast
Wait, How Many Metrics? Monitoring at Quantcast Count what is countable, measure what is measurable, and what is not measurable, make measurable. Galileo Galilei Quantcast offers free direct audience measurement
Storage of Network Monitoring and Measurement Data
Storage of Network Monitoring and Measurement Data A report submitted in partial fulfillment of the requirements for the degree of Bachelor of Computing and Mathematical Sciences at The University of Waikato
AXIGEN Mail Server Reporting Service
AXIGEN Mail Server Reporting Service Usage and Configuration The article describes in full details how to properly configure and use the AXIGEN reporting service, as well as the steps for integrating it
Fritz Speed Documentation
Fritz Speed Documentation Release 0.1.1 Thomas Westfeld February 04, 2016 Contents 1 What it does 1 2 How It Works 3 3 Contents 5 3.1 Installation................................................ 5 3.2
Introduction to Open Atrium s workflow
Okay welcome everybody! Thanks for attending the webinar today, my name is Mike Potter and we're going to be doing a demonstration today of some really exciting new features in open atrium 2 for handling
Redundant and Failover Network Monitoring This section describes a few scenarios for implementing redundant monitoring hosts an various types of network layouts. With redundant hosts, you can maintain
Using SNMP to Obtain Port Counter Statistics During Live Migration of a Virtual Machine. Ronny L. Bull Project Writeup For: CS644 Clarkson University
Using SNMP to Obtain Port Counter Statistics During Live Migration of a Virtual Machine Ronny L. Bull Project Writeup For: CS644 Clarkson University Fall 2012 Abstract If a managed switch is used during
Deploying Microsoft Operations Manager with the BIG-IP system and icontrol
Deployment Guide Deploying Microsoft Operations Manager with the BIG-IP system and icontrol Deploying Microsoft Operations Manager with the BIG-IP system and icontrol Welcome to the BIG-IP LTM system -
Newton Linux User Group Graphing SNMP with Cacti and RRDtool
Newton Linux User Group Graphing SNMP with Cacti and RRDtool Summary: Cacti is an interface that can be used to easily manage the graphing of SNMP data. These graphs allow you to visualize performance
IP Subnetting: Practical Subnet Design and Address Determination Example
IP Subnetting: Practical Subnet Design and Address Determination Example When educators ask students what they consider to be the most confusing aspect in learning about networking, many say that it is
1.2 Using the GPG Gen key Command
Creating Your Personal Key Pair GPG uses public key cryptography for encrypting and signing messages. Public key cryptography involves your public key which is distributed to the public and is used to
An Email Newsletter Using ASP Smart Mailer and Advanced HTML Editor
An Email Newsletter Using ASP Smart Mailer and Advanced HTML Editor This tutorial is going to take you through creating a mailing list application to send out a newsletter for your site. We'll be using
There are numerous ways to access monitors:
Remote Monitors REMOTE MONITORS... 1 Overview... 1 Accessing Monitors... 1 Creating Monitors... 2 Monitor Wizard Options... 11 Editing the Monitor Configuration... 14 Status... 15 Location... 17 Alerting...
Visual FoxPro Accessing MySQL. presented to The Atlanta FoxPro Users Group June 19 th, 2007
Visual FoxPro Accessing MySQL presented to The Atlanta FoxPro Users Group June 19 th, 2007 Comparing MySQL with the VFP DBC, SQL Server, PostgreSQL VFP DBC Pros: 1. Very fast 2. Free DBC multiuser access
Part II. Managing Issues
Managing Issues Part II. Managing Issues If projects are the most important part of Redmine, then issues are the second most important. Projects are where you describe what to do, bring everyone together,
A whole new stream of income - For many, affiliate commissions are like "found money."
ActionPlan.com Affiliate Marketing Handbook by Robert Middleton The purpose of this handbook is to give you the tools you need to become a successful ActionPlan.com Affiliate. Being an affiliate and earning
Getting Started with WebSite Tonight
Getting Started with WebSite Tonight WebSite Tonight Getting Started Guide Version 3.0 (12.2010) Copyright 2010. All rights reserved. Distribution of this work or derivative of this work is prohibited
Okay, good. He's gonna release the computers for you and allow you to log into NSLDS.
Welcome to the NSLDS hands-on session. My name is Larry Parker. I'm from the Department of Education NSLDS. Today, I have with me a whole host of folks, so we're gonna make sure that if you have any questions
Deploying the BIG-IP LTM with the Cacti Open Source Network Monitoring System
DEPLOYMENT GUIDE Deploying the BIG-IP LTM with the Cacti Open Source Network Monitoring System Version 1.0 Deploying F5 with Cacti Open Source Network Monitoring System Welcome to the F5 and Cacti deployment
Helsinki University of Technology Department of Electrical and Communications Engineering Networking Laboratory
Helsinki University of Technology Department of Electrical and Communications Engineering Networking Laboratory S-38.3133 - Networking Technology, laboratory course Spring 2007 Work number 32: SNMP Instructions,
Network Monitoring & Management Measuring Delay with Smokeping
Network Monitoring & Management Measuring Delay with Smokeping Network Startup Resource Center www.nsrc.org These materials are licensed under the Creative Commons Attribution-NonCommercial 4.0 International
Introduction. What is RAID? The Array and RAID Controller Concept. Click here to print this article. Re-Printed From SLCentral
Click here to print this article. Re-Printed From SLCentral RAID: An In-Depth Guide To RAID Technology Author: Tom Solinap Date Posted: January 24th, 2001 URL: http://www.slcentral.com/articles/01/1/raid
Grids & networks monitoring - practical approach
Session 2 Networking for the Grid Grids & networks monitoring - practical approach Jedrzej Jajor Cezary Mazurek Wiktor Procyk INGRID 2007 Instrumenting the Grid Outline Introduction to monitoring Tools:
Cacti-ReportItv0.6.1. A small introduction to the use of ReportIt
Cacti-ReportItv0.6.1 A small introduction to the use of ReportIt Copyright (c) 2009 Andreas Braun Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation
The $200 A Day Cash Machine System
The $200 A Day Cash Machine System Make Big Profits Selling This Opportunity From Home! This is a free ebook from Frank Jones. You should not have paid for it. COPYRIGHT Frank Jones. All Rights Reserved:
StorageCraft ShadowStream User Guide StorageCraft Copyright Declaration
StorageCraft Copyright Declaration StorageCraft ImageManager, StorageCraft ShadowProtect, StorageCraft Cloud, and StorageCraft Cloud Services, together with any associated logos, are trademarks of StorageCraft
How To Change Your Site On Drupal Cloud On A Pcode On A Microsoft Powerstone On A Macbook Or Ipad (For Free) On A Freebie (For A Free Download) On An Ipad Or Ipa (For
How-to Guide: MIT DLC Drupal Cloud Theme This guide will show you how to take your initial Drupal Cloud site... and turn it into something more like this, using the MIT DLC Drupal Cloud theme. See this
Budget Main Window (Single Bank Account) Budget Main Window (Multiple Bank Accounts)
Budget Main Window (Single Bank Account) Budget Main Window (Multiple Bank Accounts) Page 1 of 136 Using Budget Help Budget has extensive help features. To get help use Budget's Help > Budget Help menu
How To Analyze Log Data On A Web Site
WSG Log Analysis Review I've reviewed ten different log analysis packages based on the following criteria: Analysis priorities (must include ) Traffic patterns (daily, hourly, etc) Most visited urls Referrer
mylittleadmin for MS SQL Server 2005 from a Webhosting Perspective Anthony Wilko President, Infuseweb LLC
mylittleadmin for MS SQL Server 2005 from a Webhosting Perspective Anthony Wilko President, Infuseweb LLC April 2008 Introduction f there's one thing constant in the IT and hosting industries, it's that
SNMP and MRTG Monitoring. SNMP and MRTG Monitoring. Introduction. Start. 1 de 9 07/06/2013 10:33. Contents
1 de 9 07/06/2013 10:33 Log in / create account Navigation Main Page Community portal Current events Recent changes Random page Help Search the Wiki SNMP and MRTG Monitoring Contents 1 SNMP and MRTG Monitoring
Network and Server Statistics Using Cacti
Network Management & Monitoring Network and Server Statistics Using Cacti These materials are licensed under the Creative Commons Attribution-Noncommercial 3.0 Unported license (http://creativecommons.org/licenses/by-nc/3.0/)
Dwebs IIS Log Archiver Installation and Configuration Guide for Helm 4
Dwebs IIS Log Archiver Installation and Configuration Guide for Helm 4 Dwebs Ltd http://www.dwebs.ltd.uk January 2009 Doc: Helm 4.1.0.0 Mod Ver: 1.0.0.0 The names of actual companies and products mentioned
Managing User Accounts and User Groups
Managing User Accounts and User Groups Contents Managing User Accounts and User Groups...2 About User Accounts and User Groups... 2 Managing User Groups...3 Adding User Groups... 3 Managing Group Membership...
Transcription. Crashplan vs Backblaze. Which service should you pick the short version
Transcription Crashplan vs Backblaze Hey and welcome to cloudwards.net and another exciting video of two major unlimited online backup services namely Backblaze and CrashPlan or CrashPlan or Backblaze.
Network Monitoring. Dhruba Raj Bhandari (CCNA) Manager Systems Soaltee Crowne Plaza Kathmandu NEPAL [email protected]
Network Monitoring Dhruba Raj Bhandari (CCNA) Manager Systems Soaltee Crowne Plaza Kathmandu NEPAL [email protected] Welcome! Network Management Workshop Network Monitoring Concepts, Tools And
>> My name is Danielle Anguiano and I am a tutor of the Writing Center which is just outside these doors within the Student Learning Center.
>> My name is Danielle Anguiano and I am a tutor of the Writing Center which is just outside these doors within the Student Learning Center. Have any of you been to the Writing Center before? A couple
Transcription. Founder Interview - Panayotis Vryonis Talks About BigStash Cloud Storage. Media Duration: 28:45
Transcription Founder Interview - Panayotis Vryonis Talks About BigStash Cloud Storage Media Duration: 28:45 Feel free to quote any passage from this interview for your articles. Please reference cloudwards.net
0 Introduction to Data Analysis Using an Excel Spreadsheet
Experiment 0 Introduction to Data Analysis Using an Excel Spreadsheet I. Purpose The purpose of this introductory lab is to teach you a few basic things about how to use an EXCEL 2010 spreadsheet to do
Kaseya 2. User Guide. for Network Monitor 4.1
Kaseya 2 Ping Monitor User Guide for Network Monitor 4.1 June 5, 2012 About Kaseya Kaseya is a global provider of IT automation software for IT Solution Providers and Public and Private Sector IT organizations.
Network and Server Statistics Using Cacti
Network Management & Monitoring Network and Server Statistics Using Cacti These materials are licensed under the Creative Commons Attribution-Noncommercial 3.0 Unported license (http://creativecommons.org/licenses/by-nc/3.0/)
Jenesis Software - Podcast Episode 3
Jenesis Software - Podcast Episode 3 Welcome to Episode 3. This is Benny speaking, and I'm with- Eddie. Chuck. Today we'll be addressing system requirements. We will also be talking about some monitor
Multi-Router Traffic Grapher (MRTG)
Multi-Router Traffic Grapher (MRTG) MULTI-ROUTER TRAFFIC GRAPHER (MRTG)... 1 Overview... 1 Installing MRTG... 1 Viewing the Results... 10 Notes... 13 Troubleshooting... 13 Document Revision History...
Cobian9 Backup Program - Amanita
The problem with backup software Cobian9 Backup Program - Amanita Due to the quixotic nature of Windows computers, viruses and possibility of hardware failure many programs are available for backing up
Cacti complete network graphing solution. Oz Melamed E&M Computing Jun 2009
Cacti complete network graphing solution Oz Melamed E&M Computing Jun 2009 About Cacti Cacti is a complete network graphing solution designed to harness the power of RRDTool's data storage and graphing
BetInfo24 Betfair Trading Strategy
BetInfo24 Betfair Trading Strategy All rights reserved BetInfo24 has been identified as the author of this work. The right has been asserted in accordance with the copyright, designs and patents act 1988.
Simple Document Management Using VFP, Part 1 Russell Campbell [email protected]
Seite 1 von 5 Issue Date: FoxTalk November 2000 Simple Document Management Using VFP, Part 1 Russell Campbell [email protected] Some clients seem to be under the impression that they need to
Configure ActiveSync with a single Exchange server (Exchange sync for an iphone)
Configure ActiveSync with a single Exchange server (Exchange sync for an iphone) I recently picked up a first-generation iphone from a friend and after playing around with it for a while I decided that
KretchmarBook 2003/8/29 19:05 page 39 #51
KretchmarBook 2003/8/29 19:05 page 39 #51 Chapter 3 MRTG 3.1 Overview of MRTG MRTG is the Multi Router Traffic Grapher, a piece of free software released under the GNU General Public License. 1 It was
Over the past several years Google Adwords has become THE most effective way to guarantee a steady
Killer Keyword Strategies - Day 3 "How To Virtually Guarantee That You Find Profitable Keywords For Google Adwords..." Over the past several years Google Adwords has become THE most effective way to guarantee
Using New Relic to Monitor Your Servers
TUTORIAL Using New Relic to Monitor Your Servers by Alan Skorkin Contents Introduction 3 Why Do I Need a Service to Monitor Boxes at All? 4 It Works in Real Life 4 Installing the New Relic Server Monitoring
Computer Programming In QBasic
Computer Programming In QBasic Name: Class ID. Computer# Introduction You've probably used computers to play games, and to write reports for school. It's a lot more fun to create your own games to play
GelAnalyzer 2010 User s manual. Contents
GelAnalyzer 2010 User s manual Contents 1. Starting GelAnalyzer... 2 2. The main window... 2 3. Create a new analysis... 2 4. The image window... 3 5. Lanes... 3 5.1 Detect lanes automatically... 3 5.2
12Planet Chat end-user manual
12Planet Chat end-user manual Document version 1.0 12Planet 12Planet Page 2 / 13 Table of content 1 General... 4 1.1 How does the chat work?... 4 1.2 Browser Requirements... 4 1.3 Proxy / Firewall Info...
PotPieGirl's Pinterest SEO Tips
PotPieGirl's Pinterest SEO Tips This info is sold as is and does not guarantee any results. You do not have rights to share, sell, and/or give away this guide. This guide is for YOUR use only. All content
Backing Up TestTrack Native Project Databases
Backing Up TestTrack Native Project Databases TestTrack projects should be backed up regularly. You can use the TestTrack Native Database Backup Command Line Utility to back up TestTrack 2012 and later
Practical Example: Building Reports for Bugzilla
Practical Example: Building Reports for Bugzilla We have seen all the components of building reports with BIRT. By this time, we are now familiar with how to navigate the Eclipse BIRT Report Designer perspective,
RULESET DEFINING MANUAL. Manual Version: 0.9.1. Software version: 0.6.0
RULESET DEFINING MANUAL Manual Version: 0.9.1 Software version: 0.6.0 Last Revision: June 09, 2011 Contents 1 Introduction...4 1.1 What is a F-Deets ruleset...4 1.2 Where are they located...4 1.3 How to
Written by CiscoNET Thursday, 07 May 2009 00:57 - Last Updated Wednesday, 02 December 2009 00:50
STG is stand for SNMP Traffic Grapher. This is a well-known free real-time monitoring SNMP tool for quick real-time bandwidth monitoring. It was coded by Leonid Mikhailov. Yes... Yes... it's old, but it
SolarWinds. Understanding SolarWinds Charts and Graphs Technical Reference
SolarWinds Understanding SolarWinds Charts and Graphs Technical Reference Copyright 1995-2015 SolarWinds Worldwide, LLC. All rights reserved worldwide. No part of this document may be reproduced by any
Step One: Installing Rsnapshot and Configuring SSH Keys
Source: https://www.digitalocean.com/community/articles/how-to-installrsnapshot-on-ubuntu-12-04 What the Red Means The lines that the user needs to enter or customize will be in red in this tutorial! The
Cal Answers Analysis Training Part III. Advanced OBIEE - Dashboard Reports
Cal Answers Analysis Training Part III Advanced OBIEE - Dashboard Reports University of California, Berkeley March 2012 Table of Contents Table of Contents... 1 Overview... 2 Remember How to Create a Query?...
VI Performance Monitoring
VI Performance Monitoring Preetham Gopalaswamy Group Product Manager Ravi Soundararajan Staff Engineer September 15, 2008 Agenda Introduction to performance monitoring in VI Common customer/partner questions
One of the most important topics in any discussion of TCP/IP is IP. IP Addressing
IP Addressing 125 machine, called a RARP server, responds with the answer, and the identity crisis is over. RARP uses the information it does know about the machine s MAC address to learn its IP address
Solving the Rubik's Revenge (4x4x4) Home Pre-Solution Stuff Step 1 Step 2 Step 3 Solution Moves Lists
Solving your Rubik's Revenge (4x4x4) 07/16/2007 12:59 AM Solving the Rubik's Revenge (4x4x4) Home Pre-Solution Stuff Step 1 Step 2 Step 3 Solution Moves Lists Turn this... Into THIS! To solve the Rubik's
Action Steps for Setting Up a Successful Home Web Design Business
Action Steps for Setting Up a Successful Home Web Design Business In this document you'll find all of the action steps included in this course. As you are completing these action steps, please do not hesitate
REDUCING YOUR MICROSOFT OUTLOOK MAILBOX SIZE
There are several ways to eliminate having too much email on the Exchange mail server. To reduce your mailbox size it is recommended that you practice the following tasks: Delete items from your Mailbox:
Capturing Network Traffic With Wireshark
Capturing Network Traffic With Wireshark A White Paper From For more information, see our web site at Capturing Network Traffic with Wireshark Last Updated: 02/26/2013 In some cases, the easiest way to
Elixir Schedule Designer User Manual
Elixir Schedule Designer User Manual Release 7.3 Elixir Technology Pte Ltd Elixir Schedule Designer User Manual: Release 7.3 Elixir Technology Pte Ltd Published 2008 Copyright 2008 Elixir Technology Pte
virtualization.info Review Center SWsoft Virtuozzo 3.5.1 (for Windows) // 02.26.06
virtualization.info Review Center SWsoft Virtuozzo 3.5.1 (for Windows) // 02.26.06 SWsoft Virtuozzo 3.5.1 (for Windows) Review 2 Summary 0. Introduction 1. Installation 2. VPSs creation and modification
MONITORING PERFORMANCE IN WINDOWS 7
MONITORING PERFORMANCE IN WINDOWS 7 Performance Monitor In this demo we will take a look at how we can use the Performance Monitor to capture information about our machine performance. We can access Performance
Kaseya 2. User Guide. Version 7.0. English
Kaseya 2 Monitoring Configuration User Guide Version 7.0 English September 3, 2014 Agreement The purchase and use of all Software and Services is subject to the Agreement as defined in Kaseya s Click-Accept
Version Control with. Ben Morgan
Version Control with Ben Morgan Developer Workflow Log what we did: Add foo support Edit Sources Add Files Compile and Test Logbook ======= 1. Initial version Logbook ======= 1. Initial version 2. Remove
Start Learning Joomla!
Start Learning Joomla! Mini Course Transcript 2010 StartLearningJoomla.com The following course text is for distribution with the Start Learning Joomla mini-course. You can find the videos at http://www.startlearningjoomla.com/mini-course/
PentaMetric battery Monitor System Sentry data logging
PentaMetric battery Monitor System Sentry data logging How to graph and analyze renewable energy system performance using the PentaMetric data logging function. Bogart Engineering Revised August 10, 2009:
New 3ds transfer with a larger SD card
Jan 18, 2015 I've seen the video Nintendo released showing the best method of transfering data from one system to another, but it wasn't very clear about what to do if the SD card on your older system
Load-Balanced Merak Mail Server
Load-Balanced Merak Mail Server The most powerful solution for high traffic performance is to deploy Merak Mail Server in a load-balanced environment. This solution is shown in the demonstration below:
GUITAR THEORY REVOLUTION. Part 1: How To Learn All The Notes On The Guitar Fretboard
GUITAR THEORY REVOLUTION Part 1: How To Learn All The Notes On The Guitar Fretboard Contents Introduction Lesson 1: Numbering The Guitar Strings Lesson 2: The Notes Lesson 3: The Universal Pattern For
aspwebcalendar FREE / Quick Start Guide 1
aspwebcalendar FREE / Quick Start Guide 1 TABLE OF CONTENTS Quick Start Guide Table of Contents 2 About this guide 3 Chapter 1 4 System Requirements 5 Installation 7 Configuration 9 Other Notes 12 aspwebcalendar
Beginning BGP. Peter J. Welcher. Introduction. When Do We Need BGP?
Beginning BGP Peter J. Welcher Introduction At the time I'm writing this, it is time to register for Networkers / CCIE recertification. I just signed up for Denver. You'll probably be reading this around
Setting Up Your FTP Server
Requirements:! A computer dedicated to FTP server only! Linksys router! TCP/IP internet connection Steps: Getting Started Configure Static IP on the FTP Server Computer: Setting Up Your FTP Server 1. This
WP Tweets PRO User's Guide
WP Tweets PRO User's Guide Contents: 1. Installation 2. Configure WP to Twitter 3. Basic Settings 4. Advanced Settings 5. WP Tweets PRO Settings 6. Strategy Tips Installation WP Tweets PRO is an add-on
Gladinet Cloud Backup V3.0 User Guide
Gladinet Cloud Backup V3.0 User Guide Foreword The Gladinet User Guide gives step-by-step instructions for end users. Revision History Gladinet User Guide Date Description Version 8/20/2010 Draft Gladinet
