Cisco IOS XR Diagnostics

Cisco IOS XR online diagnostics allow you to test and verify hardware functionality while connected to a live network. Health-monitoring and scheduled diagnostics help ensure system high availability (HA). If a problem is detected, online diagnostics help isolate the location of the problem allowing you to identify and replace the hardware. Online diagnostics are also used for system acceptance of new hardware. The online diagnostics contain tests that check different hardware components and verify the data path and control signals. Nondisruptive online diagnostic tests can be scheduled or run on-demand. On-demand diagnostics run from the command-line interface (CLI); scheduled diagnostics run at user-designated intervals or specified times when the system is connected to a live network. Online diagnostics is one of the requirements for high availability (HA). High availability is a set of quality standards that seek to limit the impact of equipment failures on the network. A key part of high availability is detecting hardware failures and taking corrective action while the system continues to run in a live network. Online diagnostics detect hardware failures and provide feedback to high availability software components. Online diagnostics is supported on all cards except Shared Port Adapter (SPA) cards. Offline diagnostics is supported on all nodes. For line card service processor (SP), SPA and physical layer interface module (PLIM), offline diagnostics must be run from line card CPU node. Each CPU node has its own diagnostic process, or test execution engine, but is part of the overall diagnostic system. Diagnostic messages are logged in syslog. See the Implementing Logging Services on Cisco IOS XR Software module of Cisco IOS XR System Monitoring Configuration Guide or use the show logging inc diag command to display the contents of the logging buffer. Integrated Field Diagnostics (IFDs) allow you to load and unload offline diagnostic images. Loading a diagnostic image places the specified location out of service. When an offline diagnostic image is loaded, the diagnostic start command is used to start the test, and the show diagnostic content and show diagnostic results commands are used to show the test list and results. See the Cisco IOS XR Interface and Hardware Components Command Reference for diagnostic command information. The integrated field diagnostics detect problems in the following areas: Hardware components Connectors (loose connectors, bent pins, and so forth) Solder joints Memory (failure over time) 1

The Package must be installed to access the Cisco IOS XR diagnostics. See Cisco IOS XR Getting Started Guide for information on installing packages. For more information about diagnostics on the Cisco IOS XR software and complete descriptions of diagnostics commands listed in this module, see the Related Documents of this module. To locate documentation for other commands that might appear during the execution of a configuration task, search online in the Cisco IOS XR software master command index. Feature History for Release Release 3.3.0 Release 3.4.0 Modification This feature was introduced on the Cisco CRS-1. Configuring health-monitoring diagnostics, including support for monitor syslog, monitor intervals, and the failure count thresholds of test, was added. The show diagnostic result and show diagnostic content command output was modified to include diagnostic monitoring support. The show diagnostic content command output for nodes in the FDIAG RUNNING state has expanded to provide more control over execution of offline test suites. The following online diagnostics tests were added: Control Ethernet Inactive Link Test Self-Ping over Fabric RommonRevision Fabric Diagnosis Test The following information is provided: Diagnostics Framework, page 3 Diagnostics Operations, page 3 Available Online Diagnostic Tests, page 4 Running Cisco IOS XR Online Diagnostics, page 6 Running Integrated Field Diagnostics, page 10 Displaying Diagnostic Test Results, page 15 Displaying the Diagnostic Schedule, page 15 Examples for Running, page 16 Additional References, page 22 2

Diagnostics Framework Diagnostics Framework Cisco IOS XR online diagnostics defines a common framework for diagnostic operations across Cisco platforms running Cisco IOS XR software. Figure 1 illustrates the diagnostics framework and how it interacts with platform dependant diagnostics operations. The command-line interface (CLI) categorizes diagnostics as runtime diagnostics. Cisco IOS XR online diagnostics also provides for easy addition of platform specific tests that contribute in reducing the Mean Time To Repair (MTTR). The Cisco IOS XR online diagnostics framework specifies platform-independent fault detection architecture for centralized and distributed systems. This framework includes common diagnostics CLI and platform-independent fault detection procedures for runtime diagnostics. Figure 1 Diagnostic Framework Provide generic diagnostics & health monitoring framework Online diagnostics layer Generic online diagnostics subsystem framework Runtime diagnostics On-demand Scheduled Health monitoring Platform specific diagnostic Detect bad HW Disruptive and non-disruptive online diagnostics Non-disruptive health monitoring HW layer Hardware 158710 Diagnostics Operations Cisco IOS XR online diagnostics are implemented using tests to check the health of hardware components and to verify proper operation of the system data and control paths. As illustrated in Figure 1, tests can be categorized into runtime and offline diagnostic tests. Runtime Diagnostics Defects can be diagnosed during system operation or runtime. A series of diagnostic checks can be enabled to determine the condition of an online system. Care must be taken to distinguish between disruptive and non-disruptive diagnostics tests. While non-disruptive tests occur in the background and do not affect the data or control plane of a system, disruptive tests will affect live packet flows and should be scheduled during special maintenance windows. The impact of disruptive tests can be minimal or significant. Disruptive tests can take varying amounts of time to complete. Runtime diagnostic checks can be run on demand, can be scheduled to run at a specific time, or can run in the background: 3

Available Online Diagnostic Tests Online diagnostics Online diagnostics tests are triggered by entering a diagnostic start command. You can specify how many times a test runs and whether to continue the test on failure detection. Online diagnostics is useful as a troubleshooting tool that allows you to verify hardware functionality when a hardware fault is suspected. that online diagnostics will not cause bad hardware to reset or power down on the Cisco CRS-1. Syslog messages warn about bad hardware and you need to check the diagnostics results to understand if the tests passed or failed and take further action. Scheduled diagnostics Scheduled diagnostic tests are scheduled to run either at one specific time or periodically. This is especially useful when scheduling disruptive tests during maintenance windows. When failures are detected, appropriate syslog messages are displayed and diagnostic results are saved. Scheduled diagnostics do not cause bad hardware to reset or power down on the Cisco CRS-1. Health-monitoring diagnostics Health-monitoring diagnostic tests are nondisruptive and run in the background while the system is in operation. The role of health-monitoring diagnostics is to proactively detect hardware failures in the live network environment and inform appropriate entities of a failure. It is up to you to determine the number of health-monitoring checks and the interval at which to run health-monitoring checks. The interval can be as granular as 50ms. By default, health-monitoring tests include data path and control path verification as well as proper hardware registers functionality. Offline Diagnostics Offline diagnostic tests run field diagnostics on a node. Available Online Diagnostic Tests The following online diagnostic tests are supported: Control Ethernet Ping Test A nondisruptive test that "pings" each control ethernet node in a chassis from the node where the test is started. The test sends a ping to one node at a time, waiting for a response or until the maximum timeout of 2 seconds is reached, before proceeding to send a ping to the next node. The returned ping response is verified by comparing it byte by byte with the sent ping. Pings are sent only to active nodes within the same chassis as the node from which the test is run. Only one ping is sent per node. Each ping has a 100 byte payload. The test result is PASS if all nodes return the ping and the response matches the sent ping. Fabric Ping Test A nondisruptive test that "pings" each fabric node in a chassis from the node where the test is started. The test sends a ping to one node at a time, waiting for a response or until the maximum timeout of 2 seconds is reached, before proceeding to send a ping to the next node. The returned ping response is verified by comparing it byte by byte with the sent ping. Pings are sent only to active nodes within the same chassis as the node from which the test is run. Due to the undeterministic path of traffic in the fabric, 72 pings are sent per node to maximize coverage. Each ping has a 1-kilobyte payload. The test result is PASS if all nodes return all the pings and all the responses match the sent pings. Due to the 2 second wait timeout for a lost ping, a node that is unreachable or intermittently working impacts the total run time for the test. Therefore, in a worst case scenario where a node has lost all fabric connectivity, the test time can be increased by 2.5 minutes for that node. The total test time depends on how many active nodes there are in the tested rack and how many of the nodes have failing fabric connections. Control Ethernet Inactive Link Test A nondisruptive test that verifies the inactive control Ethernet links between a standby RP and all other nodes in the same rack. This test is available only on RPs and can only be started on the standby RP. The test sends a "ping" packet to a test target node, using the inactive control Ethernet (CE) link. It expects three responses from each node: 4

Available Online Diagnostic Tests One response that travels along the inactive CE return link, which verifies the inactive CE link from test target to standby RP One response that travels along the active CE link through the active RP CPU to the standby RP, which verifies the external inactive CE link from the standby RP to the test target One that travels along the active CE link to the standby RP, which verifies the internal test target CE path Each returned response is verified by comparing it byte-by-byte with the sent ping. When either all three responses from the test target have returned, or a timeout of 2 seconds has expired, the test repeats this procedure for the next node in the rack until all nodes connected to inactive CE links are tested. The test result is PASS if all nodes return all the pings and all the responses match the sent pings. This test can be used to "qualify" the CE connectivity of the standby RP prior to a switchover. Self-Ping over Fabric Test A nondisruptive test that lets a node "ping" itself over the fabric. The test sends a fabric ping to the node itself, waiting for a response or until the maximum timeout of 2 seconds is reached. The returned ping response is verified by comparing it byte by byte with the sent ping. Each ping has a 100-byte payload. This single ping is repeated 100 times with an interval of 300 ms in between each ping. The test result is PASS if all the pings are returned and all the responses match the sent pings. This test is equivalent to the existing health monitoring fabric ping test that runs automatically on all LC nodes. It differs in that it is available on all nodes with fabric connectivity, not only LCs, and in that its health monitoring characteristics (enabled/disabled, run interval, and so on) can be configured by the user. The normal run time for this test is 30 seconds. Due to the 2 second wait timeout for a lost ping, the test time for a node that has failing fabric connectivity may be increased by up to 3.5 minutes. RommonRevision Test A nondisruptive test that verifies that a node is running the minimally acceptable version of ROM Monitor (ROMMON). When a node is rebooted, its current version of ROMMON is retrieved and saved in a shared memory space. This shared memory space is queried for the running version of ROMMON and that version is compared to the minimally acceptable version of ROMMON. If the running version is not greater than or equal the minimal version, the test fails. Fabric Diagnosis Test A non-disruptive, fault isolation test that "pings" each fabric node in a chassis from the standby RP node. The test steers ping test packets through different fabric planes, aggregates ping (pass or fail) results with fabric plane information, analyzes the results, and points out the most logical point of failure (if any) in the chassis. The test can only be executed from the standby RP on demand. Executing this test in a Cisco CRS-1 Carrier Routing System Multishelf System may help determine which fabric stage (s1, s2, or s3) is the most logical point of failure in the system. This test must be run in each LC rack standby RP in the system. For example, if the test reports failures on multiple LC racks, and the failure information points to the same fabric plane, then the most likely point of failure is the S2 stage, which is the card in fabric chassis of the system. We highly recommend that you only run this test when the system experiences fabric issues and needs to isolate the fault. To debug problems reported by the ping tests on the Cisco CRS-1, use the command for internal ping. See Cisco IOS XR Interface and Hardware Components Command Reference for further information on the internal ping command. 5

Running Cisco IOS XR Online Diagnostics Running Cisco IOS XR Online Diagnostics The following options are available for running online diagnostics: Running On-Demand Online Diagnostics, page 6 Scheduling Online Diagnostics, page 7 Configuring Health-Monitoring Diagnostics, page 9 For complete syntax and usage information for the diagnostic commands, see Cisco IOS XR Interface and Hardware Components Command Reference. Running On-Demand Online Diagnostics To start and stop on-demand online diagnostics, perform the following procedure. A message is displayed when the diagnostic tests are completed. The diagnostic stop command is used for stopping a test before it has completed. For example, you can stop a test that takes longer than desired. See Cisco IOS XR Interface and Hardware Components Command Reference for syntax information. Prerequisites The Package must be installed before starting this procedure. See Cisco IOS XR Getting Started Guide for information on installing packages. SUMMARY STEPS 1. admin 2. show diagnostic content location node-id 3. diagnostic start location node-id test {id all basic non-disruptive} 6

Running Cisco IOS XR Online Diagnostics DETAILED STEPS Step 1 Command or Action admin Purpose Enters administration EXEC mode. Step 2 RP/0/RP0/CPU0:router# admin show diagnostic content location node-id RP/0/RP0/CPU0:router(admin)# show diagnostic content location 0/RP1/CPU0 Step 3 diagnostic start location node-id test {id all basic non-disruptive} Displays the available tests and their attributes, including which commands are in the nondisruptive category. Runs the specified on-demand diagnostic test or series of tests. RP/0/RP0/CPU0:router(admin)# diagnostic start location 0/RP1/CPU0 test 1 What to Do Next Proceed to the Displaying Diagnostic Test Results section on page 15 to display the results from the on-demand diagnostic test. Scheduling Online Diagnostics Prerequisites SUMMARY STEPS You can schedule online diagnostics to run at a designated time of day or on a daily or weekly basis for a specific node. You can schedule tests to run only once or to repeat at an interval. Use the no form of the diagnostic schedule command to remove the scheduling. To schedule online diagnostics, perform the following procedure. The Package must be installed before starting this procedure. See Cisco IOS XR Getting Started Guide for information on installing packages. 1. admin 2. show diagnostic content location node-id 3. configure 4. diagnostic schedule location node-id test {id all basic non-disruptive} {daily on month day year weekly day-of-week} hour:minute 5. end 6. show diagnostic schedule location node-id 7

Running Cisco IOS XR Online Diagnostics DETAILED STEPS Step 1 Command or Action admin Purpose Enters administration EXEC mode. Step 2 Step 3 RP/0/RP0/CPU0:router# admin show diagnostic content location node-id RP/0/RP0/CPU0:router(admin)# show diagnostic content location 0/RP1/CPU0 configure Displays the available tests and their attributes, including which commands are in the nondisruptive category. Enters administration configuration mode. RP/0/RP0/CPU0:router(admin)# configure Step 4 diagnostic schedule location node-id test {id all basic non-disruptive} {daily on month day year weekly day-of-week} hour:minute Schedules on-demand diagnostic tests for a specific location and date and time. Step 5 Step 6 RP/0/RP0/CPU0:router(admin-config)# diagnostic schedule location 0/RP1/CPU0 test 1 daily 12:42 end RP/0/RP0/CPU0:router(admin-config)# end show diagnostic schedule location node-id Saves configuration changes. When you issue the end command, the system prompts you to commit changes: Uncommitted changes found, commit them before exiting(yes/no/cancel)? [cancel]: Entering yes saves configuration changes to the running configuration file, exits the configuration session, and returns the router to EXEC mode. Entering no exits the configuration session and returns the router to EXEC mode without committing the configuration changes. Entering cancel leaves the router in the current configuration session without exiting or committing the configuration changes. Displays the scheduled diagnostics for a specific location. RP/0/RP0/CPU0:router(admin)# show diagnostic schedule location 0/RP1/CPU0 8

Running Cisco IOS XR Online Diagnostics What to Do Next Proceed to the Displaying Diagnostic Test Results section on page 15 to display the results from the scheduled diagnostic test. Proceed to the Displaying the Diagnostic Schedule section on page 15 to display the scheduled diagnostic tasks. Configuring Health-Monitoring Diagnostics Prerequisites SUMMARY STEPS DETAILED STEPS To configure health-monitoring diagnostic testing on specified cards while the router is connected to a live network, perform the following procedure. You can configure the: Execution interval for each health-monitoring test Failure threshold for each health-monitoring test Generation of system messages upon test failure Location where each health-monitoring test is run The Package must be installed before starting this procedure. See Cisco IOS XR Getting Started Guide for information on installing packages. 1. admin 2. configure 3. diagnostic monitor syslog 4. diagnostic monitor location node-id test {id test-name} [disable] 5. diagnostic monitor interval location node-id test {id test-name} number-of-days hour:minutes:seconds.milliseconds 6. diagnostic monitor threshold location node-id test {id test-name} failure count failures Step 1 Command or Action admin Purpose Enters administration EXEC mode. Step 2 RP/0/RP0/CPU0:router# admin configure Enters administration configuration mode. RP/0/RP0/CPU0:router(admin)# configure 9

Running Integrated Field Diagnostics Step 3 Command or Action diagnostic monitor syslog RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor syslog Step 4 diagnostic monitor location node-id test {id test-name} [disable] Purpose (Optional) Enables the generation of a syslog message when any health-monitoring test fails. Configures the health-monitoring diagnostic testing for a specified location. Step 5 RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor location 0/1/cpu0 test 1 diagnostic monitor interval location node-id test {id test-name} number-of-days hour:minutes:seconds.milliseconds Configures the health-monitoring diagnostic testing for a specified interval for a specified location. Step 6 RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor interval location 0/1/cpu0 test 1 12:55:25.100 diagnostic monitor threshold location node-id test {id test-name} failure count failures Configures the health-monitoring diagnostic testing failure threshold. RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor location 0/1/cpu0 test 1 failure count 25 What to Do Next Proceed to the Displaying Diagnostic Test Results section on page 15 to display the results of the health-diagnostic monitoring output. Running Integrated Field Diagnostics To load and unload an offline diagnostics image for integrated field diagnostics, perform one of the following procedures: Autostart Integrated Field Diagnostics, page 11 Manually Executing Integrated Field Diagnostics, page 13 Loading an image places the specified card out of service. Unloading the image returns the specified card to service. Caution To run offline diagnostics successfully, ensure there are no external fiber cables connected to the line card PLIM ports and that all PLIM ports are populated with optics or small form-factor pluggable (SFP) optic modules. See the Cisco IOS XR hardware documentation on Cisco.com for information on PLIM ports. 10

Running Integrated Field Diagnostics For complete syntax and usage information for the diagnostic commands, see Cisco IOS XR Interface and Hardware Components Command Reference. Prerequisites The Package must be installed before starting this procedure. See Cisco IOS XR Getting Started Guide for information on installing packages. Autostart Integrated Field Diagnostics When enabled, the autostart feature for Integrated Field Diagnostics automatically starts running the tests when the image is loaded. A message is displayed when the diagnostic tests are completed. The diagnostic start command is not required. When offline diagnostics are loaded with the diagnostic load location node-id command, the specified node remains in the "FDIAG RUNNING" state until diagnostics are unloaded using the diagnostic unload location node-id command. The following commands can be used to unload the diagnostic test, display the test results, and display diagnostic test content when the tests are completed: show diagnostic result show diagnostic content diagnostic unload The show diagnostic result location node-id command output for a specified node can be viewed until the diagnostic unload location node-id command is used. Once the test is unloaded, the test results can no longer be viewed. While a node is in the "FDIAG RUNNING" state, tests are run in response to the optional autostart keyword of the diagnostic load location node-id command or the diagnostic start location node-id command. When an individual test finishes, a message is printed and results are updated. The results can be read using the show diagnostic result location node-id command. When a test finishes, a new diagnostic start location node-id command can be invoked, because the card remains in the "FDIAG RUNNING" state until it is explicitly unloaded using the diagnostic unload location node-id command. SUMMARY STEPS 1. admin 2. show platform 3. diagnostic load location node-id autostart {basic all} 4. show platform 5. show diagnostic result location node-id [test {id all}] [detail] 6. diagnostic unload location node-id 7. show platform 11

Running Integrated Field Diagnostics DETAILED STEPS Step 1 Command or Action admin Purpose Enters administration EXEC mode. Step 2 Step 3 RP/0/RP0/CPU0:router# admin show platform RP/0/RP0/CPU0:router(admin)# show platform diagnostic load location node-id autostart {basic all} Displays the status of cards in the system. Verify that all the cards are in the IOS XR RUN state. Loads an offline diagnostic image for integrated field diagnostics, placing the specified card out of service. RP/0/RP0/CPU0:router(admin)# diagnostic load location 0/3/CPU0 autostart all Caution This command places all modules in the slot of the specified node out of service. Step 4 Step 5 Step 6 Step 7 show platform RP/0/RP0/CPU0:router(admin)# show platform show diagnostic result location node-id [test {id all}] [detail] RP/0/RP0/CPU0:router(admin)# show diagnostic result location 0/3/CPU0 test 1 diagnostic unload location node-id RP/0/RP0/CPU0:router(admin)# diagnostic unload location 0/3/CPU0 show platform RP/0/RP0/CPU0:router(admin)# show platform The distributed route processor (DRP) does not support the automatic running of tests when the image is loaded for CPU0 and CPU1. After the diagnostic image is loaded, use the diagnostic start location node-id test {id all basic non-disruptive} command to execute the tests. Displays the status of cards in the system. Verify that the card state goes to the FDIAG RUNNING state. For modular services cards (MSCs), the service processor (SP) and CPU0 instance should go to the FDIAG RUNNING state. For distributed route processors (DRPs), the SP, CPU0 and CPU1 instances should all go to the FDIAG RUNNING state. Displays result of tests run at the specified location. Possible results are shown as: Passed (.) Failed (F) Untested (U) Unloads the offline diagnostic image, placing the specified card back in service. Displays the status of cards in the system. Verify that all cards are in the IOS-XR RUN state. 12

Running Integrated Field Diagnostics Manually Executing Integrated Field Diagnostics Manually executing Integrated Field Diagnostics requires you to enter the diagnostic start command to start the diagnostic testing. A message is displayed when the diagnostic tests are completed. The following commands can be used to unload the diagnostic test, display the test results, and display diagnostic test content when the tests are completed: diagnostic unload show diagnostic result show diagnostic content The show diagnostic result location node-id command output for a specified node can be viewed until the diagnostic unload location node-id command is used. Once the test is unloaded, the test results can no longer be viewed. When offline diagnostics are loaded with the diagnostic load location node-id command, the specified node remains in the "FDIAG RUNNING" state until diagnostics are unloaded using the diagnostic unload location node-id command. While a node is in "FDIAG RUNNING" state, tests are run in response to the optional autostart keyword of the diagnostic load location node-id command or the diagnostic start location node-id command. When an individual test finishes, a message is printed and results are updated. The results can be read using the show diagnostic result location node-id command. When a test finishes, a new diagnostic start location node-id command can be invoked, because the card remains in the "FDIAG RUNNING" state until it is explicitly unloaded using the diagnostic unload location node-id command. SUMMARY STEPS 1. admin 2. show platform 3. diagnostic load location node-id 4. show platform 5. diagnostic start location node-id test {id all basic non-disruptive} 6. show diagnostic result location node-id [test {id all}] [detail] 7. diagnostic unload location node-id 8. show platform 13

Running Integrated Field Diagnostics DETAILED STEPS Step 1 Command or Action admin Purpose Enters administration EXEC mode. Step 2 Step 3 RP/0/RP0/CPU0:router# admin show platform RP/0/RP0/CPU0:router(admin)# show platform diagnostic load location node-id RP/0/RP0/CPU0:router(admin)# diagnostic load location 0/3/CPU0 Displays the status of cards in the system. Verify that all the cards are in the IOS-XR RUN state. Loads an offline diagnostic image for integrated field diagnostics, placing the specified card out of service. Caution This command places all modules in the slot of the specified node out of service. Step 4 show platform RP/0/RP0/CPU0:router(admin)# show platform Step 5 diagnostic start location node-id test {id all basic non-disruptive} Displays the status of cards in the system. Verify that the card state goes to the FDIAG RUNNING state. For modular services cards (MSCs), the service processor (SP) and CPU0 instance should go to the FDIAG RUNNING state. For distributed route processors (DRPs), the SP, CPU0 and CPU1 instances should all go to the FDIAG RUNNING state. Starts the selected diagnostic tests. Step 6 Step 7 Step 8 RP/0/RP0/CPU0:router(admin)# diagnostic start location 0/3/CPU0 test all show diagnostic result location node-id [test {id all}] [detail] RP/0/RP0/CPU0:router(admin)# show diagnostic result location 0/3/CPU0 test 1 diagnostic unload location node-id RP/0/RP0/CPU0:router(admin)# diagnostic unload location 0/3/CPU0 show platform RP/0/RP0/CPU0:router(admin)# show platform Displays result of tests run at specified location. Possible results are shown as: Passed (.) Failed (F) Untested (U) Unloads the offline diagnostic image, placing the specified card back in service. Displays the status of cards in the system. Verify that all cards are in the IOS-XR RUN state. 14

Displaying Diagnostic Test Results Displaying Diagnostic Test Results To display test results for online diagnostics, perform the following procedure. For complete syntax and usage information for the diagnostic commands, see Cisco IOS XR Interface and Hardware Components Command Reference. SUMMARY STEPS 1. admin 2. show diagnostic result location node-id [test {id all}] [detail] DETAILED STEPS Step 1 Command or Action admin Purpose Enters administration EXEC mode. Step 2 RP/0/RP0/CPU0:router# admin show diagnostic result location node-id [test {id all}] [detail] RP/0/RP0/CPU0:router(admin)# show diagnostic result location 0/RP1/CPU0 test 1 Displays result of tests run at specified location. Possible results are shown as: Passed (.) Failed (F) Untested (U) Displaying the Diagnostic Schedule To display the diagnostic schedule, perform the following procedure. For complete syntax and usage information for the diagnostic commands, see Cisco IOS XR Interface and Hardware Components Command Reference. SUMMARY STEPS 1. admin 2. show diagnostic schedule location node-id 15

Examples for Running DETAILED STEPS Step 1 Command or Action admin Purpose Enters administration EXEC mode. Step 2 RP/0/RP0/CPU0:router# admin show diagnostic schedule location node-id RP/0/RP0/CPU0:router(admin)# show diagnostic schedule location 0/3/CPU0 Display the current scheduled diagnostic tasks for 0/3/CPU0. Examples for Running This section contains the following examples: Running Health-Monitoring Diagnostics: Example, page 16 Running Cisco IOS XR On-Demand Online Diagnostics: Example, page 17 Scheduling Cisco IOS XR Online Diagnostics: Example, page 18 Running Cisco IOS XR Integrated Field Diagnostics: Example, page 18 Displaying Diagnostics Test Results: Example, page 21 For complete syntax and usage information for the diagnostic commands, see Cisco IOS XR Interface and Hardware Components Command Reference. Running Health-Monitoring Diagnostics: Example In the following example, the generation of a syslog message when any health-monitoring test fails is enabled: RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor syslog In the following example, health-monitoring diagnostic test 1 for 0/1/CPU0 is enabled: RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor location 0/1/cpu0 test 1 In the following example, health-monitoring diagnostic test 1 for 0/1/CPU0 is set to an interval of every 15 days at 6:00: RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor interval location 0/1/cpu0 test 1 15 6:0:0.0 In the following example, the health-monitoring diagnostic testing failure threshold is set to 25 failures for test 1 on 0/1/CPU0: RP/0/RP0/CPU0:router(admin-config)# diagnostic monitor threshold location 0/1/CPU0 test 1 failure count 25 16

Examples for Running Running Cisco IOS XR On-Demand Online Diagnostics: Example In the following example, on-demand diagnostics are run on route processor (RP) card 0/RP1/CPU0. The tests available for the card are checked, then one of the available tests is run. RP/0/RP0/CPU0:router(admin)# show diagnostic content location 0/RP1/CPU0 RP 0/RP1/CPU0: Diagnostics test suite attributes: M/C/* - Minimal bootup level test / Complete bootup level test / NA B/* - Basic ondemand test / NA P/V/* - Per port test / Per device test / NA D/N/* - Disruptive test / Non-disruptive test / NA S/* - Only applicable to standby unit / NA X/* - Not a health monitoring test / NA F/* - Fixed monitoring interval test / NA E/* - Always enabled monitoring test / NA A/I - Monitoring is active / Monitoring is inactive Test Interval Thre- ID Test Name Attributes (day hh:mm:ss.ms shold) ==== ================================== ============ ================= ===== 1) ControlEthernetPingTest ---------> *B*N****I 001 00:00:00.000 1 2) SelfPingOverFabric --------------> *B*N****I 001 00:00:00.000 1 3) FabricPingTest ------------------> *B*N****I 001 00:00:00.000 1 4) ControlEthernetInactiveLinkTest -> *B*N****I 001 00:00:00.000 1 RP/0/RP0/CPU0:router(admin)# diagnostic start location 0/RP1/CPU0 test all RP/0/RP1/CPU0:Jul 25 16:22:49.692 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-TEST_RUNNING : RP 0/RP1/CPU0: Running ControlEthernetPingTest{ID=1}... RP/0/RP1/CPU0:Jul 25 16:22:49.838 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-TEST_OK : RP 0/RP1/CPU0: ControlEthernetPingTest{ID=1} has completed successfully RP/0/RP1/CPU0:Jul 25 16:22:49.844 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-TEST_RUNNING : RP 0/RP1/CPU0: Running SelfPingOverFabric{ID=2}... RP/0/RP1/CPU0:Jul 25 16:23:19.948 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-TEST_OK : RP 0/RP1/CPU0: SelfPingOverFabric{ID=2} has completed successfully RP/0/RP1/CPU0:Jul 25 16:23:19.951 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-TEST_RUNNING : RP 0/RP1/CPU0: Running FabricPingTest{ID=3}... RP/0/RP1/CPU0:Jul 25 16:23:20.643 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-TEST_OK : RP 0/RP1/CPU0: FabricPingTest{ID=3} has completed successfully RP/0/RP1/CPU0:Jul 25 16:23:20.646 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-TEST_RUNNING : RP 0/RP1/CPU0: Running ControlEthernetInactiveLinkTest{ID=4}... RP/0/RP1/CPU0:Jul 25 16:23:20.927 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-TEST_OK : RP 0/RP1/CPU0: ControlEthernetInactiveLinkTest{ID=4} has completed successfully In the following example, an on-demand diagnostics test is stopped on RP card 0/RP1/CPU0: RP/0/RP0/CPU0:router(admin)# diagnostic start location 0/RP1/CPU0 test all RP/0/RP1/CPU0:Jul 25 16:28:08.801 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-TEST_RUNNING : RP 0/RP1/CPU0: Running ControlEthernetPingTest{ID=1}... RP/0/RP1/CPU0:Jul 25 16:28:08.957 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-TEST_OK : RP 0/RP1/CPU0: ControlEthernetPingTest{ID=1} has completed successfully RP/0/RP1/CPU0:Jul 25 16:28:08.963 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-TEST_RUNNING : RP 0/RP1/CPU0: Running SelfPingOverFabric{ID=2}... RP/0/RP0/CPU0:router(admin)# diagnostic stop location 0/RP1/CPU0 RP/0/RP1/CPU0:Jul 25 16:28:39.068 : online_diag_hfr_rp[359]: 17

Examples for Running %DIAG-DIAG-6-TEST_OK : RP 0/RP1/CPU0: SelfPingOverFabric{ID=2} has completed successfully RP/0/RP1/CPU0:Jul 25 16:28:39.068 : online_diag_hfr_rp[359]: %DIAG-DIAG-6-DIAG_STOPPED : RP 0/RP1/CPU0: Diagnostic is stopped. Scheduling Cisco IOS XR Online Diagnostics: Example In the following example, no diagnostic test schedule is configured for 0/0/CPU0: RP/0/RP0/CPU0:router(admin)# show diagnostic schedule location 0/0/CPU0 Current Time = Wed Nov 30 15:50:56 2005 Diagnostic for LC 0/0/CPU0 is not scheduled. In the following example, diagnostic testing is scheduled for 0/0/CPU0: RP/0/RP0/CPU0:router(admin)# configure RP/0/RP0/CPU0:router(admin-config)# diagnostic schedule location 0/0/CPU0 test 1 daily 14:40 RP/0/RP0/CPU0:router(admin-config)# end Uncommitted changes found, commit them before exiting(yes/no/cancel)? [cancel]: yes RP/0/RP0/CPU0:Nov 30 15:55:10.211 : config[65738]: %MGBL-LIBTARCFG-6-ADMIN_COMMIT : Administration configuration committed by user 'administrator'. Use 'show configuration commit changes 2000000001' to view the changes. RP/0/RP0/CPU0:Nov 30 15:55:11.549 : config[65738]: %MGBL-SYS-5-CONFIG_I : Configured from console by administrator RP/0/RP0/CPU0:router(admin)# In the following example, the diagnostic test schedule for 0/0/CPU0 shows that test 1 will run daily at 14:40: RP/0/RP0/CPU0:router(admin)# show diagnostic schedule location 0/0/CPU0 Current Time = Wed Nov 30 16:03:38 2005 Diagnostic for LC 0/0/CPU0: Schedule #1: To be run daily 14:40 Test ID(s) to be executed: 1. Running Cisco IOS XR Integrated Field Diagnostics: Example In the following example, the state of the nodes in the system are checked before running the integrated field diagnostics. The integrated field diagnostics are loaded on the card, placing the card out of service. While the diagnostics are running, the show platform command for the card is used to check that the diagnostics are running and the card is in the FDIAG RUNNING state. The diagnostics are then unloaded, returning the card to service. RP/0/RP0/CPU0:router(admin)# show platform Node Type PLIM State Config State ----------------------------------------------------------------------------- 0/0/SP MSC(SP) N/A IOS XR RUN PWR,NSHUT,MON 0/0/CPU0 MSC 4OC192-POS/DPT IOS XR RUN PWR,NSHUT,MON 0/RP0/CPU0 RP(Active) N/A IOS XR RUN PWR,NSHUT,MON 0/RP1/CPU0 RP(Standby) N/A IOS XR RUN PWR,NSHUT,MON 0/SM0/SP FC/S(SP) N/A IOS XR RUN PWR,NSHUT,MON 0/SM1/SP FC/S(SP) N/A IOS XR RUN PWR,NSHUT,MON 18

Examples for Running RP/0/RP0/CPU0:router(admin)# diagnostic load location 0/RP1/CPU0 diagnostic load will bring requested slot out of service. [confirm(y/n)] y User has confirmed diagnostic load request Preparing UUT for Diagnostics software. Downloading IDS diagnostics image /pkg/gsr/ucode/hfr-diag-rp-fdiags Please wait for UUT image downloading... diagnostic load in progress. RP/0/RP0/CPU0:Nov 30 16:23:56.845 : shelfmgr[298]: %PLATFORM-SHELFMGR-3-USER_RESET : Node 0/RP1/CPU0 is reset due to user reload request RP/0/RP0/CPU0:Nov 30 16:23:56.904 : redcon[287]: %HA-REDCON-6-STANDBY_CONNECTION_DOWN : connection to standby card is DOWN RP/0/RP0/CPU0:Nov 30 16:23:57.444 : syncfs2[309]: %MEDIA-SYNCFS2-7-LRD_NOTAVAILABLE : Standby has been rendered UNAVAILABLE - STOP SYNC RP/0/RP0/CPU0:Nov 30 16:24:11.462 : redcon[287]: %HA-REDCON-4-STANDBY_NOT_READY : standby card is NOT ready RP/0/RP0/CPU0:Nov 30 16:24:35.072 : online_diag_hfr_rp[341]: %HFRDIAG-6-INFO : [UUT 0/RP1/CPU0: INFO: CRS-1 RP Field Diagnostics V5.1] RP/0/RP0/CPU0:router(admin)# show platform Node Type PLIM State Config State ----------------------------------------------------------------------------- 0/0/SP MSC(SP) N/A IOS XR RUN PWR,NSHUT,MON 0/0/CPU0 MSC 4OC192-POS/DPT IOS XR RUN PWR,NSHUT,MON 0/RP0/CPU0 RP(Active) N/A IOS XR RUN PWR,NSHUT,MON 0/RP1/CPU0 RP(Standby) N/A FDIAG RUNNING PWR,NSHUT,MON 0/SM0/SP FC/S(SP) N/A IOS XR RUN PWR,NSHUT,MON 0/SM1/SP FC/S(SP) N/A IOS XR RUN PWR,NSHUT,MON RP/0/RP0/CPU0:router(admin)# show diagnostic content location 0/RP1/CPU0 HQRP(FD) 0/RP1/CPU0: Diagnostics test suite attributes: M/C/* - Minimal bootup level test / Complete bootup level test / NA B/* - Basic ondemand test / NA P/V/* - Per port test / Per device test / NA D/N/* - Disruptive test / Non-disruptive test / NA S/* - Only applicable to standby unit / NA X/* - Not a health monitoring test / NA F/* - Fixed monitoring interval test / NA E/* - Always enabled monitoring test / NA A/I - Monitoring is active / Monitoring is inactive Test Interval Thre- ID Test Name Attributes (day hh:mm:ss.ms shold) ==== ================================== ============ ================= ===== 1) CRSLinecard.QuickTests ----------> *B*D****I 000 00:00:00.000 n/a 2) CRSL3SP.MemoryTest --------------> ***D****I 000 00:00:00.000 n/a 3) CRSL3SP.FuncTests ---------------> ***D****I 000 00:00:00.000 n/a 4) L3SPTests.FuncTests -------------> ***D****I 000 00:00:00.000 n/a 5) PPCModule.MemoryTest ------------> ***D****I 000 00:00:00.000 n/a 6) PPCModule.FuncTests -------------> ***D****I 000 00:00:00.000 n/a 7) LCMotherboard.FuncTests ---------> ***D****I 000 00:00:00.000 n/a 8) SharqModule.MemoryTest ----------> ***D****I 000 00:00:00.000 n/a 9) SprayerModule.MemoryTest --------> ***D****I 000 00:00:00.000 n/a 10) SprayerModule.FuncTests ---------> ***D****I 000 00:00:00.000 n/a 11) Sponge0Module.MemoryTest --------> ***D****I 000 00:00:00.000 n/a 12) Sponge0Module.FuncTests ---------> ***D****I 000 00:00:00.000 n/a 13) Sponge1Module.MemoryTest --------> ***D****I 000 00:00:00.000 n/a 14) Sponge1Module.FuncTests ---------> ***D****I 000 00:00:00.000 n/a 19

Examples for Running 15) IngressMetroModul.MemoryTest ----> ***D****I 000 00:00:00.000 n/a 16) IngressMetroModul.FuncTests -----> ***D****I 000 00:00:00.000 n/a 17) EgressMetroModule.MemoryTest ----> ***D****I 000 00:00:00.000 n/a 18) EgressMetroModule.FuncTests -----> ***D****I 000 00:00:00.000 n/a 19) L3BIST.MemoryTest ---------------> ***D****I 000 00:00:00.000 n/a 20) L3FCRAMMarch.MemoryTest ---------> ***D****I 000 00:00:00.000 n/a 21) PacketTests.FuncTests -----------> ***D****I 000 00:00:00.000 n/a 22) 10GEPLIM(SP).FuncTests ----------> ***D****I 000 00:00:00.000 n/a 23) PLIM8x10GE.MemoryTest -----------> ***D****I 000 00:00:00.000 n/a 24) PLIM8x10GE.FuncTests ------------> ***D****I 000 00:00:00.000 n/a 25) L3PacketTests.FuncTests ---------> ***D****I 000 00:00:00.000 n/a RP/0/RP0/CPU0:router(admin)# diagnostic start location 0/RP1/CPU0 test 2 RP/0/RP0/CPU0:Nov 30 16:30:21.978 : online_diag_hfr_rp[341]: %DIAG-6-TEST_RUNNING : IFD 0/RP1/CPU0: Running Quick Test List{ID=2}... RP/0/RP0/CPU0:Nov 30 16:30:22.103 : online_diag_hfr_rp[341]: %HFRDIAG-3-ERROR : [UUT 0/RP1/CPU0: ERROR: Ids System - Slots - 33 RP - Ingress Module - External Memory - Bank 0 - Memory Address Pins Test: after writing byte pattern 20 to addr=0x00a00000, content at addr=0x00800000 changed from 0x00000000 to 0x14141414] RP/0/RP0/CPU0:Nov 30 16:30:39.747 : online_diag_hfr_rp[341]: %HFRDIAG-6-INFO : [UUT 0/RP1/CPU0: RESULT: UUT RP Field Diagnostics status: FAIL] RP/0/RP0/CPU0:Nov 30 16:30:39.747 : online_diag_hfr_rp[341]: %HFRDIAG-6-INFO : [UUT 0/RP1/CPU0: RESULT: tests_run=72, tests_fail=1, num_tests=72] RP/0/RP0/CPU0:Nov 30 16:30:39.899 : online_diag_hfr_rp[341]: %DIAG-3-TEST_FAIL : IFD 0/RP1/CPU0: Quick Test List{ID=2} has failed. Error code = 0x1 RP/0/RP0/CPU0:router(admin)# show diagnostic result location 0/RP1/CPU0 test 2 detail Current bootup diagnostic level for HQRP(FD) 0/RP1/CPU0: minimal Test results: (. = Pass, F = Fail, U = Untested) 2 ) Quick Test List -----------------> F Error code ------------------> 1 (DIAG_FAILURE) Total run count -------------> 1 Last test execution time ----> Wed Nov 30 16:30:21 2005 First test failure time -----> Wed Nov 30 16:30:21 2005 Last test failure time ------> Wed Nov 30 16:30:21 2005 Last test pass time ---------> n/a Total failure count ---------> 1 Consecutive failure count ---> 1 RP/0/RP0/CPU0:router(admin)# diagnostic unload location 0/RP1/CPU0 RP/0/RP0/CPU0:Nov 30 16:39:07.711 : online_diag_hfr_rp[341]: %DIAG-6-GOLDXR_GENERAL : card_remove-d(529,-1) RP/0/RP0/CPU0:Nov 30 16:39:07.711 : shelfmgr[298]: %PLATFORM-SHELFMGR-3-USER_RESET : Node 0/RP1/CPU0 is reset due to user reload request RP/0/RP0/CPU0:Nov 30 16:39:27.740 : shelfmgr[298]: %PLATFORM-MBIMGR-7-IMAGE_VALIDATED : 0/RP1/CPU0: MBI tftp:/hfr-os-mbi-3.3.80/mbihfr-rp.vm validated RP/0/RP0/CPU0:Nov 30 16:41:27.615 : syncfs2[309]: %MEDIA-SYNCFS2-7-LRD_AVAILABLE : Standby has become AVAILABLE - START SYNC RP/0/RP0/CPU0:Nov 30 16:41:28.354 : redcon[287]: %HA-REDCON-6-STANDBY_CONNECTION_UP : connection to standby card is UP... 20

Examples for Running RP/0/RP0/CPU0:Nov 30 16:43:07.822 : redcon[287]: %HA-REDCON-4-STANDBY_READY : standby card is ready RP/0/RP1/CPU0:Nov 30 16:43:04.877 : redcon[288]: %HA-REDCON-6-STBY_STANDBY_READY : This card is standby and is ready RP/0/RP0/CPU0:router(admin)# show platform Node Type PLIM State Config State ----------------------------------------------------------------------------- 0/0/SP MSC(SP) N/A IOS XR RUN PWR,NSHUT,MON 0/0/CPU0 MSC 4OC192-POS/DPT IOS XR RUN PWR,NSHUT,MON 0/RP0/CPU0 RP(Active) N/A IOS XR RUN PWR,NSHUT,MON 0/RP1/CPU0 RP(Standby) N/A IOS XR RUN PWR,NSHUT,MON 0/SM0/SP FC/S(SP) N/A IOS XR RUN PWR,NSHUT,MON 0/SM1/SP FC/S(SP) N/A IOS XR RUN PWR,NSHUT,MON Displaying Diagnostics Test Results: Example In the following example, the overall online diagnostics test results for 0/0/CPU0 are displayed: RP/0/RP0/CPU0:router(admin)# show diagnostic result location 0/0/cpu0 Current bootup diagnostic level for LC 0/0/CPU0: minimal LC 0/0/CPU0: Overall diagnostic result: PASS Current bootup diagnostic level for LC 0/0/CPU0: minimal Test results: (. = Pass, F = Fail, U = Untested) 1 ) Control Ethernet Ping Test ------>. In the following example, the detailed results of test 1 on 0/0/CPU0 are displayed: RP/0/RP0/CPU0:router(admin)# show diagnostic result location 0/0/CPU0 test 1 detail LC 0/0/CPU0: Overall diagnostic result: UNTESTED Diagnostic level at card bootup: minimal Test results: (. = Pass, F = Fail, U = Untested) 1 ) Control Ethernet Ping Test ------>. Error code ------------------> 0 (DIAG_SUCCESS) Total run count -------------> 1 Last test execution time ----> Wed Nov 30 17:05:21 2005 First test failure time -----> n/a Last test failure time ------> n/a Last test pass time ---------> Wed Nov 30 17:05:21 2005 Total failure count ---------> 0 Consecutive failure count ---> 0 21

Additional References Additional References The following sections provide references related to Cisco IOS XR diagnostics. Related Documents Related Topic Diagnostics commands: complete command syntax, command modes, command history, defaults, usage guidelines, and examples Initial system bootup and configuration information for a router using Cisco IOS XR software Document Title Cisco IOS XR Interface and Hardware Components Command Reference, Release 3.4 Cisco IOS XR Getting Started, Release 3.4 Standards Standards No new or modified standards are supported by this feature, and support for existing standards has not been modified by this feature. Title MIBs MIBs MIBs Link To locate and download MIBs using Cisco IOS XR software, use the Cisco MIB Locator found at the following URL and choose a platform under the Cisco Access Products menu: http://cisco.com/public/sw-center/netmgmt/cmtk/mibs.shtml RFCs RFCs Title There are no applicable RFCs for this module. 22

Additional References Technical Assistance Description The Cisco Technical Support website contains thousands of pages of searchable technical content, including links to products, technologies, solutions, technical tips, and tools. Registered Cisco.com users can log in from this page to access even more content. Link http://www.cisco.com/techsupport 23

Additional References 24