Junos Space. Network Management Platform Monitoring and Troubleshooting Guide. Release Published:

Transcription

1 Junos Space Network Management Platform Monitoring and Troubleshooting Guide Release 13.3 Published:

2 Juniper Networks, Inc North Mathilda Avenue Sunnyvale, California USA All rights reserved. Juniper Networks, Junos, Steel-Belted Radius, NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United States and other countries. The Juniper Networks Logo, the Junos logo, and JunosE are trademarks of Juniper Networks, Inc. All other trademarks, service marks, registered trademarks, or registered service marks are the property of their respective owners. Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice. Junos Space Network Management Platform Monitoring and Troubleshooting Guide All rights reserved. The information in this document is current as of the date on the title page. YEAR 2000 NOTICE Juniper Networks hardware and software products are Year 2000 compliant. Junos OS has no known time-related limitations through the year However, the NTP application is known to have some difficulty in the year END USER LICENSE AGREEMENT The Juniper Networks product that is the subject of this technical documentation consists of (or is intended for use with) Juniper Networks software. Use of such software is subject to the terms and conditions of the End User License Agreement ( EULA ) posted at By downloading, installing or using such software, you agree to the terms and conditions of that EULA. ii

3 Table of Contents About the Documentation xiii Documentation and Release Notes xiii Documentation Conventions xiii Documentation Feedback xv Requesting Technical Support xv Self-Help Online Tools and Resources xvi Opening a Case with JTAC xvi Part 1 Monitoring Chapter 1 Pre-requisites for Monitoring Network Monitoring Network Monitoring Workspace Overview Network Monitoring Reports Overview Resource Graphs Key SNMP Customized (KSC) Performance Reports, Node Reports, and Domain Reports Database Reports Statistics Reports Admin: Configuring Network Monitoring Network Monitoring System: System Information Generating a Log File for Troubleshooting Notification Status Fabric Management Fabric Management Overview Single-Node Functionality Multinode Functionality Specialized Node Functionality Node Function Availability Adding a Node to an Existing Junos Space Fabric Viewing Nodes in the Fabric Changing Views Viewing Fabric Node Details Performing Fabric Node Actions Configuring the Network Settings of a Node in the Junos Space Fabric Network Settings Configuration Guidelines Changing the VIP Interface in the Same Subnet Changing the Node Management IP in the Same Subnet Changing the Default Gateway Changing the Management IP to a Different Network Adding the Device Management IP Address iii

4 Junos Space Network Management Monitoring and Troubleshooting Guide Changing the Device Management IP Address in the Same Subnet Changing the Device Management IP Address to a Different Network Deleting a Device Management IP Address Changing the VIP Interface to a Different Network Changing the Node Management IP Address of All Nodes in the Fabric to the Same Subnet Changing the VIP Interface of a Multiple-Node Fabric to a Different Network Shutting Down or Rebooting a Junos Space Appliance Node From Junos Space Deleting a Node from the Junos Space Fabric Replacing a Failed Junos Space Node Chapter 2 Monitoring the Software Monitoring Junos Space Network Management Platform Usage Junos Space Network Management Platform Widgets Devices Device Templates CLI Configlets Images and Scripts Reports Configuration Files Jobs Role Based Access Control Audit Logs Chapter 3 Monitoring the Fabric Overall System Condition and Fabric Load History Overview Overall System Condition Fabric Load History Active Users History Monitoring Nodes in the Fabric Viewing and Modifying the SNMP Configuration for a Fabric Node Starting SNMP Monitoring on Fabric Nodes Stopping SNMP Monitoring on Fabric Nodes Restarting SNMP Monitoring on Fabric Nodes Adding a Third-Party SNMP V1 or V2c Manager on a Fabric Node Adding a Third-Party SNMP V3 Manager on a Fabric Node Deleting a Third-Party SNMP Manager from a Fabric Node Chapter 4 Monitoring your Network Monitoring Devices and Assets Viewing the Node List Resyncing Nodes Turning SNMP Data Collection Off and On Searching in the Network Monitoring Workspace Viewing the Dashboard iv

5 Table of Contents Tracking and Searching for Assets Working with Topology Using the Search Option to View Nodes Working with Topology Map Views Viewing Alarms and Node Details for Nodes Viewing Nodes with Active Alarms Managing Alarms Associated with Nodes Viewing the Topology Map with Different Layouts Automatic Refresh of Topology Map Pinging a Node Viewing the Alarms Associated with the Node Viewing the Events Associated with the Node Viewing the Resource Graphs Associated with the Node Working With Events, Alarms, and Notifications Viewing and Tracking Outages Viewing and Managing Events Events Landing Page Advanced Event Search Configuring an Events Filter Viewing the Events List Viewing Event Details Exporting Events Purging Events Viewing and Managing Alarms Viewing Alarms Acknowledging Alarms Clearing Alarms Escalating Alarms Unacknowledging Alarms Viewing Acknowledged Alarms Viewing, Configuring, and Searching for Notifications Notification Escalation Working With Reports and Charts Creating Reports Creating Key SNMP Customized Performance Reports, Node Reports, Domain Reports Creating a New KSC Report from an Existing Report Viewing Reports Viewing Resource Graphs Viewing Key SNMP Customized (KSC) Performance Reports, Node Reports, Domain Reports Viewing Database Reports Sending Database Reports Viewing Pre-run Database Reports Viewing Statistics Reports Generating a Statistics Report for Export v

6 Junos Space Network Management Monitoring and Troubleshooting Guide Deleting Reports Deleting Key SNMP Customized Reports Deleting Pre-run Database Reports Viewing Charts Managing Network Monitoring Operations Configuring SNMP Community Names by IP Configuring SNMP Data Collection per Interface Managing and Unmanaging Interfaces and Services Managing Thresholds Creating Thresholds Modifying Thresholds Deleting Thresholds Selecting and Sending an Event to the Network Management System Configuring Notifications Configuring Event Notifications Configure Destination Paths Configure Path Outages Configuring Scheduled Outages Compiling SNMP MIBs Uploading MIBs Compiling MIBs Viewing MIBs Deleting MIBs Clearing MIB Console Logs Generating Event Configuration Generating a Data Collection Configuration Managing Events Configuration Files Adding New Events Configuration Files Deleting Events Configuration Files Modifying Events Configuration Files Managing SNMP Collections Add a New SNMP Collection Modify an SNMP Collection Managing Data Collection Groups Adding New Data Collection Files Deleting Data Collection Files Modifying Data Collection Files Managing Devices Managing Surveillance Categories Modifying Surveillance Categories Deleting Surveillance Categories Adding Surveillance Categories Configuring Alarm Notifications Alarm Notification Configuration Overview Basic Filtering Guidelines for Configuring Alarm Notifications vi

7 Table of Contents Advanced Filtering Configuring Alarm Notification Configuring a Basic Filter for Alarm Notification Activating Alarm Notification Configuration Files for Basic Filtering Reloading a Filter Configuration to Apply Filter Configuration Changes Part 2 Troubleshooting Chapter 5 Pre-requisites for Troubleshooting Maintenance Mode Maintenance Mode Overview Maintenance Mode Access and System Locking Maintenance-Mode User Administration Restoring the Database in the Maintenance Mode Systems of Record Systems of Record in Junos Space Overview Systems of Record Implications System Snapshot Creating a System Snapshot Deleting a System Snapshot Restoring the System to a Snapshot Backup and Restore Backing Up and Restoring the Database Overview Backing Up a Database Restoring a Database Backing Up the Junos Space Network Management Platform Database Backing Up the Junos Space Network Management Platform Database to a Local Directory Backing Up the Junos Space Network Management Platform Database to a Remote Host Restoring the Junos Space Network Management Platform Database Through the Junos Space User Interface Restoring a Local Junos Space Network Management Platform Database Through the Junos Space User Interface Restoring the Junos Space Network Management Platform Database from a Remote File Through the Junos Space User Interface Restoring the Database from the CLI Chapter 6 Troubleshooting using Log Files System Status Log File Overview System Status Log File Customizing Status Log File Content Downloading System Log Files for a Junos Space Appliance Customizing Log Files to Download Junos Space Network Management Platform Log Files JBOSS Application Server Log Files MYSQL Database Log Files Apache Web Server Log Files vii

8 Junos Space Network Management Monitoring and Troubleshooting Guide Node Management Agent Log Files Troubleshooting Log File Overview Downloading the Troubleshooting Log File in the Server Mode Downloading the Troubleshooting Log File in the Maintenance Mode Downloading Troubleshooting System Log Files Through the CLI Downloading a System Log File by Using a USB Device Downloading System Log File by Using SCP Customizing Node System Status Log Checking Customizing Node Log Files To Download Chapter 7 Troubleshooting Login Related Issues Troubleshooting Not Able to Login from the Junos Space Login Page Issue Chapter 8 Troubleshooting Network Monitoring Related Issues Network Monitoring Troubleshooting Overview Backing up and Restoring the Network Monitoring Database Updating Network Monitoring After Upgrading the Junos Space Network Management Platform Overview Step 1: Monitoring the Software Install Status Window for File Conflicts Step 2: Identifying Files with Conflicts Step 3: Merging Files with Conflicts Step 4: Verifying the Manual Merge Status of Configuration Files Step 5: Final Steps After Upgrading Network Monitoring Troubleshooting Network Monitoring Page is Not Available Issue Chapter 9 Troubleshooting Device Management Related Issues Troubleshooting Device Discovery Failure Troubleshooting Device Data Collection Issue Troubleshooting Devices Discovered Twice using Device Discovery Issue Part 3 Index Index viii

9 List of Figures Part 1 Monitoring Chapter 1 Pre-requisites for Monitoring Figure 1: Fabric Nodes Figure 2: Fabric with One Node Figure 3: Fabric with Two Nodes Figure 4: Fabric with Three Nodes Chapter 3 Monitoring the Fabric Figure 5: Overall System Condition Gauge Figure 6: Fabric Load History Chart Figure 7: Active Users History Chart Figure 8: Network Monitoring Details for the Selected Fabric Node ix

10 Junos Space Network Management Monitoring and Troubleshooting Guide x

11 List of Tables About the Documentation xiii Table 1: Notice Icons xiv Table 2: Text and Syntax Conventions xiv Part 1 Monitoring Chapter 1 Pre-requisites for Monitoring Table 3: Fields for the Fabric Monitoring Inventory Page Chapter 3 Monitoring the Fabric Table 4: Logical Component Monitoring Chapter 4 Monitoring your Network Table 5: Alarms Table Table 6: Notifications Table Table 7: Node Status Table Table 8: Resource Graphs Table Table 9: Topology Map Options Table 10: Information Displayed in the Alarms List Part 2 Troubleshooting Chapter 5 Pre-requisites for Troubleshooting Table 11: Backup Schedule Units and Increments Chapter 6 Troubleshooting using Log Files Table 12: Log Files included in the troubleshoot File Table 13: Junos Space Network Management Platform Log Files Table 14: JBOSS Directories Table 15: JBOSS Log Files Table 16: MYSQL Database Log Files Table 17: Apache Web Server Log Files Table 18: NMA Log Files Table 19: List of Log Files in the Troubleshooting Log File Table 20: Data and Log Files in troubleshooting log File Chapter 8 Troubleshooting Network Monitoring Related Issues Table 21: Network Monitoring Log Files xi

12 Junos Space Network Management Monitoring and Troubleshooting Guide xii

13 About the Documentation Documentation and Release Notes Documentation and Release Notes on page xiii Documentation Conventions on page xiii Documentation Feedback on page xv Requesting Technical Support on page xv Documentation Conventions To obtain the most current version of all Juniper Networks technical documentation, see the product documentation page on the Juniper Networks website at If the information in the latest release notes differs from the information in the documentation, follow the product Release Notes. Juniper Networks Books publishes books by Juniper Networks engineers and subject matter experts. These books go beyond the technical documentation to explore the nuances of network architecture, deployment, and administration. The current list can be viewed at Table 1 on page xiv defines notice icons used in this guide. xiii

14 Junos Space Network Management Monitoring and Troubleshooting Guide Table 1: Notice Icons Icon Meaning Description Informational note Indicates important features or instructions. Caution Indicates a situation that might result in loss of data or hardware damage. Warning Alerts you to the risk of personal injury or death. Laser warning Alerts you to the risk of personal injury from a laser. Table 2: Text and Syntax Conventions Table 2 on page xiv defines the text and syntax conventions used in this guide. Convention Description Examples Bold text like this Represents text that you type. To enter configuration mode, type the configure command: user@host> configure Fixed-width text like this Italic text like this Represents output that appears on the terminal screen. Introduces or emphasizes important new terms. Identifies guide names. Identifies RFC and Internet draft titles. user@host> show chassis alarms No alarms currently active A policy term is a named structure that defines match conditions and actions. Junos OS CLI User Guide RFC 1997, BGP Communities Attribute Italic text like this Text like this Represents variables (options for which you substitute a value) in commands or configuration statements. Represents names of configuration statements, commands, files, and directories; configuration hierarchy levels; or labels on routing platform components. Configure the machine s domain name: [edit] root@# set system domain-name domain-name To configure a stub area, include the stub statement at the [edit protocols ospf area area-id] hierarchy level. The console port is labeled CONSOLE. < > (angle brackets) Encloses optional keywords or variables. stub <default-metric metric>; xiv

15 About the Documentation Table 2: Text and Syntax Conventions (continued) Convention Description Examples (pipe symbol) Indicates a choice between the mutually exclusive keywords or variables on either side of the symbol. The set of choices is often enclosed in parentheses for clarity. broadcast multicast (string1 string2 string3) # (pound sign) Indicates a comment specified on the same line as the configuration statement to which it applies. rsvp { # Required for dynamic MPLS only [ ] (square brackets) Encloses a variable for which you can substitute one or more values. community name members [ community-ids ] Indention and braces ( { } ) ; (semicolon) Identifies a level in the configuration hierarchy. Identifies a leaf statement at a configuration hierarchy level. [edit] routing-options { static { route default { nexthop address; retain; } } } GUI Conventions Bold text like this Represents graphical user interface (GUI) items you click or select. In the Logical Interfaces box, select All Interfaces. To cancel the configuration, click Cancel. > (bold right angle bracket) Separates levels in a hierarchy of menu selections. In the configuration editor hierarchy, select Protocols>Ospf. Documentation Feedback We encourage you to provide feedback, comments, and suggestions so that we can improve the documentation. You can send your comments to techpubs-comments@juniper.net, or fill out the documentation feedback form at If you are using , be sure to include the following information with your comments: Document or topic name URL or page number Software release version (if applicable) Requesting Technical Support Technical product support is available through the Juniper Networks Technical Assistance Center (JTAC). If you are a customer with an active J-Care or JNASC support contract, xv

16 Junos Space Network Management Monitoring and Troubleshooting Guide or are covered under warranty, and need post-sales technical support, you can access our tools and resources online or open a case with JTAC. JTAC policies For a complete understanding of our JTAC procedures and policies, review the JTAC User Guide located at Product warranties For product warranty information, visit JTAC hours of operation The JTAC centers have resources available 24 hours a day, 7 days a week, 365 days a year. Self-Help Online Tools and Resources For quick and easy problem resolution, Juniper Networks has designed an online self-service portal called the Customer Support Center (CSC) that provides you with the following features: Find CSC offerings: Search for known bugs: Find product documentation: Find solutions and answer questions using our Knowledge Base: Download the latest versions of software and review release notes: Search technical bulletins for relevant hardware and software notifications: Join and participate in the Juniper Networks Community Forum: Open a case online in the CSC Case Management tool: To verify service entitlement by product serial number, use our Serial Number Entitlement (SNE) Tool: Opening a Case with JTAC You can open a case with JTAC on the Web or by telephone. Use the Case Management tool in the CSC at Call JTAC ( toll-free in the USA, Canada, and Mexico). For international or direct-dial options in countries without toll-free numbers, see xvi

17 PART 1 Monitoring Pre-requisites for Monitoring on page 3 Monitoring the Software on page 33 Monitoring the Fabric on page 37 Monitoring your Network on page 47 1

18 Junos Space Network Management Monitoring and Troubleshooting Guide 2

19 CHAPTER 1 Pre-requisites for Monitoring Network Monitoring Network Monitoring on page 3 Fabric Management on page 10 Network Monitoring Workspace Overview on page 4 Network Monitoring Reports Overview on page 7 Admin: Configuring Network Monitoring on page 8 3

20 Junos Space Network Management Monitoring and Troubleshooting Guide Network Monitoring Workspace Overview The Network Monitoring workspace enables you to assess the performance of your network, not only at a point in time, but also over a period of time. This feature enables you to determine trending and diverse other things; for example, whether service-level agreements (SLAs) have been violated. NOTE: Junos Space Release 13.2 supports SNMP monitoring of devices using SNMP v1 and SNMPv2c. CAUTION: Although additional network monitoring functionality can be accessed by customizing its XML files, editing these files can affect the functionality of the Network Monitoring workspace. We recommend that you do not edit these XML files unless you are directed to do so by Juniper Networks. To grant a Junos Space user full privileges to access and perform tasks from the Network Monitoring workspace, the user must be assigned the FMPM Manager role. To grant a Junos Space user read-only access to the Network Monitoring workspace, the user must be assigned the FMPM Read Only User role. The Network Monitoring workspace supports the following three types of users: Administrator role: A user assigned the FMPM Manager role and with access to Global domain can view and administer all devices in the Network monitoring workspace, including all devices that exist in other sub-domains. Regular user role: A user assigned the FMPM Manager role but without access to global domain can only view and administer devices in their selected domain. This type of user can also acknowledge and clear alarms. Read only user role: A user assigned the FMPM Read Only User role (or a customized role with FMPM access capability except admin tab) in Junos Space. This type of user can only view devices in the selected domain, but cannot access the Network Monitoring > Admin workspace and cannot acknowledge or clear alarms. When a remote user (with the FMPM manager role) logs in from the Junos Space user interface, Junos Space authenticates the user from the remote authentication server as follows: If the remote authentication is successful, Junos Space uses the user s login credentials to authenticate with the network monitoring server and either creates or updates the network monitoring local user. If the remote authentication fails and the user previously existed on the network monitoring server, Junos Space removes the network monitoring local user. To analyze and aggregate device-level performance data, and to detect device faults, 4