ESPDS Scalable and Secure Infrastructure Ensuring the NOAA/NESDIS Environmental Satellite Processing and Distribution Capabilities Meet the Growing User and Data Demands of Today and Tomorrow Rich Baker Solers, Inc. ESPDS Development Chief Architect 2013 AMS Annual Meeting
What is ESPDS? ESPDS: Environmental Satellite Processing and Distribution System Developed by the NESDIS Office of Systems Development (OSD), with Solers ( Team Solers ) as the development contractor Will be operated by the NESDIS Office of Satellite and Product Operations (OSPO) Modernizes the NESDIS Environmental Satellite Processing Center (ESPC) Single enterprise solution that meets the needs of existing (legacy), Suomi NPP, JPSS, and GOES-R, with scalability to meet future environmental satellite needs No more stovepipes! Includes modernization of the Ingest, Product Generation (PG), Product Distribution (PD), and Infrastructure segments of the ESPC Provides environmental satellite data and services to a growing user community including: NOAA Line Offices (NWS, NMFS, NOS, NIC, NESDIS, etc.) DoD (AFWA, NAVO, etc.) Other U.S. and international users (government agencies, universities, foreign partners, etc.) Will be implemented at the primary and backup ESPC sites: Primary ESPC site is the NOAA Satellite Operations Facility (NSOF) in Suitland, MD Future ESPC backup site is the Consolidated Back-Up (CBU) facility in Fairmont, WV Provides a scalable and secure infrastructure as a foundational building block upon which all other system functions reside 2
Traits of a Scalable Infrastructure No Single Point of Failure Redundancy and fault tolerance as key design tenants throughout Line Replaceable Units Can upgrade or replace existing hardware and software components without impacting operational availability Business Process Flexibility and Extensibility Can change existing business processes within the system, and integrate new business processes into the system, without impacting operational availability Horizontal Scalability Can add additional hardware resources (computing, network, storage) and software business processing instances without impacting operational availability 3
Traits of a Secure Infrastructure Complies with applicable IT security policies, procedures, and controls: NIST SP 800-53 DOC/NOAA IT Security Handbook Center for Internet Security (CIS) Benchmarks DISA STIG Etc. Provides a defense-in-depth foundation for securing the system that includes: Network security Centralized identity/account management, authentication, and authorization Host-based intrusion detection and prevention Anti-malware Integrated monitoring, logging, and reporting (Security Incident and Event Management [SIEM]) 4
ESPDS Scalable and Secure Infrastructure Suomi NPP and JPSS (via IDPS) GOES-R GS PD Non-NOAA Satellites (MSG, MTSAT, INSAT) Ancillary Data Providers Legacy GOES Legacy POES Future Missions Satellite Ingest Computing Cluster Scalable x86 hardware cluster with specialized adapters to interface with satellite antenna systems and perform RF/IF to IP conversion of the data Resource Management Communications Framework Logging & Reporting Monitoring Product Generation Computing Cluster Scalable x86 hardware cluster that leverages a grid computing scheduler to perform PG algorithm execution and report applicable status/metrics Common Infrastructure Services Identity/Account Management HIDS Anti-Malware/HIPS Network Management Converged 10Gb IP Networking Virtualized Computing Cluster Scalable x86 hardware cluster that hosts the distribution and access, PG management, common infrastructure, and other services as Virtual Machines (VMs) Database Data Intake & Transmission Scheduling System Backup Enterprise Shared Storage Scale-Out Enterprise Network Network Attached Attached Storage Storage (NAS) (NAS) solution solution with with standard standard IP-based IP-based file file access access protocols protocols (NFS, (NFS, CIFS, CIFS, HTTP, HTTP, FTP) FTP) (EMC Isilon) Includes switches, firewalls, and Network IDS components (Cisco) NOAA Line Offices DoD CLASS Other U.S. and International Users Ancillary Data Users (PG Systems) 5
Common Infrastructure Services The following slides provide an overview of the Common Infrastructure Services depicted in the previous diagram Resource Management Communications Framework Logging & Reporting Monitoring Identity/Account Management HIDS Anti-Malware/HIPS Network Management Database Data Intake & Transmission Scheduling System Backup 6
Resource Management VMware vsphere/esxi, vcenter, and Orchestrator 7
Communications Framework User /Operator/ Admin HTTPS (S)FTP(S) Load Balancer (S)FTP(S) Client (S)FTP(S) VM (S)FTP(S) Server GOES-R GS PD WS Client Other System WS Client ESPDS SOAP over HTTP HTTPS Portal Portal (S)FTP(S) (S)FTP(S) (S)FTP(S) Server Server (S)FTP(S) Other Other (S)FTP(S) PDA Client Client Service ESPDS Service Load Balancer SOAP over HTTPS VM SOAP over HTTP(S) ESB ESB SOAP over JMS 1.1 JMS Broker VM SOAP over JMS 1.1 JMS Broker VM SOAP over JMS 1.1 Application Server ESPDS Service VM Application Server ESPDS Service VM Application Server ESPDS Service VM WSO2 ESB and Application Server Apache ActiveMQ Java Message Service (JMS) Broker Red Hat Linux Virtual Server (LVS) Load Balancer 8
Logging & Reporting Windows Event logs (WMI) Microsoft Windows Layer 3 Switch (Cisco) Firmware logs (Syslog) Monitoring (SolarWinds Orion) Directory Server (Microsoft Active Directory) vcenter, ESXi, Resource Coordinator, Red Hat Repository logs (WMI) Resource Management (VMware vcenter) Computing HW (Cisco UCS) Firmware logs (Syslog) SolarWinds Orion logs (WMI) Windows Event logs (WMI) Apache SSHD/FTPD logs (Rsyslog) Data Intake/Data Transmit ([S]FTP[S] Server/Client) NAS Storage (Isilon) SAN Storage (EMC VNX) Firmware logs (Syslog) Firmware logs (Syslog) Linux OS logs (Rsyslog) Red Hat Enterprise Linux Administrator Portal Logging & Reporting (Tripwire Log Center) Web server logs (Rsyslog) User Portal Custom Java Service logs (Rsyslog) Other Java Components (e.g. Subscription, Product Tailoring, Ad- Hoc search) WSO2 ESB, WSO2 AS, Red Hat LVS logs (Rsyslog) Oracle logs (Rsyslog) Communications Framework (WSO2, ActiveMQ, Red Hat LVS) Database (Oracle RDBMS) Tripwire Log Center Rsyslog (Linux-based syslog client) Windows Management Interface (WMI) 9
Monitoring Microsoft Windows resource and service status (WMI) Microsoft Windows Layer 3 Switch (Cisco) Computing HW (Cisco UCS) SAN Storage (EMC VNX) NAS Storage (EMC Isilon) Interface Status and Bandwidth Usage (SNMP) Blade Resource Utilization (SNMP) I/O Data and Storage Usage (SNMP) I/O Data and Storage Usage (SSH) Red Hat OS resource and service status (SNMP) Red Hat Enterprise Linux Administrator Portal Logging and Reporting (Tripwire Log Center) Service/Process Status and Resource Allocation (SSH/RMI) Directory Server (Microsoft Active Directory) Monitoring (SolarWinds Orion) User Statistics (SSH) User Portal Web Interface Authentication (LDAPS) Service/Process Status and Resource Allocation (SSH/RMI) VM CPU, Memory, and Network performance measurements (SNMP) Other Java Components (e.g., Subscription, Product Tailoring, Ad- Hoc search) Connection Status and Transfer Rate (SNMP) Service/Process Status and Resource Allocation (SSH/RMI) Service/Process Status and Resource Allocation (SSH/RMI) Resource Management (VMware vcenter) Data Intake/Data Transmit ([S]FTP[S] Server/Client) Comm Framework (WSO2, Red Hat LVS) Data Management (Oracle RDBMS) SolarWinds Orion Network Performance Monitor (NPM) and Application Performance Monitor (APM) Red Hat Simple Network Management Protocol (SNMP) Agent and Secure Shell (SSH) Server Windows Management Interface (WMI) 10
Identity/Account Management Centralized identity and account management solution Manages human user accounts (internal and external users, operators, administrators) Manages machine and operating system accounts Provides Kerberos and web services-based authentication and authorization services Compatible with NOAA/NESDIS HSPD-12 solution (DoD CAC PIV token, X509 PKI certificates) Microsoft Active Directory Centrify ForgeRock OpenAM 11
HIDS Centralized Host-based Intrusion Detection System (HIDS) solution Ensures integrity of critical system and configuration files across the infrastructure, including: Computing device firmware Networking device firmware Storage device firmware Operating systems Applications and services Tripwire Enterprise 12
Anti-Malware/HIPS Provides virus scanning and Host-based Intrusion Prevention System (HIPS) capabilities across all machines and operating systems Centralized virus signature and HIPS policy management (automated deployments and updates) McAfee VirusScan Enterprise, HIPS, and epolicy Orchestrator 13
Network Management Domain Name Service (DNS) Server Dynamic Host Configuration Protocol (DHCP) Server Network Time Protocol (NTP) Server Microsoft Windows DNS and Time Services (integrated with Active Directory) Red Hat DCHP Server Red Hat NTP Server 14
Database Highly Available Relational Database Solution Two Oracle Database 11gR2 Enterprise Edition Database Server instances One primary instance providing client access One identical standby instance to receive/apply redo operations from primary database Oracle Data Guard configuration established between primary & standby database servers to maintain duplicate copy of operational database Supports high database availability and fast start failover Oracle Database 11gR2 Enterprise Edition with Data Guard Hibernate (database client access) 15
Data Intake & Transmission FTP, FTPS, and SFTP client and server solutions Used to obtain product and ancillary data from providers (intake), and deliver product and ancillary data to consumers (transmission) via push or pull Apache FtpServer (FTP and FTPS Server) Apache SSHD (SFTP Server) Apache Commons Library (FTP, FTPS, and SFTP Client) 16
Scheduling Schedules periodic operations to be performed within the infrastructure Product and ancillary data inventory cleanup (expired files) Subscription-specific product and ancillary data acquisition Extensible to accommodate future scheduling needs Terracotta Quartz Scheduler 17
System Backup Performs periodic backup of specific system data and files to support on-site archive and recovery Backups include: VM image files Database contents Log files Configuration files EMC NetWorker 18
ESPDS Scalable and Secure Infrastructure Benefits To End Users Ensures highly available and reliable access to human and machine interfaces that scales to accommodate the growing user and data demands Provides flexibility to quickly adapt to changes in end user requirements To System Operators/Administrators Easily scalable hardware and software Provides automated operations Compliant with IT security requirements for a High Impact system To NOAA/NESDIS As A Whole Scalable and secure foundation to support enterprise environmental satellite services across NOAA/NESDIS Removes mission-specific stovepiping Paving the path toward modernized data centers 19
Questions 20