WW INFO-02 Wonderware Historian Best Practices

WW INFO-02 Wonderware Historian Best Practices social.invensys.com Ray Norman @InvensysOpsMgmt / #SoftwareRevolution /InvensysVideos /Wonderware /company/wonderware North American Solutions Consultant 2013 Invensys. All Rights Reserved. The names, logos, and taglines identifying the products and services of Invensys are proprietary marks of Invensys or its subsidiaries. All third party trademarks and service marks are the proprietary marks of their respective owners.

Slide 2

Slide 3

Questions??? Slide 4

Best Practices - Caveat Use to Keep the Alligators at Bay Best Practices are Subject to Change Product Changes OS Changes SQL Server Changes Best Practices As We Know Them Today!! Slide 5

Agenda Historian Review Historian 2012 R2 Hardware and Virtualization Practices Upgrade and Historian Maintenance Practices Slide 6

Agenda Historian Review Historian 2012 R2 Hardware and Virtualization Practices Upgrade and Historian Maintenance Practices Slide 7

A Historian Is A storage repository for time-based information a Database But a Historian is much more than a database A Historian stores process data - lots of it A Historian lets you retrieve the process data sensibly A Historian Transforms the process data into Information A complete system to enable you to make the best use of this data Wonderware Historian does this, elegantly Store data at the resolution of your Process Retrieve at the resolution of the Problem you are trying to Solve -Mike Brost Slide 8

A Historian Is A Time Machine The future ain't what it used to be. Yogi Berra Slide 9

A Historian Is A Gold Mine! 1 Invensys 23 Invensys 0 October 2013 proprieta ry & Slide 10

Why Wonderware Historian? Low Customer Risk Installed Base over 70,000 licenses sold Optimal use of COTS - Microsoft SQL Server Fast Installation and Quick ROI Checkbox configuration from Application Server Tag importer for conventional InTouch applications CSV/SQL bulk Load options Automatically manages historical storage Circular, Alternate, Buffer, Permanent Intuitive Historian Client Tools Immediate value to the end user Complete Plant Performance Management with System Platform Slide 11

Historian 10.0 Historian 10.0 Wonderware introduced key new functionality Tiered Storage Capability Retrieval Enhancements Improved System Platform Namespace Integration The new multi-tiered architecture capability Enables smaller tier 1 Historians to feed to tier 2 for replication Enables tier 1 Historians to send aggregated or summary data to tier 2 Enables tier 1 Historian to send aggregated or summary data to itself Enables local data access for tier 1 data in distributed architectures Slide 12

Historian 10.0 Architecture SQL Server Retrieval IDAS/ SuiteLink History Blocks Storage Engine Pull Data Acquisition Storage Push Data Acquisition Replication Acquisition Application Server Historian New historian hidden within 10.0 Slide 13

Tiered Historian Simple Data Replication Replicate all data for selected or all tags Tier 2 Example: 1-second data Tier 1 Example: 1-second data Slide 14

Tiered Historian Simple replication & Summary Data Tier 2 Example: 5-minute, hourly, daily data Summary Tag Many aggregate values for each Tier 1 Example: 1-second data Slide 15 Local Tiered Summary Replication

Tiered Historian Architecture Business Domain Enterprise Tier 2 Historian Historian Client Historian Client Corporate Network Open Outbound Replication (single TCP port) SCADA Domain Local/Std Tier 1 Historian Historian Client I/O Slide 16 InTouch Control Network Application Server

Tiered Historian- TCP Port Tier2 Historian Configuration (Parameters) Tier1 Historian Configuration (Replication Servers) Slide 17

Tiered Historian Under the Covers Simple Replication Data Packet Payload: Real - 34 Bytes Integer 32 Bytes Discrete 31 Bytes Data Chunk Size (2-bytes), tagid (16-byte GUID), FILETIME (8 bytes), OPC Quality (2 bytes), QualityDetail (2 bytes), and value bytes (for example 4 bytes for 32-bit integer tags) Plus Zip Compression (~ 30%) Summary Replication Data Packet Payload: Analog - 96 Bytes/Sample Discrete State - 68 Bytes/Sample Analog State - 71 Bytes/Sample >50% Compression ~ 39 Bytes/Sample over time Slide 18

AnalogSummaryHistory Slide 19

StateSummaryHistory Slide 20

Analog and String StateSummaryHistory Slide 21

Tiered Historian Summary Historian Servers Two Main Roles in a System Operational Historian (Short Term Trending, Reporting, Statistics) Business Historian (Long Term Storage, Process Analysis, Advanced Reporting) Same Historian Instance for Both Roles Historian Placed in a DMZ Tiered Historians offer a Better Solution Local Operational Historian (25K Tags, 7 Days) $2.5K Enterprise Business Historian (Part of System Platform Bundle) No Need for a DMZ Single Outgoing TCP Port on SCADA Firewall Supports Domain Isolation Security Model (No Shared Credentials) Push of Configuration and Data from SCADA to Business LAN Slide 22

Agenda Historian Review Historian 2012 R2 Hardware and Virtualization Practices Upgrade and Historian Maintenance Practices Data Retrieval and Transformation Special Sauce Slide 23

The Most Amazing Wonderware Historian Ever Continued World class desktop tools Rich query capability Low management cost & effort All new integration with Application Server Significantly higher tag counts Redundant Historians SQL Server 64-bit support (2008 R2 and 2012) New Toolkit Slide 24

Historian 2012 R2 Architecture SQL Server Retrieval IDAS/ SuiteLink Most changes apply only here History Blocks Pull Data Acquisition Push Data Acquisition Application Server <3.6 Storage Engine Classic Storage Application Server >3.5 Push Data Acquisition Replication Acquisition Historian Slide 25

Historian 2012 R2 Communications SQL Server Retrieval SuiteLink (Single TCP Port) Storage Engine Pull Data Acquisition Application Server >3.5 Push Data Acquisition Storage Push Data Acquisition Application Server <3.6 COM/DCOM Named Pipes Slide 26 Replication Acquisition Historian WCF (Single TCP Port)

Engine Data Acquisition Throughput 150,000 100,000 10.0 2012 R2 50,000 0 Sustained Burst Values Per Second Slide 27 Late

Reliable Data Acquisition On failover, history gap from: Detecting failure Starting engine from checkpoint* Subscribing to I/O* Initializing history* N/A for 2012 R2 Historian Client Wonderware Historian Redundant Engines Application Server Redundant DI Objects Control System 140 120 100 80 60 40 20 0 System Platform 2012 System Platform 2012 R2 * Varies by number of objects Slide 28

High Availability Historian Client Multiple Clients Redundant Historians (2012 R2) Stratus VMware/Hyper-V Cluster Wonderware Historian Redundant Engines Application Server Redundant DI Objects Control System Slide 29 Reliable Access Reliable Collection

Configuring Redundant Historians MYHISTORIAN02 MYHISTORIAN01 Slide 30

Using CSV Files For Data Acquisition 500 CSV files, each for 1,000 tags 10.0 2012 R2 100% 0% Processing Time Slide 31 # Streams Retrieval Time

Business Historian as a R/O Real-Time DAServer Customers 2012 R2 Release InTouch Corporate Engineering RDS/WIS Server Application Server DAServer Advanced Alarming Enterprise Tier 2 Historian Historian Client Corporate Network Open Outbound Replication (single TCP port) DMZ Required Local Tier 1 Historian Slide 32 Historian Client

Historian 11 Under the Covers Slide 33

AI (Active Image) Tag Ownership Classic storage (AITag = 1) All system tags All IDAS tags Created by WAS 3.1 or SDK 1.0 MDAS2 (HCAL) Storage (AITag = 0) All tier-2 tags Created by WAS 2012 R2 or SDK 2012 R2 Manual tags created via Config Editor Slide 34

AITag Upgrade AITag = 1 AITag = 0 Automatic for WAS 2012 R2 / SDK 2012 R2 Manual for manually created tags IDAS and System tags cannot be upgraded Downgrade is not supported (but TS does anyway) Slide 35

Retrieving AITag Data AITag = 0 (after) AITag = 1 (before) transition point AIHistory = 1 instructs Retrieval to search for Classic Storage data Slide 36 time now AIHistory = 0 instructs Retrieval to search for New Storage data ONLY

Retrieving AITag Data (Real World) AITag = 0 (after) AITag = 1 (before) transition point AIHistory = 1 instructs Retrieval to search for Classic Storage data time now AIHistory = 1 May have to run SQL Update if new Historian AIHistory = 1 to retrieve old history Blocks!!! Slide 37

Channel Status Tag Historian Client (Trend) Behavior ChannelStatus =1 : Null s are injected into data stream on Disconnect Trend shows a Gap ChannelStatus =0 : Null s are NOT injected into data stream on Disconnect Trend shows no gap on channel disconnect Slide 38

Agenda Historian Review Historian 2012 R2 Hardware and Virtualization Practices Upgrade and Historian Maintenance Practices Slide 39

Useful Documents Historian Install PDF Historian Admin PDF Tech Note 850 Historian Hyper-V Guest Image and Time Synchronization Tech Note 817 Moving the Historian Runtime Database from One Machine to Another System Platform Virtual Implementation PDF The Role of Storage in HMI/SCADA Systems (Topic#: 002684) Virtualization and Storage Considerations (Topic#: 002686) Slide 40

Specifying Historian Host Hardware Level 1 Server - Hardware A Level 1 server can handle a load of about 5,000 tags. For example, 2,600 analogs, 2,200 discretes, 300 strings, and 20 non-i/o Server (manual) tags. OS: Win7/Server2k8R2 Note Client Connection Limit on Win7 CPU: Dual-core CPU RAM: 4 GB NIC: 100 Mbps network interface card (NIC) Slide 41

Specifying Historian Host Hardware Level 2 Server - Hardware Level 2 server can handle a load of about 63,000 tags. For example, 40,000 analogs, 20,000 discretes, 300 strings, and 5,000 non-i/o Server (manual) tags OS: Server2k8R2 CPU: Quad-core CPU RAM: 6 GB NIC: 1 Gbps network interface card (NIC) Slide 42

Specifying Historian Host Hardware Level 3 Server - Hardware A Level 3 server can handle a load of 130,000 tags. For example,70,000 analogs, 50,000 discretes, 6,000 strings, and 4000 non-i/o Server (manual) tags OS: Server2k8R2 CPU: Dual-Quad-core CPU (Xeon) RAM: 10 GB NIC: 1 Gbps network interface card (NIC) Slide 43

Specifying Historian Host Hardware Level 4 Server (Historian 11)- Hardware A Level 4 server can handle a load of 400,000 tags. For example:200,000 analogs, 150,000 discretes, 15,000 strings, and 35,000 non-i/o Server (manual) tags OS: Server2k8R2 CPU: Dual-Quad-core CPU (8 Core @ 2.6 GHz-Xeon) RAM: 16 GB NIC: 1 Gbps network interface card (NIC) Slide 44

Historian Disk Sizing and Data Storage Considerations How important is the data? Is anyone in the organization going to require operating data that is older than a month? Older than a year? How long can the system be off-line in the event of a component failure? What happens if the system stops storing data? What happens if stored data is lost as a result of a hard drive failure? Can the server equipment be taken off-line to perform repairs? Slide 45

Storage Hardware SCSI/SAS/SATA drives configured using hardware RAID is optimum. RAID5/RAID1+0 More Heads = Better! 10,000/15,000 RPM Design to use < 60% of available space Consider RAID1 SSD for Circular Storage RAID5/1+0 for Alternate Storage NTFS is the only officially supported file system for a production System Enable file compression for the historical data storage locations - Circular, Alternate, Buffer, and Permanent. Slide 46

Tiered Historian Under the Covers Simple Replication Data Packet Payload: Real - 34 Bytes Integer 32 Bytes Discrete 31 Bytes Data Chunk Size (2-bytes), tagid (16-byte GUID), FILETIME (8 bytes), OPC Quality (2 bytes), QualityDetail (2 bytes), and value bytes (for example 4 bytes for 32-bit integer tags) Plus Zip Compression (~ 30%) Summary Replication 84 Bytes/Sample >50% Compression ~ 34 Bytes/Sample over time Slide 47

Network and Storage Calculator Slide 48

Configure and Use Alternate Storage Slide 49

Historian Virtualization ESX/vSphere 5 Hyper-V Proper Host Hardware/Drive Selection Required Watch Checkpointing! History Block Changeover History Block Error on Restore Slide 50

Virtualization Example: Large System 32 cores over two R710 Rack Servers. 192 GB RAM Total 1.5 TB on Storage Capacity enough to host for example: 1 GR, 1HIST, 4 AOS s, 4 RDS Servers, 1 Info Server (11 Machines) Slide 51

VM Cores and Memory Considerations Cores and Memory Spare Resources The host server should always have spare resources of 25% above what the guest machines require. For example, if a configuration with five nodes requires 20GB of RAM and 10 CPUs, the host system should have 25GB of RAM and 13 CPUs. If this is not feasible, choose the alternative closest to the 25% figure, but round up so the host server has 32GB of RAM and 16 cores. Hyper-Threading Hyper-Threading Technology can be used to extend the amount of cores, but it does impact performance. An 8-core CPU will perform better than a 4core CPU that is Hyper-Threading. Slide 52

VM Storage Recommendations -1 Plan for proper Storage. A best practice is to dedicate a local drive or virtual drive on a Logical Unit Number (LUN) to each of the VMs being hosted. We recommend SATA or higher interfaces. The host OS also should have a dedicated storage drive. A basic storage topology would include: Host storage VM storage for each VM A general disk large enough to hold snapshots, backups, and other content. It should not be used by the host or by a VM. Slide 53

Storage Recommendations -2 Recommended Storage Speed Boot times and VM performance are impacted both by storage bandwidth and storage speed. Faster is always better. Drives rated at 7200 rpm perform better than those rated at 5400 rpm. Solid-state drives (SSDs) perform better than 7200-rpm drives. Keep in mind that multiple VMs attempting to boot from one hard drive will be slow, and your performance will significantly degrade. degrade Attempting to save on storage could well become more costly in the end. Slide 54

Network Considerations Networking is as important as any other component for the overall performance of the system. Recommended Networking for Virtualization If virtualization is your only requirement, your network topology could include the following elements: Plant network Storage network Virtualization network. A best practice is to establish, on every node, an internalonly Static Virtual Network. In the event that the host and the guest VMs become disconnected from the outside world, you will still be able to communicate through an RDP session independent of external network connectivity. Slide 55

Virtual Host Recommendations - Review Do NOT use Shared Resources Pre-Allocate dedicated Memory for every VM Pre-allocated Cores for every VM Separate Drive or R/W heads for every VM SAS, FibreChannel SAN Flash Memory for VMHost (ESX/vSphere/Win2K8R2) Use Solid State Drives for Circular Storage HP Servers 1GB Flash Backed Write Cache is HIGHLY recommended Slide 56

It s About Time (Stamps) Timestamps are propagated from IO/DA Source Can be several hops removed from Historian Pick One!! Domain Time Synch (Windows Time Service) W32TM Net Time NTP (Network Time Protocol) 3rd Party VM Technology can affect Server Time Drift vsphere5 is currently better than Hyper-V (2K8R2) Slide 57

Agenda Historian Review Historian 2012 R2 Hardware and Virtualization Practices Upgrade and Historian Maintenance Practices Slide 58

Upgrading Your Historian In-Place Upgrade Backup the RT Database!!! Run Historian 10.0 SP1/Historian 11 Install New Server/SQL Server Upgrade Backup the RT Database!!! Restore Runtime DB to New Server Run Historian 10.0 SP1/Historian 11 Install Copy History Blocks to New Server Set AIHistory=1 Configure Parameters as required Slide 59

Upgrading In-Place Upgrade Application Server <3.6 Upgrade Historian First 1. During upgrade, Engine goes into store-forward 2. After upgrade is complete, Engine forwards data and resumes 3. Engine continues using Classic Storage until it is upgraded 4. After Engine upgrade, using new Storage Historian 10.0 Upgrade Platform First 1. After upgrade is complete, Engine immediately goes into store-forward 2. Remains in store-forward until Historian is upgraded 3. After Historian upgrade, using new Storage Not Recommended No data loss in either scenario ** Slide 60

Runtime DB Maintenance The Runtime DB size is not affected by the amount of history data. General settings let you purge event and summary data. You need to manually purge the change log (ModLogTracking). Avoid creating custom tables in the Runtime DB. Slide 61

Active Image AITag = 1 Memory Buffer of Real-time Data Default is 65 samples Automatic Resize (Default) One Minute of data = xx Samples Query spanning Active Image Window Returns Active Image samples first Manually setting AI Sample Size Can be GOOD Blazing Retrieval (From Memory) Can be BAD! String tag -1038 Bytes/Sample Slide 62

What s the most important online measurement at your site? How do you monitor it? Webaccessible Status Pager Alarms Real-time Displays Email Trends Slide 63

How important is the Historian to your site? How should you monitor it? Webaccessible Status Pager Alarms Real-time Displays Email Trends Slide 64

Historian System Tags System Resource Monitoring Memory Processor Disk Historian Server Monitoring Subsystems: Event, Storage, OLE DB, Indexing, etc. Items: Status, resource usage, etc. Data Acquisition Monitoring by Source Item throughput Bad data quality Outside real-time Testing Tags (one per type, date/time) Slide 65

Historian I/O Server: aahiosvrsvc Slide 66

Using System & Platform Tags InTouch/System Platform Historian Alarms Event Tags Status indicators Email Action Slide 67

Historian Event Tag Slide 68

Faster Diagnostics With Trend Slide 69

Memory Management for Storage aahindexsvc Manages Tag and History Block Information Can be Memory Intensive Use Perfmon to observe HistoryCacheSize and HistoryDaysAlwaysCached Parameters SysHistoryCacheFaults and SysHistoryCacheUsed system tags Slide 70

Questions??? Slide 71

The most amazing Wonderware Historian Thank You! EVER! Ray.norman@invensys.com Slide 72