TASK. Windows Edition version Edition Tidal Analysis Software Kit

Transcription

1 TASK Windows Edition version Edition Tidal Analysis Software Kit Developed by the Marine Data Products team at the UK National Oceanography Centre. Please read this user guide thoroughly before using the TASK software.

2 Table of Contents TABLE OF CONTENTS... 1 Warranty and Liability... 4 Software Use and Licensing... 5 THE NATIONAL OCEANOGRAPHY CENTRE... 6 TIDES, ANALYSIS AND PREDICTION... 7 Tides... 7 Observing Sea Level... 7 Mean Sea Level... 7 Surge Level... 8 Tidal Level... 8 From Tide Gauge to Tide Table... 9 Tidal Harmonics Constituents Tidal Harmonic Analysis TASK WINDOWS EDITION Introduction Before Using the Software Workflow Data File Formats NOC CSV Format University of Hawaii Sea Level Center Format TASK 2000 File Format Advisory Notes, Warnings and Errors INSTALLING THE SOFTWARE Minimum Requirements Installation Where Files Are Installed Commercial Licences STEP BY STEP HARMONIC ANALYSIS General comments before you start STEP 1: Start the Main TASK Application STEP 2: Using TASK Toolkit to create the TASK 2000 file Setting the options for converting the data to TASK 2000 format STEP 3: Looking at the Data with the TASK Graph Module Assessing Data Quality and Flagging Bad Data Changing the Y Axis Scaling STEP 4: Harmonic Analysis Performing a Harmonic Analysis Selecting the number of harmonics Page 1

3 Analysing Short Time Periods of Data (less than 6 months) Looking Beyond the TASK Predictions Limitations of the TASK Analyse program Possible Warning / Error Messages STEP 5: Looking at the Analysis Output The Primary Results (PR) File The Time Series (TS) Output File STEP 6: Graphing the Data with the TASK Graph Module STEP 7: Deriving a prediction Setting up TASK Analyse and POLTIPS 3 Integration Using TASK Analyse and POLTIPS 3 Integration REFERENCE SECTION Errors and Warnings TASK Toolkit TASK Analyse Additional Automatic Checking of Results Daily and Monthly Means Submission of monthly means to PSMSL Handling Time Zones Daylight Saving Time Handling Datums Data Management and Quality Control Issues Data Management Quality Control The Data Manipulation Tool What is TASK DMT used for? Specifying the file Data Value Adjustment Time Adjustment Flag Bad Data Splitting Files Channel swapping and comparison Using Other Software to Manipulate Data Analysis of Tidal Currents ADVANCED TIDAL ANALYSIS TOPICS How Daily Means Are Computed Data Epochs and Historical Calendar System Sa Phase Calculations in Different Software Creating your own Harmonic Constant Sets Inferred Harmonics Revisited S 2 and K Related / Inferred Harmonics Example Page 2

4 Table of Equilibrium Tide Relationships Adding Inferred Harmonics to the HCSelect.txt file Amplitude Adjustments APPENDICES Appendix A Wave Terminology Appendix B Glossary of Tidal Terms Appendix C1 A Table of Standard Harmonics Appendix C2 Table of Standard Harmonics with Speed Appendix C3 Day Numbers Appendix D Additional Reading Material Book List Appendix E Using the Microsoft Excel Graph Template Enabling Macros General Troubleshooting Appendix F Updating Your Dongle Extending your subscription / updating the dongle Page 3

5 Warranty and Liability 1. If the licence disk or CD ROM is defective NOC will replace it at no charge provided the defective item is returned within thirty (30) days from the date of despatch from the National Oceanography Centre (NOC). 2. NOC will refund the purchasing price provided all items (CD ROM, licence disk and user manual) is returned within thirty (30) days from the date of despatch from NOC and provided that all seals remain intact. No refund will be made if the seal has been broken on any items. 3. NOC will endeavour to contact all registered customers (by ) informing them that a new update is available for the software. This will be made available on CD or for download from the NOC FTP site. 4. NOC will provide basic technical support for all TASK Windows Edition users. This will be limited to installation problems only. 5. Technical support on issues relating to harmonic analysis, data processing and data quality control as well as support on using the various software modules will only be provided to users who have an appropriate support contract with NOC. 6. We do not guarantee that the functions contained in TASK Windows Edition will meet your requirements or that the operation of the program will be either error free or appear precisely as described in any documentation describing TASK Windows Edition. 7. No responsibility can be accepted by the UK Natural Environment Research Council (NERC) for any consequential loss or damage arising from any use that is made of TASK Windows Edition and output from the software howsoever caused. 8. All implied warranties, terms, representations and conditions, including (by way of example only) the implied conditions of satisfactory quality and fitness for any purpose of the information, are excluded to the fullest extent permitted by law. 9. This Agreement is governed by English law and the parties submit to the exclusive jurisdiction of the English courts. It is recommended that you only use this software if you have a good understanding of tidal harmonic analysis. If you are unsure of the processes involved in harmonic analysis or how to interpret tidal time series, you should contact the Tidal Analysis and Prediction Service at the UK National Oceanography Centre for advice. Although the National Oceanography Centre has done its best to make TASK Windows Edition as user friendly as possible, it does not take away the requirement for the user being familiar with the process. It is a tool to help you in much the same way a hammer and saw is to a carpenter. Page 4

6 Software Use and Licensing The TASK Windows Edition software is available under a range of licenses which permit the software to be used in different ways. In some cases, the different licenses will have restrictions imposed as to how the software can and can t be used, and this may be enforced by one or more of the following: Different versions of the software containing hard wired restrictions / limitation, or having features disabled. The requirement to have a USB security dongle (supplied by NOC) present on the computer before the software can be run. The signing of a licence agreement which must be returned to the National Oceanography Centre before the software is made available. The different licences come under the following broad types: Commercial use (annual subscription) Academic use (for commissioned work) Academic use (for government funded blue skies research) Academic use (for teaching within a marine science or related programme) GLOSS / IOC / PSMSL related activities Users are generally required to sign a licence and return this to the Marine Data Products Team at the National Oceanography Centre at the address given in the box. Use of the software within university departments for the purposes of teaching requires a separate licence to be signed ( Use of TASK Windows Edition software for teaching purposes ). Marine Data Products National Oceanography Centre Joseph Proudman Building 6 Brownlow Street Liverpool L3 5DA United Kingdom. [email protected] Fax: +44 (0) If you have any questions relating to the software licensing or terms and conditions of use, please contact the Marine Data Products Team who will be more than happy to answer any questions. An evaluation (demo) version of the software is available for evaluation purposes. This is restricted to generating a maximum of 2 harmonics (usually M2 and S2) from the data. Page 5

7 The National Oceanography Centre TASK Windows Edition The National Oceanography Centre (NOC) is a component laboratory of the UK s Natural Environment Research Council (NERC). The NOC has provided a national and international tidal analysis and prediction service since 1924, in fact the modern methods of tidal analysis and prediction used throughout the world are based on the work carried out by Dr. A.T. Doodson F.R.S. between 1919 and 1960 while he was working at the Liverpool Tidal Institute (the forerunner to the National Oceanography Centre). The NOC has continued to develop and refine the methods used to analyse tide gauge data and compute tidal levels and currents. Coupled with advances in modern computer hardware and software, we now have the most accurate tidal analysis and prediction software available. The NOC has offices in Liverpool (left) and Southampton (right), and combined they house a staff of around 500 scientists, engineers, IT and administrative support staff. It also houses various national and international facilities such as the British Oceanographic Data Centre (BODC), the Permanent Service for Mean Sea Level (PSMSL), the National Tidal and Sea Level Facility (NTSLF) and the British Ocean Sediment Core Research Facility (BOSCORF). The Liverpool Site formerly known as the Proudman Oceanographic Laboratory The Southampton Site Page 6

8 Tides, Analysis and Prediction Tides Anyone who has spent more than a few days at the coast will be familiar with the regular rise and fall of the tide. If you were to measure the tidal level every hour for a whole year, you would notice some regular patterns in the rise and fall. For the majority of locations around the UK you would notice: the time between one high water and the next is, on average, around 12 hours 25 minutes thereby giving two high waters and two low waters on most days, with the time of high water moving on about 50 minutes each day. often, the high waters alternate between a slightly higher high water and a slightly lower one. every 14 days, the tides appear to be slightly higher than average. Observing Sea Level There are a large number of natural phenomena that cause the sea level to change. These include: Tides Surges Long term changes in the mean level Waves Seiches Tsunamis For most purposes, you can consider the level to be made up of the first three. Observed sea level = tidal level + surge level + mean sea level. Let s briefly look at each of these influences on the observed sea level. Mean Sea Level This is the average level of the tide calculated over a long period of time (usually a minimum of one year). This has generally increased globally by about 15cm over the past century due to the melting of ground ice (ice sheets and glaciers) and to the thermal expansion of the warming seas. There are some regional variations where the sea level appears to be falling; however, this is caused by the land rising faster than the sea. Page 7

9 Surge Level The surge level is the name given to the change in sea level caused by meteorological effects such as variation in the atmospheric pressure, prolonged onshore or offshore winds etc. As such, surge levels are very hard to predict accurately and require complex computer simulations. The UK National Oceanography Centre is a world leader in surge modelling, however even using some of the most powerful computers in the country, predicting a surge level more than a few days ahead is a futile exercise as the errors start to become significant and the accuracy drops off rapidly. Tidal Level The tidal level is the change in sea level caused by the ocean s response to the gravitational attraction of the Moon and the Sun. The tides are the predominant signal in a sea level record and are easy to distinguish from other components of sea level variation as they have such well defined periods. The tidal level is affected by numerous factors such as: the relative position of the three bodies in the Earth Moon Sun system. the variation in the Moon s and Sun s distance from Earth. the variation in the position (declination) of the Moon and the Sun. the local land masses and the shape of the coast. the depth of water at the location and surrounding areas. As you can see, the phenomena that affect tidal level are highly predictable (we can compute the position of the moon to an incredible accuracy), or change very slowly over time (coastlines rarely change overnight). Therefore it makes sense that tidal levels should also be predictable with a high level of accuracy and for dates well into the future. This is indeed the case. In this user guide, tidal level will often just be referred to as tide, and surge level as surge. It was discovered as far back as Isaac Newton ( ) that the individual influences on the tide can be modelled by summing up combinations of sine waves a mathematical function very closely related to circular motion (or in our case, motion of water on a rotating Earth), however the modern harmonic method of analysing and computing the tide was developed by Arthur Thomas Doodson and published in 1921 when he was secretary of the Liverpool Tidal Institute (one of the many previous names for what is now the Liverpool site of the National Oceanography Centre). Page 8

10 From Tide Gauge to Tide Table The complete process of deriving a tide table showing the tides at any date and time into the future starts with having a good quality record of the actual tidal levels observed over as long a period as possible (ideally 18.6 years). Tide Gauge The tide gauge records the vertical movements of the sea level at a regular interval (e.g. the height every 15 minutes). Quality control process The data is carefully quality controlled to ensure any incorrect data is flagged. Data reformatting & harmonic analysis Derive a set of harmonic constants (a unique set of amplitudes and phases for each of the constituents). Software: TASK Windows Edition Harmonic tidal calculation Use harmonics to derive the tide at any point in time. Software: POLTIPS 3 Tide tables / tidal graphs etc. Reproduce the data in a user friendly format. The process starts with the installation of a tide gauge at the coast to record the variation in the sea level. To a casual observer, surface waves are the most obvious variation as they have a period measured in tens of seconds. The oscillations due to the tide have much longer periods (ranging from 30 minutes to over a day with a few seasonal oscillations thrown in for good measure). These are sometimes not easy to observe due to the high frequency and amplitude of surface waves, therefore one of the principle jobs of a tide gauge is to remove this high frequency information (by applying a lowpass filter to the tidal signal). This can be done either physically or digitally on the tidal data. Traditionally a stilling well would have been used with an oldfashioned float gauge (in fact stilling wells are still sometimes used with modern acoustic and radar gauges). However, the Page 9

11 UK s National Tide Gauge Network developed and maintained by the National Oceanography Centre uses pressure transducers to record the sea level. Although the pressure transducer can be installed in a stilling well, the normal method of low pass filtering is carried out in the digital domain by sampling the tide at a 1 second interval and then averaging the data to 15 minute values. Other types of tide gauge that have become very popular are the acoustic gauge and the radar gauge, primarily due to their relatively low cost. They are also quite easy to install and maintain, and usually come supplied with all the relevant data processing software. The output from these gauges will be compatible with the majority of data loggers. Once installed, they simply bounce a radar signal off the water s surface to measure the height. For detailed information on the A radar gauge installed near Liverpool Docks, UK relative merits of each type of tide gauge, please refer to the Intergovernmental Oceanographic Commission report Manual on Sea Level Measurement and Interpretation volume 4 which is available for download from the Permanent Service for Mean Sea Level. ( Once the sea level record has been recorded by the tide gauge it needs to undergo some quality control. For example the clock in the data logger may drift (or even stop completely) or the recording Page 10

12 device by not be reporting the correct tidal height. The example below shows a sudden jump in the data for about 14 hours. This turned out to be a boat which someone had moored directly underneath a radar gauge the radar, instead of bouncing off the surface of the water, bounced off the deck of the boat which was about 1.7m higher. Data in the tide gauge record which is identified as wrong should either be fixed (if this is possible) or flagged as bad data so that it does not get used in the harmonic analysis. Following the quality control stage, the data can be harmonically analysed using a package such as TASK Windows Edition. This is used to generate a set of tidal harmonic constants unique to the location of the tide gauge. Tidal Harmonics Constituents A tidal harmonic constituent is an individual sine wave that describes, either fully or in part, one of the numerous cycles that make up the overall variation in the tides. (See Appendix A for an explanation of the various terms related to waves.) To accurately predict the tides for a location you need to take into account all the different phenomena that influence the tide, and therefore many locations will make use of over 100 individual harmonic constituents to derive the resultant tide. These individual tidal cycles all have quite distinct frequencies related to the astronomical phenomena that influence the tides. For example: the rotation of the Earth relative to the Sun is /hour the rotation of the Earth relative to the Moon is /hour the variation in lunar distance is /hour (a day cycle) Each of the distinct frequencies is related directly to a tidal harmonic constituent, or in some cases a pair of harmonic constituents which are close in frequency. These pairs of harmonics move in and out of phase to create a much slower oscillation as shown in the diagram below. Page 11

13 The result of adding two sine waves whose frequency differs by only 5% and where one has an amplitude of 2m and the other of 1m. When in phase they reinforce each other to give an amplitude of 3m. When out of phase, they partially cancel each other out to give an amplitude of 1m. In the three examples in the bullet list on the previous page: the rotation of the Earth relative to the Sun is signified by a harmonic constituent called S 1 the rotation of the Earth relative to the Moon is signified by a harmonic constituent called M 1 the variation in lunar distance is constructed from a pair of harmonics, Q 1 combined with M 1. Many harmonics also have precise multiples which in many cases are far more significant in the mathematical reconstruction of the tide. For example, the harmonics M 2 and S 2 have frequencies exactly twice that of M 1 and S 1, but due to the semi diurnal (twice daily) nature of the tide, are far more significant (i.e. have a much greater amplitude than their diurnal counterparts). In the simplest terms, this is due to successive high and low tides generated by the Earth rotating beneath two bulges of water (see diagram below) and therefore generating approximately two tides per day. So although M 1 defines the speed of a whole rotation of the Earth relative to the moon (360 in 24 hours and 50 minutes = 1 lunar day), that single rotation will show two complete oscillations of the lunar tide, and therefore M 2 (with a speed of 360 in 12 hours 25 minutes, or per hour) is far more dominant than M 1. Tidal Harmonic Analysis Analysis involves reducing a set of sea level recordings from a tide gauge (usually at least 8760 hourly values covering one whole year) into a more manageable set of parameters which completely defines the tidal component of the tide gauge record. Since the tide is made up of a finite number of oscillations (sine waves) of fixed known frequencies, we can take this tide gauge data and compute the amplitudes and phases for each of the harmonic constituents using standard mathematical techniques. The amplitude is often given the mathematical symbol H, and the phase lag G. As the tidal influences change very slowly and the values are derived to a very high precision, these amplitude/phase pairs are often called harmonic constants. For a location with a long tidal record, we Page 12

14 will usually derive up to 115 harmonic constants for each location (a complete list is given in the Appendix of this document). The diagram above shows the energy density for a typical tidal record. You can see that there are some quite distinct spikes and these correspond to the frequencies of the known harmonics. The table below shows the 10 most significant harmonic constants for the Port of Liverpool. Constituent Speed ( /hr) Amplitude (H) (units: metres) Phase lag (G) (units: degrees) M S N K μ 2 (mu 2 ) M MS L ν 2 (nu 2 ) K O Once the set of harmonic constants is derived, they can be used with a tidal prediction program such as POLTIPS 3 to compute the tide forwards or backwards in time for many years at a time if required. It is the high precision with which astronomical parameters (such as the speeds in the table above) are known that allows the tides to be computed so far in advance. Page 13

15 Tidal prediction is basically the process of taking the harmonic constants derived from the analysis process and summing them together for the appropriate time. For the mathematically inclined, the equation is: cos For more information on the mathematics involved, please refer to one of the books in Appendix D (Additional Reading Material). Page 14

16 TASK Windows Edition Introduction TASK Windows Edition is a suite of programs for Microsoft Windows based PCs that will enable you to take the data from a tide gauge, carry out the necessary data processing, compute a set of harmonic constants for that location and ultimately finish up with a fully formatted tide table for any date or time into the future. The modules included are: TASK Analyse TASK Toolkit POLTIPS 3 TASK Graph TASK Loader The main harmonic analysis program. A program to help process and resample the data. A tidal prediction program designed to use custom sets of harmonic constants. An Excel template for looking at the time series data. A program used to launch the other programs and give access to help/tutorials. Before Using the Software Before you use the TASK software, it would be advantageous to have at least a rudimentary understanding of tides and tidal analysis. The information in this user guide will go some way to providing this, but if there is anything you are unsure of, it is worth seeking clarification from another source (see Appendix D Additional Reading Material at the end of this user guide). It would also be useful to have some experience in manipulating data files. If you don t have experience in computer programming, then a package such as Microsoft Excel 2007 (and any later version) will be capable of carrying out most of the data manipulation you might need to do. For advice and possible assistance in data processing, please send an e mail to the Marine Data Products Team at the UK National Oceanography Centre ([email protected]). Workflow The diagram on the right shows the basic workflow in carrying out a harmonic analysis starting with getting the data from the tide gauge through to producing a set of tidal harmonics, and ultimately through to the production of a printed tide table. Since much of the tidal analysis process is to do with manipulating data files, it will be useful to look at the different file formats before taking a more detailed look at the step by step process. Page 15

17 Data File Formats There are three standard formats that are used by TASK Windows Edition for processing and manipulating data which are described below. The various formats need to be processed by TASK Toolkit before the data can be analysed. This will convert the data into the TASK 2000 format the input format for the TASK Analyse module. NOC CSV Format The default TASK Toolkit Input Format file is a simple comma separated values (CSV) file that is very easily created from any data manipulation or spreadsheet package such as Microsoft Excel. There is an optional number of header lines (which can contain any text) followed by the tide gauge data to be analysed. The order of values on the data lines is: date time data1 data2 (optional) quality flag (0 or 1, optional) any additional information (optional, ignored) Here s a typical file (it has only one data value, a data quality flag and some other information which is always ignored by the TASK Toolkit processing software. Header line 1 Header line 2 01/01/2001,00:00:00,1.396,0,Anything here is ignored 01/01/2001,00:15:00,1.161,0 01/01/2001,00:30:00,1.066,0 01/01/2001,00:45:00,0.936,0 01/01/2001,01:00:00,5.791,1,This data point flagged as bad 01/01/2001,01:15:00,0.626,0 01/01/2001,01:30:00,0.501,0 Note that in the example above, the timing interval is the same (15 minutes), however this does not need to be the case with a NOC CSV file. TASK Toolkit will resample the data to a user specified fixed interval before outputting the TASK 2000 file. It is advisable to use a.csv file extension for this type of file. University of Hawaii Sea Level Center Format The University of Hawaii Sea Level Center (UHSLC) serves multiple roles in supporting real time oceanographic operations as well as climate and oceanographic research. The UHSLC is an active contributor to the Intergovernmental Oceanographic Commission (IOC) Global Sea Level Observing System (GLOSS), and participates in operational and scientific oversight through the GLOSS Group of Page 16

18 Experts. The UHSLC is primarily concerned with the implementation of the GLOSS Core Network of stations, the primary global observing system for in situ sea level. Data from the UHSLC global tide gauge network are supplemented by contributions of hourly data from agencies around the world on a monthly basis in support of the GLOSS Fast Delivery (FD) System. The rapid availability lends to limited quality control and calibration. The data support near real time oceanographic projects such as the Global Ocean Data Assimilation Experiment (GODAE) and preliminary comparisons for calibration of altimeters. You can visit the UHSLC at the following address: Data can be directly downloaded from the UHSLC website which looks like this... Saving the hourly CSV data will allow you to read it straight into TASK Toolkit and reformat it into TASK Format, as well as checking for any gaps in the data. The latitude, longitude and time zone of the data is not present in the CSV file however so you will need to also look at the information in the Data column where the position and time zone appear on the first line of the file. Each line has the year, month, date, hour and sea level height (in millimetres). An example of the UHSLC CSV format is shown below: 2001,12,16,6, ,12,16,7, ,12,16,8, ,12,16,9, ,12,16,10, ,12,16,11, ,12,16,12, Page 17

19 TASK-2000 File Format This is the native data file format for the TASK Windows Edition software all TASK users should read this section carefully and familiarise themselves with this format. All data should be converted to TASK 2000 format before harmonic analysis using the TASK Toolkit module. One of the principle jobs of the TASK Toolkit module is to convert the data from one of the three accepted input formats into TASK 2000 format prior to harmonic analysis. TASK 2000 files are text files that consist of a number of header lines (which can contain any descriptive information relating to the data) and then one data record per line. Each data record consists of data corresponding to a single date and time as shown in the example below: Header line 1 blank header line 20 (always 20 header lines in total) data channels must be 8 characters, see comment below on data channel spacing Each data record contains the following information: Record number (sequential) Data quality flag (0 = good data ; 1 = bad data) Year (must be 4 digits and no earlier than 1910) Day number (1 to 366 table of day numbers given in Appendix C3) Time as a decimal hour (e.g. 3:45pm = 15.75) 5 data columns. These are referred to as channels 6 to 10 throughout this document There are some strict requirements for the TASK 2000 file format some of the program modules require strict adherence to the format, whereas other parts allow some flexibility. Following these guidelines will ensure all programs in the TASK suite will work correctly. Record numbers must be sequential and start at 1 the software uses this as part of the data quality control. All modules have this requirement. If you use TASK Toolkit to generate the TASK 2000 file, the record numbers will always be sequential beginning at 1. TASK DMT can be used to extract a subset of a TASK 2000 and provides the option to auto renumber the records so that they always start from 1. The data quality flag should be 0 (for good data) or 1 (for bad/suspect data), however the TASK Toolkit program will often generate a file with values of greater than 1 as advisory flags. These should be assessed and the analyst should make a decision as to whether these should be converted to a 0 or 1. However the TASK Analyse module does have the option to treat all nonzero flags as bad data. In this situation, all the non zero flags will be changed to a 1 in the TASK Analyse TS output file and thereby making it conform to strict TASK 2000 format. Page 18

20 A strict TASK 2000 file should have 20 headers lines, however TASK Toolkit and TASK Analyse will allow any number. If there are less than 20 header lines in the input file, TASK Toolkit will generate additional header lines to create a strict TASK 2000 file. If there are more than 20 header lines in the input file, a warning will be given but all header lines will be copied. The time interval between successive lines of data should be identical throughout the file. Decimal time is rounded to 3 decimal places; therefore the data interval needs to be such that decimal rounding does not create an uneven spacing. (e.g. intervals of 6 minutes, 12 minutes and 15 minutes are permitted, but 10 minutes are not as the values would be 0.000, 0.167, and this is not perfectly even due to rounding). Convention is that observation data goes in channel 6, and the predictions and residuals computed by TASK Analyse are put in channels 7 and 8 respectively. You must use this convention if you want to view the data in TASK Graph (the supplied Microsoft Excel based graphing module). It is also good practice to be consistent from one analysis to the next. TASK Toolkit will always write any processed data to channel 6. TASK Analyse is flexible in the number of data channels on each line (it can handle from one to six data channels per line). The t2k file output by TASK Analyse will always have a fixed number of data channels on each line. TASK Analyse can handle flexible spacing of the data channels as long as there is a space or tab between each number. However to use TASK Graph, the spacing of the data channels must be exactly 8 characters each with at least one space before each number. Therefore for numbers greater than two decimal places should be used instead of three (TASK Toolkit will automatically reduce the number of decimal places when converting CSV files if required). It is not uncommon for FORTRAN programmers to read/write numeric data with a format statement that does not leave a space between the numbers. This will cause problems with TASK Analyse Advisory Notes, Warnings and Errors Throughout the software and this user guide the following message levels are used: Advisory Note: Warning: Error: This is usually a technical note about some aspect of the analysis or data processing. It is often just to provide a piece of useful advice or a recommendation to the analyst. A message informing the analyst that there may be an issue with the data, the processing or the software. These are usually not critical, and will not usually prevent the program from completing the task at hand. This is usually more critical in that there is a problem with the data, and the program being used is unable to complete the task. TASK Windows Edition will often make checks on the data in the background and only give a warning / error if it detects something that should be investigated further. A summary of all the advisory notes, warnings and errors that might be issued by the software is given in the Reference section of this user guide. Page 19

21 Installing the Software Minimum Requirements Operating System: Windows XP, Vista, Windows 7 or Windows 8 (The software may work on Windows 2000 but is untested) Hard Disk Space: 20MB for software ; recommended 50MB minimum workspace Screen Resolution: Minimum of 1280x1024 any smaller can make using the application awkward Additional software: Microsoft Excel 2007 or later to use TASK Graph. Template was tested with Excel Note: Microsoft Excel is only required to use the Graph module it is not needed for the harmonic analysis or using any of the other software tools. Installation The software is normally supplied on CD which will contain the setup file. Insert the CD and double click on the setup file to run the software installation. You may get a warning message stating that the software is from an unknown publisher. Click on Yes to continue with the installation. Then work through the steps to install the software. You will need to accept the licence agreement before you can continue with the installation. Please read this before agreeing to it. It is recommended to accept the default installation folder for the files. The installation will create a TASK group on your start button with at least eight program shortcuts in it: TASK Windows Edition The main program launcher TASK Toolkit Main toolkit for format conversion and checking TASK Data Manipulation Tool Data Manipulation Tool TASK Analyse Harmonic Analysis module TASK Graph Graphing Template for Microsoft Excel TASK User Guide This document in PDF format POLTIPS Tidal Predictions POLTIPS 3 Tidal Prediction software POLTIPS User Guide POLTIPS 3 User Guide in PDF format Dongle Update Commercial/Academic version only Page 20

22 Where Files Are Installed Due to the nature of the program in the way some of the programs update other parts of the program, and the way that different organisations have their systems locked down to allow user access only to specific parts of the system, it is beneficial to know how and where on your system the files have been installed. The main program files are, by default, installed into: or C:\Program Files\National Oceanography Centre\TASK\ C:\ Program Files (x86)\national Oceanography Centre\TASK\ if using a 64 bit computer The POLTIPS software consists of numerous files and has therefore been installed in it s own subfolder: or C:\Program Files\National Oceanography Centre\TASK\poltips3\ C:\ Program Files (x86)\national Oceanography Centre\TASK\poltips3\ The installation process will also create a folder in the user files area called: or C:\Documents and Settings\<USERID>\My Documents\TASK Data\ Windows XP C:\Users\<USERID>\Documents\TASK Data\ Windows Vista, 7, 8 In this folder will be placed the TASK Graph Microsoft Excel template (also available directly from the TASK group on the Start Button), and a subfolder called TestData containing some basic test data. When working on a harmonic analysis, we would recommend creating a subfolder under TASK Data for each analysis job that you carry out. In the following step by step section of the User Guide, the screenshots of the software are taken from different versions of the Microsoft Windows Operating system and therefore might not look exactly the same as they appear on your screen. Commercial Licences If you purchased a commercial licence for TASK, you may have received a USB dongle (shown right). This must be inserted into a USB port on your computer before you can run the TASK Analyse module. Educational teaching and evaluation versions of TASK do not require a dongle, but are limited in terms of functionality. See Appendix F for details on renewing your subscription and updating the dongle. Page 21

23 Step By Step Harmonic Analysis This section of the user guide will work through a complete analysis. It will explain each step / program function in detail (in normal black text) as well as letting you know what you need to select for the sample data set included with the software (shown in blue boxed text like this). If you look in your Documents folder you will see a folder called TASK Data, and in this a further folder called Test Data. This contains the following two files: TestData1.xls TestData1.csv An Excel spreadsheet with 1½ months of data for a UK location. A NOC CSV format file of the same data (saved directly from Microsoft Excel) General comments before you start Make sure your system meets the minimum requirements of the software, and that if you are using the standard version, you have the appropriate USB dongle plugged into your PC (if required). Familiarise yourself with the TASK 2000 file format covered on page 18. Specifically take note of the spacing and limits on number size. If you want to use the POLTIPS 3 software which is included in TASK, the data you are analysing must be in metres. If not, you can convert it into a TASK 2000 file using TASK Toolkit, and then use the TASK DMT (Data Manipulation Tool) to change the units (to convert mm to metres, set the data value adjustment scaling to 0.001, and set it to process all records). Note that if use TASK Toolkit to convert a UHSLC data set into TASK format the conversion from millimetres to metres is done automatically. Read the section on handling time zones and datums on page 61. STEP 1: Start the Main TASK Application TASK Windows Edition is actually a suite of individual programs. To make starting each of these easier, there is a module selection application (simply called TASK Windows Edition.exe). Running this to start with will allow you to run any of the other TASK modules. STEP 2: Using TASK-Toolkit to create the TASK-2000 file The first step is to convert the tide gauge data into a TASK 2000 format file, and carry out some initial quality checking. To help achieve this, you can use TASK Toolkit, a collection of data manipulation and processing functions (the Program Function is selected from the drop down list at the top of the window). The main window for TASK Toolkit looks like this Page 22

24 Run TASK Toolkit (either double click on TASK Toolkit.exe or use the Module Launcher Application) and under program function, make sure Convert from CSV to TASK format (with data processing) is selected (this is the default). The main jobs of this program function are: Read in the input file. Generate the sequential record number. Convert the date and time into the year / day number / decimal hour. Resample the input data to the requested time interval so that the timing interval is perfectly consistent. If either of the real data points either side of the resampled data point is flagged as bad, the resampled data point will also be flagged as bad. Check for any serious problems with the input data such as time jumping backwards, or excessively large gaps in the input data. Check for data outside a specified range or for spikes in the data Either copy flags across or generate additional advisory flags to allow the analyst to see where potential problems may be. Generate the TASK 2000 (.t2k) file. Note that the first data value output by TASK Toolkit will always be on the hour, and therefore you may lose up to 59 minutes of data from the start of your file. Page 23

25 Setting the options for converting the data to TASK-2000 format. 1. Select the input data file (TestData1.csv) by clicking the Browse button in the input options or dragand drop the input file from an Explorer window to anywhere on the TASK Toolkit window. Select the appropriate format from the list (NOC CSV, UH SLC, TASK 2000). TestData1.csv is a NOC CSV file. Note that when you select the input file (either using the browse button or by drag and drop, the output filename is also created for you. This will be the same name and the input file but with the filename extension changed to.t2k. If the input file is already a.t2k file, then _TT will be added to the filename before the.t2k. 2. Set the other input options 2.1. Header Lines gives the number of lines in the file before the main data starts these lines will either be copied to the output file or ignored completely Channel selection 1 channel only : only one data channel in the input file so use it Use ch.1 of 2 : two channels of data in input file use the first Use ch.2 of 2 : two channels of data in input file use the second 2.3. Flag present If checked, the data quality flag is copied to the output file. If not, all data is assumed to be good and given a flag value of zero All non zero flags > 1 Allow the flag in the input file to be any numeric value. Any value other than zero will be converted to Copy headers The header lines will be copied to the output file additional headers will be added if required to make a total of 20 lines. For TestData1.csv there is only 1 header line before the data begins, 1 data channel and the data quality flags are present. Therefore enter 1 for Lines to Skip, and ensure 1 channel only is selected and Flag present is checked (ticked). 3. Set the output options 3.1. Filename Enter filename. It is advisable to use the same filename as the input file but with a.t2k extension instead of the.csv one. This makes it easy to identify files that are in TASK 2000 format. Note that the.t2k extension is not added by default if no extension is specified Flag Options When processing the data, TASK Toolkit will look for potential problems (for example by looking at gaps in the input data). The flag options are used to select how these are communicated to the user. These are called advisory flags as they neither indicate good or bad data, but should be looked at by the analyst. Page 24

26 No change to input flags (other than resampling): No additional flags will be generated other than those required for the resampling process Advisory (advice flags set to 2, can overwrite a 1): Any potential causes for concern identified by the software will be flagged with a 2 in the flag column. This will overwrite whatever value is there to start with (whether it is a 0 or 1) Advisory (value indicates advice type) [default / RECOMMENDED]: Values greater than 1 can be added to the flag column. The number will indicate the type of warning. This is the recommended option for most analysis work. 1: Original data flagged as bad 2: Gap detection threshold this data point falls between a gap that exceeds the threshold level set by the user. 4: This data points falls between a gap that exceeds the allowed variance from the mean. 8: Data range warning or spike warning. 9: Other warning or multiple warnings Advisory (with additional flag column): As for but an additional column is added at the end of any line with << to indicate an advisory flag has been added, or JB if there is a jump backwards in the time stamps of the input file << << This option if to make manual viewing of the file in a text editor easier to do. It creates a non compliant TASK 2000 file and therefore should be run with option before analysis. The extra column is there to make identification of the advisory flags easier and to indicate backward time jumps (indicated by a JB ) Advisory (advice flags set to 1) NOT RECOMMENDED Page 25

27 Suspect data is flagged with a 1. This is not recommended as advisory flags are only potential problems that should be looked at by the analyst, and using this option makes it impossible to tell which flags were set by TASK Toolkit and which were in the original data (assuming the original data had data quality flags) Resampling interval Set this to the fixed interval to which the input data should be resampled. This should generally be set to a value close to the interval of the tide gauge data. Only certain values are permitted here the most common ones being 360s (6 minutes), 720s (12 mins), 900s (15 mins), 1800s (30 mins), 3600s (hourly) Autorun data manipulation tool The data manipulation tool (a separate program in the TASK software suite) can be used for a range of different data processing tasks. If this is selected, the program is automatically launched when TASK Toolkit has finished creating the TASK 2000 file. The filename and number of header lines are automatically passed to the TASK DMT program. See later section for more information on the data manipulation tool. For TestData1.csv, we would recommend keeping the filename the same except change the.csv extension to.t2k (for TASK 2000 files). Therefore enter TestData1.t2k in the Output filename box. Set flag options to Advisory (value indicates advice type), and set the resampling interval to 900 seconds. The Test Data is already 15 minute (900 second) data, so by setting this interval to the same value no resampling will actually take place. 4. Processing Options 4.1. Gap Detection Threshold This is the value above which any time interval between two points in the input data would be considered a genuine gap and flagged as suspect in the output file, rather than just a normal fluctuation in the time intervals. This is not necessarily related to the resampling interval. If your input data is typically every one minute you might set this value to 180 seconds to indicate that any gap in the input data above 3 minutes is a genuine gap and data output within this interval should be flagged as suspect. On the other hand, if your input data has intervals that generally vary between 12 and 18 Page 26

28 minutes, you might set the gap detection threshold to 20 minutes (1200 seconds). In both cases you might set the resample interval to 900 seconds (15 minutes). Any data in the output file that is interpolated from two data points that exceeds this gap will have an advisory flag attached to it Acceptable variance from the running mean This is similar to the gap detection threshold but compares the time gap between two values with the average gap for all the data read in up to this point. The user can specify the amount by which the gap can differ from the average value. After a minimum n values (data intervals) have been averaged together, the software will start comparing the data gap with the mean interval of all the data so far. If the gap in the data is less than the lower limit multiplied by the mean or greater than the upper limit multiplied by the mean, an advisory flag is put into the output file. For most purposes, the default values of 0.7 and 1.4 can be used, however if the time interval of the original data is very small (say 1 minute), values of 0.1 and 10.0 will probably work better. For the supplied test data (TestData1.csv), the data interval has very regular 15 minute spacing so the gap detection threshold can be set to an interval slightly longer than this say 950 seconds. For more erratically spaced data, you will need to put a realistic value for what you consider an unacceptable gap in the data. Leave the acceptable variance values to their default values (the default values specified here will work well for most data sets and should rarely need changing) Data range Use this to specify an upper and lower limit on valid data. Any value outside of the range specified will be flagged as bad data. For example if this is set to a minimum of 10.0 and maximum of +10.0, values such as and (values often used to indicate bad or missing data in the tide gauge record) will be flagged as bad and so be excluded from the analysis. Data range checking can be enabled or disabled using the check box. 4.4 Spike Detect This is used to detect single values in the data that do not seem to fit with the two data values either side of it. It does this by seeing how much the data value differs from the value obtained by averaging the data point with the two either side of it. The size parameter allows you to specify how much of a variation is allowed (the units are the Page 27

29 same as for the data), however if the value is set too low, real data may be misinterpreted as spikes and flagged as bad. The group parameter allows you to specify how close two spikes should be for TASK Toolkit to also exclude all the data between the two spikes. The default value of 4 means that for 15 minute data, two spikes up to an hour apart will force TASK Toolkit to flag the data between the two spikes as bad also. Spike detect only works with spikes in the data which last one data sample. A jump in the data of more than one sample will not be detected as a spike. For a brief reminder of what this does, click the Exp button for a simple explanation. 5. Click Start to begin processing the data. After a couple of seconds you will either get confirmation that the task has completed successfully, or you will get an error message indicating that a problem has been encountered. See the Reference section of this user guide for full details of all the errors and warnings you are likely to encounter in the TASK modules. 6. Look carefully at the output in the Program output box for any warning messages that may have been reported by the software. Major errors will usually pop up in a separate dialog box as well as in the output window. 7. Check the output file has been created correctly (A basic text editor such as Windows Notepad or Wordpad are useful here). The TestData1.t2k file is output for the sample data (TestData1.csv) and should look like this: Date,Time,Level,Flag header line added by TASK Toolkit header line added by TASK Toolkit.. total of 20 header lines altogether.. header line added by TASK Toolkit header line added by TASK Toolkit Page 28

30 You will notice that since the original data file only had one header line (which was set to be copied to the output file), and fully compliant TASK 2000 files have 20 header lines, 19 additional lines have been added in. If you process data that has values greater than , TASK Toolkit will automatically reduce the number of decimal places to two (and down to one if > ) so that the numbers still fit in the 8 character column width of the file format. If you are confident that the data you have processed is of a high quality (i.e. the timing information is accurate and any gaps in the data are correctly flagged), then you can skip the next stage of graphing the data and go straight to the harmonic analysis. However it is good practice to take a look at the data in the TASK Graph module and we recommend you always do this. Page 29

31 STEP 3: Looking at the Data with the TASK-Graph Module TASK Windows Edition The graph module included with the software uses Microsoft Excel 2007 (and later versions although not as thoroughly tested on them) to provide some basic plotting facilities for the data. As all the file formats used by TASK are plain text files, you could quite easily import the data into any other graphing software you choose. The Excel template file is called Task Graph.xltm. This can be used to view the data before you carry out the initial analysis as well as to inspect the time series output from the analysis (see Step 5). NOTE: The Excel template provided with TASK Windows Edition uses embedded macros to provide some basic functionality. Therefore you must have macros enabled to use the Excel template. If you try using the template and get a message stating that macros are disabled, refer to Appendix E for details on how to enable them. To view the data in the graph module, follow these steps: 1. Double click on the file Task Graph.xltm to load it in to Microsoft Excel (or click on Graph in the Module Selection Window). The window should open up and look something like this... Important Note: Ensure that double clicking on the Task Graph file creates a new blank document rather than loading the template for editing. The title bar of the program should say Task Graph1, and NOT Task Graph.xltm. If the latter is shown, see Appendix E of this user guide. Page 30

32 2. Click on Import Task Format Data and select the file you wish to load. This needs to be a strict TASK 2000 file with 20 header lines (as generated by the TASK Toolkit or the TASK Analyse programs). 3. The data will automatically be loaded into the spreadsheet at the appropriate location. This template always assumes that the observation data is in column F (the 6 th column in the text file), the prediction data in column G, the residual data in column H, and the quality flag in column B. At this stage of the workflow, you have not yet carried out the harmonic analysis (that s the next step), so the prediction and residual channels will contain zeros. 4. Click on the chart tab at the bottom of the spreadsheet (see right) to bring up the graph showing the data. It is best to have the window maximised to see the whole graph. Page 31

33 If the whole graph is not visible within your Excel windows, scale the chart using the zoom slider in the bottom right corner as shown on the right. You will notice that the graph has two Y axes. The left axis is used for the observation data and the predicted data (at this stage the predictions will be all zero as the harmonic analysis has not yet been carried out). The axis on the right is used for the residual (also all zero) and data quality flag. You can scroll through the data using the scroll bar, and select how much is visible by clicking one of the Auto Zoom buttons. Clicking the arrows at the end of the scrollbar moves the window by a quarter of the visible region (i.e. if zoomed in to show 4 weeks, the graph will scroll one week). Clicking the area of the scroll bar inside the arrows will scroll a full screen at a time. Click in this area to scroll by a specified zoom amount i.e. one screenful. This is called a page jump. Click here to scroll by 25% of the visible graph Page 32

34 Change the amount of data in view by clicking one of the Auto Zoom buttons. The graph can show up to four traces simultaneously, although at this stage, the prediction and residual are all zero, and so only the data (in red) and the Error Flag (in black) will be shown. The prediction and residual traces will appear when viewing the TS output file from the analysis (see Step 6). Assessing Data Quality and Flagging Bad Data When trying to assess the quality of data, you may want to visually scan through all the data. At NOC, we usually find that setting the auto zoom to 1 week, and then using the page jump to move through the data is quite effective for spotting any significant problems in the data. If you see data that looks suspect, you may which to flag this data as bad so that it is not taken account of in the harmonic analysis. Below is an example that has been artificially created just for the purpose of this documentation (i.e. it does not exist in the test data supplied with the software). There is clearly data here that is not correct and should be flagged as such. By zooming in a bit closer (maybe using the 1 day AutoZoom button) you can see that in this example, the data from Page 33

35 approximately record 1290 to 1305 is incorrect. To ensure that erroneous data is not used in the harmonic analysis, it is important that it is flagged as bad. To do this you should switch back to the data tab and scroll down until you find the suspect data records 1290 to 1305 (the record numbers are in column A they are not the same as the row numbers of the spreadsheet). In this example you can see that the data from records 1291 to 1304 are all close to 5.0. Once you have identified the bad data records, change the value in the flag column to 1 (Excel column B). If the Enable auto toggle for bad data flags option is set (at the top of the data worksheet and is on by default), positioning the cursor in any cell in this column toggles the value between zero and one. Therefore in the example shown above, you would click on the flag column for record 1291 and press the down arrow until you get to record The values in the flag column will all change to 1. If you now flip back to the chart, you will see that the black line representing the flag column is set to 1, and this data will no longer be included in any harmonic analysis. Page 34

36 If you make any changes to the data, you will need to export the data from Excel using the Export button at the top of the Data worksheet. This will ask you for a filename for the data, and will save the file in the same folder as the input data file. Changing the Y-Axis Scaling The graph uses two different Y axes to display the data due to the differences in the ranges of values between the data/predictions and the residuals. The primary Y axis to the left of the graph is used for the observation data and the predictions whereas the secondary axis on the right is used for the residual and the data flag. These are set automatically when the data is read in. The values selected for the maximum and minimum of both the primary and the secondary axis are shown at the top of the data spreadsheet. However you can choose to override the auto settings but specifying new axis maximum and minimum values in the blue boxes. Page 35

37 STEP 4: Harmonic Analysis Once the data is in TASK 2000 format, a harmonic analysis can be carried out. Start the TASK Analyse module by clicking the Analyse button in the module launcher or double clicking on the TASK Analyse.exe program in the explorer window. The program should look something like this: Performing a Harmonic Analysis To run the harmonic analysis, follow these steps Select the TASK 2000 file containing the data to be analysed (click the Browse button and select the file, or drag and drop the file anywhere on the window). The data must be in TASK 2000 format (although there are additional buttons for future expansion if necessary). 2. As soon as the file is selected, the program will count the number of records in the file and display this information in the Start Record and End Record boxes. While counting, it will also carry out some basic checks such as ensuring the record numbers are all sequential (if not, ERROR: Non consecutive record numbers will be displayed in the output window). The TestData1.t2k file should have 4416 records. If you wish to analyse only a subset of the data, you can enter the start and end record numbers in the boxes. Clicking the Count records button to reset the numbers to select all the data records in the file. 3. The number of header lines will default to 20 which is the normal amount for a TASK 2000 file generated by the TASK Toolkit module. However TASK Analyse will allow some flexibility regarding the number of header lines present, and therefore this number can be set here. Page 36

38 4. Specify which channel contains the observations, and where the analysis module should put the tidal predictions (generated from the derived harmonic constants) and residuals. The TASK 2000 file usually has 10 columns (values per data line). The first five values are for record number, quality flag and date/time information. The next five (called channels 6 10) contain data and one of these should contain the tide gauge data to be analysed. The default channel numbers for a TASK 2000 file are: Channel 6: Channel 7: Channel 8: Tide gauge observations Tidal Predictions (from the computed harmonic constants) Residuals (observation minus tidal prediction) The input file at this stage will usually only contain data values in the observations column. However on completion of the harmonic analysis a copy of the input file will be created with the prediction and residual columns filled in. 5. A TASK 2000 file will usually only contain a value of 0 or 1 in the flag column (0 being good data, and 1 being bad). However the TASK Toolkit program can create a TASK 2000 file with additional flag values to indicate an advisory condition on that piece of data. If you want the analysis module to treat all non zero flags as bad data (i.e. to ignore the data when deriving the harmonic constants), then tick the box:. The output TASK 2000 file which is created by the analysis module will change all such flags to Select the number of harmonic constants you want to extract from the analysis using the drop down list (the actual list in the software may differ slightly from the one shown below). For the test data in TestData.t2k, a suitable number is no more than the set of 35. If you used TASK Toolkit to process the data and create the TASK 2000 file, it will have recommended a suitable number of harmonic constants to use. This is based on the number of days of good quality data in the input file but it is only a guideline. As a general rule it is better to analyse for too few rather than too many (but read the section entitled Selecting the Number of Harmonics below). The table below is a guide to which set should be selected based on the amount of good quality data in the input file. It lists standard sets that are configured for use with the software however advanced users can create their own configurations of harmonic constants. Read the appropriate section of the manual (under the Advanced Tidal Analysis Topics section) before doing this. Page 37

39 Minimum recommended duration for data Minimum recommended number of hours Number of harmonics which can be derived 1 day 24 hours 1 (M2 only) 15 days 360 hours 15 harmonics (N 2 not included) 30 days 720 hours 26 harmonics Use with caution 30 days 720 hours 26 harmonics + 8 inferred 6 months 4,380 hours 54 harmonics 12 months 8,760 hours 62 harmonics 12 months 8,760 hours 102 harmonics (good quality data) 4.5 years 39,000 hours 114 harmonics 7. If you require daily and monthly means to be calculated, tick the box:. There are two possible output formats for these files these will be covered later in the user guide. 8. Begin the harmonic analysis by clicking the button:. This can be a lengthy process depending on the amount of data and the number of data blocks (data marked as bad will create additional block of data). The progress indicator will show the progress of the analysis. Stages 2 and 3 (data blocking and matrix generation) can take much longer than the other stages. The program may freeze for up to 10 seconds during stage 1 (reading data). The analysis can be stopped by clicking on the stop button or the exit button in the top right corner. 9. If the analysis completes successfully, you will get the message Analysis Complete in the output window. Any problems will give an error message and the analysis will not run to completion. Even if the analysis completes successfully, some warning messages may be issued as part of the quality checking process. Selecting the number of harmonics One of the most important decisions a tidal analyst must make is to decide how many harmonic constituents to analyse for. If you analyse for too few, you will not capture all the periodic tidal energy that is in the tide gauge record, and predictions based on these harmonics will not be as accurate as they could be. However if you have a short time series, it may not be possible to separate two different harmonics that are close in frequency. Trying to analyse for too many harmonics will lead to one of the following situations: You will get an error message: Matrix is too ill conditioned to be solved, and the analysis will end prematurely with no harmonic constants generated. The analysis will run to completion however the harmonic constants calculated by TASK Analyse could be wrong. They may well fit the period of data that was analysed but, if used to compute tides outside of this period, could give very inaccurate results. For example, analysing significantly less than a full year of data for a set of harmonics that include S 1, K 1 and ψ 1 [PSI 1 ] (all of which are very close in frequency and require a minimum of 1 year of data to separate) may lead to harmonics with amplitudes that are considerably larger than they should be. Page 38

40 So is it always better to select too few harmonics rather than too many? Sometimes, but extreme caution must be used... Analysing Short Time Periods of Data (less than 6 months). Analysing any data set for too few harmonics also has its disadvantages as will be discussed in this section, and again in more detail on page 75 in the section Inferred Harmonics Revisited. Any two constituents need a minimum period of data if they are to be independently identified by the analysis process. The closer the two constituents are in frequency, the longer the span of data needs to be to get both of them accurately out of the analysis. If two harmonics both play an important contribution to the tide but there is insufficient data to separate them, a different approach to the analysis must be taken. TASK Analyse lets you use what are called inferred harmonics (sometimes known as related harmonics) in which you specify an amplitude and phase lag relationship between the two harmonic constituents. TASK Analyse can then calculate both of them correctly using this additional information. One set of harmonics which incorporates eight inferred harmonic constituents is included in the default configuration that is supplied with the software. For a more detailed understanding of inferred/related harmonics and to find out how to use inferred harmonics, see Inferred Harmonics Revisited on page 75. Looking Beyond the TASK Predictions The TASK Analyse program automatically generates the predictions and residual for the period of the observations. However if there is less than 1 year of data being analysed, it is highly advisable to carry out at least a full year of predictions using the POLTIPS 3 prediction software included with TASK Windows Edition. This may highlight a problem which would not necessarily appear when just predicting the period of the observations (especially if this is 3 months or less). If the analysis completes successfully and you have the option to use POLTIPS 3 enabled in the settings dialog, you will be asked whether you want to automatically add the analysis to the POLTIPS 3 software. See the section on page 47 for further details on how to do this. Limitations of the TASK-Analyse program There are a few limitations of the TASK Analyse program: The number of data blocks must not exceed 20,000. A new data block is generated every time bad data is encountered, or after every 59 days of good data. Header lines can contain a maximum of 255 characters. Page 39

41 Certain time intervals are not acceptable in the input file (e.g. 10 minute, 20 minute). The most common time intervals are 360s (6 minutes), 720s (12 mins), 900s (15 mins), 1800s (30 mins), 3600s (hourly). Use TASK Toolkit to resample to one of these intervals. In addition to these TASK Toolkit will only process data from the year 1910 up to 2037 (due to use of the Microsoft date and time functions). Possible Warning / Error Messages There are a range of warning and error messages that might be displayed while running a harmonic analysis. Some of these will not affect the completion of the analysis and are just warnings to draw your attention to something that appears a little out of the ordinary. Other errors may cause the analysis to terminate before it has completed. For a full list of warning and error messages, see the Reference Section of this user guide. Page 40

42 STEP 5: Looking at the Analysis Output On successful completion of the analysis, two output files will have been created (with two additional files for daily and monthly means if the option was selected). These are automatically named to prevent any files being overwritten using a convention that allows multiple analyses to be carried out in quick succession and the user to easily identify which output file corresponds to each analysis. The format of the output file names is: HA_nnn_aaaaaa_bbbbbb_mm_ss_tt.ext nnn aaaaaa bbbbbb mm ss tt ext Number of harmonics requested in the analysis Start record number End record number Analysis method invoked ( GS =Gauss Seidel ; PC =pivotal condensation) Sequence number (increments to prevent files with otherwise identical parameters being overwritten). File type: PR: The primary results file of the analysis. This file also acts as a log file for any messages that arose during the analysis, as well as containing various analysis quality statistics. TS: The time series file. This is a copy of the input data file but with the prediction and residual channels filled in. The prediction is computed from the derived harmonics, and the residual is the observation data value minus the prediction data value. DM: The daily means computed from the original tide gauge data. MM: The monthly means computed from the original tide gauge data. File extension is either.txt or.t2k (for the TS file which is in TASK 2000 format). For example: HA_007_000001_002976_GS_01_PR.txt This is the primary results file ( PR ) for a 7 harmonic constant analysis from record 1 to record Gauss Seidel was used sequence number is 01 indicating this is the first analysis of this type. The Primary Results (PR) File Below is an example of the primary results (PR) file from an analysis of the test data supplied with the software. In this example, the analysis was done for just 11 harmonics (just to keep the length of the output file short for this document). Each section of the PR file are shown in the purple boxes with comments beneath. Page 41

43 TASK/Win ver 1.0 Harmonic Analysis ================================== Input data filename: C:\Users\Analyse\Port001\TestData1.t2k Observations channel: 6 Tidal Predictions channel: 7 Residuals channel: 8 Analysis starts at record: 1 Analysis ends at record: 4412 Harmonic Set selected: 11 harmonics This section gives some of the initial parameters of the analysis that were set by the user, along with the full pathname of the input file containing the tide gauge data. Major Constituents ================== Z Q O MP K N OP M L S M MS This is a list of the constituents that were selected for the analysis, along with their speeds (in /hour). Header information from input file ================================== 01: Date,Time,Level,Flag 02: blank header line 03: blank header line lines removed for the user guide 19: blank header line 20: blank header line The header lines in the input file are copied here they may contain useful information regarding the tide gauge site, recording parameters etc. and are useful to have copied into the output file. Data interval computed as: hours The data time interval is computed by looking at the first two records in the input file. This time interval should then exist between all data records in the file. Page 42

44 Data Block Information ====================== Blk# SRec ERec ts te Reason for block ending ==== ====== ====== ========== ========== ======================== Bad data encountered Bad data encountered End of data reached Blocking Complete. 3 blocks identified. All data read in The data block information shows the details of the individual blocks of data in the file. For most purposes, this information can be ignored, although it is useful if you want to quickly find out where any data is flagged as bad since a block will end whenever a bad data flag is encountered. Observation Stats for selected data records =========================================== Maximum obs value: Minimum obs value: Mean (average): Standard Deviation: Delta (interval): Hours of data: Hours of good data: The Observation Stats section gives some basic statistical information on the tide gauge data such as the largest and smallest values found in the data, the average value (mean), and the amount of data. The statistics apply only to good data within the range of records specified in the dialog box. Bad data is ignored for both the analysis and these statistics. Solving by pivotal condensation method. Shows which of the two matrix reduction methods are used: Gauss Seidel or Pivotal Condensation. Good data analysed for the recommended number of harmonic constants will generally use Gauss Seidel, whereas pivotal condensation may be an indication that too many harmonics are being used for the length and quality of the data therefore check the results carefully. Harmonic Constants ================== Z Q O MP K N OP M L S M MS Page 43

45 The Harmonic Constants section is the main part of the output. For each harmonic constituent that was requested it lists: a sequential number (1 to n) the amplitude of the constituent (in whatever unit the input data was in) the phase of the constituent (relative to the same time zone as the input data see the section on Handling Time Zones for more information) the harmonic constituent name the speed of that constituent (in degrees per hour) the Doodson number (used by many tidal prediction software packages including POLTIPS 3 which is included with TASK Windows Edsition). Quality Control Statistics ========================== Residual Mean: Residual S Dev: Residual Max: Residual Min: Obs Max Min: G(S2) G(M2): Max harmonic amplitude threshold: Actual largest harmonic amplitude: (name: M2) O1+K1/M2+S2: (semidiurnal) Daily and monthly means have been requested as part of the analysis. This may also provide useful information for data quality control. The last section is the quality control statistics these look at the results of the analysis and display a range of values useful for quality control purposes. Residual Mean: This should be very close to zero (within ±0.0003). Residual S Dev: Residual standard deviation (is often in the range 0.1 to 0.4m). Residual Max/Min: The most extreme values in the residual (i.e. the largest negative and positive surge). It is rare for the maximum to be greater than 3.0m and the minimum to be less than (more negative than) 3.0m. Obs Max Min: The difference between the maximum and minimum values in the observations. Predicted max harmonic This is an estimate of the absolute maximum amplitude of any amplitude: harmonic constituent this is used as the alert threshold for the actual largest harmonic constant amplitude rather than an estimate of what the largest should be. Actual max harmonic The amplitude of the largest harmonic with the Doodson number K amplitude: indicating which constituent shown in brackets see appendix C. For most locations, this will be constant number 31 (M2), however places with a diurnal tidal regime may show 10 (O1) or 17 (K1). Page 44

46 O1+K1/M2+S2: This is a common formula which takes the amplitudes of the four principle harmonic constituents and computes a value which gives an indication of the type of tidal regime the location exhibits. Value 0.25 Tide is semi diurnal 0.25 > Value 1.5 Tide is mixed (mainly semi diurnal) 1.5 > Value 3.0 Tide is mixed (mainly diurnal) Value > 3.0 Tide is diurnal It is useful to check this value, as sometime the value will register too high if the analysis is computed for too many harmonics. For example, UK ports will predominantly give a value of <0.25, however if a very short period of observations is analysed with too many harmonics, the value can increase dramatically due to the amplitude of K1 becoming too large. If the daily and monthly means box was ticked when the analysis was done, this will be stated in the main output file and two additional files will have been created. The monthly means may prove useful in quality controlling the original tide gauge data as you may be able to identify shifts in the datum. See the section later in the manual for more information on daily and monthly means. Note that the PR file from TASK Windows Edition is different from the main output file from the older TASK software package which was DOS (command line) based and is now obsolete. The Time Series (TS) Output File This is simply a TASK 2000 format file that is output from the TASK Analyse module. It is basically identical to the input file supplied for analysis, however two additional channels will be filled in with the prediction (based on the harmonic constants just computed) and the residual (the observation value minus the predicted value). This file can be plotted using the TASK Graph module. Page 45

47 STEP 6: Graphing the Data with the TASK-Graph Module TASK Windows Edition For a basic introduction to the Excel based graphing module that is included with the software, see step 3 (page 30) of this user guide. The main difference between step 3 and step 6 is that, if you import the TS file which comes out of the analysis, you will be able to see the prediction (blue line) and the residual (green line) as well as the observation and data flag (provided the default channels 7 and 8 were used for the prediction and residual respectively). Looking at the graph will give you some indication of the quality of the analysis, however you should also make use of the other information given in the analysis output files. One of the most common signs of a possible problem is when there is a significant oscillation in the residual of the same frequency as the tidal data for just a portion of the complete time series. This may be (but is not always) an indication of a timing error where the clock on the tide gauge data logger was incorrect for that period. If the clock is wrong for the whole of the observation period, you will have no way of knowing since the analysis will generate the harmonic constants based on this timing information. Page 46

48 STEP 7: Deriving a prediction For many users of the TASK software, the ultimate output will be the production of tidal predictions derived directly from the harmonic constants computed by the TASK Analyse program. To carry out the predictions, the TASK package includes a copy of the National Oceanography Centre s POLTIPS 3 tidal prediction program. This is used as a standalone package by hundreds of clients and some of the features include: Calculation of many years of tidal predictions at once Save data to file or copy & paste the data into any other Windows program Calculation of turning points, interval predictions or general statistical information Graphing the data A selection of fixed layout tide tables Powerful custom tide table layout tools User friendly interface allows scroll and zoom across graphs, maps etc. Auto integration with the TASK Analyse program Batch mode processing (available only under special licence from NOC) Page 47

49 For users of TASK Windows Edition, one of the most useful feature is how the TASK Analyse module and the POLTIPS 3 software integrate together. As soon as an analysis has been successfully carried out, it can be set up to make the harmonic constants available in the POLTIPS 3 software and you can compute a fully formatted tide table or plot a graph showing a full year of data in seconds. Even the parameters of the analysis such as the number of harmonic constants and the start and end data of the observation data used to derive the harmonics are passed to the POLTIPS 3 software and displayed in the status bar of the Port Selection dialog box. Setting up TASK-Analyse and POLTIPS-3 Integration The first time the TASK Analyse module is used, the POLTIPS 3 integration is not setup, and therefore you need to carry out the following steps if you want the results of your analysis to be instantly available within POLTIPS Click the Settings button in the TASK Analyse window. The dialog box shown on the right will appear. 2. Ensure that the Use POLTIPS box is checked, and enter the full location of the POLTIPS 3 software in the text box. It is usually installed in a subfolder of the main TASK software modules. 3. Click OK. TASK Analyse will check to see if it can find POLTIPS 3 in the specified location and if so, you will get confirmation that the software has been found. If the POLTIPS software can t be found in the location specified, check you have typed it correctly, and that the software is where it is meant to be. 4. Once this has been done, the settings will be remembered the next time you run the TASK Analyse software so this process will not need to be done again. Using TASK-Analyse and POLTIPS-3 Integration Once the previous steps have been carried out you will be given the option to automatically update the POLTIPS 3 software with the harmonic constants at the end of each analysis. Once the analysis has completed (see STEP 4 on how to do this), you will prompted if you want to add this to POLTIPS 3. Page 48

50 Answering Yes will have TASK Analyse bring up an additional dialog box to gather some additional information required by POLTIPS. The following information is required: The location name (maximum of 31 characters are permitted) Primary datum name this is the datum to which the tide gauge observations were recorded. The Z0 value from the analysis is used to shift the predictions from mean sea level (MSL) to this datum. If left black, POLTIPS will assume this is Chart Datum. The optional Secondary Datum information. Four predefined UK based Ordnance Datums are available additional international datums can be specified by selecting SEC and putting the datum name in the text box. Time zone information specify if the original data (and therefore the harmonic constants) are in local time zone (box ticked) or UTC/GMT (box unticked). For non UTC/GMT locations... you should specify the local time zone as a decimal value. Use the traditional tide table convention i.e. specify the number of hours required to shift the data from local time zone back to GMT (e.g. most of Europe is in time zone UTC + 1 hour therefore you would enter the value 1 0 ). This is required whether the harmonic constants are in UTC/GMT or already in local time as it enables POLTIPS 3 to display the appropriate time zone label on the outputs. Consult the table below for advice on how to achieve the different time zone outputs in POLTIPS 3. Location TZ is UTC/GMT (e.g. UK, Ireland, Iceland, Portugal) Location TZ is not UTC/GMT but harmonic constants are in UTC/GMT Location TZ is not UTC/GMT and the harmonic constants are in local time You require POLTIPS 3 to output data in UTC/GMT Since Time Zone specified as zero, POLTIPS will label the outputs as GMT. Not available Technically you could leave the box unchecked, and put an opposite T Zone value in the box which would change the data from the local time back to GMT. However POLTIPS would then label the outputs incorrectly so this is definately NOT ADVISABLE. You require POLTIPS 3 to output data in local time Local time is GMT therefore not applicable POLTIPS will make the T Zone adjustment for you and label outputs accordingly POLTIPS will not make any T Zone adjustment to the data but will use the information specified here to label the outputs correctly. For details on how to use the POLTIPS 3 software, see the separate POLTIPS User Manual. Page 49

51 Reference Section Errors and Warnings This section covers all the errors and warning messages that can be issued by the various software modules in TASK. Some of these will appear in a dialog box, and some will appear in the output window of the applications. Errors will generally halt the process at the point it occurs whereas warnings will usually allow the program to continue. TASK-Toolkit Message ERROR: No output filename specified ERROR: Problem opening input file [HC_Convert] ERROR: Problem reading line in input file [HC_Convert] ERROR: problem reading harmonics [HC_Convert] ERROR: Unknown harmonic name check list of allowed names ERROR: Duplicate harmonic constant found in.pri file Warning: XML compliance check failed... ERROR: Problem opening the output XXX file ERROR: Problem reading line in input file ERROR: Problem reading TASK 2000 format input data ERROR: Channel n missing from file Description Ensure a valid filename has been entered into the output filename box. The file does not exist check you are specifying the correct location for the file. Windows reported an unspecified error reading the input file. TASK Toolkit seems to have a problem reading the analysis output file. Make sure this has not been manually edited in any way. One of the harmonic constants that appears in the analysis output file is invalid. A duplicate harmonic has been found in the analysis output file. The creation of the XML file has caused a problem. One of the required pieces of information does not conform to the correct format. Problem opening one of the output file formats during the harmonic constant format conversion. XXX will either report D, CSV or XML depending on which file could not be opened. Check you have sufficient disk space and permission to write to the disk. Check all lines of the input file conform to the specified format and that there are no anomalous characters. The error will sometimes give an indication of the approximate line number in the input file check a few lines either side of this. Data not in strict TASK 2000 format check the input file carefully. The channel selected for the observation data in a TASK 2000 format file is missing. This can be channel 6, 7 or 8 only. Page 50

52 ERROR: Problem reading UH SLC input data. Check format is: yyyy mm dd hh:mn:sc,z where z is an integer value for sea level in millimetres. ERROR: Problem reading input data (ch n/n + flag) ERROR: 'year' value read in is outside allowed range ERROR: 'month' value read in is outside allowed range ERROR: 'date' value read in is outside allowed range ERROR: Resample interval is not a valid value ERROR: Problem opening the input file ERROR: Problem opening the output file ERROR: More than 20 lines in the input headers. This will make the output file non compliant ERROR: Problem skipping required number of lines WARNING: Data interval not consistent at 3DP { the WARNING occurs only when convert from CSV to TASK format (no data processing) is selected. See next line for when this occurs as an ERROR} ERROR: Data interval not consistent at 3DP The UH SLC website supports different output formats. If the data is downloaded from the site, make sure that the correct option was selected for the output format. The data being read in had an error in it. Check the data conforms to the correct format on all lines. The error will sometimes give an indication of the approximate line number in the input file check a few lines either side of this. Data must be in the range 1910 to We hope to extend this range further back in the near future. Approximate line number in the input file is given. Invalid value found somewhere in the month field of the data. Approximate line number in the input file is given. Invalid value found somewhere in the date field of the data. Approximate line number in the input file is given. Only certain values are permitted for the resample interval. These are: 10800, 7200, 5400, 3600, 1800, 900, 720, 450, 360, 180, 90, 72, 36. The Windows operating system has reported a problem opening this file. Check the file exists and the correct location has been specified. The Windows operating system has reported a problem opening this file. Check that you have write permission on the disk and that there is sufficient space. A TASK 2000 compliant file should have 20 lines in the header. If there are less than 20 lines, additional lines will be written to make the number up to 20. If there are more than 20 lines, they will all be copied to the output file, however the file will not be TASK 2000 compliant. There was a problem reading the input data file. Check the number of header lines you have specified matches the number in the file. The interval between the first two values sets the interval which should continue throughout the file. Data has been found that is not consistent with this to the full 3 decimal places. This would make analysis not possible. Select convert from CSV to TASK format (with data processing) to resample to data to a fixed time interval. Approximate line number in the input file is given. This error should not occur and indicates an internal calculation error. Please contact the NOC Marine Data Page 51

53 ERROR: Invalid value for flag ERROR: Data before 1910 not currently supported ERROR: Position value out of range. Please report this error to NOC. WARNING: Gap Errors found and flagged in output file WARNING: Backwards time jump found. May be caused by the data having a daylight saving time adjustment? WARNING: Block count exceeds analysis limit of 1000 Products Team to report this error. The only permitted values for the flag are 0 and 1. If you have values other than this. Approximate line number in the input file is given. The current version of the software does not support data prior to This limitation may be removed in later versions of the software. This is an internal mathematical error which occurred while trying to analyse the data. Please report this error to NOC. This is a warning message that gaps were found in the input data that exceeded the value set in the gap detection threshold. This will not stop the conversion process from CSV to TASK format. There was a jump backwards in time in the input data. If Advisory flags with additional flag column is selected, an additional 'JB' flags in the output show the approximate vicinity of the backward timestep in the input data. If you intend to analyse the data using TASK Analyse, you will not be able to do it as a single job as the number of data blocks exceeds the limit of 1000 blocks. This is probably due to lots of bad data points in the input file. TASK-Analyse Message CRITICAL ERROR: Can't find HCselect.txt ERROR: Input check reports recnum out of range ERROR: Input check reports year out of range ERROR: Input check reports interval differs from expected ERROR: Input check reports flag value invalid CRITICAL ERROR: Harmonic in HCselect.txt not identified (in TiraUFset) Description One of the vital files required by the TASK Analyse program is missing. Please reinstall the software. If the problem persists, contact the National Oceanography Centre Marine Data Products Team. The record number read in from the input file is outside of the expected range. Check the file uses only sequential record numbers. The year must be in the range 1801 to Make sure years are represented as 4 digit numbers and not 2 digit numbers. The time interval is not consistent throughout the input file. Please use TASK Toolkit to generate the input file that is compatible. Error flag must have the value 0 or 1 unless the extended flags option is selected. Either correct the input file, or select extended flags and all non zero flags will be converted to 1. An unknown harmonic constituent has been identified in the HCselect.txt file. If this file has been edited or Page 52

54 changed in any way, please revert back to the original or manually correct the error. ERROR: Problem counting the data records Unexpected error encountered when trying to count the number of records in the input file. The input file must be a TASK 2000 type file. ERROR: Problem opening the input file The Windows operating system has reported a problem opening this file. Check the file exists and the correct location has been specified. ERROR: Input file header lines >255 characters One of the header lines in the input file exceeds the 255 character limit. Shorten the line and try again. ERROR: Start record number after end record number When specifying the start and end record numbers to be used for the analysis, ensure the end number is larger than the start number. CRITICAL ERROR: Problem allocating memory to hold Microsoft Windows has reported a problem allocating data sufficient memory to hold all data. Try restarting your computer before attempting to do the analysis again. ERROR: Problem finding record number in data line. When selecting the input file or clicking Count Records Check the number of header lines is correct. various checks are done on the input data including checking the record numbers. If you get this error it is most likely the file is not a TASK 2000 format file, or the number of header lines specified is incorrect and therefore it is trying to read data in one of the header lines. ERROR: First data record must be record number 1. The data in the file does not appear to start at record Check the number of header lines specified is correct. number 1. Check that the file specified is a valid TASK 2000 format file, and that the number of header lines you have specified is correct. ERROR: Non consecutive record numbers Record numbers in the input file must be consecutive (which should be the case if TASK Toolkit was used to generate the input file). This error can also occur if you do not have sufficient values on each line (a TASK 2000 file should always have 10 values on every line of data). CRITICAL ERROR: Too many major constants (max 115) There is a limit of 115 harmonic constants (114 + Z 0 ). This error should only occur if you have attempted to create your own set of harmonics. Check that you have not made an error in the hcselect.txt file. CRITICAL ERROR: Non standard constituent speed in selection ERROR: End record number must be >1. Suggest you click the 'Count' button. ERROR: Start record number must be >=1 ERROR: All channel numbers must be in the range 6 to 10 The hcselect.txt file contains an error. Make sure that any changes made to this file are accurate. You should only alter this file if you fully understand the theory of tidal analysis. The end record number for the analysis must be greater than 1. You have probably forgotten to click the count button which will count the records and fill the start and end record numbers in for you. The first column in the TASK 2000 file is the data record and numbering always begins at 1. Therefore the start record specified should be greater than or equal to 1. The last five columns in the TASK 2000 file are referred to as channels 6 to 10. Therefore when specifying the channel number for the observations, predictions and Page 53

55 ERROR: The observation, prediction and residual channels must all be different and in the range 6 to 10 ERROR: Problem opening TiraOut.txt / Tira.pri CRITICAL ERROR: Harmonic in HCselect.txt not identified (in main) CRITICAL ERROR: Non continuous record pointers WARNING: Matrix is too ill conditioned to be solved. WARNING: One or more data values outside specified normal range and/or difference between data max & min is too large. See primary results file for more details. WARNING: Amplitude of largest harmonic exceeds alert threshold. A likely cause is that you are analysing for too many harmonics try reducing the number. See the 'PR' output file. WARNING: Maximum (or minimum) residual exceeds the alert threshold WARNING: Residual mean exceeds the alert threshold residuals, enter a value between 6 and 10. The same channel number has been specified for two or more of the channel boxes. Make sure the observations, predictions and residuals have a unique channel number. The Windows operating system has reported a problem opening this file. Check that you have write permission on the disk and that there is sufficient space. These files are temporary filenames. Problem with one of the harmonics specified in the hcselect.txt file. This error should only occur if you have attempted to edit or create your own set of harmonics. Check that you have not made an error in the hcselect.txt file. Record numbers are not consecutive in the input file. Use TASK Toolkit to help prepare a fully compliant input file. The analysis could not be completed dues to a problem solving the internal matrix. It is most likely that you have attempted to analyse for too many harmonic constituents. Pick a reduced set of harmonics and try again. Please read the section on Selecting the number of harmonics in this user guide (page 30). TASK Analyse has scanned the data for the largest and smallest data values. One or more of the following has occurred: largest data value exceeds the limit (default: 20.0) smallest data value less than the limit (default: 20.0) difference between largest and smallest data values exceeds the limit (default: 16.0). More details are put in the primary results file (the PR file). TASK Analyse predicts the absolute largest amplitude value that any harmonic constants should have based on the maximum range in the observation data. One or more of the harmonics computed for this analysis exceeds this limit. It is likely that you have analysed for too many harmonics. The most likely culprit is the harmonic constituents K 1, and the two either side S 1 and PSI 1. Select a set with less harmonics and try again. The largest positive or negative residual exceeds the alert threshold which is set in the software at ±4.0 metres. It is possible that this could be genuine but quite rare. The mean value for the residual is usually very close to zero (10 5 metres). If this mean of the residual exceeds metres, this warning is triggered. A likely cause is that you have used a set of custom harmonics that does not include a Z0 component. This is the mean level value and should be included in every set of Page 54

56 WARNING: Difference between M2 and S2 phase outside of the checking range. WARNING: M2 / S2 / O1 / K1 harmonic amplitude exceeds the alert thresholds WARNING: Amplitude of K1 seems too large. Data should be in UTC/GMT when computing daily/monthly means WARNING: File rename failed. File left as <filename> ERROR: Blank line found in input file (near record n). WARNING: Please drop just one file Problem opening daily/monthly means file Error reading settings.dat file please delete this file and then restart this program. POLTIPS location invalid. Please delete the settings.dat file and restart this program. Can't seem to find POLTIPS in specified location. Check harmonic constituents. The value of (S2 phase M2 phase ) is outside of the normal checking range. The default value for this range is from 10.0 to Therefore if M2 is 180, then S2 must be in the range 190 to 250 otherwise this warning is triggered. A check of the amplitude of the four principle harmonics M2, S2, O1, K1 has revealed one or more of them to exceed the normal limit. The default values for the limits are M 2 = 4.5m, S 2 = 1.75m, O 1 and K 1 = 1.5m. If the results of the analysis give a value greater than these, this warning message is triggered. Due to the nature of the K1 harmonic constituent and the proximity (in frequency) to other constituents, special checks have been built into the TASK Analyse program to identify potential problems. If you get an error relating specifically to K1, there is a good chance that you have analysed for too many harmonics. Try selecting a smaller set of harmonics. If the problem is still flagged up by the software, it would be worth checking the harmonics against an independent data source. Means have not been computed. It is a standard scientific convention for daily and monthly means to be computed for data in UTC/GMT (i.e. daily mean value centred on 12:00 GMT and not the local time zone). The TASK Analyse software writes results to temporary files and then renames them at the end of the analysis if it completes successfully. This error indicates that Windows was unable to rename one of the files to the standard naming convention. It is likely that Windows has failed to release the file. Try running the analysis again (this is often easier than trying to rename the files manually). Blank lines are not permitted in the input file remove any blank lines and try again. You have attempted to drag and drop more than one file onto the TASK Analyse window. Please just drop the input file to be used. The Windows operating system has reported a problem opening this file. The daily and monthly means have not been computed. Deleting this file will cause it to be recreated when you rerun the program. You will then need to click the Settings button and specify the POLTIPS location again. Deleting this file will force TASK Analyse to recreate it when you rerun the program. You will need to click the Settings button and specify the POLTIPS location again. When you change the POLTIPS location in the settings Page 55

57 the program settings. Can't seem to find usercon.dat / index.dat in specified location. Check the program settings. Problem opening settings file for writing Can't open POLTIPS index.dat / usercon.dat file to update it. Port of that name already found in the POLTIPS index dialog box, TASK Analyse will check to see if the POLTIPS software is in the specified location. If not, this warning is triggered. One of the required POLTIPS files is missing. Reinstall the software. Check the file is not set to readonly and you have permission to write to this file. Check the file is not set to readonly and you have permission to write to this file. The POLTIPS index file can t contain 2 locations with an identical file name. Use a different name for the port (or append a number to the name). Page 56

58 Additional Automatic Checking of Results On completion of an analysis, the TASK Analyse program can perform some additional checks on the data to see if the results seem sensible, and issue warnings if certain values are outside what would be sensible. The extra checking can be enabled/disabled in the settings dialog (accessible using the button in the main TASK Analyse dialog box). Because the tides in different parts of the world vary so much, what is sensible for one part of the world might not be for another. Therefore the default values at which various warnings are triggered can be changed by the user by putting new values in a file called checkresults.txt which must reside in the same folder as the TASK Analyse.exe program. The format of the file looks like the one shown on the right (the precise spacing is not important). All checks along with a description and the default value are shown below. It is not necessary to specify all checks in the file, just the ones that you want to change from the default value. CHK_RESIDMAX 0.5 CHK_RESIDMIN 0.5 CHK_M2_MAXAMPL 2.0 CHK_S2_MAXAMPL 1.0 CHK_DATAMAX 15.0 CHK_DATAMIN 0.0 We would not expect the analyst to change these values for every analysis they are really just a guide for helping to identify possible erroneous values in the data or analysis results. For example maybe a few data values have been recorded with extremely high values somewhere in the middle of the file and this might trigger a CHK_DATAMAX or CHK_DATAMIN warning. Check name Default value Description CHK_RESIDMAX 4.00 The maximum size of the residual before a warning is triggered. CHK_RESIDMIN 4.00 The minimum size of the residual before a warning is triggered. CHK_RESIDMEAN The maximum absolute value of the residual mean average. For example a value of means that any value for the residual mean outside of the range to will trigger a warning. CHK_M2_MAXAMPL 4.50 The maximum sensible amplitude for the M 2 harmonic constituent. CHK_S2_MAXAMPL 1.75 The maximum sensible amplitude for the S 2 harmonic constituent. CHK_O1_MAXAMPL 1.00 The maximum sensible amplitude for the O 1 harmonic constituent. CHK_K1_MAXAMPL 1.00 The maximum sensible amplitude for the K 1 harmonic constituent. CHK_M2_S2_PHASEDIFFMIN There is usually a common relationship between the phase of S2 and the phase of M2. When the following Page 57

59 is calculated (PHASE S2 PHASE M2 ), this sets the minimum value allowed. Wraparound at 360 is accounted for. CHK_M2_S2_PHASEDIFFMAX This sets the maximum value allowed for (PHASE S2 PHASE M2 ). CHK_M2_S2_AMPLLIMIT 0.5 This sets the amplitude of M2 at which the phase checks described in CHK_M2_S2_PHASEDIFFMIN come into effect. In locations where M2 and S2 are very small, the phases values can differ considerably. CHK_DATAMAX 20.0 The maximum value for the data included in the analysis (data flagged as bad is not included). CHK_DATAMIN 20.0 The minimum value for the data included in the analysis (data flagged as bad is not included). CHK_DATAMAXMINDIFF 16.0 The maximum value allowed for the difference between the largest and smallest data values included in the analysis. CHK_MAXAMPL_FACTOR 0.6 The maximum value amplitude that any harmonic constituent could possibly have as a factor of the difference between the largest and smallest values in the data. For example if the largest data value is 6.5 and the smallest is 1.1, the difference is 5.4, and therefore the maximum amplitude of any of the harmonics will be set as = Sometimes requesting too many harmonics from a short period of data can lead to excessively large amplitude harmonics which will trigger this warning. Page 58

60 Daily and Monthly Means When carrying out an analysis, there is the option to compute daily, monthly and annual means of the observation data. As well as being useful for quality assurance of the data (identifying shifts in datum for example), they are a useful statistic in their own right. Organisations exist that bank such data as part of the global monitoring programme for changes in mean sea level. One such organisation is the Permanent Service for Mean Sea Level (see the box for more information). Means are computed in the same units as the original tide gauge data, and by convention are normally carried out centred on 12:00 UTC/GMT. Therefore the software will ask you if the original data is in UTC/GMT before proceeding to compute the daily and monthly means. If you answer no, then the calculation of daily and monthly means is skipped. The Permanent Service for Mean Sea Level Established in 1933, the Permanent Service for Mean Sea Level (PSMSL) has been responsible for the collection, publication, analysis and interpretation of sea level data from the global network of tide gauges. It is based in Liverpool at the National Oceanography Centre (NOC), which is a component of the UK Natural Environment Research Council (NERC). The PSMSL is a member of the Federation of Astronomical and Geophysical Data Analysis Services (FAGS), which is currently restructuring under the World Data System. Funding for the PSMSL comes from FAGS, the Intergovernmental Oceanographic Commission (IOC) and NERC. The current director of the PSMSL is Dr. Lesley Rickards. Contact Information Permanent Service for Mean Sea Level National Oceanography Centre 6 Brownlow Street Liverpool L3 5DA United Kingdom Telephone: +44 (0) Fax: +44 (0) [email protected] When daily and monthly means are requested, two additional output files are generated (the DM and MM files). The DM files contain the daily mean values in the following format: 17/08/ /08/ /08/ /08/ /08/ /08/ /08/ /08/ /08/ Any gaps or bad data will usually mean at least two of the days will not have a daily mean value (due to the way daily means are calculated see Advanced Topics section of this user guide). Once the daily means are computed, monthly means can then be derived from a simple arithmetic averaging of the daily means. The format of the file is to first give the monthly means with four values Page 59

61 per line (year, month, mean value and number of daily means used to compute this value). Annual means show the year, mean value, total number of days and how many months this was spread across. Monthly Means ============= Annual Means ============ (in 2 months) The annual means are computed from the monthly means. Each month is weighted by the number of days used in the calculation of the monthly mean (and not the number of days in the month). In the example above, August 2011 has 28 good days of data whereas September has only 12 days. Therefore the annual mean based on those two months is much closer to the value for August since there is was more daily means used in calculating this value. For details of how the daily means are computed, refer to the section in this user guide under Advanced Tidal Analysis Topics. Submission of monthly means to PSMSL The Permanent Service appreciates contributions from all organisations supplying mean sea level data and does not seek to impose unnecessary conditions upon contributors. Nevertheless a minimum level of quality control must be exercised if the data bank is to be an authoritative reference. To this end the PSMSL requests the following information be included with each set of monthly and annual mean sea level values supplied: the units used (metres, rarely feet) a statement of the datum to which the values refer a statement of the measured depth of that datum below the primary tide gauge bench mark an indication of incomplete or deduced data the number of observations per day used to calculate the monthly means any information of changes in datums, bench marks or relevant procedures since the previous batch of data submitted any information on the availability of more frequent readings (e.g. hourly heights) Although data will be accepted in any format, mean heights should preferably be in the metric system to the nearest millimetre, and the datum to which the means refer should preferably be the tide gauge zero. Data will be gratefully received in any form (e.g. as paper tabulations or digital formats). For further information on submitting data, please go to or contact the PSMSL directly at the address in the box above. Page 60

62 Handling Time Zones This section explains how time zones are handled in the analysis and prediction software, and the tools available for working with data in different time zones. When data from a tide gauge is analysed, it will generate a set of harmonic constants with the phase values defined to match the time zone of the original data. For example, if you have data for Calais, France which is in the local time zone (UTC + 1hr) and you carry out a harmonic analysis, it will generate a set of harmonic constants that, if used in POLTIPS 3 to predict the tide levels, will calculate the predictions also in local time zone. Likewise, data in UTC/GMT will lead to predictions in UTC/GMT. If you are using the analysis software without integrating it with the POLTIPS 3 prediction program, you just need to be aware that the results of the analysis are in the same time zone as the data. However if the TASK Analyse software is set to automatically update the POLTIPS 3 database, you will be prompted to specify information about the time zone. This is vital if POLTIPS is to correctly label (and adjust if necessary) the time zones of the predicted data. See page 47 on this user guide for more information on TASK Analyse / POLTIPS 3 integration and time zones. Daylight Saving Time Any data set to be analysed must be in the same time zone throughout the whole time series. If the date/time stamps on the data change to account for daylight saving time, then this must be manually corrected so that it remains in the same time zone. You can use the TASK DMT (Data Manipulation Tool) program to help you do this, or use a third party program such as Microsoft Excel that can carry out calculations on date and time fields. Handling Datums All sea level data recorded by a tide gauge must be given relative to a specific zero level. This is called the datum of the observations. This will usually be decided at the time the tide gauge is installed. For tidal heights, the most common datums are: National datum often defined by a country s national mapping authority Chart datum (CD) Mean sea level (MSL) Lowest Astronomical Tide (LAT) Arbitrarily defined tide gauge zero (TGZ) When you carry out a harmonic analysis, the harmonic constants generated define a series of sine waves that are centred on a zero level. Therefore if the data is not centred on zero also, there will be a difference between zero and the mean of the observations (as shown in the image below). Page 61

63 In this illustration the mean level of the data is 5.16 metres however the harmonics when used to compute the tidal level will always generate a level with a mean value of zero. The difference shown by the purple arrow which also is computed as part of the harmonic analysis is required. This value is called Z 0 and must be added onto the predicted tides to get them to match the observations. A positive value for Z 0 means that the observation data mean value is above the zero level (as in the illustration above where the value of Z 0 is metres). When you carry out an analysis and select to add the analysis into the POLTIPS tidal prediction software the value of Z 0 from the analysis will be used automatically as the primary datum. You can specify an alternative name for this, but the value is as computed during the harmonic analysis process. A secondary datum can also be included which can then be selected once a tidal prediction has been made. Various UK National Datums are built in to the software (called Ordnance Datums) but to can also specify your own datum (name and the height difference). Be careful to specify this value with the correct sign. If the secondary datum is below the primary datum, this number will be positive. If the secondary datum is above the primary datum the number is negative. When secondary datum is selected in POLTIPS, the number here is effectively added to the primary datum prediction (making the datum higher makes the values on the tide table lower). Page 62

64 Data Management and Quality Control Issues TASK Windows Edition The complete process of tidal analysis and prediction involves numerous steps. Therefore it is beneficial to be organised in the way you work through the data. Our general suggestions for managing your data and the analysis process are given in this section. Data Management For each new analysis job, we would suggest creating a complete copy of a clean installation of the files. This prevents you accidentally using a data file from a previous analysis, and allows multiple copies of the data to be kept at each step while minimizing the confusion that can arise from having tens or even hundreds of data files in one location. Beneath the main folder containing the various TASK modules, we suggest you have a folder called OriginalData which stores the raw tide gauge data along with any notes concerning the operation of the tide gauge at the time. Devise an appropriate filenaming convention for your files making use of the file name extension to identify the different types of file. For example: o.txt the raw tide gauge data (although this might be in a non text format such as Microsoft Excel (.xls /.xlsx). o.csv a comma separated file generated by Excel from the raw tide gauge data and ready to be processed by TASK Toolkit. o.t2k a TASK 2000 file which has been generated by TASK Toolkit o HC*.txt files that begin with HC are generated by the TASK Analyse module. The two letter code before the.txt extension indicates the type of file this is. Keep some notes of the stages of the analysis. This can either be hand written notes, or maybe a simple notepad file called AnalysisLog.txt could be created in the main analysis folder. Quality Control A software suite like TASK Windows Edition is essential for carrying out the mathematics of the tidal analysis process, it is not able to provide every tool that you might require in the overall analysis process. A variety of problems can arise with the data (e.g. datum offset changes, timing errors, calibration corrections etc.) and will require some additional software or programming skills to rectify. A tidal analyst with a reasonably high level of knowledge in Microsoft Excel (or an equivalent spreadsheet package) will be able to carry out almost every type of data manipulation required. If you encounter a problem with the data that you are unable to solve, the National Oceanography Centre may be able to advise and assist. Send an e mail with the subject line TASK User Support to [email protected] and we will try and answer any questions you have. If the problem is relating to data, please attach the data file and explain what the problem is. Page 63

65 The Data Manipulation Tool The TASK Data Manipulation Tool (or TASK DMT for short) is a module in the TASK software suite for changing the data and quality flags in a TASK 2000 format file in certain ways. The program can be used in two ways: Launched directly from the TASK Toolkit on completion of data processing. The name of the output file from TASK Toolkit will automatically be passed to TASK DMT. Run as a standalone program the user can browse for any TASK 2000 file to manipulate. Other than the automatic passing of the filename across, there is no difference between the way the program operates in these modes. What is TASK-DMT used for? The program allows you to make changes to all the records ( ) or a subset (a single continuous block) of records in a TASK 2000 file. For the latter, you need to specify the start and end record numbers (the first value on each data line in the TASK 2000 file). If you choose to operate on only a subset of the records, then you have the option to output all the records including those that have not changed ( ) or only the records selected (from Start Rec to End Rec). Electing to output only the selected records will normally retain the same record number (the first value on each line). However for use in the TASK Analyse module, the record numbering must begin at 1. Therefore you may wish to have the program automatically renumber the data records starting at 1 ( ). The program works on one data channel at a time which you must specify. You can carry out more than one type of manipulation at a time by ticking the appropriate active boxes. Specifying the file If you opted to autorun TASK DMT from within TASK Toolkit, the filename should already appear in the filename box. However if you are running it in standalone mode, or wish to process a different file, simply click the browse button and select the file, or drag and drop the file from an Explorer window anywhere onto the TASK DMT window. Page 64

66 Data Value Adjustment This allows the data in the specified channel to be scaled or shifted. Scaling the data multiplies the values in the selected data channel by the value specified. A typical use for this might be to convert data between different units. Since old tidal records may be in feet rather than metres, there is a button to automatically fill in the correct values for this conversion. Shifting the data in one of the channels adds the value specified by the user to the value in the data channel. This might be used to correct a problem with the datum, or even change the datum completely. Both of these combined work like this: New data value = ( Scaling Old data value ) + Shift You can also use the data value adjustment to set one of the channels to a fixed value by setting the scaling value to 0.0 and the shift value to the fixed value required for that channel. A scaling value of 0.0 will cause the program to issue a warning message. Time Adjustment This allows the user to enter a decimal hour value and have the time values shifted by this amount. For example, entering 1.5 will shift the time stamp for each record back 1½ hours (a time of 05:00 for example would become 03:30). Time adjustments are useful for changing the time zone of the data in a file by applying the appropriate time shift to the whole file. For example if you have data in the US Eastern Time Zone (UTC 5hr) and you want this in UTC/GMT, you should enter the value of If you have data that has been corrected for daylight saving time, you can time shift a subset of the records. Cautionary Note: If you have a strict TASK 2000 file, the time interval between adjacent records is fixed throughout the file. Adjusting the time interval of a subset of records can corrupt this fixed spacing. Therefore you may need to rerun the output file back through TASK Toolkit to resample the data back into a continuous fixed interval file. TASK DMT will warn you if you attempt to carry out a time shift with the process all records turned off. Flag Bad Data This option allows you to scan the file for data outside of a specified (acceptable) range of values, and flag this as bad data. It is possible that the data file already contains data which is flagged as bad and it is usually advisable to retain these flags even if the data falls outside the specified acceptable range. To do this, ensure retain existing bad flags is selected ( ). Page 65

67 Another Cautionary Note: If you have both the data value adjustment AND flag bad data active at the same time, the data adjustment is made before the flag check is made. If you want the scan for bad data done on the data prior to the value adjustment, you will need to carry it out in two steps. Splitting Files <May be implemented in a later version of the software> Channel swapping and comparison <May be implemented in a later version of the software> Page 66

68 Using Other Software to Manipulate Data Although TASK Windows Edition comes with various tools to help you manipulate data (TASK Toolkit and TASK DMT), there will be times when they can t do everything necessary to get the data into the appropriate format and you will need to resort to using other means. One option is to write your own software however for this you obviously need to have programming skills and the appropriate development software on your PC. One of the most useful programs to have at your disposal is Microsoft Excel (2007 or later). Not only can this be used to graph the data (by using the supplied Excel template), but it will allow you to manipulate the data in a variety of ways (date and time calculations, simple arithmetic operations on the data), as well as adjusting formats and layout. If you have data that is not in the TASK format, then you need to convert the data into the NOC CSV format (see page 16 for information on this format) before you can process it using TASK Toolkit and create the TASK format file ready for analysis. Manipulate the data into a format that looks something like this (4 columns : date, time, level, quality flag) Page 67

69 You can now save the file as a comma separated values (CSV) file by selecting this option from the Save as type drop down list. The resulting file will look something like this: Date,Time,Level,Flag 01/08/2011,00:00:00,2.69,0 01/08/2011,00:15:00,2.664,0 01/08/2011,00:30:00,2.61,0 01/08/2011,00:45:00,2.527,0 01/08/2011,01:00:00,2.396,0 01/08/2011,01:15:00,2.216,0 01/08/2011,01:30:00,2.012,0 01/08/2011,01:45:00,1.755,0 01/08/2011,02:00:00,1.489,0... This file can now be processed using TASK Toolkit to generate the TASK format file ready for harmonic analysis. Useful Excel Tip #1 If you use Microsoft Excel to manipulate a file that is dependent on spacing rather than comma separation (such as a TASK format file), you should save the file in the Formatted Text (space delimited) format. This has a.prn file extension. Page 68

70 Useful Excel Tip #2 If you work with a large time series, it is often difficult to find a problem in the file. One of the most useful tools in Microsoft Excel is the filter. This will show you all the values found in a particular column. For example in the test data set there are two data values of which are flagged as bad. If they weren t already flagged, it might be tricky to find them. However using the Excel filter tool is a quick way to find unusual values in a column. On the Home ribbon, click on Sort and Filter and select Filter. This will put an arrow at the top of each column (in row 1). Click the arrow next to the column containing the sea height data. You can now see all the possible values that occur in this column. The clearly stands out as unusual and looks like it has been used to indicate bad data so uncheck the (Select All) and check just the box. You will now see just the rows of the spreadsheet that have in column C, and if they weren t already flagged as bad, you could flag them appropriately. Page 69

71 Analysis of Tidal Currents The harmonic analysis of tidal current data is similar to the analysis of sea level data however there is one fundamental difference tidal currents require the analysis of 2 separate components of the current flow. This is usually done by analysing the flow in two perpendicular directions the most common being the east west flow and the north south flow. You will therefore end up with two sets of harmonic constants for each location. You can use these to predict the east west and north south flows separately at any date and time, and recombine them to give a speed and direction. Note: At present there is no facility within TASK to split current speed and direction into east/north components, or recombine them after a prediction of the individual components has been made. If there is sufficient demand, NOC may develop some additional tools to assist in the analysis of currents in the next version of TASK. The POLTIPS 3 tidal prediction software is only designed to work with sea levels (a single set of harmonics). If you want to predict current speed and direction, you will need to have both sets of tidal current harmonics in POLTIPS, run a prediction for both and save the data into a text file. The data can then be combined in a program such as Microsoft Excel where a speed and direction time series can be derived from the separate east west and north south time series. This will need to be done manually NOC will probably develop an additional module to support current predictions in the next version of the software. Page 70

72 Advanced Tidal Analysis Topics How Daily Means Are Computed Means are computed directly from observation data. The process only uses data flagged as good in calculating the mean values. In conventional sea level research, a daily mean is defined to be data low pass filtered to give values corresponding to each day centred on 12:00 noon GMT (not local time). If you intend to submit the data to the PSMSL, you should use the toolkit application to adjust the time zone of the original data into GMT before computing the daily and monthly means. Daily means are computed using a special low pass filter called the Doodson X0 filter the purpose of which is to remove the tidal energy at diurnal and higher frequencies from sea level records. It requires 39 hours of data for each daily mean value to be calculated (i.e. the mid day value ± 19 hours). Because the filter takes data from the two days either side of the day being calculated a single bad data point will often prevent the calculation of a daily mean value on two adjacent days. The filter is defined as: F(t) = (2, 1, 1, 2, 0, 1, 1, 0, 2, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1) for 1 t 19. The filter is symmetric, therefore F(t) = F( t), and is applied for each day giving a mean computed by: 1 30, 0 Where H(t) are the sea level elevations, and T=12:00 hours. Example worked by hand Taking a central time of 1200 hours on the 2 nd January, data from 3 days are involved: 1 January Time Height January Time Height January Time Height January Time Height Summing up all the values in the blue boxes gives a total of m. Page 71

73 Dividing this by 30 gives a mean sea level value for 2 nd January of 3.789m. Monthly and annual means are calculated using a simple arithmetic mean. The number of days used in the calculation will usually be shown along with the computed value. In calculating the annual means, the 12 months are weighted based on the number of daily mean values used to calculate the monthly mean. Data Epochs and Historical Calendar System The TASK Windows Edition package is written to be as flexible as possible. The calculation of the astronomical parameters within the software works back to 1620 however it has not been tested with data prior to 1915 (using the early tidal records from Newlyn in the UK). The programs have not been adapted to accommodate the changes in the calendar systems (from Julian to Gregorian) that occurred at different times throughout the world. See the page on Gregorian Calendar on Wikipedia.org for more information. Sa Phase Calculations in Different Software Each harmonic constituent is defined by its speed. For example the M 2 harmonic constituent is defined as being degrees per hour (i.e. a complete 360 cycle takes which is approximately 12 hours 25 minutes). For most harmonic constituents, the name given to it is unique for example the constituent with the speed of is known as M 2, and no other constituent is given that name. However there are some harmonics that have different speeds but that have been given the same name. The reason that this has been deemed acceptable is that they are so close in terms of their speed that you would need an incredibly long observation time series to separate them at the analysis stage, so therefore only one of the two harmonics can ever be used at any one time. However, although the speeds are the two harmonics are very close, the values for the phase that are derived from the analysis can be very different as the two alternatives will move in and out of phase over a long time period. In the majority of cases, a given name uniquely identifies one and only one harmonic constituent (for example, the name M2 always refers to the harmonic with a speed of /hour. However two different harmonic constituents have been given the name Sa, and software from different organisations are not consistent on which one should be used. The implication of this is that if you generate a set of harmonic constants using the analysis software from one organisation, and use it with the prediction software from another you must check that they both use the same version of Sa. Page 72

74 The two different Sa constituents go in and out of phase only once every 20,942 years (i.e. the phase difference between the two changes by / year. Currently (as of 2012), this phase difference is about 76.9, and each year it reduced by so that in the year 6480, they will be the same. Name Speed Extended Doodson Number Numerical Alphabetical Sa /hr Z ZAZ ZYZ < Used by M.Foreman kernel Sa /hr Z ZAZ ZZZ < Used by NOC / UKHO If you need to use the results of a harmonic analysis derived by TASK Windows Edition in prediction software that uses the first definition in the box above, you will need to manually adjust the phase of Sa. The POLTIPS 3 tidal prediction software used by the National Oceanography Centre (NOC) and included with TASK Windows Edition should only be used with the NOC definition. If you manually adjust Sa, do not then use it in the POLTIPS 3 software which only supports the latter definition of Sa. Table of adjustment to be made to Sa based on the first year of observation data used in the analysis. First year of observation data To convert from TASK Sa phase value to the alternate Sa definition, add the following to the Sa phase value Select the nearest year to the first year of your tide gauge data, then use the value in the second column. For example, if TASK Windows Edition derives an Sa constituent with a phase of (based on an analysis of a year of data from 1986), and you want to use this with software that is known to be compatible with T Tide (which uses the other definition), you would select the phase difference value for the nearest year (1985) which is 77.31, and add this to the phase computed by TASK, giving a new phase value of Page 73

75 Creating your own Harmonic Constant Sets TASK Windows Edition comes configured to carry out analysis for some standard sets of harmonics: 1 harmonic (M2 only) 4 harmonics (O1,K1,M2,S2) [15+ days] 7 harmonics (O1,K1,M2,S2,N2,M4,MS4) [30+ days] 15 harmonics (does not include N2) [15+ days] 26 harmonics [30+ days of data] 54 harmonics [6 months of data] 62 harmonics [1 year of data] 102 harmonics [1 year of high quality data] 114 harmonics [4.5 years of data] If you want to create your own custom set of harmonic constants, you must edit the file included with the software called hcselect.txt. We advise this is used with extreme caution and only if you fully understand the mathematics behind tidal analysis. You cannot pick any combination of harmonic constituents and expect an analysis to work. To separate any given pair of harmonics, you will need a specific length of input data, and if the harmonics are very close in speed it can require a significantly longer data period. Attempting to analyse for too many harmonics or certain combinations of harmonics may at best fail with an error message, but at worst could succeed in producing a set of harmonics constants that generate incorrect data when used outside the period of the original observations. To create a new harmonic configuration in the TASK Analyse Harmonic Constituent Selection dropdown list you need to create a new entry in the hcselect.txt file. Let s look at one of the existing entries to understand for format. This example sets up a configuration for 7 harmonics harmonics (O1,K1,M2,S2,N2,M4,MS4) [30+ days] Z0 0 O K N M S M MS First line contains three numbers: The first is always set to zero, the second is the number of constituents required (add 1 for Z0), and the third as a three character code (can contain any alphanumeric characters) that is used in autogenerating the output filenames. Page 74

76 Since most people don t think of Z 0 as actually being a harmonic constant, the example shown will often be described as a 7 constituent set even though the second number on the first line shows there are 8 including the Z 0. All data sets however must include a Z 0 term after the analysis this will contain the datum offset for the data. Any prediction software will add this value back in and will give you predictions to the same datum as the observations. If you omit the Z 0 term, you will most likely end up with a large residual mean, and the other harmonics generated may not be as accurate. The second line is the description that will appear in the drop down list (a maximum of 78 characters). This is followed by the list of harmonics required (one per line) the name of the constituent followed by its speed. It is advisable to copy the entry for 114 harmonic constants and just delete the ones not required. This will prevent typo errors creeping in when specifying the constituent speeds. The output filenames are in this format: HA_007_000001_008760_GS_01_TS.t2k The number shown in blue is where the 3 character code specified on the first line of the block is written. If you have any concerns or questions about configuring a custom set of harmonic constants, please contact the Marine Data Products Team at the National Oceanography Centre for assistance or advice. Inferred Harmonics Revisited This section of the user guide follows on from the introduction to inferred harmonics given on page 39. The TASK Analyse module in installed pre configured with about 8 different selectable sets of harmonics. Which one the user should select depends on the duration and the quality of the data being analysed. One criterion that is often quoted for selection of harmonics is the Rayleigh criterion 1 which states that only constituents which are separated by at least a complete period over the length of the data should be included in the analysis. For example M 2 has a speed of /hr and S 2 has a speed of /hr. Therefore to separate these two, you need: 360 / ( ) hours = hours = 14.7 days This is not a particularly long span of data and so M 2 and S 2 can usually be separated in most analyses. However S 2 and K 2 are two harmonics that are both required for accurate tidal predictions in most parts of the world, yet require a substantially longer period of observations to extract from the analysis. 1 Some argue that the Rayleigh Criterion is overly restrictive and that when noise in the signal from meteorology and the tide gauge instrument itself are low, you can separate two harmonics with less data than the criterion suggests. However it is a good rule of thumb to follow. Page 75

77 S 2 and K 2 K 2 represents the 27% modulation of the main solar harmonic S 2 due to changes in solar declination over a 6 month cycle. Due to how close K 2 and S 2 are in frequency, at least 6 months of data are required to separate them. If you have significantly less data than this, then you will only be able to independently include S 2. However, missing out K 2 from the analysis will mean that the energy which should be in K 2 will be captured in the resulting S 2 harmonic. As a result the apparent S 2 amplitude could be 27% less than it should be for a June or December analysis, and 27% higher than it should for a March or September analysis. Predicting the tide using the resulting harmonics for dates outside of the data observation period can lead to significant errors potentially in magnitude up to 50% the amplitude of S 2 alone. Related / Inferred Harmonics When there are pairs of harmonics at a port for which both play an important role but which can t be separated using the amount of data available (such as trying to derive S 2 and K 2 from only a month of data), a custom harmonic data set can be created which relates the amplitude and phase lag of the weaker term to the amplitude and phase lag of the stronger term. The weaker term is called an inferred harmonic or related harmonic, and the stronger one the reference harmonic. By specifying this relationship, TASK Analyse can generate both harmonics correctly. It is vital that you do not do the analysis with both constituents specified as standard harmonics as the analysis will either fail or produce very inaccurate values for both constituents. In the custom harmonic data set, you will specify the stronger of the two harmonics as a normal harmonic, and the weaker one as an inferred harmonic. The first step is to find the relationship between the two harmonics you want to include. The most accurate way to achieve this is to look at the relationship of the two constituents for another nearby location, however if this information is not available, the relationships from what is known as the Equilibrium Tide can be used. Either method is considerably more preferable than just leaving out the weaker of the two harmonics constituents altogether. The relationship is expressed using two parameters: α and β (alpha and beta). related amplitude reference amplitude related phase reference phase Example You are required to analyse 1 month of data for Location X which is just a few miles up the coast from Location Y for which there exists a full set of harmonics. Both S 2 and K 2 are important and are required in the analysis results. Begin by looking up the amplitude and phase values of S 2 and K 2 for location Y and calculating α and β for their relationship. Page 76

78 Amplitude S2 Phase of S2 Amplitude of K2 Phase of K2 Location Y 1.14m m S 2 is always the reference harmonic in this pairing (it has the bigger amplitude) and K 2 is the related harmonic (i.e. we are going to infer K 2 from S 2 ). α = 0.32 / 1.14 = β = = 2.7 These values can now be used to create a custom set of harmonics specifically for location X. Table of Equilibrium Tide Relationships The table below shows a set of 8 inferred harmonic constituents which are often used for analysis of very short periods of data on the north west European Shelf however they are valid for many other areas too). Use these relationships if you have no specific data for the region or from a nearby reference location. The set of harmonics included with the software (26 standard + 8 inferred) uses the values in the table below. Equilibrium relationship Inferred harmonic related to... α E β E π 1 (PI 1 ) K P 1 K ψ 1 (PSI 1 ) K ϕ 1 (PHI 1 ) K N 2 N ν 2 (NU 2 ) N T 2 S K 2 S Adding Inferred Harmonics to the HCSelect.txt file The box on the right shows how a set of harmonics would be set up in the HCSelect.txt file (see also Creating your own harmonic constant data sets on page 74). This example shows 26 standard harmonics and 8 inferred harmonics (using the relationships given in the equilibrium table above). Line 1 is the number of inferred, the number of standard (+1 for Z0) and the 3 character code to use in the file naming of the output i 34 harmonics (26 standard, 8 inferred) PI1 K P1 K PSI1 K PHI1 K N2 N NU2 N T2 S K2 S Z0 0 MM MSF Q <other harmonics removed for this document>... files. Including an i in the code is a useful reminder that some of the 35 harmonics were inferred from others. Page 77

79 Line 2 is the text that appears in the TASK Analyse select harmonics drop down list (max 78 characters). Lines 3 10 are the inferred harmonics. This should specify the name of the inferred harmonic, the name of the reference harmonic, the speed of both of these (in degrees per hour) and the values for α and β. If you make any changes to the HCselect.txt file, you need to restart the TASK Analyse program for the changes to take effect and the new set of harmonics to appear in the list. If the relationship specified is specific to a particular location or stretch of coastline it is advisable to put this in the text description on line two (a maximum of 78 characters). For example Location X only 35 harmonics (8 inferred) Amplitude Adjustments TASK can make an amplitude adjustment on the resulting harmonics from an analysis. This was implemented into the software at the specific request of a science researcher, and although probably of little use to most users of the software, has been included in the documentation for completeness. To scale specific harmonics, a text file should be created in the main TASK program folder called AdjustAmplitude.txt and should have the harmonic constant Doodson number followed by the scale factor (see example on the right which shows a file set to make an adjustment to M 2, S 2, O 1 and K 1 ). Only the harmonic constituents that need adjusting should be included any not listed in the file will be output with no adjustment applied Note that adjusting the harmonics is done after the analysis is complete therefore adjusting one will not have any knock on effect on the others. When you run the TASK Analyse module, a check is made for the presence of this file. If it is found, then you will be asked if you want to apply the amplitude adjustments in the file. If you create or edit the file, you must restart TASK Analyse to load the changes. Page 78

80 Appendices Appendix A Wave Terminology Sine Wave The standard smoothly changing wave form that many people recognise is mathematically known as a sine wave and is something that occurs frequently in nature, as it is very closely related to circles and circular motion. Wave period The period of a wave can either be expressed in the amount of time taken to complete a full oscillation, or the fraction of an oscillation that is completed in a given time period. In tidal science, the latter is normally quoted when referring to harmonic constituents. For example the M2 harmonic has a period of /hour. Amplitude The amplitude is a way of measuring the magnitude of the wave it is specified as the height in metres from the centre line to the peak. Page 79

81 Phase Phase is the time difference between two waves. In tidal science the phase of a harmonic constituent is the difference between the harmonic at that specific location, and the wave from the theoretical equilibrium tide. Waves are often described as being in phase or out of phase. The diagram below shows two sets of 5 waves. In each case the period and amplitude of all 5 waves is identical only the phase of the waves differs. The 5 waves are all perfectly in phase. If combined they would create a sine wave with 5 times the amplitude of an individual wave The 5 waves are all out of phase by different amounts. Waves 2 and 3 are said to be completely out of phase. When combining sine waves (as is done when predicting tidal levels from a set of harmonic constants), waves that are in phase reinforce each other waves that are out of phase can often partially or even completely cancel each other out. The two illustrations above each show a set of five sine waves, all of the same frequency. However with tides, waves of different frequencies are being combined. Therefore a wave will be in phase at one point in time and out of phase a certain time later. Combining waves of different frequencies leads to the rather varied, and sometimes complex, tidal patterns that we see throughout the world. In harmonic analysis, the term phase is commonly used to describe the time lag of a constituent relative to the Equilibrium Tide passing the Greenwich Meridian, though strictly speaking, the term phase lag is more appropriate. Sources of additional information about tidal theory (including the concept of the Equilibrium Tide) and about tidal analysis and prediction methods are detailed in the Further Reading section of this guide. Page 80

82 Appendix B - Glossary of Tidal Terms Chart Datum Data Logger Datum Diurnal Tides UTC / GMT The level to which depths on a nautical chart are measured. It is the most common datum for tidal predictions. A device which is connected to a tide gauge and records the tidal level at a fixed interval. A level which is defined as having the value zero for the time series. All heights must be given to a specific datum (often Chart Datum). Tides having a period of approximately 1 day. Coordinated Universal Time and Greenwich Mean Time are basically the same. UTC is the primary time standard by which the world regulated clocks and time. For most purposes, UTC is synonymous with GMT but GMT is no longer precisely defined by the scientific community. Extreme High or Low Water The highest or lowest elevation reached by the sea during a given period. Harbour Datum Harmonic Analysis Harmonic Constant Harmonic Constituent Highest Astronomical Tide Long Period Tides Lowest Astronomical Tide Mean Sea Level The horizontal plane, defined by the local harbour authority, from which levels and tidal heights are measured by that authority. The process of mathematically identifying a series of sine waves of varying amplitude, phase and speed which best defines the period of observations being analysed. See page 10 for more information. A pair of values (amplitude and phase lag) that defines a specific harmonic constituent for a given location. Every location will have a unique set of harmonic constants. See page 9 for more information. One of the individual sine waves that the process of harmonic analysis attempts to identify within a tide gauge record. The highest level which can be predicted under average meteorological conditions. This is not the same as the extreme high water as storm surges may cause considerably higher levels to occur. Individual tidal constituents that have a period of longer than 1 week. The lowest level which can be predicted under average meteorological conditions. This is not the same as the extreme low water as negative storm surges may cause considerably lower levels to occur. The average level of the tide calculated over a long period of time. Page 81

83 Mixed Tide National Datum Neap Tide Oscillation Seiche Semi diurnal Tides Spring Tide Surge Tide Gauge Tide Gauge Bench Mark Tide Gauge Zero Tides that are neither totally semidiurnal or diurnal, however one or the other is usually predominant. A fixed reference adopted by a country as a standard geodetic datum for elevations determined by levelling. A lower than average high tide (or higher than average low tide) occurring around the time of first and third quarter of the Moons phase. A regular and often fixed periodic change of a parameter between two extreme values. A sine wave is an example of an oscillating mathematical function. A short period oscillation occurring in a harbour, bay or gulf and unrelated to actual tides. Tides having a period of approximately half a day. A higher than average high tide (or lower than average low tide) occurring around the time of full moon and new moon. One of the components of sea level caused by meoteorological effects such as prolonged winds, changes in atmospheric pressure etc. Surges are not easily predicted, requiring accurate weather forecasts and complex computer models of the seas and oceans. Tide tables do not included predictions of the surge, and explains why they may differ from the observed sea level. A device used to measure the level of the water at a specific location. The gauge is normally installed on a rigid stable structure, and connected to a data logger that will record the level at a fixed time interval (usually a minimum of hourly, but often more frequently). A fixed vertical point near a gauge used to check levelling of the gauge. Repeated levelling from the tide gauge bench mark to other bench marks in the vicinity check the stability of the tide gauge installation with respect to the level of the land. The datum which corresponds to a reading of 0.0 on a tide gauge. When a harmonic analysis is carried out, the Z0 constituent gives the height of this datum relative to the mean level. Sometimes called Tide Gauge Datum. Page 82

84 Appendix C1 A Table of Standard Harmonics All harmonics listed below are supported by the TASK Analyse program. No. Name No. Name No. Name No. Name 1 Sa 30 OP SM MN 8 2 Ssa 31 M 2 60 MSK 6 89 M 8 3 Mm 32 MKS MN 2 S MSN 8 4 Msf M(SK) MS 8 5 Mf 34 L M 2 S MK 8 6 2Q 1 35 T 2 64 MNK 2 S 2 93 MSNK (sigma) 36 S 2 65 SNK (MS) 8 8 Q 1 37 R SK MSK (rho) 38 K MS 2 N MS O 1 39 MSN 2 68 MQ M2S MP 1 40 KJ MP MSN M SM MQ MS (chi) 42 MO MK M 2 S (pi) 43 M MS MVS 2 15 P 1 44 SO MSK MK 2 16 S 1 45 MK MK MA 2 17 K 1 46 SK 3 75 M MB (psi) 47 MN MO MSV (phi) 48 M (MN)S SKM (theta) 49 SN MNS MNS 4 21 J 1 50 MS MK MV 4 22 SO 1 51 MK MS MN 4 23 OO 1 52 S MSNK MSN 4 24 OQ 2 53 SK MV NA 2 25 MNS MN MSK NB N 2 55 M MN MSO (mu) 56 MSN MSN MSK 5 28 N MS 6 86 MKL (nu) 58 2MK (MN) 8 Some harmonic constants have alternative names. For a complete list of all harmonic constants, please contact NOC who can provide you with a document listing all possible harmonics. However any additional ones in the list not shown above are not currently supported by TASK. Page 83

85 Appendix C2 Table of Standard Harmonics with Speed No. Name Speed ( /sec) No. Name Speed ( /sec) No. Name Speed ( /sec) 1 Sa MSN (MN)S Ssa KJ MNS Mm SM MK Msf MO MS Mf M MSNK Q SO MV (sigma) MK MSK Q SK MN (rho) MN MSN O M MKL MP SN (MN) M MS MN (chi) MK M (pi) S MSN P SK MS S MN MK K M MSNK (psi) MSN (MS) (phi) MS MSK (theta) MK MS J SM M2S SO MSK MSN OO MN 2 S MS OQ M(SK) M 2 S MNS M 2 S MVS N MNK 2 S MK (mu) SNK MA N SK MB (nu) MS 2 N MSV OP MQ SKM M MP MNS MKS MQ MV MK MN L MS MSN T MSK NA S MK NB R M MSO K MO MSK Note that the harmonic constituent Sa has two different definitions (and therefore speeds). See the section on page 72 for more information. Page 84

86 Appendix C3 Day Numbers This table is useful for converting the day numbers (used in the TASK file format). Common Year (non leap year) Jan Feb March April May June July August Sept Oct Nov Dec Page 85

87 Leap Year Jan Feb March April May June July August Sept Oct Nov Dec Page 86

88 Appendix D Additional Reading Material Each of the books listed below is given a rating to indicate the complexity level. This book is recommended for anyone working in tidal analysis. It provides a good overview of the subject whilst keeping the mathematics to a minimum. This book goes into more detail and is recommended for those with a reasonably strong mathematical background. This is a higher level text which should be used by those wanting a comprehensive and extensive understanding of tidal analysis. Book List This User Guide. It may be obvious but this user guide should be read thoroughly before attempting to use the TASK Windows Edition software to carry out any tidal analysis work. The IOC Manuals (volumes I to IV) on Sea Level Measurement and Interpretation. These manuals can be downloaded at no charge from All manuals are in English, although Volume 1 is also available in French, Russian, Portugese and Spanish, and Volume 4 is available in Arabic. Volume 1 (pub: 1985): Basic Procedures Volume 2 (pub: 1994): Emerging Technologies Volume 3 (pub: 2000): Reappraisals & Recommendations as of year 2000 Volume 4 (pub: 2006): An update to 2006 Glossary of Terms used in Tidal Measurement and Analysis Changing Sea Levels: Effects of Tides, Weather and Climate David Pugh, This is an excellent book written by a world renowned authority on Tides. Tides, Surges and Mean Sea level: A Handbook for Engineers and Scientists David Pugh, To many scientists, this book is the standard reference for all things relating to tides and sea level. It s not the easiest read, but it is thorough and covers almost every aspect of tides. The Analysis of Tides Gabriel Godin, Liverpool University Press. Although published in 1972, this book is still one of the most detailed texts on tidal analysis. Do not attempt to read this book unless you have a very strong mathematical background. Page 87

89 Appendix E Using the Microsoft Excel Graph Template TASK Windows Edition The Excel Graph template was developed under Microsoft Excel It has not been tested or used on any earlier versions of Excel. Enabling Macros To use the Microsoft Excel Graph Template provided with TASK Windows Edition, you must have macros enabled within Excel. If you do not, you will get a warning message. The notes below show you how to enable the macros on Microsoft Excel Start up Excel and click on the large round Office button in the top left corner of the window. Select Excel Options. Select Trust Centre down the left hand side, then click on the button Trust Centre Settings 2. Select Macro Settings on the left and click on the option to Enable all macros. The template should now run correctly. Page 88

90 General Troubleshooting With Microsoft Office products such as Word and Excel, double clicking on a template file (such as a.dotx or.xltm file) should create a new blank document based on the template rather than actually opening the template for editing. The TASK Graph module is supplied as an Excel template file and so double clicking the file should give you a new blank document based on this template (i.e. the text in the title bar of the program should look like the image on the left and not the one on the right). left: Task Graph1 indicates a new document which has not been saved ; right: Task Graph.xltm means the template is open for editing and if you save and changes, you overwrite the supplied template. If you get the template loaded for editing, then it indicates the behaviour of the file association has been changed from the default Microsoft behaviour either manually or most likely by the installation of the Microsoft Office Compatibility Pack. If you use Windows XP, you can change this back to be able to use the TASK Graph module. Windows XP users should follow these steps carefully: 1. Load up an Explorer windows, go to the Tools menu and select Folder Options. Select the File Types tab and find the extension XLTM in the list. 2. Click on the Advanced button and you should see the dialog box shown on the right. If the option open is shown in bold, it means that double clicking on the template will open it for editing rather than creating a new document. Change this to New and click Set Default. 3. Now click on the Edit button and you should see the dialog shown on the right. The Application used to perform action should be set to something like this (ensure double quotes are in the right place too): "C:\Program Files\Microsoft Office\Office12\EXCEL.EXE" /e /n "%1" The location may be slightly different depending on where your Microsoft Office is installed. However there must be /e /n %1 at the end. Also ensure that the Use DDE is unchecked, or the DDE Message is blank. Page 89

91 Windows Vista / Windows 7 / Windows 8 users : Microsoft has unfortunately removed the graphical user interface to be able to change and configure these settings it can now only be done through the Windows Registry which is not recommended. So if you have installed something like the Microsoft Compatibility Pack that alters the behaviour when double clicking a template file you will either need to reinstall Microsoft Office, or manually start Excel by typing the following command directly into the Search Programs and Files edit box on the start button (replacing the pathname in quotes to the full location of the TASK Graph template: excel /t "C:\Users\Joe Bloggs\TASK\Task Graph.xltm" Note that the /t is required to force Excel to treat the specified file as a template (omitting this will open the template in Excel for editing which is not what you require). Also make sure you use double quotes around the full pathname for the file otherwise any spaces in the path will cause it to fail. Page 90

92 Appendix F Updating Your Dongle If you are using the commercial version of TASK, then it is likely that you have a USB dongle. This needs to be inserted into a USB port on your machine for the TASK Analyse software to work. If the software cannot find the dongle, you will get a message asking you to insert it. When you obtain the TASK software for commercial use, it is purchased on an annual subscription basis, and the software will expire at the end of the subscription period. However you can update the expiry date (which is stored inside the dongle) by contacting the Marine Data Products Team at the National Oceanography Centre to renew your subscription and obtain an update code. You can renew your subscription before the current one has expired and we will ensure that a full year is added to the current expiry date and not from the date of purchase. Extending your subscription / updating the dongle 1. Contact the Marine Data Products Team at the NOC to extend your TASK subscription. If you have more than one dongle to permit running the software on multiple machines, let the TASK support specialist know if you require just one or all of the dongles updating. 2. Once the licence renewal has been accepted, we will send you an e mail with an attachment. Save this attachment to your local PC (the desktop is probably the most convenient). 3. Ensure the dongle is inserted into a USB port on your computer and run the software in the TASK group on your Start Button called DongleUpdate. You should see the dialog box shown on the right. 4. Go to File > Open Update Code File, and select the file that you were sent by NOC. Then click on the button Make changes to dongle. The dongle should update successfully. If you have any problems updating your dongle, then please e mail the Marine Data Products Team at NOC (with the subject line: TASK dongle update ). In the e mail, include the Product Code, Dongle Number and Update Number. All three pieces of information are available by inserting the dongle into a USB port and running the DongleUpdate software. Page 91