Operationalizing Open-Source Electronic Monitoring Systems in New England Groundfish Sectors: Year 2 Public Report

Transcription

1 2015 Operationalizing Open-Source Electronic Monitoring Systems in New England Groundfish Sectors: Year 2 Public Report Marine Monitoring Initiative Ecotrust Canada 4/21/2015 1

2 CONTENTS Commonly used Acronyms Introduction Project Description and Rationale Project Objectives Year Two Overview Methods Data Collection Data Handling Data Analysis Vessel responsibilities Results Non-video Data Assets Video Assets System Reliability Data Quality Comparisons between Logbook and ASM/Observer and EM Data EM vs. Logbook EM vs. Logbook vs. Observer Data Agreement Discussion and Conclusions References Appendix 1: Example Vessel Monitoring Plan Appendix 2: Visualizations of Other EM Data Capture Appendix 3: Fixed and Variable Costs Associated with EM.37 2

3 Commonly used Acronyms ACE Annual Catch Entitlement ACLs Annual Catch Limits ASM At-Sea Monitor EC Ecotrust Canada EM Electronic Monitoring Fps Frames per second FY Fishing Year GMRI Gulf of Maine Research Institute MCCS Maine Coast Community Sector NEFOP Northeast Fishery Observer Program NEFSC Northeast Fisheries Science Center NMFS National Marine Fisheries Service NOAA National Oceanic and Atmospheric Administration TNC The Nature Conservancy 3

4 1 Introduction 1.1 Project Description and Rationale Through a collaboration between Ecotrust Canada, Gulf of Maine Research Institute (GMRI), The Nature Conservancy (TNC) and Maine Coast Community Sector (MCCS), an innovative and cost effective Electronic Monitoring (EM) system is being developed and operationalized in the New England groundfish fishery that aims to meet federal performance standards, currently under development, and collect certain data sets comparable to the At-Sea Monitoring (ASM) program of the National Marine Fisheries Service (NMFS). In Year 1 (Fishing Year [FY] 2013), Ecotrust Canada developed EM technology with open source data collection software for boats using gillnet and trawl gear and installed EM systems on two MCCS vessels, one of each gear type. Systems were installed in 2013 and a server was set up at GMRI for data downloading and analysis. In Year 2, (FY 2014) the system was deployed on seven active MCCS vessels. In Year 2, the system was upgraded to be able to capture and create an electronic log of vessel activity using a National Marine Fisheries Service (NMFS) approved electronic reporting software program developed by Ecotrust Canada and installed on EM systems. The New England groundfish fleet will soon begin facing significantly increased costs for monitoring when NMFS starts to transition ASM costs to the industry. The new requirements are likely to be phased in, with the possibility for the industry to pay a percentage of the overall ASM costs as early as September of FY These costs are being transferred from NMFS to harvesters over the same period as the industry is facing reductions in annual catch limits (ACLs) of over 70% for some stocks. In order to maintain the economic viability of groundfish fishing enterprises, while at the same time delivering the required catch monitoring data to fisheries managers, innovation in monitoring program delivery is required. EM systems, either as a replacement for human observers or in conjunction with them, could offer a way to reduce the costs associated with ASM without compromising data quality or integrity. Cost savings are dependent on regulatory requirements, the appropriateness of existing EM technology for those requirements and overall EM program design. In addition, video footage of catch and discards is collected on 100% of the trips using EM, which is an improvement from the current combined coverage level of 25% 1, and opens up multiple opportunities to increase the use of fishery-dependent data for science and management. Finally, the funding for the EM pilot study conducted through the Northeast Fisheries Science Center (NEFSC) by Archipelago Marine Research was shifted to cover ASM costs in FY 2014, and thus the project ended in This study will not replicate the pilot, but will build on their lessons learned and operationally test methods introduced in this work. During Year 1 the EM system included all the necessary equipment for collecting video, vessel tracking, and hydraulic sensor data. In Year 2 the system included an additional camera. In Year 1 Ecotrust Canada developed and began testing an electronic logbook (elog) system in order to be able to test the feasibility of integrating EM data with haul-by-haul fishing log data in Year 2. The Ecotrust Canada elog system was certified by NMFS for use in the NE groundfish fishery in FY 2014 and was planned to be 1 The 25% is derived from 8% coverage under the Northeast Fishery Observer Program (NEFOP) and 17% under the groundfish sector at-sea monitoring program. In March 2015, NMFS announced the combined coverage rate for FY15 is 24%. 4

5 integrated into the EM system of one vessel during Year 2 of the pilot. However, since testing of this software took longer than anticipated, no data was captured using the elog in FY2014. One unforeseen challenge was that the small screen and portable vessel keyboards (initially installed for us with EM systems without integrated elogs and therefore intended for simple navigation and viewing of EM system status only) were not ideal for onboard data entry into an elog. Future systems that have active elogs installed will come with a regular sized monitor for easier inputting and existing systems will be retrofitted with same. Use of elogs should be fully functional in FY2015 and data delivery will be fully field tested and implemented. One expected outcome of this project was to outfit MCCS vessels with voluntary EM systems that could be utilized to capture certain data sets comparable to those collected by the ASM program. Upon completion of this project, the vessels may be able to keep the hardware installed on their boats at the discretion of the project partners. While NMFS may only approve the use of EM for certain gears and types of fishing behavior (i.e., single stock area fishing trips or for day trips), and no approval will come before FY 2016, we believe this is a step towards facilitating the implementation of EM in this region. Further, as there are currently only a few operational EM providers operating in North America, Ecotrust Canada, through this pilot work and the partnerships involves, endeavors to provide fishermen with an effective and affordable option for monitoring service provision that draws on local fisheries expertise and technical capacity. A second, longer-term outcome is that the software and hardware developed for the MCCS vessels can be expanded in the future for other vessels in the groundfish fleet, both for different fishing operations and for the varying species catch composition in the broad stocks areas within the fishery. 1.2 Year Two Project Objectives To develop a cost effective EM system using open source technology that would meet federal technical specifications and collects certain data sets comparable to those collected by the ASM program of NMFS. To test open source EM technology on seven active sector vessels of two different gear types. To compare data collected by EM systems with those self-reported by MCCS fishermen and with those collected by the NMFS Observer and ASM programs. To further the work commissioned by NMFS in EM weight estimation and species identification by applying length to weight relationships in an operational environment. To transfer knowledge through partnerships and to increase the capacity for the support of EM programs in Maine. 1.3 Year Two Overview In Year 1, EM systems were installed on one trawl vessel and one gillnet vessel. In Year 2, seven vessels took part in the project, five gillnet vessels and two trawl vessels. The gillnet vessel from FY 2013 continued in FY 2014, but the trawl vessel did not. In FY 2014, two Ecotrust Canada technicians made a trip to Portland, Maine to install EM systems on four gillnet vessels and two trawl vessels. The fifth gillnet vessel had an EM system from FY Ecotrust Canada technicians upgraded and repaired this system so that it was up to Year 2 standards. Additional server hardware was set up at GMRI in 2014 for video archives. A member of GMRI staff helped with the system installs and was retrained by EC on how 5

6 to collect and download EM data for analysis, thus building increased capacity in New England to support EM programs. EM data was collected from April 2014 to February All video downloads were first screened to assess usability, and a feedback letter was provided to GMRI and the harvester seven days after the date upload that outlined any issues that may have come up during a trip. Next, random hauls were selected from usable video and were analyzed to determine the species and weight of discarded ACE species. For both trawl and gillnet vessels, length/weight ratios were used to determine weight, and the total estimated weights of all ACE discarded species for those hauls were compared to fisherman s log and, where applicable, to data collected by Northeast Fisheries Observer Program (NEFOP) observers and at-sea monitors. The review level for this pilot was 10% of ALL 2 hauls from each trip, with a minimum of one haul per trip being randomly selected and reviewed within three weeks of the data capture. If more than one haul was required to be fully reviewed in order to reach the 10% review level, additional hauls would be selected and reviewed. Hauls that had coverage by an at-sea monitor or NEFOP observer coverage were selected in order to be able to comparisons with a third source of data. In Year 1 we developed and began testing an electronic logbook (elog) system in order to be able to test the feasibility of integrating EM data with haul-by-haul fishing log data in Year 2. This elog was integrated into the EM system in Year 2, although for several reasons, one mentioned above, we were not able to test elog on any active vessels in FY We plan to make elog a requirement for most participating vessels in Year 3/FY Methods 2.1 Data Collection EM System Description In Year 2 EM systems were employed on five gillnetters and two trawlers. During Year 2 the EM systems hardware components were: Central control box o removable data drive o Uninterrupted power supply (UPS) Keyboard & monitor GPS (See Figure 1) Hydraulic pressure sensor Three digital (IP) cameras One powered IP switch 2 If some hauls in a trip had unusable video data and they could not be reviewed, they were taken out of the pool of hauls that could be randomly selected for review but they were still counted in the total hauls made in that trip, i.e. if 30 hauls were made in a trip but only 20 hauls had useable video, 3 hauls randomly selected from the 20 needed to be reviewed to meet our 10% and not just 2 hauls. 6

7 Figure 1: Schematic showing EM system. Drum rotation sensors were not used in Year 1 or 2. Two cameras were used during Year 1. One camera was positioned to capture an overview of the deck activity and fish handling. The second camera focused on discards for the purpose of species identification. The video capture rate was frames per second (fps), allowing for multiple views of fish. In Year 2 a third camera was installed on each vessel to capture an outboard view; this allowed for video capture of catch that drops out of the fishing gear or is handled and discarded at the rail and is never brought aboard. Due to video storage concerns, in Year 2 systems were upgraded to only record fishing activity at high frame-rates, all other activity was recorded at reduced frame-rates. High framerate capture started when hydraulics reached a certain threshold, and frame-rates were reduced 2 hours after the last detected fishing activity. High frame rate video data takes up a lot of room on a hard drive. Hard drives do not have the capacity to carry data at high frame rates for the data collection period average 2 week time period, so the low frame rate during non-fishing times was required. In Year 2 the trawl vessel s camera focused on the measuring strip on the edge of the side of the vessel, and deckhands would pass each ACE discard over the measuring strip before throwing it overboard (see Figure 2). The other two cameras were focused on the entire deck and the discard area, respectively (see Figures 3 and 4). On gillnet vessels, the measuring strip was adhered to the sorting tray, and as such the camera was focused on that area (see Figure 5). As with trawlers, the other two cameras on gillnet vessels were focused on the entire deck and the discard area. Each individual vessel was set up to have cameras in the optimum location to record fishing activity and fish handling and these specifications were captured in individual vessel monitoring plans (VMPs) (see Appendix 1). 7

8 Figure 2: View of the camera angle used for species identification on the trawl vessel Figure 3. Camera view of the entire deck on a trawl vessel 8

9 Figure 4. Camera view of the general discard area on a trawl vessel Figure 5: View of the camera angle used for species identification on the gillnet vessel 9

10 EM Data Collection In Year 2, EM data was collected in a similar manner to Year 1, and according to the following software configuration: Data was logged from the GPS and hydraulic pressure sensor every second while system was powered. Video recording triggered when the vessel leaves Port. The video from all cameras was captured at 15 frames per second. Project logbook data was captured by participating captains on a haul-by-haul basis using a paper log. The EM system s integrated elog was tested near the end of Year Data Handling EM system hard drives store up to 4 6 weeks of data. Hard drives were retrieved on a weekly basis by a local GMRI technician. The data from each hard drive was copied, archived and then uploaded to a server and accessed securely over the Internet for analysis. When logbook data was not entered directly into the elog or collected by ASM/NEFOP programs it was converted into an appropriate format for analysis. At the end of the project, all raw project data will be destroyed except for the raw data released by the project participants (e.g., sample video clips approved by the vessel owner). 2.3 Data Analysis Sensor Data Analysis 100% of the sensor data was automatically analysed to identify hauls. Hauls identified in the EM data were automatically aligned with hauls reported in the logbook and, when available, hauls identified by the ASM/NEFOP programs. An analyst verified the data alignment. Video Analysis 10% of all hauls were required to be reviewed in Year % of the gillnet hauls and 18.6% of the trawl hauls were reviewed. o For each haul reviewed the following was recorded: vessel name length per fish of discarded ACE managed species piece counts of discarded ACE managed species video date video name Fish lengths were converted to weights using accepted weight to length ratios by species. Video Analysis Feedback Loop In general, the Ecotrust Canada tech team reviewed downloaded data and technical summaries, and provided feedback on technical issues to all participating captains within seven days of upload of each batch of video footage. A more detailed analysis of species IDs and discard weight estimates with a comparison to captain s vessel trip reports, and ASM data if applicable, were to be provided to each participating fisherman no later than three weeks from the upload date of each batch of data. This was not always possible due to a number of outside factors, including high frequency of video uploads during 10

11 the busy season, but any urgent issues were discussed in the feedback letters and were dealt with promptly. Video Review Methodology Video reviewers estimated lengths of individual fish by measuring the fish in centimeters using a measuring strip that was adhered to the tray (gillnet) or the edge on the side of the boat (trawl). Reviewer lengths were converted to weights using NEFSC length-weight relationships from Wigley et al. (2003). This methodology is consistent with the work commissioned by NMFS in EM weight estimation and species identification (Pria et al. 2012) and allows us to test this methodology in an operational instead of experimental setting. Those fish identified only to a species group could not be included in the weight comparison as it was not possible to determine which specific species weight-length relationship to apply. Some species can only be identified using identifying features not captured by video (e.g. gillraker counts for some hake species) and therefore can only be identified to a species group. Other times, condition of fish and/or fish handling did not allow for a clear enough image for identification to species. Methods to account for these issues and/or avoid them altogether are being developed for FY2015. Data Comparison We compared the following data between EM and logbook and the ASM/Observer program when available: the date, time and location of each haul and ACE managed discarded species weights. ACE Regulated Species: Atlantic cod (Gadus morhua) Haddock (Melanogrammus aeglefinus) Pollock (Pollachius virens) Redfish (Sebastes spp) White hake (Urophycis tenuis) American plaice flounder (Hippoglossoides platessoides) Winter flounder (Pseudopleuronectes americanus) Witch flounder (Glyptocephalus cynoglossus) Yellowtail flounder (Limanda ferruginea) Other fish species involved in project: Monkfish (Lophius spp) 2.4 Vessel responsibilities The deck was required to be well lit during fishing activity. Operators were required to monitor the system to make sure that it was functioning during fishing (screen with camera display and sensor indicators was installed in the wheelhouse, to ensure pictures were clear and all sensors showed a green light). Operators were also responsible for keeping cameras clean and unobstructed, and were to report any system problems immediately. Since audits are most effective by comparing EM data and selfreported data on a tow-by-tow level, operators also recorded paper logs on a tow-by-tow level, for later 11

12 comparison to EM data. During installation an individual VMP is developed for that vessel. The VMP includes vessel and fishery specific information; the objectives for the monitoring program as well as the data capture objectives, and the vessel responsibilities to help achieve these objectives. Vessel responsibilities and procedures are broken down into fishing gear deployment details; catch handling details and camera system handling details. The VMP also includes the type, location and rationale of all installed sensors, including photographs of installation locations. For the cameras, images of areas of focus/views are also included. Finally, the VMPs include a section where technicians can record any work done or changes to the initial set-up as means to create a vessel specific log of system functionality. A blank VMP is attached to this document as Appendix 1. Fish Handling Procedures All ACE discards were required to be handled in view of the discard camera. ACE managed species to be discarded were to be passed over a measurement strip. The measurement strip was adhered to tray (gillnet) or on the edge on the side of the boat (trawl). All non-compliant catch was to be discarded and returned to the sea as soon as practicable. 3 Results 3.1 Non-video Data Assets Vessel position data, pressure logs, and events of fishing activity were recorded during 44 data collection periods. The completeness of this dataset is shown in Appendix Video Assets The resolution of technical and workflow issues in Year 1, and in particular the implementation of a tight feedback loop for data quality control, saw a much-improved data capture record in Year 2. While the video assets weren t perfect, the improved process allowed issues to be addressed quickly and no single issue instance persisted for a substantial portion of the fishing season on any one vessel or across the entire pilot fleet. Some issues were recurrent throughout the season, and although none were critical, they may merit some action taken for their prevention in subsequent years. Issues that recurred intermittently included: Defective or mis-calibrated pressure sensor leading to extra video review requirement to discover fishing activity Mismatch between logbooks and apparent fishing activity as detected by the control box. Video camera restarting and truncating videos (software was patched early in the season to prevent this from occurring through the year) Incorrect or changed vessel name or hull numbers, leading to mis-categorized data sets. 12

13 3.3 System Reliability System reliability is measured by number of times the system shut off unexpectedly during fishing activity. When the system is shut down for any length of time, the date, time and location (using GPS coordinates) of both when the system turned off and when the system turned back on, are automatically recorded and are highlighted by the analysis software when data is collected. These downtimes can then be assessed to determine whether or not they are indicative of major system errors, major compliance events or are minor system reboots that do not require any follow-up action. Downtimes, which are only a few minutes in length, and while at shore, are likely a relatively harmless system reboot often caused by a switching from vessel power to shore power. These short reboots while in port are not a major cause for concern regarding either system functionality or for compliance reasons. Repeated reboots at sea, even if only for short periods of time, could however be a cause for concern with regard to both compliance and system robustness. There were also some longer periods of downtime than can be caused by a reboot. System downtime longer than a determined threshold may be considered a compliance issue because it could potentially contain unmonitored fishing activity. These could be caused by the system freezing up without notice due to power supply issues or issues with sensor inputs. Any data loss due to these issues can be minimized by ensuring the monitor is visible in the wheelhouse to promote early detection, and with prompt reporting of problems and repair or replacement of equipment as needed. The following information is given to demonstrate the low number of system restarts: During an 11 day trip at the start of July, 30 hours of video of was recorded and the EM system did not restart or shut-off at any time. During a 12-day trip at the end July, 59 hours of video was recorded and the EM system restarted once. 3.4 Data Quality In Year 1, the focus was on obtaining high quality video that could be used to identify fish to the species level and would further the work commissioned by NMFS in EM weight estimation and species identification by applying length to weight relationships in an operational environment. While our Year 1 results proved that the video captured could be used to accurately identify fish to species and get length estimates on individual fishes, the total amount of usable video was too low in order to make any statistically sound comparisons between data collected by EM systems with those self-reported by MCCS fishermen and with those collected by the NEFOP and ASM programs. In Year 2, an emphasis was made on reviewing video in greater detail early in the fishing season allowing for faster iterative correction of video quality problems in order to capture a large enough number of hauls on video in order to make significant comparisons. Examples of the video quality problems that were addressed and corrected for in Year 2 are: Ensuring video from both cameras is present during fishing activity. Ensuring a usable scale reference object is kept in frame. Testing camera angles are conducive to species identification. 13

14 3.5 Comparisons between Logbook and ASM/Observer and EM Data EM vs. Logbook The graphs below use data from all participating vessels 3. The graphs are April to December aggregations of discards only. These graphs show that some species have good comparisons across vessels, gear types and time of year, while others do not. This information is helpful in determining where discrepancies between data sources come from. Figure 6: Comparison of All Discarded Weights Recorded Between April and December from All Vessels (Logbook Data vs. Video Review Data) 3. Sets of data from two hauls of one vessel data were excluded because the video data did not differentiate between discarded and retained species due to fish handling occurring outside of the field of view. The video was analyzed but not included in the data set. 14

15 Figure 7: Comparison of All Discarded Piece Counts Recorded Between April and December from All Vessels (Logbook Data vs. Video Review Data) The graph of piece counts offers insight into the sources of error for the graph of discarded weights. In video review, weights are calculated from the measured length of each fish. If a fish is damaged or the reviewer is otherwise unable to determine the length of a fish, the calculated weight for video may be inaccurate or nonexistent. Pollock is a good example: the difference between video and logbook data is proportionally smaller for piece counts than it is for weight. There are several possible reasons why there are these differences: it could be because of ability to estimates difference species weight, in general the flatfish species are closer in weights and pieces counts than round fish. These two types of fish are often handled in different ways by crew and stored in different containers on the vessel (totes v. baskets for example). A basket of flounder would be more uniform than a tote of cod and therefore estimates for flatfish could be more consistently accurate than those of groundfish. Other sources of error could be that round fish are gutted where flatfish are not and this could lead to discrepancies in the way they are recorded. There are also error sources from video review data; these include misidentification of species and the inability to determine if fish are discarded or not. Finally, since weight estimates from EM video are calculated from a combination of piece counts and fish lengths using a species-specific estimation formula; any of the three may produce discrepancies between data sources, particularly with very large hauls/values needing to be converted. EM vs. Logbook vs. Observer Not every trip had an at-sea monitor or NEFOP observer in attendance. Of the 57 trips we analyzed, 8 (14%) also carried either a monitor or an observer. The following graphs compare results from each of the three sources. All Observed trips from all vessels are aggregated. Note that Observer data does not provide piece counts. 15

16 Figure 8: Comparison of All Discarded Weights Recorded Between April and December from All Vessels That Had Observer Trips and All Data Sources Data Agreement The following graphs show the degree to which logbook and video data agreed on what was caught the closer a point is to the diagonal line, the more the sources agreed. Points move away from the diagonal line as the difference between the two sources increases. Missing values are graphed as zero, and thus appear along the x- and y-axes. These missing values may result from a video reviewer not seeing a discard, a fisherman not noting a discard, or a disagreement between sources over which species was caught. Each data point in these graphs represents the discard of a single species from a single haul. Data points come from all hauls that we analyzed from all participating boats (gillnetters and trawlers combined) in the 2014 fishing season. Graphs in following sections break these data down according to gear type. 16

17 Figure 9A: Data Agreement between Observer and Video Review Discarded Weights Recorded Between April and December from All Vessels That Had Observer Trips The number of points above the diagonal line suggests that Observers were able to identify more ACE discards than video reviewers. Points along the y-axis indicate instances where Observers saw ACE species discarded and video reviewers did not, implying that video reviewers may have been unable to see the fishermen s act of discarding if the fate of the fish was unknown, video reviewers had to assume that the fish was retained. Figure 9B breaks this dataset down by gear type. Figure 10A gives us a comparison of discarded weights from logbook and EM video data. The few outlying points reflect unusually large hauls. Note that weight estimates from EM video are calculated from a combination of piece counts and fish lengths using a species-specific estimation formula; any of the three may produce discrepancies between data sources, particularly with very large values. Figure 10B shows the same data agreement but only for those data points below 25 pounds of harvested weight (i.e. with the major outliers removed). Finally, Figure 10C gives us the same agreement as Figure 10A but broken down by gear type. Again, breaking comparisons down by gear type allows for a better understanding of where data discrepancies may be coming from. 17

18 Trawl Gillnet Figure 9B: Data Agreement between Observer and Video Review Discarded Weights Recorded Between April and December from All Vessels That Had Observer Trips (by gear type) Figure 10A: Data Agreement between Logbook and Video Review Discarded Weights Recorded Between April and December from All Vessels and all Hauls. 18

19 Figure 10B: Data Agreement between Logbook and Video Review Discarded Weights BELOW 25lbs Recorded Between April and December from All Vessels. Trawl Gillnet Figure 10C: Data Agreement between Logbook and Video Review Discarded Weights Recorded Between April and December from All Vessels and all Hauls (by gear type). 19

20 Figure 11A: Data Agreement between Logbook and Video Review Discarded Pieces Recorded Between April and December from All Vessels and all Hauls. Figure 11A gives us a comparison of discard piece counts from logbook and EM video data. As before, outliers reflect unusually large hauls. More points appear along the x-axis in this graph than in weight comparisons. These points indicate cases where only video data provided piece counts, reflecting the logbook data s inconsistent level of detail. Figure 11B shows a close-up of data points below 20 discarded pieces for the same data agreement. Figure 11C shows the agreement from the entire data set broken down by gear type. 20

21 Figure 11B: Data Agreement between Logbook and Video Review Discarded Pieces BELOW 20 Recorded Between April and December from All Vessels. Trawl Gillnet Figure 11AC Data Agreement between Logbook and Video Review Discarded Pieces Recorded Between April and December from All Vessels and all Hauls (by hear type). The tables below show the R 2 values for the data points shown in results section. R 2 values indicate the degree to which two data sources "agree." A perfect match would have an R 2 value of 1, while totally random datasets with no relationship whatsoever would have an R 2 value of 0. These tables evaluate data in two ways: with zeroes included and excluded. A zero value in the dataset indicates that one data source observed a discard of a particular species but the other source did not. This missing value may be due to inability to see the discard take place, misreporting, misidentification of species, or other causes. Figures 9 through 11 illustrate these zero values as points along each axis. R 2 values including zeroes give a good sense of the full dataset. Excluding the zeroes gives a better sense of accuracy when both sources agree that something was caught i.e., how well does each measure the amount of discarded fish? We know that there will be a disagreement if one data set thinks that something was caught and the other does not, but by removing the zeroes we can see if how well the data sets agree with each other when there is data to compare. This allows for further investigation into discrepancies between data sets and helps determine how to correct for these in future. 21

22 Table 1: R 2 values for all weight comparisons Weight Trawl Gillnet All Gear Types With 0 Without 0 With 0 Without 0 With 0 Without 0 EM Video - Observer EM Video - Logbook Observer - Logbook Table 2: R 2 values for all piece count comparisons Piece Count Trawl Gillnet All Gear Types With 0 Without 0 With 0 Without 0 With 0 Without 0 EM Video - Logbook The R 2 values very clearly show that EM is far more effective for gillnet vessels than it is for trawl. R 2 values for the trawl fishery indicate minimal agreement between EM video and both Observer and logbook sources. The gillnet fishery, however, shows a high level of agreement between EM video and Observer and logbook data. The exceptionally low level of agreement between Observer and logbook data in this fishery is due in part to one vessel that did not record any discards on their logbook form during an observed trip. However, this data has not been excluded from this comparison as it is required in order to have an n value of 4 4 for this comparison. 4 Discussion and Conclusions The Year 2 objectives of this pilot program are bulleted below with discussion and conclusions regarding each objective detailed underneath. To test open source EM technology on seven active sector vessels of two different gear types. In Year 2 Ecotrust Canada successfully installed and tested EM technology on five gillnet vessels and two trawl vessels. To ensure that Year 2 collected more usable video, regular feedback was given to fishermen regarding their systems performance and their interactions with it. This feedback helped users be more educated about and engaged with the systems, and reduced the percentage of video made accidentally unusable by fishermen and deck conditions. As in Year 1, the system collected high quality data including both the close up high resolution video necessary for identifying fish to species and capturing individual fish lengths, and the wide angled high resolution video necessary for observing fish handling techniques, estimating volumes of fish and determining discard rates. In Year 2 a third 4 Comparisons made between fewer than 4 data sets are not deemed statistically significantly. 22

23 camera was added to the system to capture an outboard view allowing for video capture of catch that drops out of the fishing gear or is handled and discarded at the rail and is never brought aboard. There was still some unusable video in Year 2, but it was discovered quickly and generally remedied within time to collect usable data on subsequent trips. Unusable video data from was due to a number of physical factors, including water spots on camera covers, fishermen obstructing camera views, fish handling or discarding occurring outside the field of view, and due to some technical EM system malfunctions listed in Section 3.4. Overall, the EM systems proved to be reliable as there was very little downtime when systems were triggered to be on and recording sensor data. It should be noted that there was limited data for trawl vessels. Only one trawl vessel did its full 25- fishing day requirement. The other vessel was unable to complete all 25 trips due to engine issues. An EM system was installed on a third trawl vessel, but that vessel had technical issues and only took one trip with the system, which yielded no usable data. Transfer knowledge through partnerships to increase the capacity for the support of EM programs in New England To develop a cost effective EM system using open source technology that meets federal technical specifications and collects data comparable to those collected by the ASM program of NMFS. One of the goals of this program is to increase the local capacity to support EM programs in New England like this one. In Year 2 Ecotrust Canada continued to work with local New England residents to develop the knowledge base in the region. The more work we do in this regard, the more efficient the program becomes, allowing local people to play an active role in the management of their regional fisheries and reducing the need for travel and flights between British Columbia and New England. Thanks to the partners in this project (GMRI, MCCS and TNC), Ecotrust Canada has been able to engage with regulators continually throughout Year 2 of this project. In 2013/2014 this engagement facilitated the approval and integration of Ecotrust Canada s elog software into the EM system, which gives the overall program the potential to meet regulatory requirements without increasing user inputs. While the elog was not approved in time for fleet wide use it in Year 2, it is hoped that it will be employed in Year 3. Throughout 2014 Ecotrust Canada s engagement with local partners and regulators allowed us to participate in policy discussions and costs analyses and provide feedback regarding the economic realities of different policy options as well as the technical requirements for different types of data collection, analysis and delivery. It s apparent that the cost of EM is very highly dependent on the specific monitoring objectives, required rates of video review, and other policy considerations. As a result, depending on details of the program design, EM may present fishermen with substantial cost savings compared with traditional observers or none at all. Appendix 3 details the fixed and variable costs associated with an EM program that uses a 4 camera system and reviews 1-2 hauls per trip for species level estimates of discards at sea. Based on a monitoring program model proposed in 2014 by NMFS for a cost analysis of ASM and EM for NE multispecies groundfish sectors, EM may not in fact offer a cost savings to harvesters. Ecotrust Canada 23

24 was contracted by NMFS to do a cost analysis of proposed regulations and came up with recommendations related to both discard compliance standards and data retrieval standards, a summary of this recommendations is included below. The budget in Appendix 3 is NOT associated with the cost analysis done for NMFS but, as explained, is based on a program similar to this pilot. Discard Compliance: The cost of video review implied by the standards put forth in the NMFS costs analysis piece was extremely high and may be prohibitive if performed on (nearly) 5 100% of trip data, greatly outweighing the cost of other service deliverables, and likely being performed on a much higher percent of trips/hauls than the use of traditional onboard human observers (which is normally done on < 30% of trips/hauls). We recommend that the necessity of 100% review coverage be questioned and placed in the context of other objectives such as reducing costs to industry. Disk Retrieval: The weekly disk retrieval is the 2 nd largest source of ongoing program cost. If permitted, this cost could be nearly halved by allowing video retrieval to occur once every two weeks instead. For example, in another fishery 6 served by EC s Monitoring systems, program video is collected on a monthly basis. An increased reporting latency may be reasonable in the NE groundfish fishery considering that some data is already collected weekly via logbooks, and EM only serves to audit that data for inaccuracy which may not be as time sensitive an operation. To compare data collected by EM systems with those self-reported by MCCS and SHS fishermen and with those collected by the NMFS Observer and ASM programs. To further the work commissioned by NMFS in EM weight estimation and species identification by applying length to weight relationships in an operational environment. In Year 2 we were able to make comparisons between EM data, self-reported logbook data and Observer data; although observed trips accounted for only 14% of the trips analyzed. Video to be reviewed was chosen randomly but attention was paid to observed trips so that those comparisons could be made as often as possible. In order to troubleshoot sources of logbook/em data disagreement through potential comparisons of results between studies, the same length-to-weight relationships that were used by NMFS were applied to our data. Results from Year 1 demonstrated that the length-weight methodology employed was not highly effective but worth redesigning and investigating further in Year 2. In Year 2 measurement strips were adhered to vessels so that lengths did not have to be calculated using a screen to reality length conversion therefore removing one potential source of error. In Year 2 we were still reliant on the same length-to-weight conversions that were used in Year 1 and in the NMFS pilot. As shown in Table 1, weight estimates of discards in Year 2 had higher R values than those in Year 1, hopefully indicative of a 5 Possibly excluding traveling to fishing grounds which makes up < 10% of the trip time in our estimation. 6 The Area A Crab fleet in British Columbia, Canada, a pioneer of video-based compliance reporting who have been using EM technology since

25 more accurate conversion now that length strips are being used and not a screen to reality length conversions as well as a length to weight conversion. In general, discarded weight comparisons between logbook and EM data were high when both data sets recognized the presence of a species. However, when one data source identified a species and another did not (either because the species was not seen at all or was not seen being discarded) then R 2 values dropped substantially. Some species and vessel/gear combinations exhibited substantial disagreement in discard counts in logbooks and EM video review due to this. An important aspect of the next pilot year should be to more clearly identify and understand sources of this disagreement, while keeping in mind that we don t have a control or baseline for the accuracy of the logbook. This underlines the importance of having multiple bases for comparison such as the Observer data. Sources of disagreement that should be investigated in 2015 include: Misidentification of catch, specifically are there species that are more difficult to accurately identify on camera and if so how can they be classified to capture a repeated problem, e.g. is white hake always classified as miscellaneous hake and should we consider comparing those two classifications? Catch discarded outside of camera view, if all catch is being identified on camera could some of these disagreements be corrected and/or explained by a secondary comparison, i.e. both data sources identified species X but video data did not see it being discarded. This would speak to the need for a behavioral change from operators or a readjusting of a camera instead to implying EM systems inability to capture good enough data for species identification and/or the video reviewer s inability to accurately identify fish on video. Transcription errors Inaccuracy of estimation formulae Inaccuracy of length estimation Similar to the methodology for improving video usability in 2014, in 2015 Ecotrust Canada recommends real time generation and review of agreement charts and metrics to identify vessels not only with problematic video footage, but any repeated source of error in general. This will allow us to focus on identifying error sources on specific vessels with specific symptoms throughout the season. This will also increase video review efficiency and reduce transcription error by reducing the data entry workload of the reviewer. Finally, in 2015 we would also like to look at other types of data being collected by the EM system and determine if there are ways to use this information for better system functionality and/or to determine if there are innovative ways to meet NMFS proposed data collection requirements without huge increases in the cost of implementing EM systems. Examples of other data collected by the system are included as Appendix 2 25

26 References Pria, M.J., McElderry, H., Fedoruk, A. and McVeigh, R Estimating Weight and Identifying Species Through Electronic Monitoring (EM): A Preliminary Comparison of Electronic and Observer-Based Reporting. Interim Technical Review prepared for the Fisheries Sampling Branch of the Northeast Science Center by Archipelago Marine Research Ltd., Victoria, BC Canada. 37 p. Wigley, S.E., McBride, H.M., and McHugh, N.J Length-Weight Relationships for 74 Fish Species Collected during NEFSC Research Vessel Bottom Trawl Surveys, NOAA Technical Memorandum NMFS-NE p. Appendix 1: Example Vessel Monitoring Plan VESSEL INFORMATION Vessel Name: Vessel ID Number: Vessel Length: Home Port: Number of Crew: Vessel Master: Gear Type: Vessel Master 2: EM SYSTEM MONITORING OBJECTIVES To develop a cost effective EM system using open source technology that meets federal technical specifications and collects data comparable to those collected by the ASM program of NMFS. Objectives for Data Capture: Collect time and location of fishing events in EM sensor data and in vessel fishing logs Collect imagery of catch processing of such quality that ACE managed discarded species can be enumerated and measured. Collect imagery of catch disposition with enough quality to tell when ACE managed species are being discarded Capture weights of discarded ACE species in fishing logs. VESSEL PROCEDURES Details on the Vessel s operation are necessary to ensure an optimum arrangement of EM system components. Fishing Gear Deployment Details: Gillnet Setting 26

27 String of nets deployed of the stern of vessel using no mechanics only vessel speed. 4 to 5 strings deployed and left to soak. Hauling A flag marks each end of a string of gillnets, one flag is pulled aboard at the start of a hauling event. The nets are then brought on board by the hauler near the bow on the starboard side of the vessel. Catch is removed from the net as it moves from the hauler to a flake bin at the stern of the vessel. Catch Handling Details: Catch is either retained or discarded as it is removed from the net. Large animals that cannot be brought aboard are released from the net at the side of the vessel. All catch to be discarded must be handled in view of the camera. All catch must be passed lengthwise to the measuring strip on the tray. Camera and System Handling Details: Ensure the EM system is always powered while the vessel is at-sea. Ensure camera domes are kept clean and views clear. Ensure working area is well lit at night. Report any issues with the EM system in a timely way. Report all logbook information at the haul level. 27

28 INSTALLED EM SYSTEM COMPONENTS EM Control Box Monitor Keyboard Installed Yes No Yes No Yes No Type Location Rationale GPS Other Sensor Hydraulic Sensor Length Strip Installed Yes No Yes No Yes No Yes No Type Location Rationale 28 Prepared by Ecotrust Canada s Marine Monitoring Initiative

29 Comments Camera 1 Camera 2 Camera 3 Other Installed Yes No Yes No Yes No Yes No Lens Size FPS Codec Location Rationale Goals 29 Prepared by Ecotrust Canada s Marine Monitoring Initiative

30 CURRENTLY INSTALLED COMPONENTS Control Box Monitor View Hydraulic Pressure Sensor Other Sensor GPS Vessel 30

31 CURRENT CAMERA LOCATIONS AND VIEWS Camera 1 Location Camera 1 View Camera 2 Location Camera 2 View Camera 3 Location Camera 3 View 31

32 FIELD SERVICES CORRECTIVE ACTION PLAN Details of what changes are needed and why Date Initiated Corrective Action Plan Date Complete 32

33 CHANGE HISTORY This should reflect the changes made during the creation of the EM system as well as changes made afterwards that reflect refinements due to feedback from field services, data services or the fishermen. 33

34 Appendix 2: Visualizations of Other EM Data Capture The following graphs all come from a single participating vessel (the same vessel used for haul evaluations). Datasets cover that vessel s entire 2014 fishing season. Distribution of Pressure By Speed Boats typically conduct hauls at speeds of 5 m/s (10 knots) or less. This sample boat, a gillnetter, only shows low-pressure activity at speeds above 5 m/s. This could help the system to automatically weed out false positives; any pressure readings above a vessel speed of 5 m/s are likely non-fishing activities (cleaning and maintenance). 34

35 Distribution of Pressure Readings by Vessel Heading This graph shows the boat s heading when operating hydraulics. The y-axis shows direction based on compass heading, while the x-axis is hydraulic pressure. Points to the right of the graph show increases in pressure associated with hauling activity. The solid bands on the left side of the graph indicate lowpressure activities (cleaning and maintenance), often carried out while the vessel is underway. The variable compass heading reflects changing wind patterns in the area, as boats tend to head into the wind when slowing down or maintaining position to conduct a haul. In contrast, the graph below comes from a boat fishing a different gear type on the west coast of North America, in a region where wind patterns are far more consistent. The three large spikes show that the vessel typically faces either north (0 /360 ) or south (180 ) when conducting their hauls. 35

36 36

37 Appendix 3: Fixed and Variable Costs Associated with EM Example Budget/EM Costs FIXED Budget Items (per vessel) Description/Detail Cost/System VMS Box Electronic monitoring control box $ Cameras Outdoor Network Camera with D/N, IR, Vari-focal lens 4 * $ = $ Camswitch 4 Mobile PoE switch for cameras 4 * $ = $ Sensors Hydraulic pressure sensor & GPS $ Wheelhouse station Monitor & keyboard $ Installation & testing Technician time & support for 4 camera system install (est.) $ Total $ Example Budget/EM Costs VARIABLE Budget Items Description/Detail Cost/Vessel Annual Tech. Support Testing parts, technical support after installation etc. $ $ Video Analysis Estimated cost per trip for haul level review of ACE managed discards (assuming 20% review level) $ $ Program Admin Administering program, ensuring data delivery to all parties $ $ Research & Development Upgrades/customizations of hardware & software (depends on data collection and delivery standards) $ $ The table of fixed costs above shows the 2015 purchase and installation costs of a 4 camera electronic monitoring system from Ecotrust Canada. This system includes all the hardware and software necessary to collect the sensor and video data required to make at-sea discard estimates of ACE managed species. These systems also include elog software, which is a NOAA approved evtr that can be used by skippers to collect and report out their trip information. 37