New Zealand Dairy. Herd Improvement Database Review

Transcription

1 New Zealand Dairy Herd Improvement Database Review 2009

2 Commissioned by DairyNZ

3 NEW ZEALAND DAIRY HERD IMPROVEMENT DATABASE REVIEW ANDERSON COMMITTEE REPORT MAY

4 2

5 CONTENTS Page Chairman s Introduction... 4 Recommendations A. Herd testing... 6 B. Structural principles... 7 C. Transition and role of LIC D. High level maps of data flows Background to the Review Review Committee Responses to Questions in Terms of Reference Review Committee A. Personnel 17 B. Meetings.. 17 C. Central issues Submissions and Interviews A. Commissioned International Experts B. Volunteered Submissions C. Interviews 20 APPENDICES A Terms of Reference B Submission received from Dr Wickham C Submission received from Professor Garrick D Submission received from Dr Weigel E List of parties making written submissions F List of interviewed parties 3

6 CHAIRMAN S INTRODUCTION This particular review is the latest in a series that has been undertaken periodically ever since the herd improvement movement began in New Zealand last century. The fact that such reviews have been regularly conducted in one case having the status of a Royal Commission is indicative of an industry that has long realised the value of objective animal performance recording to underpin critical decision making, for both animal selection, in particular, and farm management purposes, more generally. This realisation is arguably the fundamental reason for the New Zealand dairy industry being amongst the most efficient and competitive anywhere in the world. The various reviews on herd recording and related matters over the decades have all been motivated by the need to enhance on-farm profitability and to optimally-position the wider sector for the future, especially in terms of harnessing new technological developments. This review is intended to build upon that tradition. The Review Committee was also mindful of the fact that what might be termed the non-genetic fields in the database to inform decisions pertaining to animal health, greenhouse gas emissions, biosecurity, compliance, general farm management and to facilitate traceability, will grow in importance. Accordingly, in inviting submissions from a wide range of interested parties, all were asked to specifically address the following question: Given your knowledge of the features of New Zealand dairy farming industry, and in the light of impending technological developments and the changing business landscape, what is the preferred design for a national data recording and information management system to ensure optimal genetic gain in the national herd primarily, and also to provide scope for advancing on-farm decision making practice, over the next 25 years? In the conduct of its deliberations, the Review Committee was mindful that, the acknowledged success of the nation s dairy herd recording and related activities notwithstanding, it is an arena that hasn t been without controversy and occasional legal dispute. Accordingly, the Review Committee took special care to follow a framework for its decision making that, as far as possible, was underpinned by sound scientific knowledge and experience. It decided to solicit (and publish) technical submissions from three internationally-renowned experts, together with obtaining input from a wide range of stake-holding individuals and organisations. It also met face-to-face with a substantial number of the latter group. A pivotal recommendation arising from this Review concerns the future custodial and ongoing developmental responsibilities for the New Zealand Dairy Core Database. The Review Committee proposes that these responsibilities, hitherto discharged by the Livestock Improvement Corporation (LIC), be re-assigned to an industry-good agency having no interest in utilising the Core Database for competing commercial services. In making this recommendation, the Committee wishes to record its appreciation to the Directors and Senior Management of LIC on two counts: First, for the leadership in developing and managing the Core Database over the years, and; secondly, for the insightful and constructive advice to the Committee in helping it to formulate its proposals for change. The willingness expressed by the Directors and Senior Management of LIC to assist with the (extensive) transitional agenda that will be necessary to drive this particular recommendation to a successful conclusion stands to be enormously helpful to those who will be involved. 4

7 Significantly, the vision of the Review Committee extends beyond just a recommended re-assignment of the custodial / trusteeship and the on-going developmental responsibilities for the New Zealand Dairy Core Database. Indeed, it envisages the creation of a one-stop New Zealand Dairy Industry data warehouse within which the data pertaining to herd improvement considerations will be a subset. As identified above, the industry now needs a national repository of accurately recorded data encompassing many non-genetic fields. Harmonisation with NAIT will also be essential. Obviously, this vision needs to be tested in detail through the design and transition phases going forward. Logic would dictate that responsibility for such a warehouse should also be vested in an industry good agency, rather than one particular commercial enterprise whose core business doesn t naturally extend to embracing these additional non-genetic contexts. It is also the view of the Review Committee that since some of these additional data recording activities derive from priorities deemed to be in the national interest beyond just the dairy industry, a case for (part) funding by the Government exists. In addition, the Review Committee envisages expanded opportunities to generate revenue from the appropriate sale of data and information (under a cost-recovery, rather than a profit centre, motive). The Committee is strongly of the view that a commitment industry-wide to a National Breeding Objective (NBO) remains essential. It respects the fact that individual breeding companies will always wish to differentiate their services to clients through developing and promoting alternate breeding objectives. Nevertheless, it is of the view that the Breeding Worth (BW) pertaining to the National Breeding Objective must be determined for all animals and should be added as an additional field in a re-defined National Dairy Database. The availability of that particular BW is important not only for the purposes of informed animal selection, but also, it is increasingly important for the valuation of animals that are being traded throughout the industry. The correlation between the NBO BW and any breeding indices derived from the breeding objectives set by those companies seeking to secure a sustainable tenure within the industry will be high. Achieving these aspirations will demand first-class industry governance arrangements. Finally, it remains for me to express genuine appreciation to the fellow members of the Review Committee, Harry Bayliss and Graeme Milne, for their substantial insight and effort. Thanks are also due to Bill Montgomerie (NZAEL), Bruce Thorrold (DairyNZ), and Russell Knutson and Gillian Mangin (MAF) for their excellent support of the workings of the Committee. Robert D. Anderson, ONZM May,

8 RECOMMENDATIONS The Review Committee s focus has been on identifying actions that will advance the future benefits to be derived from a new National Dairy Database in New Zealand. A. Herd testing and other animal records A.1 The New Zealand Dairy Core Database as it exists today should be retained, it should be refreshed with new data on an ongoing basis, and it should be maintained, owned, developed and controlled as an industry good facility. A.2 Herd testing should continue to be regulated by statute over the next five years, requiring that all herd testers must be certified and that they must continue to submit a defined minimum data set that captures the essential features of the dairy production performance of animals in the national herd. However, the Review Committee recommends that the dairy industry should aim for self-regulation of herd testing within the next five years. This recommendation derives from the recognition that there are long lead times associated with amending regulations that are governed by statute, a situation that is not well suited to circumstances where technology is evolving rapidly. A.3 The current herd testing standard for certified herd testers (New Zealand Standard Dairy Herd Testing [NZS 8100:2007]) is not suitable for the circumstances where farm dairies are fitted with automatic recording devices for milk yields and/or milk components. The Standard needs to be up-dated in view of the increasing farmer use of these devices. Key issues with automated systems include the assurance of data integrity and the establishment of cost-effective database interfaces. A.4 Certified herd testers should continue to be required to validate farm, animal, event and production data prior to submission to the New Zealand Dairy Core Database. A.5 The dairy industry has operated an effective scheme for recording non-production traits (described as Traits Other than Production and commonly known as TOP Traits ) since These records include liveweight measurements obtained outside the certified herd testing system and are an integral part of the national genetic evaluation system (the Animal Evaluation System). Accordingly, the future data maintenance and access arrangements must safeguard the on-going availability of these records for the Animal Evaluation System. The governance of the data collection, maintenance and access arrangements in this particular context should be vested in the users of the TOP Scheme, which includes participating breeding companies and breed societies. Future scope A.6 Moving forward, and subject to authorisation by an industry governing entity such as the DairyNZ subsidiary New Zealand Animal Evaluation Limited, new database fields should be defined and added, or deleted where appropriate, as new animal traits of relevance for genetic evaluation purposes emerge. A.7 As a specific example, the national database requirements for genetic improvement purposes should be reviewed in five years to identify industry good opportunities associated with advances in bovine genomics technology. 6

9 A.8 As appropriate, new database fields should also be defined and added to support farm management decisions, or industry supply chain requirements, or to conform with government and/or industry requirements for animal recording for disease prevention or quality assurance purposes. B. Structural and operational principles for a future national database The Review Committee has noted and agrees with the expert submission of Professor Garrick that database design and operations to date have facilitated annual rates of genetic gain that are in line with international best practice. However, stakeholder needs have changed over the past decade and data recording technology is evolving rapidly. Accordingly the optimal structure for a national database will be different over the next 25 years. B.1 The existing national herd improvement database should be re-structured to be operated as an industry good facility (a new National Dairy Database) that is owned, managed and further developed by an industry good organisation established for the purpose through the agreement of the key industry stakeholders, including DairyNZ. The precise design and operation of this facility should be the subject of a DairyNZ study referred to in Recommendation C1. While it is possible that such an arrangement will increase overall costs compared with current arrangements, benefits will also arise from transparent independence of such a new National Dairy Database from any individual service provider and these benefits should be recognised in the DairyNZ analysis. Funding B.2 The new National Dairy Database industry good organisation should arrange on-going funding for the re-structured new National Dairy Database. A mixture of industry and user-pays funding is envisaged. There is also a case for government co-funding in order that the new National Dairy Database can effectively support the biosecurity, animal health and trade access policies of the government. Mandate B.3 The mandate of the new National Dairy Database industry good organisation should preclude that organisation from competing in the commercial markets within which its stakeholders operate. This provision is necessary both from the definition of industry good services, and also to avoid disincentives for immediate service providers to invest in developing new services or to commit to data sharing. B.4 A key function of the new National Dairy Database should be the provision of estimated breeding values and indices of genetic merit for all dairy animals in New Zealand as generated by the National Breeding Objective and genetic evaluation system sanctioned by DairyNZ. Submissions strongly supported industry good provision of a single national method for estimating breeding values and provision of a National Breeding Objective. B.5 The Review Committee supports the current policy for the National Breeding Objective. Economic valuation of milk might in the future include attributes not currently used in payment systems, providing opportunities beyond a single kind of commodity milk. A single National Breeding Objective will not be able to represent all these value propositions. DairyNZ needs to be able to support breeding objectives for 7

10 every distinctive and significant circumstance (while acknowledging that more than one National Breeding Objective for identical production, management and economic circumstances is not warranted). B.6 The new National Dairy Database industry good organisation should continue the current policy that parties sourcing breeding value information from the national genetic evaluation system for the purposes of publication, including provision of herd reports, are required to publish the industry s benchmark index of total merit (the industry good service provision currently represented by the BW index). Access B.7 National Dairy Database services should be extended to include industry good information, defined by the new National Dairy Database industry good organisation, for providers of advisory services about the management of dairy cattle in New Zealand. B.8 As a basic principle, New Zealand commercial service providers should have access to the new National Dairy Database on behalf of their clients, subject to meeting the costs for the services provided by the new National Dairy Database. B.9 The new National Dairy Database industry good organisation should review the access protocols for the new National Dairy Database, with a view to achieving more widespread use of such a facility. B.10 New Zealand research organisations should have ready access to the new National Dairy Database subject to meeting the costs for the services provided. The conditions set by the new National Dairy Database industry good organisation for access in this category should be minimally restrictive. However, criteria for granting access should retain the conditions in the existing regulations that access is only granted if the governors of access protocols are satisfied that to do so is likely to be beneficial to the New Zealand dairy industry or are satisfied that to do so would not be harmful to the New Zealand dairy industry. C. Transition and role of LIC Transition to a new National Dairy Database framework that is operated as an industry good facility and is owned, managed and further developed by a new National Dairy Database industry good organisation, will involve a large number of issues. The LIC custodianship of the New Zealand Dairy Core Database, and its experience with its own database over the past 25 years, mean that it has a deep understanding of the issues associated with transition. In moving forward it is in the interests of the wider industry that this extensive experience and understanding is harnessed effectively. LIC has provided the Review Committee with valuable insights into the transition issues. The Review Committee makes the following recommendations about ways to move forward to a new database framework. C.1 The Board of DairyNZ should undertake a design and transition study (including costs and benefits) of establishing and maintaining a new National Dairy Database with the features described in recommendations A1 through to B10. 8

11 C.2 Informed by the results of that study, a small establishment team (headed by a person outside the employment of any key commercial stakeholders) should be appointed to design the new National Dairy Database framework. Widespread consultation with industry stakeholders will be essential. C.3 The establishment team should operate in accordance with terms of reference established by DairyNZ and agreed by key stakeholders. C.4 The establishment team (in consultation with DairyNZ and key stakeholders) should define the industry good purposes of the new National Dairy Database, identifying the sources and types of data needed for the identified purposes. C.5 The establishment team should ensure that the design features of the new National Dairy Database harmonise with the emerging design of NAIT (National Animal Identification and Tracing). C.6 The establishment team should draw upon a range of expertise. Central to this will be expertise from LIC on the current operations of the core database system, and the national breeding value and breeding index estimation procedures. C.7 The establishment team s conceptual design of the new National Dairy Database (including costings, expected revenues, timelines for implementation, and implications for legislation) should be subjected to assessment by dairy farmers, key stakeholders and the Ministry of Agriculture and Forestry before implementation. C.8 Dairy farmers (through their various representative agencies), other key stakeholders and the Ministry of Agriculture and Forestry should agree on the structure of the industry good entity responsible for the new National Dairy Database. 9

12 D. High level maps of data to information flows Figure 1 depicts the current transformation of raw data to genetic information showing multiple data sources, with some of the data needed for genetic evaluation subject to Ministry of Agriculture and Forestry regulation, while other data is subject to industry selfregulation. The current arrangement is characterised by a lack of clarity on ownership and control of some of the input data sources, and by a lack of clarity around access arrangements to the derived genetic information. Figure 2 depicts the proposed transformation of raw data to genetic information showing all necessary input data and the genetic information outputs of the system: accessible from the new National Dairy Database, and under the control of the whole industry. 10

13 Figure 3 depicts the proposed transformation of raw data to farm management information showing delivery of derived information to stakeholders for incorporation into their service provision to customers. A current example of this form of data sharing and information delivery is provided by the InCalf project. 11

14 BACKGROUND TO THE REVIEW PREAMBLE History New Zealand has a long history of combined actions by dairy farmers to advance the effectiveness of herd improvement (Herd Testing and Related Services, Macdonald Committee Report, 1992, Appendix F). A common thread in this history has been the efforts to achieve data collection and sharing about the characteristics of cows in the national herd. Restructuring 2001 At the time of the dairy industry restructuring in 2001, Government recognised the importance of maintaining a comprehensive database of dairy herd information. The Dairy Industry (Herd Testing and New Zealand Dairy Core Database) Regulations 2001 followed. The Herd Testing Regulations provided the foundation for the collection of a specified set of data characterising the performance of the national bovine dairy herd. NZS 8100:2002 Dairy herd testing, was subsequently developed to define the inputs required to meet the objectives of the Herd Testing Regulations. This has subsequently been superseded by NZS 8100:2007 Dairy herd testing published by Standards New Zealand. Certified herd testers are required to populate the Core Database fields. LIC has been a certified herd tester for many years. CRV AmBreed became a certified herd tester in Purpose of Core Database fields The forty-six data fields in NZS 8100:2007 Dairy herd testing are known as core data and are held in the New Zealand Dairy Core Database, which, under the Herd Testing Regulations, is managed by Livestock Improvement Corporation Limited. The purpose of the data held in the core database is to facilitate herd performance recording and enable research and animal evaluation of the genetic productive potential of the national herd and its members, for the benefit of all New Zealand dairy farmers (Paul Reynolds, Assistant Director-General (Policy), MAF, 23 May 2007, Preface to NZS 8100:2007 Dairy herd testing). The Animal Evaluation System The national genetic evaluation system for dairy cattle is known as the Animal Evaluation System (Animal Evaluation Technical Advisory Committee Report, 1996). The Animal Evaluation System makes use of core data and other non-core animal records, such as liveweight data. The utilised records include information gathered by certified herd testers under the Herd Testing Regulations, information gathered as part of traits other than production services and information provided by dairy breed associations. These performance records both core and non-core data are used to derive outputs of the Animal Evaluation System. These outputs are estimates of genetic merit (breeding values) for over twenty traits of New Zealand dairy animals, and of productive merit (production values) for a smaller number of traits. Additional outputs are indices for profit ranking of animals. The Breeding Worth index (BW) ranks animals on their expected ability to breed replacements that are efficient converters of feed into profit. The Production Worth index (PW) ranks females on their expected lifetime ability to convert feed into profit. 12

15 New technologies Data sharing is not the norm in the commercial world, nor is it ubiquitous in science. Since 1996, new technologies have emerged for selecting animals for breeding and for performance recording. Both categories of new technology come with in-built incentives to restrict data sharing. Selection of animals for breeding is increasingly being influenced by use of DNA data by breeding companies. These data are not readily available for sharing, due to commercial requirements for breeding companies to prohibit or severely limit data sharing to protect proprietary interests. New solutions to data sharing in this area need to be devised. Automated and in-line devices for recording performance of cows are becoming increasingly available. Developers of these emerging technologies do not have normal commercial incentives to share data. The same observation about the need for new solutions to overcome barriers to data sharing apply in this area as well. Advances in automatic data capture also offer opportunities to improve the quantity and quality of data available for animal improvement. Both areas are relevant for the purpose of a core database to facilitate herd performance recording and enable research and animal evaluation of the genetic productive potential of the national herd and its members, for the benefit of all New Zealand dairy farmers. Access and pricing With the advent of competitive herd testing and herd recording since 2001, issues have arisen about access to data, and pricing of data extracts. (i) Core data: Under the Herd Testing Regulations, the New Zealand Dairy Core Database Access Panel adjudicates upon applications for access to core data. The Panel does not set LIC s charges for access, which are subject to a gazetted methodology (available at There is an annual audit of LIC compliance with the Regulations. (ii) Non-core data: Access and pricing for outputs of the Animal Evaluation System (which are non-core data) are decided by LIC. Concern over pricing has been advanced as a key reason for CRV AmBreed developing an alternative breeding value estimation method outside the Animal Evaluation System. Establishment of the review At its meeting on 12 th August 2008, the DairyNZ Board resolved to establish a Herd Improvement Database Review Committee to inquire into animal data requirements for the Animal Evaluation System, and related matter. Terms of Reference DairyNZ established Terms of Reference for the Review Committee (Appendix A). 13

16 REVIEW COMMITTEE RESPONSES TO QUESTIONS RAISED IN TERMS OF REFERENCE (a) Is the current system effective for obtaining animal performance records for the Animal Evaluation System, including cost efficiency and relevance of records? The Review Committee agrees that the current system continues to be effective and has facilitated annual rates of genetic gain that are in line with international best practice. However, stakeholder needs have changed over the past decade and data recording technology is evolving rapidly. Accordingly, the optimal structure and operating arrangements for a national database will be different over the next 25 years. (b) What are the future opportunities for gathering performance records using automatic recording devices that are not currently linked to any national animal recording database? Local and overseas evidence is that there will be many new opportunities arising from automatic generation of raw data. Dr Weigel summarised many of the opportunities as follows: As herds grow larger, labour costs, consumer demands, and animal welfare concerns will become increasingly important. Farmers must strive to achieve optimal production and reproduction efficiency while minimizing disease incidence and severity and environmental impact, and while producing a product of desired quality. Ingvartsen (2008) discussed this issue and emphasized the need for automated precision management of dairy herds. Farmers will have numerous tools and a wealth of information at their disposal. In a review of the impact of new on-farm technologies on dairy cattle breeding, Miglior et al. (2008) noted the advantages of automated data collection, which include improved accuracy, reduced cost, and availability of new traits, as well as the disadvantages, which include recording errors and the need to compensate farmers for their investments in these technologies. Earlier, Wade (2006) noted that both short and long-term implications of the use of automated on-farm data recording systems should be considered. In the short term, automated systems will provide a wealth of data for traits that are already considered in the breeding goal, and this will increase the accuracy of selection. In the long-term, management practices will change due to increased automation, and researchers must strive to identify the types of animals that are best suited for these new management systems, such that new traits may be needed in the breeding goal. (c) Is the current Herd Testing Standard for certified herd testers appropriate in light of recording options currently available or about to become available? The current Herd Testing Standard needs to be revised, taking into consideration both recording options that are currently available and recording options that can be anticipated in the future. 14

17 (d) Should the scope of core data be enlarged to include all inputs into and outputs of the Animal Evaluation System, extending to both the PW and BW indexes? What impacts would such an enlargement have on obligations and benefits of providing input data to the Core Database and the Animal Evaluation System? The scope of data in the new National Dairy Database should include all inputs and outputs for Breeding Value estimation in the Animal Evaluation System, and associated breeding indices. Submissions made to the Review Committee did not indicate that Production Value and Production Worth outputs of the Animal Evaluation System are in strong demand to be provided as industry good services. Consideration of industry good provision of the PW index via the National Dairy Database can be undertaken as a design and transition issue for the establishment team, recognising prior investments by LIC in developing the index. (e) Is LIC s ownership, operation and control of the LIC Database (of which the New Zealand Dairy Core Database is a subset) appropriate in terms of: (i) Meeting the DairyNZ aims to secure and enhance the profitability, sustainability and competitiveness of New Zealand dairy farming; Establishment of a separate and independently managed industry good National Dairy Database stands to advance the wider industry needs more effectively than retention of the New Zealand Dairy Core Database as a subset of the LICs private database activities. DairyNZ should undertake a study to analyse the costs and benefits associated with a National Dairy Database that is re-structured to be operated as an industry good facility by a national database industry good organisation established for the purpose through the agreement of the key industry stakeholders. (ii) The breadth and completeness of data collected; Breadth and completeness of data capture has been well achieved, but emerging signs indicate that this will be less readily achieved in the new era of automatic recording devices and under the expanded scope that private software developers wish to pursue. (iii) Accessibility of data for industry users; A number of submissions to the Review Committee commented that the absence of public domain listings of top cows for consideration as bull dams, together with perceived time delays and debatable pricing arrangements, pointed to sub-optimal access to industry good data under the current database arrangements. (iv) Adaptability to the changing needs of the dairy industry; The Review Committee did not observe any material examples of rigidity in the face of changing needs. It does note, however, that the current Herd Testing Standard makes the capture of data from automatic recording devices difficult. (v) Accuracy and integrity of the data recorded; The Review Committee and all organisations making submissions recognise that maintaining high levels of accuracy in livestock recording is a never-ending challenge. The evidence provided to the Review Committee did not indicate that there are material problems affecting accuracy and integrity of data recorded on the New Zealand Dairy Core Database other than problems that are already being addressed. 15

18 (vi) The cost to industry users; The Review Committee has no reason to form any view other than that the LIC operation of the New Zealand Dairy Core Database has been cost efficient for the industry. (f) Other matters that should be considered in reviewing or inquiring into (a) to (e) above including the role of industry good/dairynz investment and ownership. The Review Committee considered the practicality of a regulated approach to national database capture of raw genomic data. It does not consider that such an approach is appropriate in the immediate future due to the rapidly changing technology for recording single nucleotide polymorphisms. 16

19 REVIEW COMMITTEE A. Personnel DairyNZ appointed the following people to the Review Committee: Professor R D Anderson (Chairman) Mr H G Bayliss Mr G R Milne B. Meetings The Review Committee met on 19 December 2008, 26 February 2009, March 2009, 7 April 2009 and 20 May Gillian Mangin and Russell Knutson (Ministry of Agriculture and Forestry), Bruce Thorrold (DairyNZ), and Bill Montgomerie (New Zealand Animal Evaluation Limited) attended these meetings as observers apart from Gillian Mangin and Russell Knutson being unable to attend the meeting on 20 May. The Chairman attended New Zealand Animal Evaluation Ltd's User Group Meeting (Meeting 17) on 5 February 2009, where he outlined the approach of the Review Committee to its task. The meeting also featured a presentation by Dr Jenny Jago, Senior Scientist, DairyNZ, on The future for automatic recording technologies on New Zealand dairy farms. The Review Committee also made an initial report at an open meeting engaging with the submitting parties which was held on 7 May C. Central issues The Review Committee identified central issues for its attention. Genetic improvement The Review Committee agreed that the National Herd Improvement Database had been, and remained, a crucially important instrument in the pursuit of optimal genetic improvement in the national dairy herd. However, the business landscape and available technologies for herd improvement service providers are now substantially different compared to the time at which the Database and related infrastructure were designed and implemented over 20 years ago. Changes in data recording methods and purposes Data recording opportunities and requirements for general farm management purposes have changed markedly over the past 20 years. The Review Committee identified that obtaining and managing data to enhance on-farm decision making, especially in the case of current larger-scale enterprises, now gives rise to an important additional dimension that the Review Committee needed to consider. At its meeting on 19 December 2008, the Review Committee agreed that optimising genetic gain in the national herd, using NZ pasture based farming systems as the reference environment, remained an overarching primary imperative. It also identified that there were likely to be additional data recording opportunities to leverage the value of a National Database. 17

20 Expert submissions The Review Committee undertook to obtain expert submissions from internationally recognised authorities based offshore. Understanding of the term database The scope of the Review Committee s task required that it adopt an agreed understanding of the term database. The Review Committee adopted the working definition provided by Dr Wickham s expert submission, namely; an integrated system for: gathering and validating data, storing data, analysing data, compiling information, and distributing information. The Review Committee also adopted Dr Wickham s description of a national database as a database in which a defined set of data pertaining to one country is held and shared between multiple users. The Review Committee confined its deliberations to data of relevance to farm production in the New Zealand dairy industry. Contacts for consultation The Review Committee established a list of interested parties in New Zealand to approach for submissions, and contacted: 16 organisations involved with milking systems; 7 organisations associated with milk processing; 29 institutions with discovered links to the dairy industry supply chain; 9 dairy farm advisory organisations; 15 dairy cattle breeding companies; 7 dairy breed societies and the NZ Dairy Breeds Federation; 333 individual contacts through the NZAEL User Group. 18

21 SUBMISSIONS AND INTERVIEWS A. Commissioned International Experts The Review Committee invited input from three internationally-recognised authorities with expertise in the fields of (i) animal breeding and genetics and (ii) on-farm decision support systems. The authorities invited to make these expert submissions were: Dr Brian Wickham Chief Executive Irish Cattle Breeding Federation Society Ltd (ICBF) Ireland Professor Dorian Garrick Lush Chair in Animal Breeding & Genetics Department of Animal Science Iowa State University USA Dr Kent Weigel Associate Professor & Extension Genetics Specialist Department of Dairy Science University of Wisconsin USA As part of the request for the submissions these experts were given the Review Committee s terms of Reference, and also asked to address the following topic: Given your knowledge of the features of New Zealand dairy farming industry, and in the light of impending technological developments and the changing business landscape, what is the preferred design for a national data recording and information management system to ensure optimal genetic gain in the national herd primarily, and also to provide scope for advancing on-farm decision making practice, over the next 25 years? The Review Committee also requested examples of best practice, or pitfalls to be avoided. The expert submissions are appended: Dr Wickham submission (Appendix B) Professor Garrick submission (Appendix C) Dr Weigel submission (Appendix D) As well as his formal submission that is reproduced in the Appendices, Dr Weigel also provided the Review Committee with detailed commentary through exchanges. 19

22 B. Submissions from New Zealand based organisations and individuals Of the 83 local organisations and 333 individuals contacted with an invitation for submissions, the Review Committee received written submissions from 18 organisations and 4 individuals (Appendix E). The Review Committee received these submissions in confidence. C. Interviews The Review Committee interviewed representatives from 10 organisations (Appendix F). 20

23 APPENDIX A DAIRYNZ TERMS OF REFERENCE FOR HERD IMPROVEMENT DATABASE REVIEW Having regard to the legislative provisions governing the dairy industry and the overall aims of DairyNZ to secure and enhance the profitability, sustainability and competitiveness of New Zealand dairy farming, the Herd Improvement Database Review Committee shall inquire into and answer the following questions: a. Is the current system effective for obtaining animal performance records for the Animal Evaluation System, including cost efficiency and relevance of records? b. What are the future opportunities for gathering performance records using automatic recording devices that are not currently linked to any national animal recording database? c. Is the current Herd Testing Standard for certified herd testers appropriate in light of recording options currently available or about to become available? d. Should the scope of core data be enlarged to include all inputs into and outputs of the Animal Evaluation System, extending to both the PW and BW indexes? What impacts would such an enlargement have on obligations and benefits of providing input data to the Core Database and the Animal Evaluation System? e. Is LIC s ownership, operation and control of the LIC Database (of which the New Zealand Dairy Core Database is a subset) appropriate in terms of: (i) meeting the DairyNZ aims to secure and enhance the profitability, sustainability and competitiveness of New Zealand dairy farming; (ii) the breadth and completeness of data collected; (iii) accessibility of data for industry users; (iv) adaptability to the changing needs of the dairy industry; (v) accuracy and integrity of the data recorded; (vi) the cost to industry users? f. Other matters that should be considered in reviewing or inquiring into (a) to (e) above including the role of industry good/dairynz investment and ownership. Conduct of review DairyNZ will advise the Committee on consultation procedures with New Zealand dairy farmers, herd recording agencies, farm advisory services, breeding companies and the Ministry of Agriculture and Forestry. This procedure will include a presentation to the stakeholders consulted before finalising the Committee s report. Completion of Review Having reviewed and inquired into the above matters, the Committee shall make whatever recommendations it considers appropriate and shall report them in writing to the DairyNZ Board. 21

24 22

25 APPENDIX B Submission by Brian Wickham. SUBMISSION TO: NEW ZEALAND NATIONAL DAIRY HERD IMPROVEMENT DATABASE REVIEW BY BRIAN W. WICKHAM (PHD) (14 TH JANUARY 2009) Chief Executive Irish Cattle Breeding Federation Society Ltd Highfield House Shinagh Bandon Co. Cork Ireland bwickham@icbf.com Web:

26 APPENDIX B Submission by Brian Wickham. TABLE OF CONTENTS 1 TERMS OF REFERENCE BACKGROUND PERSONAL TERMINOLOGY PURPOSE OF A NATIONAL DATABASE STAKEHOLDERS DESIGN CONSIDERATIONS STRUCTURE INTELLECTUAL CAPABILITY COMPUTING INFRASTRUCTURE FUNDING ACCESS SUMMARY RECOMMENDATIONS Terms of Reference This paper has been prepared in response to a request for a submission to address a question posed by the New Zealand National Dairy Herd Improvement Database Review Committee. That is; Given your knowledge of the features of New Zealand dairy farming industry, and in the light of impending technological developments and the changing business landscape, what is the preferred design for a national data recording and information management system to ensure optimal genetic gain in the national herd primarily, and also to provide scope for advancing on-farm decision making practice, over the next 25 years. 2 Background This section contains a brief summary of my background and thus the basis on which my opinions have been formed. More detail can be provided if required. 2.1 Personal The basis of this, my initial submission, is my experience and knowledge of the NZ scene and more recently in leading a redevelopment of the information systems used in Ireland. Here is a brief summary of the key points. NZ Experience. 20 years with LIC and predecessors through to December Led the development of LIC database and genetic evaluation systems. Focus was on genetic improvement and the provision of information services to dairy farmers. Ireland Experience Cattle. Last ten years (1998 to 2008) established the Irish Cattle Breeding Federation Society Ltd (ICBF) and led total redevelopment of cattle (dairy & beef) breeding information infrastructure in Ireland. Established: 24

27 Submission by Brian Wickham. APPENDIX B Integrated national database for cattle breeding shared by some forty organisations, including Herd Books (20), AI (8), Herd Testing (8), Research, Advisory, Milk Processing, Meat Processing, and Animal Health. This database is currently being used by over 90% of all cattle farms in Ireland. Breeding objectives and an across breed genetic evaluation system for all Irish cattle (dairy & beef). This system has maximising farm profit as its overall objective. It is linked with genetic evaluation systems servicing other international populations through Interbull and a number of bilateral arrangements. Theoretical optimal breeding scheme designs for dairy and beef. Implemented operational breeding schemes (G N IR LAND ), targeted at achieving optimal rates of gain for dairy and beef. The G N IR LAND program comprises a bull procurement and progeny testing service for a number of competing AI Companies, some of which have international affiliations. Ireland Experience Sheep. Over the last few years, I have become increasingly involved with Sheep Breeding in Ireland. In 2008 this culminated in a Department of Agriculture Food & Fisheries (DAFF) funded initiative to apply the experience from ICBF (for cattle) to Sheep. The contributions of the Irish Strategy Review Group led by Peter Amer (AbacusBio, NZ) are of particular relevance to the New Zealand review. A copy of the review is available in the publications section of our Sheep Ireland website ( Some of what I have to say below builds on the concept of supply chain information support contained therein. Interbull. While at LIC (and its predecessors) I was closely involved (for 13 years as chairman of the Steering Committee) with establishing this organisation that now plays a key role in facilitating the international exchange of genetic evaluation information for dairy cattle. Currently, I am leading a three-year effort to extend Interbull services to beef breeds and traits. I also have a close interest in extending Interbull services to facilitate the international use of genomic information in dairy and beef cattle breeding decisions. 2.2 Terminology Database for simplicity, I will use the word database to mean an integrated system for: Gathering and validating data, Storing data, Analysing data, Compiling information, and Distributing information. National database a national database is a database in which a defined set of data pertaining to one country is held and shared between multiple users. For the purpose of this submission the national database is restricted to the data of relevance to farm production in the NZ dairy industry. 25

28 Submission by Brian Wickham. APPENDIX B 3 Purpose of a National Database The purpose of a national database for the NZ dairy industry needs careful consideration and agreement of the key stakeholders. My assessment of the potential purposes for a national database includes the following: Information Services. These include: breeding information (from genetic evaluations), milk production (from herd testing), milk quality (from herd testing and bulk milk supply), meat production (from slaughter), reproduction (from calving, mating, pregnancy testing), and herd health (from disease testing). These information services may be provided by a number of different organisations and the database must be able to support information services to farmers from a number of different organisations. Supply Chain & Industry Profitability. The dairy industry is, in effect, a system for supplying milk products to customers around the world. The farm production sector is a key component of this supply chain. To maximise the overall profitability of the industry access is required to extensive and detailed information on a wide range of factors affecting production costs, milk quality, milk supply, quality assurance (including traceability for milk and meat) and future supply. A national database is a very effective means of meeting a large part, but not all, of this information need. Genetic Gain. The need for a national database to facilitate optimal rates of genetic gain is well established and I see little need to repeat the case here. On Farm Decision Support. Farm owners have a constant and on-going need for information to support strategic (long term), tactical (short term) and operational (day-today and minute-to-minute) decisions. Operational decisions tend to depend on locally (on-farm) accumulating data while tactical and strategic decisions require access to information derived from other farms, farm production research (using data from other farms and research farms), markets (national and international), and information about competing land (labour & capital) uses. Ready access by individual farmers to data from their own farms, from other farms nationally and internationally, is best facilitated through a well-organised highly responsive national database. The extent to which a national database can support operational decisions is partly determined by the speed and reliability of electronic networks connecting the farm to the national database. Knowledge Discovery. NZ dairy farming is rapidly evolving and is under constant pressure from international competition, a range of diseases and other threats, including environmental issues. The ability of the industry to thrive and prosper is heavily dependent on its ability to discover and exploit new knowledge. A national database is a very powerful tool for use in knowledge discovery. Quality Assurance. As the dairy supply chain becomes more sophisticated it becomes ever more important that all aspects of production, from conception to processing and distribution are traceable and any breakdowns can be readily contained. A national database is essential for world-class quality assurance for the NZ dairy industry. 26

29 Submission by Brian Wickham. APPENDIX B Animal Health. There are a wide range of conditions that can impact on animal health with consequences for production costs, product quality and human health. The identification, containment, control and eradication of animal health threats is greatly facilitated by the availability of a national database. To varying extents these reasons for a national database are complementary and synergistic. They extend beyond the terms of reference which are restricted to genetic improvement and on-farm decision support. In my view, you should consider these other benefits of a national database. While it is possible to provide many of the benefits of a national database with a number of purpose specific databases there is an inevitable increase in; duplication of effort (data collection, data storage, data processing), errors (different versions of the truth), and a degrading of functionality. Thus the alternative to a well run national database is more expensive, more error prone and less able to deliver benefits. 4 Stakeholders The creation and maintenance of an effective national dairy industry database requires strong and continuing support from a number of key stakeholder groups. These include: Farmers. Since much of the data held in a national database is farm and animal based, and a large part of the information output is directed to individual farmers, it is imperative that farmers are happy to have the data collected by them or on their behalf by service providers (AI, Herd Testing, etc), stored in the database and available for a wide range of potential uses. With the time periods involved it is important to recognise that farmers refers to those currently farming as well as those that will be farming in the future. Processors/Marketers of farm production milk & meat. Milk processors and to a lesser extent meat processors are an intermediary step in the supply chain for the NZ dairy industry. They collect data on product quality and quantity and provide signals in a number of forms including prices and quality grades. Effective management of this supply chain relies on feedback from the market to the processors and producer. The vice versa feedback from farmer to processor to market is also potentially valuable. For example, farm of origin labelling, and an organic production label. A national database fully supporting this link in both directions is only possible with processor and producer agreement. Farm Inputs Service providers. The providers of a wide range of services to farmers are key stakeholders in a national database. The key issues that must be addressed are data access, data ownership, service quality and funding. One of the greatest dangers occurs where one of the service providers exercises control over data access to the point that other service providers decide to operate outside the national database. There is 27