INVITATION TO TENDER for a Programme Evaluation of the Digital R&D Fund for the Arts Digital Arts & Media Team Request for Proposal Digital R&D Fund for the Arts Deadline for submissions: 17.00 GMT on 22 nd May 2015 1
1. Background to the Digital R&D Fund for the Arts The Digital R&D Fund for the Arts is a 7 million funding programme that supports collaboration between organisations with arts projects, technology providers, and researchers. It is a partnership between Arts Council England (ACE), the Arts and Humanities Research Council (AHRC) and Nesta, and was established in 2012, after a year-long pilot in 2011. The programme now has a portfolio of 52 funded R&D projects testing different technologies across a variety of art forms, either to reach audiences in new ways, or to develop new business models. Funding covers one year of activity, and all of the funded projects are due to complete by the end of September 2015. The fund partners are also delivering a dissemination programme designed to help the wider sector benefit from the platforms, resources, insights and processes made possible through the Fund. Arts Council England, AHRC and NESTA are jointly responsible for setting strategic objectives and for designing the Fund. NESTA is responsible for day-to-day operational delivery and grants management process through a small team based at Nesta in London. The team includes a dedicated Research Manager. Regular input is given by the Fund partners at ACE and AHRC, and consultants and researchers that include: 21Awake supporting funded projects to capture the learning from their activities MTM research consultants managing a three year longitudinal Digital Culture survey The Research Base have delivered an evaluation of the application process Elizabeth Lomas a post-doctoral research fellow investigating a definition of R&D in the Arts. There are two additional Digital R&D funds being run by Nesta and the AHRC, in Wales and Scotland. Projects supported by these funds are also on the Native web site, but for the purposes of this review the focus is on the English fund only. Purpose and objectives of the fund The overarching purpose of the Digital R&D Fund is: to enable use of digital technologies in the arts sector to engage audiences with art in new ways and/or to create opportunities for new business models for arts organisations The Co-operation and Funding Agreement between ACE, AHRC and Nesta includes the following aims and outcomes: The Digital R&D Fund will achieve a step-change in the innovative use of digital technologies by the arts sector in England. Success will entail the following outcomes: 1. A portfolio of inspiring case studies that enable wider, broader and deeper audience engagement with the arts and a better understanding of emerging business models 2. Strong networks and relationships between technology, research and the arts that inspire new collaborations beyond the Fund 2
3. A body of rich evidence, data and information that inspires and supports the capacity of the arts sector to innovate through targeted and effective dissemination of learning. 4. Demonstration of a successful model for supporting digital research and development within the arts sector 5. A rigorous definition of Research and Development in the arts sector that is used to inform policy development 6. New research insights in the arts and humanities 7. R&D funding adopted by other arts funders in the UK and beyond as an accepted model for supporting innovation in the arts sector. Funded projects Funding has been awarded to 52 collaborative R&D projects, each with its own research partner and tailored data strategy. Of the 52 projects, four are Big Data projects with a budget of up to 300,000, one is a Research+ project with a budget of less than 50,000 and the remaining 47 projects have a budget of 125,000 or less. Projects supported by the Fund cover: A range of art forms, including performing arts, literature and visual arts The commercial and subsidised sectors A mix of large and small-scale organisations All regions of England Projects which aim to explore business models and/or new ways of engaging audiences. The list of projects we are supporting can be found here: http://artsdigitalrnd.org.uk/projects/ Dissemination of learning The individual R&D projects have been funded because of their potential to generate valuable knowledge and insights on how the arts sector can use technologies to advance their missions. The dissemination plan for the Fund aims to share transferable lessons so that the wider arts and culture sector can benefit from the platforms, resources, insights and processes created by funded projects. The Fund s dissemination activities have been designed to address the outcomes listed in the Co-operation and Funding Agreement between the partners. Role of research and evaluation within the fund The programme aims to stimulate experimentation with digital technology in arts and cultural organisations, but the Fund itself is also an experiment in arts funding and programme design. Research and evaluation of the funding model has therefore been embedded from the outset, so that learnings from the administration of the programme can be shared internally amongst the partners. 3
2. What is being commissioned through this invitation to tender The purpose of the evaluation is to provide the Fund partners with an assessment of the effectiveness of the programme in meeting its overarching purpose and objectives, as described in the partners Cooperation and Funding Agreement (see, purpose and objective of the fund section). The evaluation may be used by the partners to inform their future policies and programme design, and its insights will be shared with the wider arts and culture and academic sectors. Evaluation questions In exploring the effectiveness of the Fund the evaluation will review: To what extent the fund has met its aims and objectives What factors have enabled the fund to meet its aims and objectives and what have been the barriers Evidence of whether the Fund s findings and outputs have been shared with and been useful to the arts and cultural sector Whether the design of the fund and its grant management processes were effective (including key features such as bringing together arts, tech and research, and the 12 month time period) Whether the partnership between ACE, AHRC and Nesta worked productively to support the delivery of the Fund What insights has the programme generated that might inform future funding for digital R&D in the arts and cultural sector. Following the logic model (see Appendix B), it will consider overall whether initial assumptions were correct, the inputs and activities were judiciously selected, and the outputs and outcomes were of a sufficiently high quality. The evaluation will take place concurrently with the completion of funded project. It will only, therefore, be able to make an early assessment of the outcomes achieved and draw preliminary conclusions about the potential of the programme to meet its longer-term development aims. Programme activities to be evaluated The Fund encompasses a wide range of activities that fall within the remit of this evaluation: Planning and set-up of the Fund, including pre-application support, expressions of interest, the applications process and the management of grants 1 The portfolio of 52 projects Support provided by the learning partner, 21Awake Digital Culture Survey the three year longitudinal benchmarking survey Research on the definition of R&D for the Arts 1 The application process has already been evaluated by a third party consultant The Research Base and should therefore not require additional research but the Research Base findings can inform the final report. 4
Dissemination activities, including a fund website (Native), a series of events, practical guides and toolkits and project reports. See Appendix A for a full list of dissemination activities. Methodology and available primary data We expect the process to involve a combination of: An initial workshop and initiation meetings Review of background information and programme documentation Analysis and synthesis of findings from research already commissioned in relation to the fund (process evaluation; Digital Culture Survey; post-doctoral study of R&D in the arts) Analysis and synthesis of findings emerging from funded projects (final reports) 2 Analysis of existing quantitative data Qualitative research, e.g. interviews with key stakeholders and project participants A draft findings workshop. Nesta would welcome any additional methodological or process-based recommendations for how to evaluate the programme through this invitation to tender. There is a variety of data available for analysis, generated both within the fund and by external suppliers to Nesta. This includes: Reports from individual funded projects (examples can be viewed here) Digital Culture research data from both funded and non-funded organisations over 3 years (year one report available here, year two report available here) Preliminary econometric research conducted by Nesta s policy and research team Website analytics (from the Native website) Feedback on events and other activities gathered from participants Applications data and data analysis already undertaken by the fund Evaluation report focussing on the applications process (prepared by The Research Base) Steering committee A steering committee will guide the evaluation, comprising: Sam Mitchell, Research Manager, Nesta (project manager and main point of contact) Hasan Bakhshi, Director Creative Economy Policy & Research, Nesta Emma Quinn, Senior Programme Manager, Nesta Tim Plyming, Director Digital, Arts and Media, Nesta Paul Glinkowski, Senior Manager, Creative Media, ACE Vivien Niblett, Senior Officer, Policy & Research ACE Sumi David, Strategy and Development Manager Research Impact and Sector Analysis, AHRC 2 Please note: the evaluation will not be expected to assess the impacts of all 52 projects, but present insight into a cross-section of the projects in consultation with the steering committee. 5
Draft reporting will be presented to the committee and subsequently to the Programme Steering Board. 3. Who is the audience for this work? Audiences What might their particular area of interest be? Fund partners Has the Fund met its objectives in relation to the objectives initially outlined? What was the overall impact of the Fund on the participating organisations and individuals and the wider arts sector? How useful was the Fund as a model for promoting R&D What have been the key challenges for the Fund? What are the recommendations for improvement? The evaluation is initially intended as internal document for Nesta, ACE and the AHRC, along with a summary report that can be published to an external audience. The wider audience for this summary evaluation may include: Arts and cultural organisations interested in digital R&D Other funders and policy makers Academic researchers Recipients of funding. 4. Outputs The outputs from the evaluation should include: An interim report A written final report for internal use, including a concise executive summary and conclusions A shorter summary report to publish for external audiences A PowerPoint Presentation of the findings to internal audiences Any datasheets or worksheets collated through the analysis of evidence phase, which may be of use to Nesta, ACE or the AHRC. 5. What qualities should the evaluator possess? We are looking for a programme evaluator with strong research and evaluation skills, including the ability to: Use a logic model or theory of change approach to structure programme evaluation Identify, synthesise and analyse a wide range of data sources, including programme and project data and relevant secondary data Elicit and use team insights and tacit knowledge Use a range of quantitative and qualitative research methods, and to judge the most appropriate approach to achieve strong results for the programme Identify and develop opportunities for learning and insight from the programme Present findings in a clear, compelling, accessible and credible way for our key audiences Represent data and findings using conceptual models and other visuals 6
It would also be desirable, though not essential, for the programme evaluator to possess demonstrable knowledge and experience of working with arts and cultural organisations, technology providers and academic researchers. The candidate will work from time of commission (June 2015) until four months after the funded programmes have been completed (December 2015). 6. Indicative Timetable Set out below is the procurement timetable. This is intended as a guide only. Date Timetable 01/04/2015 Issue ITT 22/5/2015 (17:00 GMT) Deadline for receipt of tenders. 26/05/2015 27/05/2015 Evaluation of submissions and notification of interviews 02/06/2015 Interviews with shortlisted applications Week commencing 08/06/2015 External evaluator appointed Week commencing 08/06/2015 11/06/2015 07/10/2015 Evaluation inception meeting Attendance of appointed organisation/individual at Digital R&D Fund for the Arts Learning Event Attendance of appointed organisation/individual at annual forum for the Digital R&D Fund for the Arts event 23/10/2015 Interim report submitted 1/12/2015 Final report submitted 7. Milestones and Budget Payments will be released in stages alongside agreed milestones, providing these have been met. The final milestones shall be clearly documented in any contract entered into as a result of this ITT, and in accordance with the terms and conditions of this ITT. The proposed budget for the above process evaluation services for the Digital R&D Fund for the Arts is up to 16,800 including all expenses and VAT (if applicable). 8. How to apply Proposals should be in English and no more than 6 A4 pages (excluding annexes) and comprise an outline of a methodology and proposals for metrics, enclosing any relevant attachments. 7
Your entire proposal including all supporting documentation should be e-mailed as a single document, to sam.mitchell@nesta.org.uk by no later than 17.00 GMT on 22 nd May 2015. Please submit your proposal to Nesta in accordance with all the instructions and terms and conditions set out within the enclosed documentation. The questions suppliers must respond to and the applicable selection and award criteria that will be applied to proposals are set out in Annex 1 of this ITT (Evaluation of Proposals and Timelines). Annex 2 sets out the Terms and Conditions. If you have a specific question not answered either above or in the Annexes to this ITT, please email sam.mitchell@nesta.org.uk Annex 1 Evaluation of Proposals 1. Evaluation Process Nesta will run this competitive procurement process in a manner which is open and transparent and ensures equal treatment of participants in the process. Decisions will be made on the basis of the stated criteria. At the first stage of this process your proposal will be evaluated against the selection criteria listed below. Nesta will select the top 3 highest scoring suppliers to the next stage which will involve interviews followed by a final selection based upon the information in your application and from the interview process. Selection Criteria Scoring/Weighting 1. Experience of organisation and individuals 25% Please provide a summary of your organisation and details of your organisations relevant experience of completing similar projects and provide up to 3 specific examples. In each case you should state: the work undertaken the timescale for completion methodology used how you dealt with issues that arose the value of the work the names and contact details of referees who will be able to comment on your work 8
If you are proposing a consortia provide the proposed management structure and details of the partner organisation. Please identify a lead that will be responsible for completing this form on behalf of all consortia members. In your response you should identify the organisation to which the experience relates. Max words: 500 2. Proposed methodology 35% Please describe the methodology and metrics you propose to use complete the project. Please reference your experience/expertise in this methodology and the relevant individuals who will be responsible for and actually performing the tasks. Max words: 500 Please include CVs for relevant individuals as appendices. 3. Overall potential of the organisation to deliver the project to time and to budget. 15% By reference to your previous experience, please describe how you ensured projects were delivered on time and within budget and of quality i.e. if the principle or lead is not doing the tasks ensuring that these are of a high standard. Max words: 300 9. Contract management arrangements 10% Please set out your proposals for: Scheduling the work within timeframes and deliverable milestones for each of the different phases of the project; and Reporting plans/arrangements describing how you will work with Nesta. Max words: 250 10. Value for money 15% Please provide a full breakdown of your proposed fees and pricing structure including the number of days, or estimated percentage of time, each team member is likely to spend on the project, set-up and future costs. Please also indicate which team member will lead on specific aspects of the work. Conflicts of Interest: 9
Not scored. Where conflicts of interest are identified Nesta reserves the right to exclude the supplier from further participation in this tender process. Please confirm if you are aware of any potential or actual conflicts of interest which may prevent you from working with Nesta. IF you consider a conflict may exist; please provide details in your response. 1.3 The following scoring methodology will be applied to responses: Score 1-3 4-6 7-8 9-10 Commentary Response provides Nesta with a poor- weak level of confidence in the supplier s ability to meet its requirements Response provides Nesta with a moderate-good level of confidence in the supplier s ability to meet its requirements Response provides Nesta with a very good level of confidence in the supplier s ability to meet its requirements Response provides Nesta with a high level of confidence in the supplier's ability to meet its requirements Annex 2 Terms and Conditions This appendix provides the terms and conditions which will govern your submission of a proposal to Nesta. You are required to read this appendix carefully together with the Invitation to Tender (ITT) brief (including all its annexes) before submitting a proposal. By submitting a proposal, you accept these terms and conditions and you agree to abide by them. If you do not agree to these terms and conditions please do not submit a proposal. 1. Submitting a proposal 1.1 By submitting a proposal you confirm that: 1.1.1 you have no actual or potential conflict of interest with Nesta (including Nesta s Trustees, officers or employees). If at any time during the ITT process you discover an actual or potential conflict of interest, please inform Nesta promptly; 10
1.1.2 you have legal capacity to submit a proposal in response to this ITT and are acting lawfully, ethically and in good faith in your dealings with Nesta; 1.1.3 you have sufficient time, skill, experience and resources to carry out the services to the highest professional standards expected of a competent supplier of services identical or similar and are able to obtain all necessary rights, licences, consents, waivers, approvals, permissions, permits, certificates and insurances necessary to provide the services to Nesta; 1.1.4 your tender will remain open for acceptance for a period of 90 days. A tender valid for a shorter period may be rejected; 1.1.5 all information contained in your proposal is true, accurate and not misleading; and 1.1.6 Nesta may share your proposal with any third party as Nesta in its absolute discretion deems necessary for the purpose of evaluation. 2 Compliance 2.1 Nesta reserves the right to reject or disqualify your tender where: 2.1.1 you fail to comply with the requirements of this ITT (including but not limited to these terms and conditions) or you are guilty of a serious misrepresentation in supplying information in response to this ITT; 2.1.2 your tender is received after the deadline set out in this ITT. Nesta will not consider any requests for an extension of the time or date fixed for the submission of responses; 2.1.3 there is a change in your identity, control, financial standing or any other factor impacting on the selection and/or evaluation of your tender; 2.1.4 you are or become insolvent or have a petition issued against you; 2.1.5 you do not have the economic and financial standing and/ or the technical and professional ability to carry out the services; 2.1.6 you are suspected either directly or indirectly of behaving in a collusive, canvassing, or anti-competitive manner or you offer or accept an inducement or reward in order to gain a commercial, contractual, regulatory or personal advantage; and/or 2.1.7 you (or if you are a commercial entity, a director or person who has the power of control or power to make representations or decisions on your behalf) have committed any offence relating to conspiracy, corruption, 11
bribery, fraud, money laundering or any other criminal offence related to your course of business or profession. 2.2 Nesta reserves the right in its absolute discretion to: 2.2.1 refuse any tender submitted; 2.2.2 disqualify any potential tenderer who has been convicted of any of the offences listed at Regulation 23(1) of the Public Contracts Regulations 2006; 2.2.3 extend the time or date for fixed submission. In such circumstances Nesta will endeavour to notify all tenderers of any change; 2.2.4 amend any aspect of this ITT (including but not limited to the evaluation criteria and the timeline) or cease the process at any time; 2.2.5 negotiate the award of additional services which are a repetition of the services advertised in this Invitation to Tender to the successful bidder; 2.2.6 limit the number of proposals invited to participate in any follow-up activity (for example, interviews) or to dispense with any follow-up altogether. Meetings will take place in London at our offices; and/ or 2.2.7 require that you clarify your tender in writing and/ or provide additional information and/or adequate references to Nesta s satisfaction. A failure to respond adequately may result in you not being selected. 3 Tenderer Responsibilities 3.1 You shall be responsible for all of your own costs, expenses and losses which may be incurred in relation to the preparation of your tender, attendance at interviews, site visits or similar. 3.2 You shall at all times treat the contents of Nesta documentation as confidential, as well as any information regarding Nesta imparted to you by any other means, and only disclose such information as may be necessary for the preparation of a compliant response. At Nesta s request you shall return or destroy all documents, other materials, working papers relating to this ITT and all copies thereof including all electronic copies. When completed you shall confirm such to Nesta in writing. 3.3 You shall not before the date and time specified within the ITT documentation disclose to any person the amount of your proposal except where the disclosure in confidence is necessary to obtain insurance premiums or guarantees required as part of any proposal to Nesta. 12
3.4 Any contract entered into as a result of this ITT will be on Nesta s standard terms and conditions (copy available on request). 4 Intellectual Property 4.1 All intellectual property rights in this ITT and all materials provided by Nesta or any third party acting on its behalf shall remain the property of Nesta. 4.2 Any intellectual property arising out of the provision of the services shall belong absolutely and exclusively to Nesta. 5 Warnings/Disclaimers 5.1 Nothing contained in this ITT or any other communication made in respect of it between Nesta or its representatives and any party will constitute an agreement, contract or representation between Nesta and any other party. For the avoidance of doubt, receipt by you of this ITT does not imply the existence of a contract or commitment by or with Nesta for any purpose. 5.2 The information contained in this ITT does not purport to contain all the information which you may require. While Nesta has taken all reasonable steps to ensure, as at the date of this ITT that the facts contained in it are true and accurate in all material respects, Nesta does not make any representation or warranty as to the accuracy or completeness or otherwise of this ITT. 5.3 Nesta accepts no liability to you whatsoever and however arising and whether resulting from the use of this ITT, or any omissions from or deficiencies in it. 6 Waiver Failure or neglect by Nesta to enforce at any time any of the provisions of these terms and conditions shall not be construed nor shall it be deemed to be, a waiver of our respective rights hereunder, nor in any way affect the validity of the whole or any part of this agreement, nor prejudice our respective rights to take subsequent action. 7 Jurisdiction This ITT shall be governed by and interpreted in accordance with the laws of England and Wales and subject to the exclusive jurisdiction of the English courts. 13
Appendix A: Dissemination activities The Fund s dissemination activities, listed below, are mapped to the Fund s objectives: 1. The Fund will share its portfolio of inspiring case studies that enable wider, broader and deeper audience engagement with the arts and a better understanding of emerging business models, through: Funding announcements Progress updates, insights, and results from funded projects Case studies of interesting R&D projects occurring internationally Features, blogs and articles on inspiring stories and examples Videos of project highlights. 2. The fund will foster strong networks and relationships between technology, research and the arts that inspire new collaborations beyond the Fund via: Learning events for funded projects and the broader arts, tech and research communities Annual showcase event 2014 Annual showcase event 2015 Native website the home of the digital R&D community Native magazine a print version of the website targeted at lessdigitally engaged stakeholders Features, articles and blogs on networks and relationships. 3. It will publish a body of rich evidence, data and information that inspires and supports the capacity of the arts sector to innovate through targeted and effective dissemination of learning. This will include: Basic project research and development reports, including data and other resources User friendly design of research and development reports, data and resources R&D toolkit, featuring tools to support collaboration beyond the fund Benchmarking survey (2013-2015) Hot topic events and research workshops Features, articles and blogs on evidence, R&D processes and management. 4. The Fund will spark discussion and evaluate its model for supporting digital R&D within the arts sector, through: Analysis of its applications and decisions Internal communications within partner organisations 14
An overall fund review with insights for policy Articles and blogs on conditions which enable to hinder R&D, and insights for funding and policy. 5. It will establish a rigorous definition of Research and Development in the arts sector, used to inform policy development, through: Conceptualising R&D post-doctoral research project Features, articles and blogs on R&D concepts and understandings Conferences and events for policy-makers and researchers. 6. It will deliver new research insights in the arts and humanities and strengthen the value and impact of arts and humanities research in the Creative Economy, via: Research insights, data and findings from projects Presentations at events Journal articles and conference papers. 7. The fund will aim to have R&D funding adopted by other arts funders in the UK and beyond as an accepted model for supporting innovation in the arts sector, through: Engaging with other funders in the UK and abroad. Demonstrating the impact of project funding. 15
Appendix B: Programme Logic Model Context Low rates of digital innovation in arts sector Lack of strong collaborations/partnership with tech companies Lack of capacity for digital R and D in arts sector Rationale To achieve a step change in the innovative use of digital technologies by the arts and cultural sector To provide a rigorous definition and understanding of digital Research and Development in the arts and cultural sector that is used to inform policy development Activities and Processes Partnership development Experimental use of digital tech for audience engagement and bus. models Strong research to test discrete hypothesis (es) as part of each project Inputs Finance from ACE, AHRC and NESTA Funds technology support Funds research support Short-term results Portfolio of R and D projects which includes some successful digital innovations for the participating organisations Large number of case studies of innovation projects, which inspire and are judged valuable by the sector Medium long-term results More partnerhsips between arts and tech orgs, resulting in better audience engagement and new digital bus models Greater audience engagement in sector using digital means Better understanding of R and D in the sector and for policymakers and researchers 16
17