The second year of value for money selfassessment:

The second year of value for money selfassessment: Insight to the sector experience and good practice January 2015 Steve Smedley with support from Liz Kenny HouseMark

Contents 1. About this publication... 2 2. Executive summary... 4 3. How was it for you? HouseMark s survey findings and observations... 7 3.1 The VFM self-assessment process... 7 3.1.1 Understanding what the VFM standard required... 7 3.1.2 The most challenging aspects of meeting the requirements... 8 3.1.3 Evidencing the return on assets... 9 3.1.4 Accessible and transparent communication... 10 3.1.5 Absolute and comparative costs and performance... 11 3.1.6 Evidencing past and future gains... 11 3.2 Comparing 2013/14 to 2012/13... 11 3.2.1 Regulator s response to members 2012/13 self-assessments... 11 3.2.2 Time and resources... 13 3.2.3 Outcome... 13 3.3 Regulating VFM... 14 3.3.1 VFM self-assessment: an appropriate regulatory tool?... 14 3.3.2 What s the alternative?... 14 3.3.3 Decouple the VFM self-assessment from the accounts?... 15 3.3.4 Improving VFM regulation... 16 3.3.5 Benefits of VFM self-assessment... 18 3.4 HouseMark data usage... 19 4. What might a good VFM self-assessment look like?... 21 4.1 Strategy: reporting the approach... 22 4.2 Assets: reporting the approach... 24 4.3 Operational cost and performance: reporting the approach... 29 4.4 Reporting VFM gains... 35 4.5 Transparent and accessible communication... 39 5. Conclusions... 47 Acknowledgements and References... 48 1

1. About this publication This publication should be of use to those with an interest in improving the approach to VFM self-assessment. You do not have to read it all. The following should help you dip in and dip out - there are two main sections: How was it for you? HouseMark s survey findings and observations (Section 3) Last year HouseMark took the view that it would be useful to understand providers experience of the first year of VFM self-assessment and publish the findings as a learning tool for the sector 1. We still think that is a good idea and have done it again, not least because the sector did not seem to get off to a good start last year. Seventy-one members responded to our survey this time against 66 last year. Respondents are self-selectors they chose whether to respond and this might infer a degree of enthusiasm (and therefore bias) for the subject. Section 3 is structured around the survey questions, as follows: The VFM self-assessment process Understanding what the VFM standard required The most challenging aspects of meeting the requirements o Evidencing the return on assets o Accessible and transparent communication o Absolute and comparative costs and performance o Evidencing past and future gains Comparing 2013/14 to 2012/13 Regulator s response to members 2012/13 self-assessments Time and resources (devoted to the self-assessment) Outcome (did members feel they did a better job?) Regulating VFM VFM self-assessment: an appropriate regulatory tool? What s the alternative? Decoupling the self-assessment from the accounts? Improving VFM regulation Benefits of VFM self-assessment HouseMark data usage What might a good VFM self-assessment look like? (Section 4) This section seeks to build on some of the observations made in section 3 and to set out what a good self-assessment might look like without resorting to a template. First we establish the hallmarks of good practice and then seek to illustrate those hallmarks with appropriate examples from the sector. We are not endorsing any individual self-assessment in its entirety 1 The first year of value for money self-assessment: insight to the sector experience http://bit.ly/vfmsayr1 2

and we are certainly not second-guessing the HCA s VFM judgement; instead we try to focus more on what might be described as objective good practice. This section is structured as follows: Strategy: reporting the approach Assets: reporting the approach Operational cost and performance: reporting the approach Reporting VFM gains Transparent and accessible communication As with last year, at the time of writing this report the HCA is conducting its own evaluation of the VFM self-assessments from the perspective of regulatory compliance. It is likely to communicate its findings early in 2015. Whilst the regulator will offer a view on the quality of self-assessments and the extent to which the sector has complied with its expectations, it is highly unlikely to accompany this with detailed guidance, as to do so would conflict with the principles of co-regulation. Conclusions (Section 5) Drawing on the previous sections, we set out the key challenges going forward. 3

2. Executive summary Section 3. How was it for you? HouseMark s survey findings and observations Who participated in the survey? Responses were received from 71 organisations. There was a good regional spread of responses, with the North West particularly well represented. The average (median) stock size of the associations that responded was around 6,300. There is a good spread across the range of stock sizes, ranging from 653 to almost 55,000. Understanding what the VFM standard required We asked whether landlords had a better understanding of what the HCA required this year, and encouragingly, 97 per cent of those answering the question said yes, although 54 per cent felt that the requirements are still not as clear as they need to be. Of these, almost six in ten said they were unsure about return on assets. The most challenging aspects of meeting the requirements: return on assets and communicating the required information Over four in ten answering this question found evidencing the return on assets the most challenging, with a further quarter having difficulties communicating the required information in an accessible and transparent matter. Those finding return on assets most challenging were asked for their reasons. They may be summarised as follows: not understanding what is required strategic asset management within the sector is still in its infancy and so a common understanding of the discipline has not been reached local asset management is in development and so there are limitations to our understanding. Those finding accessible and transparent communication most challenging were asked for their reasons. They may be summarised as follows: not understanding what is required HCA fear factor results in word count bonanza VFM self-assessments have become so lengthy and complex that they risk undermining transparency reconciling what the HCA wants with the needs of other stakeholders is this a de facto regulatory return or an exercise in public accountability? The sector appeared more comfortable this year in terms of providing cost and performance data and quantifying gains. Comparing 2013/14 to 2012/13 Whilst many respondents did not feel the HCA handled the fall-out from the first year of VFM self-assessment particularly well, a significant minority of those receiving downgrades and noncompliance letters feel that the HCA s response was justified. Over four-fifths of respondents had devoted more resources to the self-assessment this year than last year, with over half claiming to have applied a lot more time and resources to the exercise. More than nine out of ten organisations said that they felt the outcome of their VFM selfassessment this year is better than last year. No-one felt their outcome was worse. 4

VFM self-assessment: an appropriate regulatory tool? Just over three-fifths of our respondents said it was, but nearly three in ten said no, while around one in ten was unsure. Those that answered no expressed a range of preferences from a limited prescribed return to regulatory engagement. However, respondents generally see value in the exercise beyond regulation (see below) and think it is an appropriate regulatory tool, suggesting that the future should be about improving VFM self-assessment rather than devising an alternative. Decouple the VFM self-assessment from the accounts? Assuming that VFM self-assessment continues to be a regulatory requirement, we asked should it be decoupled from the annual accounts. Seventy per cent of respondents supported the proposition. Key reasons: it is not the kind of information that should be included in the audited accounts the timing makes it difficult to provide the latest comparative cost and performance data. Improving VFM regulation We asked what single thing the HCA could do to make VFM regulation work better. The responses may be summarised as follows: decouple the VFM self-assessment from the accounts improve communication and feedback from HCA increase prescription reduce prescription change the approach to regulation. Benefits of VFM self-assessment Almost nine out of ten respondents answered yes when asked do you see any benefit in going through the VFM self-assessment process beyond regulatory compliance? The benefits were described as improved: focus on continuous improvement communications cultural awareness of VFM assurance. HouseMark data usage The vast majority of HouseMark members used HouseMark data in their self-assessments. Section 4. What might a good VFM self-assessment look like? This section seeks set out what a good self-assessment might look like. First we establish the hallmarks of good practice and then seek to illustrate those hallmarks with appropriate examples. Section 5. Conclusions It is clear the sector took this year s VFM self-assessment far more seriously with the result that the output is improved. Progress is being made in the sector s approach to return on assets but we still have a way to go before we can say that evidence-based, active asset management has been mainstreamed. There is still some uncertainty as to what the HCA requires. Arguably, the biggest challenge for associations going forward in respect of the VFM selfassessment is how to make it fit for all stakeholders. For many, this means cultural change in the organisation s approach to transparency and thinking time to improve the practice of thoughtful and accessible communication. 5

In measuring VFM, we should not lose sight of the fundamental regulatory (and ethical) requirement to strive continuously to improve VFM. This means associations need to evidence performance over time as well as in comparison to others. The measurement of VFM and social value is advancing but we still have some way to go before the majority of associations can confidently point to a set of metrics and other forms of evaluation and say this is the evidence base for our VFM/social value story. In the spirit of co-regulation, there are lessons in this paper for both associations and regulator. In the VFM self-assessment, we might have the makings of the best VFM regulatory tool we have had to date. Let s make it work! 6

3. How was it for you? HouseMark s survey findings and observations Who participated in the survey? An invitation to complete an online survey about members experience in producing their VFM self-assessment was sent to around 250 Finance Directors of English housing associations, mainly HouseMark members. Responses were received from 71 organisations. There was a good regional spread of responses, with the North West particularly well represented. Table 1 Geographical distribution of responding associations Region Total Percentage East 8 11% East Midlands 4 6% London 11 15% North East 2 3% North West 19 27% South East 7 10% South West 9 13% West Midlands 4 6% Yorkshire and the Humber 7 10% Total 71 100% The average (median) stock size of the associations that responded was around 6,300. There is a good range of stock sizes, from 653 to almost 55,000. Table 2 Stock size of responding organisations Stock band Total Percentage Under 2,500 11 15% Between 2,501 and 5,000 11 15% Between 5,001 and 10,000 24 34% Over 10,000 25 35% Total 71 100% 3.1 The VFM self-assessment process 3.1.1 Understanding what the VFM standard required In our 2013 survey we found that associations had struggled to determine what the regulator was looking for in the value for money self-assessments. This is likely to have been a factor in the downgrading of the governance ratings of 14 organisations and around 160 landlords receiving a letter from the HCA telling them they needed to 7

improve their value for money self-assessments in 2014 or risk a governance rating downgrade. We asked whether landlords had a better understanding of what the HCA required this year, and encouragingly 97 per cent of those answering the question said yes, although half (54 per cent) felt that the requirements are still not as clear as they need to be. Respondents were asked which three factors contributed most to their improved understanding of the requirements. Advice from HouseMark came out as the highest ranked of the options listed in the questionnaire although, as one would expect, guidance and statements from the HCA in various forms were in combination the most important, with the guidance in the 2013 Global Accounts being the single most useful source of advice from the regulator. Other factors mentioned by respondents included direct discussions with the HCA, learning from peers though discussions and looking at good examples of selfassessments from 2013. The 54 per cent of respondents who felt the requirements are still not as clear as they need to be were asked to identify the principal area of uncertainty. Almost six in ten said they were unsure about return on assets. Table 3 Respondents' principal areas of uncertainty about what the VFM standard required Principal area of uncertainty Total Percentage Return on assets 22 58% Evidencing past and future gains 3 8% Absolute and comparative costs (and performance) 2 5% Other 11 29% Total 38 100% Eight of those answering other were either unsure of what was wanted, particularly in terms of the detail required, or how the HCA would assess their statements. The measurement of social value was cited as an area of uncertainty for three organisations. 3.1.2 The most challenging aspects of meeting the requirements Over four in ten expressing an opinion found evidencing the return on assets the most challenging issue to address in their self-assessment, with a further quarter having difficulties communicating the required information in an accessible and transparent manner. Evidencing absolute and comparative costs and performance and past and future gains were each cited by around 12 or 13 per cent of respondents. 8

Table 4 What respondents found most challenging The most challenging aspects Total Percentage Evidencing the return on assets 28 41% Communicating the required information in an accessible and transparent matter Evidencing absolute and comparative costs (and performance) 18 26% 9 13% Evidencing past and future gains 8 12% Other (please specify) 6 9% Total 69 100% Other challenges reported were: Quantifying added social value in a meaningful manner to assist the business and compare with the sector Linking past efficiency and investment decisions The amount of time taken this year to produce the statement Knowing when to stop 3.1.3 Evidencing the return on assets Those finding this aspect of the self-assessment most challenging were asked for their reasons. They may be summarised as follows: not understanding what is required, as discussed at 3.1.1 Clarity about what assets to include. Should it be primarily stock and approach to asset management strategy or should it include return from investment in staff or social inclusion activities. Also should we be using ROA formulae in this section and comparing with different types of housing, ie sheltered, supported, market rent. Arguably the return on assets encompasses the whole of our operations (as in we use our resources to deliver our service). It was hard to understand precisely what the HCA were expecting and even harder to then know whether our performance in this area was good or bad. There seems to be the presumption that yield is the most important factor when other factors such as demand are just as important. strategic asset management is still in its infancy within the sector and so a common understanding of the discipline has not been reached It means so many different things to different people and is a culture shift away from how we think about our finances generally There are a number of alternative return on asset measures and no clear best practice Because there is no universal methodology underpinning this or allowing comparison Easy enough to give an "accounting" figure on ROI, but far harder to measure all the other factors that need to be taken into account 9

Differing measures used for different purposes - so trying to cover all leads to lengthy information that may not be particularly helpful to the reader local asset management is in development and so there are limitations to our understanding We are currently developing our systems to allow us to calculate the return on assets at an individual property level Limitation of in-house systems - requires further work Haven't got a robust mechanism in place to report return on assets 3.1.4 Accessible and transparent communication Those finding this aspect of the self-assessment most challenging were asked for their reasons. They may be summarised as follows: not understanding what is required as discussed at 3.1.1 Lack of clarity from the HCA as to what they considered to be a transparent assessment. This is where there hasn't been any clear guidance from the HCA other than 'accessible and transparent' which could be interpreted in various ways. HCA fear factor results in word count bonanza Terrified of missing something out which may make the difference between fully and partially compliant, so the document becomes long and complex. From our experience last year we didn't want to omit anything in our selfassessment that would leave us open to criticism from the HCA. reconciling what the HCA wants with other stakeholders With the information the HCA seem to want it is difficult to present the level of analysis and detail in an accessible manner. We addressed this by producing a complete "detailed" version and a hopefully more user-friendly summary version. So much is required and do not want to dilute the importance of the financial statements. Our report has to be wide-ranging to cover our diverse activities, in order to embrace each of the elements of VFM this means our report has grown in size and detail. This makes the report very transparent, but probably less accessible to tenants. To describe elements such as return on assets properly does need quite detailed and technical information. I've yet to see a statement which I think ticks the HCA boxes and is accessible. Worried about meeting the requirements of different stakeholders, but need to maintain the principle of 'would your mum understand it'. Still not sure how much detail we should be providing to tenants and the level it is pitched at. 10

3.1.5 Absolute and comparative costs and performance Those finding this aspect of the self-assessment most challenging were asked for their reasons. They may be summarised as follows: unavailability of comparative data at the time the self-assessment was finalised 12/13 HouseMark data available not 13/14 which relates to the year we were reviewing. The comparative L9 report produced by HouseMark is generally not available until November which is after the VFM Statement deadline. difficulties associated with making valid comparisons [we] drive initiatives for improvements made from customer-centric feedback and not always traditional methods. Consequently it is not always 'like for like' comparatively. Difficulty in finding similar organisations in respect of diverse supporting people operation local managerial changes have disrupted the collection of data We are in the process of developing a performance management framework to improve internal reporting and scrutiny Profound changes in our structure and operation in 2013/14 made comparison analysis a challenge 3.1.6 Evidencing past and future gains Those finding this aspect of the self-assessment most challenging were asked for their reasons. They may be summarised as follows: not having the systems to adequately capture and quantify past gains We hadn't gathered enough quantitative evidence of what we've been doing and what we plan to do. Specific detail not tracked at all levels. We make savings and often reinvest and achieve huge social benefit from these investments. Overall costs for the business don't appear as low as they could so it looks as if we didn't make the savings we should. not having the systems to adequately determine and quantify future gains or targets I think we still lack detail on future gains in terms of financial outcomes and targets. Our targets are not clearly specified. Future gains - whilst we have individual initiatives to improve VFM we don't have an overarching VFM plan which states where we want to be in respect of VFM in two/ three years Organisation not good at setting targets and measuring outcomes. Particularly difficult for social value 3.2 Comparing 2013/14 to 2012/13 3.2.1 Regulator s response to members 2012/13 self-assessments We asked respondents whether their governance ratings had been downgraded last year as a result of their VFM self-assessment, or if they had received an advisory letter from the HCA and whether, looking back, they felt the regulator s decision had been justified. 11

Table 5 Did respondents feel the 2013 regulatory response was justified? Regulator s response in 2012/13 Number Decision justified? Yes Governance downgrade 4 2 2 Advisory letter 47 20 27 None 17 Totals 68 22 29 A year later, slightly fewer than half of those criticised by the regulator feel that the response was justified. One respondent whose organisation was downgraded believes there should have been some flexibility in the first year. Another is ambivalent: Looking back the HCA were under pressure from central government to give a strong message out to the sector. I find it difficult to say whether the HCA's response was justified or not. It did however give a clear signal that there was a change in attitude. The most common complaint from those who received an advisory letter was that they were judged solely on the basis of information included in the Operating and Financial Review of their annual accounts, and not on more detailed information published elsewhere or having been sent direct to the HCA. Others felt that the requirements were unclear or that they had been misled by their regulatory contacts. There were concerns about the consistency and transparency of the HCA s approach: Some of the areas where we were asked to improve were (I think) of a higher standard than the same areas in organisations that were judged to have no need to improve. So overall I accept that we could have produced a better statement, but the advisory letter did not seem particularly grounded in fact. Response may be justified as we could do better but it was unclear how they actually assessed our statement and what they used to do so. And some are clearly still baffled as to why they received the letter: We still have no idea why we got the letter, we thought that we had addressed all areas. Given the lack of clarity and guidance the HCA should have been more forthcoming in areas in which they had concerns. Nevertheless, while challenging the justification for the letters, some acknowledge that they had some positive spin-offs: it helped improve our focus the letter stimulated more interest in VFM both at Board and senior manager level. As suggested in HouseMark s last briefing on VFM, Where are we now with VFM selfassessment? 2, it is perhaps not helpful to dwell on the past other than to draw on the lessons learnt, particularly around the need for: No 2 http://bit.ly/vfmwawn 12

the HCA to think about how best to communicate its expectations to minimise uncertainty whilst looking to ensure its approach to assessment is consistent and transparent (this is touched on again at 3.3.4 Improving VFM regulation) the sector to appreciate that VFM is indeed being actively regulated and that there is no room for complacency with regards to the self-assessment. It is interesting to note the extent to which offenders felt they deserved their reprimand. Such a mind-set hopefully augurs well for a reformed and redoubled effort this time round. 3.2.2 Time and resources We asked how the time and resources devoted to carrying out and evidencing the VFM self-assessment in 2013/14 compared to 2012/13. Over four-fifths of respondents had devoted more resources to the self-assessment this year than last year, with over half claiming to have devoted a lot more time and resources to the exercise. Table 6 Comparison of time and resources taken in 2012/13 and 2013/14 This year compared to last Total Percentage A lot more 37 54% A little more 20 29% About the same 8 12% A little less 1 1% A lot less 0 0% Don't know 2 3% Total 68 100% 3.2.3 Outcome More than nine out of ten organisations responding to the survey said that they felt the outcome of their VFM self-assessment this year is better than last year s fortunately but unsurprisingly, no-one felt their outcome was worse than last year. Table 7 Comparison of outcomes in 2012/13 and 2013/14 This year compared to last Total Percentage A lot better 43 64% A little better 19 28% About the same 4 6% Don't know 1 1% Total 67 100% 13

3.3 Regulating VFM 3.3.1 VFM self-assessment: an appropriate regulatory tool? We asked whether, given that all new initiatives need time to bed in and a degree of tweaking, VFM self-assessment, in principle, is an appropriate tool for regulating value for money. Just over three-fifths of our respondents said it was, but nearly three in ten said no, while around one in ten was unsure. Table 8 Views on VFM self-assessment as a regulatory tool Is it appropriate? Total Percentage Yes 42 62% No 20 29% Don't know / no opinion 6 9% Total 68 100% 3.3.2 What s the alternative? We asked those who answered no to the previous question what they felt was the alternative. We suggested three possibilities, and gave an option for respondents to make their own suggestions. Of our suggestions, a limited, prescribed regulatory VFM return incorporating metrics and qualitative information was most popular, with seven associations supporting it. Table 9 Views on alternative regulatory tools for VFM What s the alternative? Total Percentage A limited, prescribed regulatory VFM return incorporating metrics and qualitative information Regulatory engagement, eg attendance at board meetings, structured interviews with key players A limited set of prescribed VFM metrics and ratios spanning key costs and outputs / outcomes 7 35% 3 15% 2 10% Other (please specify) 8 40% Total 20 100% However, 40 per cent of respondents answering this question had their own ideas to contribute. Surely the VFM of this sector should be linked to our long term financial plans and what we demonstrate that we can achieve in these. A formal regulatory role for resident scrutiny panels, as per the former TSA's apparent intentions under the localism bill. Our residents should be the main arbiters of value for money, not a disconnected and under resourced central regulator. A limited set of measures related to the objectives of the association. Some associations will measure VFM through tangible outcomes (new homes, SAP ratings 14

etc), while others focus on "Housing plus" services. These should reflect the vision of the association. A limited and more prescribed return incorporating metrics with qualitative information AND additional statement about RP's operating context (every organisation is different). This issue tends to excite passions. Just asking the question is likely to raise the blood pressure of some in the sector, but it is important: governments and regulators have struggled to get VFM regulation right for years and the events of last year suggest the self-assessment (just the latest in a line of tools) is not there yet. So, how should VFM be regulated? The responses indicate a diverse range of perspectives which are all dwarfed by the fact that three-fifths of respondents are happy to stick with the relative non-prescription of the self-assessment. Those who felt the self-assessment is not an appropriate tool tended to either want more prescription or for VFM to be absorbed into wider regulatory engagement (this issue is revisited at 3.3.4 under change the approach to regulation. This suggests that the future should be about perfecting the VFM self-assessment as an effective business and transparency tool which also provides regulatory assurance essentially what Section 4 is about. 3.3.3 Decouple the VFM self-assessment from the accounts? Assuming that VFM self-assessment continues to be a regulatory requirement, we asked should it be decoupled from the annual accounts. The majority (70 per cent) of respondents supported the proposition in the question. Table 10 Views on whether VFM self-assessment should be decoupled from the accounts Decouple? Total Percentage Yes 47 70% No 14 21% Don't know / no opinion 6 9% Total 67 100% Those answering yes were asked for their reasons. We suggested two responses, and gave an option for respondents to make their own suggestions. Table 11 Reasons for favouring decoupling Reasons Total Percentage 3 It is not the kind of information that should be included in the audited accounts The timing makes it difficult to provide the latest comparative cost and performance data 30 64% 23 49% Other 14 30% Total answering yes to decoupling 47 3 More than one reason could be given. 15

Almost two thirds stated that the VFM self-assessment is not the kind of information that should be included in the audited accounts, while half agreed that the timing makes it difficult to provide the latest comparative cost and performance data. Other reasons given were: In order to give a transparent VFM assessment in the OFR, it has resulted in a large increase in the size of the financial statements. We feel that this actually takes away from some of the relevance of the information in the OFR as some other key messages could be lost as readers concentrate on the VFM section. It's too detailed and makes our accounts really, really long! There is also a risk that people only publish in the accounts and therefore that a lot of tenants would never look at it It s a lengthy document and whilst a summary can be provided it works better as a stand-alone document Whilst it's linked information, it increases the size of the annual accounts and creates duplication over where things are reported. It unbalances the OFR and, more importantly, sends the message that VFM is about improving the bottom line HouseMark has devised a range of solutions 4 to enable members to include the latest data in their self-assessments. However, what is clear is that internal processes regarding accounts sign-off require the assessment to be finalised sometimes two months or more before the September deadline. The result is, notwithstanding HouseMark and its members efforts, many self-assessments have to include data that is 18 months old which clearly limits the usefulness of the exercise. Decoupling solves this problem and of course the contamination of financial documents with what many in the accounting profession feel is extraneous verbiage. The HCA needs to think through the pros and cons of decoupling. 3.3.4 Improving VFM regulation We asked what single thing the HCA could do to make VFM regulation work better. The responses may be summarised as follows: decoupling revisited Allow associations to publish a very basic summary of the self-assessment in the annual accounts on the understanding that they will publish their self-assessment in places where tenants will actually read it. improved communication and feedback from HCA Communicate better with the sector. The refusal to share a list of organisations that had been judged to have passed was frustrating and rather pointless (it just meant we all had to spend time finding out this for ourselves). The communication and feedback process was poor, unless it was deliberately designed to provoke a feeling of paranoia, in which case it was very well handled. I would very much welcome the HCA actually writing to organisations to give them the all clear, rather than worrying if you just haven't heard yet. 4 HouseMark introduced an early reporting phase into the timetable this year. Members signing up to this timetable received their benchmarking report in time to include results in their VFM self-assessment. Some members were not in a position to submit their cost data by the early data due date but still wanted to include comparative performance data in their self-assessment. Performance data submitted into PI tracking at the year-end were validated early and made available for members to use in their selfassessments. It is possible to combine this data with the previous year s inflated cost data. 16

Rather than just issuing a letter of non-compliance, feedback as to why the SA was not compliant would have assisted greatly. Whilst we believe that we have improved our [self-assessment], we are no clearer in understanding what the regulator actually wants from the process. Better individual feedback - as we were compliant last year we received no feedback at all until we asked for it and then it was vague and informal. Clarity on how they will assess VFM. Early and specific guidance that also reflects their learning this year. Be consistent in its judgement of different self-assessments Highlight statements that it feels embody the regulations. more prescription with regards to the self-assessment Prescription of a limited set of common measures Better guidance. This has become a consultant's charter. less prescription I think it is for associations to understand that the VFM agenda is an opportunity to demonstrate that they are responsible for running efficient and effective organisations Leave it to Boards change the approach to VFM regulation Tell the government that the sector could withstand a one or two year rent freeze. Come in and talk to organisations about what they have prioritised and their decision-making process. Provide a specific VFM rating rather than linking it to governance, and clarify that what represents VFM is specific to the objectives of that particular organisation. Focus on embedding of VFM rather than reporting Include VFM regulation within the regulatory engagement visits The biggest change is to stop calling it VFM Self-Assessment. It is better called Business Strategy to avoid it looking like a standalone activity instead of the "way business uses its resources and gets maximum value from resources it has". Use of resources statement would be even better than VFM. Understand that VFM is a core business activity and not a separate silo Manage the bad press and messages across the sector. The concept and tool in itself is a very good thing. Either be more or less specific - the current position leads to uncertainty about what is required/acceptable Understanding the difficulties of benchmarking specialist and diverse supported housing providers The responses here pick up issues previously covered, ie: the need for HCA to better communicate its expectations and approach to assessment and to be consistent in its approach decouple the self-assessment from the accounts 17

those wanting an alternative approach to self-assessment that is not more prescriptive tend to gravitate towards VFM being integrated into the HCA s engagement process. 3.3.5 Benefits of VFM self-assessment Almost nine out of ten respondents answered yes when asked do you see any benefit in going through the VFM self-assessment process beyond regulatory compliance? This is up on last year when 75% responded positively to this question. Table 12 Views on the wider benefits of VFM self-assessment Benefits beyond regulatory compliance? Total Percentage Yes 59 87% No 9 13% Total 68 100% The benefits they described were: improved focus on continuous improvement It is a useful exercise to focus minds on identifying efficiencies and assessing performance on VFM indicators It has been invaluable in highlighting some areas of the business that have not been performing as well as may have been thought. The SA forces us to put a spotlight on the business, ensuring we follow up on initiatives and perform a benefits realisation exercise to ascertain that services are actually delivering what they are intended to deliver It is useful to pull together all we do in the field of VFM and to force thought about and understanding of the issue. [it] allows us to mould this work to our own business objectives, which makes it valuable. All of the aspects are things we should be considering, but based specifically on our business. It focusses attention for corporate planning and offers an insight into areas to improve We will use the statement as our primary VFM monitoring tool. improved communications It also led us to realise there are good things we do that we haven't fully communicated in terms of the wider value they deliver to our stakeholders. Yes, useful to create more awareness and internal debate about costs of delivering different services and outcomes achieved. It gives us a fully rounded picture of the organisation. Helps prompt the dialogues and action about what we can do better. Provides a straight-forward way in to have a dialogue on finances and performance with residents It did have the benefit of us stopping to take stock and to realise how much we had achieved. It also gave the opportunity for us to tell our story to stakeholders. improved cultural awareness of VFM It's not about process per se it s about organisational culture. 18

2nd time around it has been owned across the business, rather than being seen as something the finance team prepare. The process enabled us to raise more awareness of VFM internally and embed it more. It has made officers more accountable and clearly links to how best to achieve our corporate objectives improved assurance It is a fabulous business assurance tool - we measure our VFM against our corporate objectives - therefore we really are looking at our overall business effectiveness. But clearly there are some that do not see the benefit: We should do this for ourselves, understanding the efficiency and value of your business should not need the regulator to tell us to do it I'm not sure the cost of putting it together is value for money. How many people actually read these assessments? Do they influence decisions? It's an expensive, time-consuming, meaningless distraction. For the very largest organisations I can understand there may be some worth, but for smaller associations, I cannot understand who in the world, beyond the HCA would be interested in the document. As with last year s survey, despite various niggles, the vast majority of respondents see value in the VFM self-assessment process as a business tool. This is very positive and again underlines the point that the future should be about perfecting the VFM selfassessment unless somebody can come up with a better idea. This means both the sector and HCA (mindful of the findings at 3.3.4) working together in a co-regulatory fashion to deliver a desired shared outcome a tool that helps drive continuous improvement whilst representing a step change in public accountability. 3.4 HouseMark data usage The vast majority of HouseMark members used HouseMark data in their selfassessments. Table 13 Use of HouseMark data in VFM self-assessment Use HouseMark data? Total Percentage Yes 62 94% No 4 6% Total 66 100% Those using HouseMark data were asked how current was the latest data used. One third used 2013/14 cost and performance data and a further 11 per cent used 2013/14 performance data and uplifted their 2012/13 cost data by inflation. 45 per cent chose to use the previous year s data for cost and performance. 19

Table 14 HouseMark data-sets used in VFM self-assessments Which HouseMark data-sets? Total Percentage 2012/13 cost and performance data 28 45% 2013/14 cost and performance data 21 34% 2012/13 cost data uplifted by inflation and 2013/14 performance data 7 11% Other 6 10% Total 62 100% Those not using HouseMark data were asked how they sourced comparative figures, and a couple of those using HouseMark data also mentioned additional sources. These included statutory accounts, HCA Global Accounts, benchmarking and assessments from other membership organisations and past organisational performance. The difficulties the sector had in terms of providing 2013/14 data whilst still meeting the HCA timescale have been discussed previously. Decoupling the self-assessments from the accounts appears to be the sector s preferred solution. As things stand the various options available to HouseMark members are explained in footnote 4 on page 16. 20

4. What might a good VFM self-assessment look like? In this section we seek to build a sense of what a good VFM self-assessment might look like drawing, where appropriate, on the findings in section 3. First we establish the hallmarks of good practice and then seek to illustrate those hallmarks with appropriate examples. We are not endorsing any individual self-assessment in its entirety and we are certainly not secondguessing the HCA s VFM judgement. We have structured this as follows: Strategy: reporting the approach Assets: reporting the approach Operational cost and performance: reporting the approach Reporting VFM gains Transparent and accessible communication But before delving into the detail it is worth standing back and considering some key principles. VFM is about the maximisation of the association s product as set out in its mission. The expectation, set out in section 2 of the regulatory framework, is that associations develop and deliver a strategy to continuously improve VFM, and then in the interests of public accountability, demonstrate it (see diagram 1). Continuous improvement implies a beneficial and measurable change in cost and/or performance or outcomes, often described in terms such as more for less, more for the same and same for less. Trend, therefore, is an important consideration, but it is not routinely addressed in all self-assessments. Arguably the key variables around which a VFM story may be told are: o movement in operational costs, and associated with this the use of gains and surpluses (potentially in conjunction with the prudent use of borrowing) to achieve additional outcomes/value o movement in performance/outcome measures: service quality, new services, new homes, better use of existing stock, lives improved, etc. Diagram 1: The regulatory principle underpinning the VFM standard VFM goes to the heart of how providers ensure current and future delivery of their objectives: boards expects to: - develop and deliver a strategy to achieve continuous improvement in their performance on running costs and the use of their assets. - be transparent and enable scrutiny of their performance. Regulatory framework page 10 Self-assessment is an exercise in transparency and accountability Continuous improvement VFM strategic approach delivery systems transparency Comparisons are important too in terms of stakeholders understanding relative cost and performance, what is possible and what needs to be improved. 21

The VFM self-assessment is an exercise in public accountability, not just a de facto regulatory return (see section 4.5) it is the means by which the recipient of public funds transparently communicates to stakeholders what it has achieved and plans to achieve (or what changed or will change) this requires honesty, openness and thought about the selection and accessibility of content. Critically, it also requires evidence where the capability to objectively measure VFM (eg through benchmarking) is essential, but qualitative data should also be considered in telling and substantiating the story. Ultimately, the self-assessment should simply be one aspect of a cultural commitment to transparency that permeates the way an association goes about its business5. The entire system of regulation is about assurance, which further underlines the importance of evidence if the archetypal reasonable board member is assured that the self-assessment is a true and fair picture of VFM, then the HCA is likely to be assured. 4.1 Strategy: reporting the approach Clearly the evidence provided in support of the HCA s three specific self-assessment requirements (return on assets, operational cost and performance and VFM gains) is the acid test of a comprehensive and strategic approach to VFM, in the same way as the proof of a good strategy is what it achieves. That said, the VFM standard requires boards to demonstrate to stakeholders how they are meeting this standard and since the standard includes items such as having a robust approach to use of resources and effective performance management and scrutiny functions, it follows that some assurance should be included as to the association s arrangements to ensure VFM, as well as what they have achieved. The hallmarks of evidencing a grip on VFM strategy and delivery are whether the selfassessment demonstrates: a comprehensive and strategic approach to VFM a robust approach to use of resources the performance management and scrutiny arrangements to achieve the strategy (including governance and tenant scrutiny) In HouseMark s view, the following examples drawn from a selection of 2013/14 VFM statements illustrate these hallmarks. 5 The Regulatory Framework states, as one of its regulatory principles, that transparency and accountability are central to co-regulation. It goes on to say, Providers and their boards and governing bodies should be transparent with their tenants, service users and other stakeholders. Providers approaches to co-regulation should be honest and robust, and include evidence-based self-assessment, external challenge, and regular reporting to tenants. All providers should run their businesses with a presumption of openness and co-operation with stakeholders, including the regulator. The Regulatory Framework for Social Housing in England from April 2012, p9. 22

Figure 1: Hyde explains and illustrates VFM roles and governance Figure 2: Paradigm visualises strategic approach to continuous improvement and structures report around it Strategic principles visualised and explained. Principles serve as basis for success measures and the structure for an evidence-based self-assessment 23

Figure 3: BCHA identifies its key stakeholders and states required value 4.2 Assets: reporting the approach It is clear from the survey results and HouseMark consultants experience that many associations are working to improve their systems for understanding asset performance as a basis for making intelligent evidence-based business decisions. A clear picture of good practice within the sector is only just starting to emerge, particularly with regards to the measurement of asset performance. To add to the uncertainty, as with last year, there is still some confusion amongst associations about what the VFM standard requires. Together these factors result in a wide variety of responses to this aspect of the VFM selfassessment. Those associations that appear to be handling it reasonably well are able to provide assurance that they understand their stock and are making intelligent decisions to improve its VFM. Some of the survey comments also allude to difficulty in interpreting the concept of return on assets: are we talking about the value derived specifically from the presence of physical assets, or about the value derived from everything we do? The problem is essentially about how best to structure and present the story, and there is no right or wrong way. HouseMark has in the past suggested basing the story on a soft split between assets and operations: the former focuses on the drive for VFM with regards to best use of physical assets whilst the latter deals with the VFM of services. The procurement subplot can be woven into each element or standalone. The hallmarks of evidencing a grip on asset management are whether the selfassessment demonstrates: an understanding of: o stock condition, associated investment needs, maintenance costs, demand, the communities and markets you operate in 24

o the performance of stock (return) at an appropriate level of detail - so that variation in performance is understood o what does it tell you? Share your analysis by product, highlighting key challenges that the above understanding informs: o an intelligent asset management strategy where decisions aim to improve returns, eg hold, invest, dispose and where the benefit of investment on future rental stream/social outcomes is understood o where a brief assessment of key asset decisions over the year will evidence active management and improved VFM how has asset performance improved? the strengths and weaknesses of current approach to assets? In HouseMark s view, the following examples drawn from a selection of 2013/14 VFM statements illustrate these hallmarks. 25

Figure 4: livin evidence the continuous improvement in asset VFM Clicking the infographic gives a chart showing improved financial performance of stock plus an explanation (Click through) 26

Figure 5: A similar story at Wrekin Housing Trust: active asset management secures continuous improvement Time Disposing of older expensive stock here to fund new stock here Result positive NPV energy efficient better quality homes lower maintenance less capital spend on improvements 27

Figure 6: The data building blocks of evidence-based business decisions: Rochdale Boroughwide Housing demonstrates understanding of return and sustainability 52 neighbourhoods mapped, underpinned by individual property data Based on 25 sustainability factors More sustainable More return Figure 7: How Savills presents the same sort of data 28

Figure 8: Improving the VFM of assets at Sanctuary: from strategy to action Objectives Data Understanding Action Other associations demonstrating that understanding stock condition and performance leads to an active asset management strategy and informed decisions: Riverside bespoke sustainability tool facilitates analysis complemented by NPV model leads to informed decisions : already disposed of 750 corporate target 6 per cent improvement in average NPV over three years rationalisation programme - >50 by LA area Own Place market sale of older stock with social twist Midland Heart intelligence-led decisions Southern HG layperson-friendly introduction to asset management 4.3 Operational cost and performance: reporting the approach HouseMark s experience suggests that the sector did far better this year in meeting this requirement. Whilst the better statements show three or five years worth of trend data and provide associated commentary which help the reader understand direction of travel, many do not include sufficient data to enable a judgement about the extent of continuous improvement, ie only one or two years data. 29

The hallmarks of evidencing a grip on operational cost and performance are whether the self-assessment: includes the absolute and comparative costs of delivering services o over time (ideally three years) and against others o stating the peer group and avoiding cherry-picking the services you are best at explains the data a brief commentary on headline unit costs, performance indicators, other evidence o what are your key cost drivers and what can you do about them? o what are you doing to contain overheads? o what can you say about the efficiency/effectiveness of your operating model? o strengths and weaknesses explains direction of travel are costs/performance up or down? remember this is about continuous improvement. In HouseMark s view, the following examples drawn from a selection of 2013/14 VFM statements illustrate these hallmarks. Figure 9: Evidence-based continuous improvement: use of a high-level VFM dashboard at Southern and A2 Dominion, Bromford, L&Q, Midland Heart, Sanctuary In-house stab at key VFM PIs Why the PI is useful - enhances accessibility & transparency But PIs not enough balanced commentary explains what is happening, business pressures, strengths, weaknesses, direction of travel etc 30

Figure 10: HouseMark s ready-cooked VFM scorecard for members available in reports and on website Figure 11: Taking it down a level: getting a grip on operational cost and performance by service at Spire amongst many others States who peers are for transparency Breakdown by specific services, forensic grip on cost and performance? Trend for continuous improvement Latest data Strengths and weaknesses 31

Figure 12: Alternative service-level approach at BCHA Cost Results Contains VFM ingredients: 3Es, comparisons & trend Figure 13: Bolton at Home says what it sees to facilitate stakeholder understanding This chart conveys by service area: comparative cost & performance magnitude of spend Excellent basis for focused commentary on strengths weaknesses, achievements, plans, trend The usefulness of this graphic is explained to aid accessibility and therefore transparency 32

Figure 14: Visualisation of continuous improvement at Paradigm Evidencing continuous improvement? Note visualisation of repairs journey Hopefully they ll get here! More direction of travel in own performance and sector quartiles. Figure 15: Riverside follows the money with three sets of results broken down by stakeholder, backed by narrative 1. resources generated 2. spent efficiently 3. achieved? 33

Figure 16: Midland Heart explains the VFM of care Figure 17: Wrekin looks to capture savings falling to Trust and partners 34

Figure 18: Trafford and others have started to quantify well-being (in this case the difference made by sheltered living) identify what tenants value (wellbeing) measure difference (before & after) apply established metrics (NEF) to provide value compare to cost of provision get multiplier Bromford also using NEF, CBA and HACT methods complemented by case studies. NB Adactus and Magenta use well-being proxies for community investment from HACT Social Value Bank. 4.4 Reporting VFM gains In HouseMark s experience the sector did better this year in quantifying past gains and, to some extent, setting future measurable targets. The latter remains a significant challenge to the extent that some might find themselves judged as not being fully compliant with the VFM standard. VFM or efficiency registers are being dusted off and re-launched as a means of capturing past gains. They also help embed VFM by serving as practical examples so that staff understand what VFM looks like in practical terms, celebrate success and to kindle further ideas and innovation. It is quite common to see qualitative targets (essentially VFM action plan items) such as the intention to undertake a service review, improve something, retender or restructure. There appears to be some reluctance to take this a step further and quantify the envisaged benefits that underpin the business case for the action. These might include anticipated improvements in performance/outcome indicators or savings. It is likely that some associations are concerned about being held to account for the quantifiable targets they set, which of course is the intention. Associations need to appreciate that the HCA is looking for quantifiable targets; the HCA and any other stakeholder need to appreciate target setting is not an exact science. Its blunt use as an accountability tool might deter associations from taking the risks inevitably associated with changing things. From the self-assessments HouseMark has seen, those associations setting targets for future gains are far more likely to quantify expected financial savings than improvements in performance or outcomes. Some associations are reporting performance against (sometimes long-held) financial targets to contain or reduce operational spend in real or absolute terms, with the rationale of augmenting surpluses to support organisational objectives. It is important to explain to readers that financial VFM gains contribute to surpluses to further your social objectives, or as David Orr would put it, it s profit for a purpose. So help the armchair auditor follow the money and understand the value of gains and surpluses. This is discussed further in section 4.5. 35

Similarly, if you have made a financial saving in one place that is then reinvested somewhere else to increase value, help the reader follow the money by explaining what you did and why, eg savings have been made in-year from operation X and reinvested in operation Y to achieve outcome Z. Own up to any diminution in X and/or knock-on effects elsewhere and Z should map to your mission. If you do not do this, some readers might be expecting to see a 1:1 relationship between financial VFM gains, surplus and operating costs. The hallmarks of evidencing VFM gains are whether, for past gains the selfassessment: quantifies gains financially, socially (performance, satisfaction, number of beneficiaries, etc), environmentally owns up to any negative impacts and costs follows the money o explains if the gain is already invested elsewhere o states what you will use surpluses for (or if it has been spent already!) and, for future gains, whether the self-assessment: sets out what you will do states the quantifiable anticipated financial, social, environmental gains are in other words the business case for action identifies any costs or diminution in service associated with them? In HouseMark s view, the following examples drawn from a selection of 2013/14 VFM statements illustrate these hallmarks. Figure 19: Quantification of gains at A2 Dominion 36

Figure 20: Quantification of gains mapped to corporate aims at Spire Economy gain and efficiency gain Corporate aim with no deterioration in fact effectiveness gain Added social value from procurement Figure 21: Bearing down on overall operating costs (and evidencing it) limits suspicion of cherry-picking isolated gains First Wessex - Efficiency Monitoring Framework - 1% annual real reduction of operation costs over 5 years - Explains origins of savings Affinity Sutton - Real reduction 69 per home - Drills down to see contributions - Explains gains and surplus essential to social mission Wrekin - Target to reduce operation costs in real terms by 10% between 2010-2014: 10.2% achieved - Drills down to see efficiency contributions 37

Figure 22: Quantifying future gains at Town & Country The list continues Figure 13: How we use surpluses A2 Dominion follows the money 38

Figure 24: How we use surpluses at Midland Heart by way of return on grant metric Figure 25: How we use surpluses at L&Q 4.5 Transparent and accessible communication It is clear from the survey and HouseMark experience that the HCA s displeasure with the sector last year has led to lengthy self-assessments this year. As some survey respondents have pointed out, in seeking to be transparent associations have increased the complexity of the documents, which risks undermining their accessibility. Indeed, the sheer volume of information may diminish transparency as key information is lost in the general word count. Can you produce something that is fit for purpose in, say, 20 pages or less? And can it tell (and illustrate) that story so your mum (or dad) would understand it (to borrow from one of our survey responses). You might if communication skills are honed as suggested in the hallmarks section 39

below. In terms of improving the self-assessment, this might be the biggest challenge for associations going forward, as in addition to making the self-assessment accessible it also requires a commitment to improve organisational transparency. Some self-assessments still have a way to go in terms of presenting a balanced and rounded picture of VFM performance. The survey responses also highlight a further tension: is this in reality a regulatory return or an exercise in public transparency and accountability? In the author s opinion, it may feel like the former but should be the latter (see Diagram 1 on page 21). The HCA should be seen as just one (albeit important) stakeholder. Ultimately what the HCA is looking for should not be any different to what any other stakeholder is looking for: assurance that you are squeezing maximum value from available resources in furtherance of your business objectives. This is not another tenant report or a regulatory return, it is an exercise in public accountability through transparent and accessible communication. Think stakeholder! A further consideration is that the current regulatory definition of VFM is more comprehensive than previously. It encompasses the use of resources with regards to both assets and operations, essentially making it about the effectiveness of the social housing business. It follows that, if you are to be transparent about your VFM efforts, your reader will need some understanding of how your business works. Unfortunately for armchair auditors (who take their job seriously) this means grasping basic housing association business principles where finance is key. Unfortunately for Finance Directors, this means finding the right form of words to explain limited and relevant extracts of the yet-to-be-written Housing association finance for dummies as part of the VFM story. For example, how surpluses are adding value to the business and not about accumulating a big pile of cash under the bed or withholding dividends from the taxpayer. It is important that the sector aims to correct misunderstandings about what associations do and how their finances work, not least because VFM self-assessment readership includes those who hold significant influence and power in Whitehall and Westminster, eg Special Advisors to No. 10 and Her Majesty s Treasury. The above discussion ranges from accessibility to audience and highlights the over-riding challenge: making such a document fit for all stakeholders. Do we really need a variety of selfassessment for different stakeholder groups? We should not expect any stakeholder to read these documents unless they are, for whatever reason, motivated to understand the business and hopefully contribute by challenging the association or suggesting ways to improve outcomes. This level of commitment means we should not expect mass consumption. But those who are motivated need a document that is of sufficient detail to understand VFM performance in the round ie spanning the VFM generated for all stakeholders 6. If the sector focuses on being open and getting its story straight in terms of passing the would your mum and dad understand this? test, it will have secured a significant victory for transparency by way of accessibility, whilst avoiding the need for a variety of documents for different audiences. The hallmarks of a transparent and accessible self-assessment are whether the selfassessment: is balanced and honest? o presents a true and fair picture o provides an even-handed, rounded commentary spanning the business and is not deliberately weighted to strengths 6 This is not to say that all tenants should not get some kind of VFM summary in the tenants report associated with the services they receive, but those interested in getting more involved in running the business or holding the association to account should be directed to the full self-assessment. 40

is clear about who you produce value for who are the key stakeholders and what are they looking for? has a clear and logical structure to facilitate understanding and navigation focuses on material issues and seeks to be succinct draws on quantitative and qualitative evidence to support assertions uses tables and data visualisation wherever possible to aid analysis and understanding uses plain English, including explaining the significance and usefulness of data and key business principles provides enough information to enable stakeholders to judge how you are performing against your peers and over time (remember continuous improvement in VFM is about evidencing beneficial change in cost and/or performance/outcomes) signposts to supporting material/evidence in the interests of transparency passes the would my mum and dad understand this accessibility test and, critically, whether: its existence has been communicated to maximise stakeholder awareness it can be found easily In HouseMark s view, the following examples drawn from a selection of 2013/14 VFM statements illustrate these hallmarks. 41

Figure 26: Radian s plain language approach is accessible and located in a wider approach to transparency Online clickable infographic Accountability: more VFM resources, eg VFM register, recent performance, etc NB Bromford do similar real commitment to transparency It s a living thing: update to self-assessment with latest benchmarking data Co-producing VFM: tenant input Accountability again Click through infographic for more detail and to understand what is going on 12/13 data Use of annotation brings key data to life: what are the cost drivers, what happened, what will you do? 13/14 data Continuous improvement? Track movement of 1 and 3 October update includes latest data and explains key movement 42

Figure 27: From headlines to detail: clickable PDF aids accessibility and navigation at Halton, Bromford and others Figure 28: Managerial accountability in the spotlight at Bromford: managers respond to customer feedback 43

Figure 29: Hi-vis: A2 Dominion satisfaction by activity Figure 30: Visual representation of corporate dashboard at Adactus PIs map to objectives and span inputs, outputs/outcomes. Accompanied by commentary. Size of box = importance 44

Figure 31: Following the money at Adactus: online story Keep following the money: pick up a thread and follow the story 45

Figure 32: WM Housing provides at-a-glance rating of individual elements of the self-assessment, accompanied by information to enable the stakeholder to form their own view 46