Organising g for Transparency. The ranking of Dutch hospital care Author: Roland Bal Erasmus University Discussant: Emmi Poteliakhoff UK Department of Health Strategy group 21 st September 2012
Overview Outline the paper Key questions the paper asks and the answers given Some thoughtsht
Outline - paper Context of Dutch healthcare e Literature on rankings Methods Analysis Question- ways health care organisations organise for transparency and to what effects? Rankings are composite measures used for comparison between entities
Context of Dutch healthcare Economisation of care Neither determines changes in healthcare practices Professional regulation The rise of rankings Preceded 2006 introduction of regulated competition Published in popular press Based on healthcare inspectorate t data, peer review (at least initially) and patient experience Came in for criticism
Literature on rankings Methodological focus validity of indicators Influence on global markets Effect of rankings on single organisations: Performativity of rankings Performative Dictionary An utterance that creates a state of affairs by the fact of being uttered Visualisation could mean placing one organisation above another in a hierarchy Performativity from paper not just creating neutral comparable information, but reshaping those practices they are meant to measure Visualisation could mean visualising (or focussing on) only one or limited aspects of the organisation
Performativity of ranking evidence and theory University sector Great impact of international rankings of business schools and law schools Set up special organisational units, devised specific policy measures in response to ranking systems Health sector Evidence of similar effects Colonisation External accountabilities become sole focus Tunnel vision Organisations come to define themselves in terms of rankings Decoupling Evidence of both Not one directional Looser ties between management and floor processes Ritualisation ti of auditing practices e.g. gaming
Guiding questions Can the concepts of colonization and decoupling form a basis for describing the effects rankings have on Dutch hospitals....and how do these mechanisms vary between hospitals? Is there indeed a growing emphasis in the hospital sector on reputation management as analyzed by Michael Power and colleagues regarding business schools, suggesting that rankings may help to carve out a new object of risk management? How does this reputation management get shaped and in what way does it affect clinical processes? In what way, if so, are rankings used for internal quality improvement? Do rankings lead to tunnel vision, that is do they make hospitals focus in those aspects of their performance that are used for rankings while neglecting others?
Method Bringing together qualitative research conducted over a number of years In depth hospital case studies (10+) Documents Reviewed Interviews Observation Interviews on rise of transparency policies i Ministry of Health Healthcare Inspectorate Council of Public Health and Healthcare organisations of professional and patient groups academic researchers healthcare quality consultants
Rankings as a new organisational context Initial impact of rankings Informing reputation Labour market importance Rankings led to investment in: Administrative processes Function and position of quality managers Internal and external dimensions Investment in IT to generate performance data Investment in IT to generate performance data Reputation management New function Communications has been restructured into marketing Seek prizes, use social marketing
The sociomaterial art of measuring Rankings require qualities to be quantified Classifying activities and defining their properties Measurement is embedded in organisational routines Measuring waiting times Evidence of this in case studies - diabetes care
The strategic use of performance measures Rankings and the practices they compare co-constitute each other Rankings Strategic responses to rankings occur hospitals design measurement strategies to score better Practice Impact of rankings- differences are reduced
Qualifying organisational performance Numbers as tin opener to start conversation Numbers require a narrative Quantitative ways of accounting for performance don t crowd out qualitative but the two sustain each other Other ways to account for the quality of care develop culture programmes
Overall conclusions Rankings and the performance indicators that they are based on have changed organizational life in Dutch hospitals in profound ways..but there is ambivalence towards rankings Policy level used strategically Colonisation and decoupling (ritualisation) both occur Ranking systems themselves are affected in the process Rankings Practice Visualizing and quantifying technologies like rankings are developed alongside qualifying techniques
Guiding questions Yes Can the concepts of colonization and decoupling form a basis for describing the effects rankings have on Dutch hospitals....and how do these mechanisms vary between hospitals?? Is there indeed a growing emphasis in the hospital sector on reputation management as analyzed by Michael? Power and colleagues regarding business schools, suggesting that rankings may help to carve out a new object of risk management? How does this reputation management get shaped and in what way does it affect clinical processes? In what way, if so, are rankings used for internal quality improvement? Yes e.g. Performance review, canteen displays Do rankings lead to tunnel vision, that is do they make hospitals focus in those aspects of their performance that are used for rankings while neglecting others? Yes but not to exclusion of ritualisation Yes
Some thoughts How the paper is presented Three strands to literature on ranking Too equivocal in defining only three, what about impact on patients The three strands are linked validity of indicators may affect their impact on organisations e.g. NYC heart surgery Is the case for the conclusions well made? Feels more ad hoc than a systematic study Could be shorter? debateable
Some thoughts Policy interpretation What is the difference between rankings and other performance measurement? Can t all be at the top - one organisation always has to be at the bottom Those with a low position - no incentive to change? Ranking is very explicitly a hierarchy of performance tension between this concrete representation and potentially weak methodology in ranking, eg WHO health system ranking Result- reducing variation- should be a good thing?
Some thoughts Policy interpretation Potential for a normative analysis of the consequences of ranking Where are rankings desirable and what sort of rankings enhance practice? What response would we want to ranking? What is the purpose of ranking? improve care reduce variation in care quality increase transparency enhance patient choice? Is colonising or decoupling consistent with this? Where ranking exercises measure the right things prefer colonising Where they measure the wrong things decoupling better where legitimacy of the measures is low it is legit to game
Highly relevant to the UK, what about other countries?