Evaluating a lean healthcare transition in a radiotherapy clinic

Transcription

1 2014 Faculty of Business Economics DOCTORAL DISSERTATION Evaluating a lean healthcare transition in a radiotherapy clinic Doctoral dissertation submitted to obtain the degree of Doctor of Applied Economic Science, to be defended by Pascale Simons Promoter: Prof. Dr Dominique Vandijck UHasselt D/2014/2451/50 ISBN:

2 Promotor Prof. dr. Dominique Vandijck Jury members Dr. Madelon Pijls-Johannesma Prof. dr. Wim Marneffe Prof. dr. Jos Benders Prof. dr. Irina Cleemput Prof. dr. Johan Hellings Prof. dr. Piet Pauwels ISBN-nummer is Copyright Pascale Simons, Diepenbeek, 2014

3

4 CONTENTS Summary 7 Samenvatting 15 Chapter 1 Introduction and outline 25 Chapter 2 Research methodologies 39 Chapter 3 Workshops as a useful tool to better understand 49 care professionals views on a lean change program Chapter 4 A factorial survey on intentions for safety behaviour 65 providing opportunities to improve safety Chapter 5 Does compliance to patient safety tasks improve and 87 sustain when radiotherapy treatment processes are standardized? Chapter 6 Does lean management improve patient safety culture? 103 An Extensive Evaluation of Safety Culture in a Radiotherapy Institute Chapter 7 Safer radiotherapy treatment 125 due to synchronized equipment A mixed methods study Chapter 8 Cost-effectiveness of reduced waiting time for head 139 and neck cancer patients due to a lean process redesign Chapter 9 Does a lean transition decrease waiting times and 161 improve outcomes for patients and employees Chapter 10 Does lean in healthcare reduce ambiguity of objectives 181 and clarify uncertainties of cause-effect relations? A mixed methods approach Chapter 11 General Discussion 199 Dankwoord 213 Curriculum Vitae 219

5

6 ABBREVIATIONS AHRQ CEAC CEO CI CT-PET HFMEA HSOPSC HNC ICER INMB KJ-technique MAASTRO CLINIC MaPSaF MUMC+ NHS NMB NWF IRS PRISMA PSC QALY RWF SD Se SOP QI VSM Agency for healthcare research & quality Cost effectiveness acceptability curve Chief executive officer Confidence interval Computed tomography positron emission tomography Healthcare failure mode and effect analysis Hospital survey on patient safety culture Head and neck cancer Incremental cost-effectiveness ratio Incremental net monetary benefit Kawakita Jiro technique Maastricht radiation oncology clinic Manchester Patient Safety Framework Maastricht university medical centre National health service Net monetary benefit New work flow Incident reporting system Prevention and Recovery Information System for Monitoring and Analysis Patient safety culture Quality adjusted life year Regular work flow Standard deviation Standard error Standardized operating procedure Quality improvement Value stream mapping

7

8 CHAPTER S Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour Summary

9 SUMMARY Quality, safety, and efficiency are highly valued in health care. Individualized, high quality care, which meets the patients needs, is demanded. However, the quality and safety of care are still not satisfactory. Although some improvements have been accomplished, many quality and safety initiatives fail to realize sustainable quality improvement. In addition, efficiency of care processes should be improved to bring healthcare costs to a minimum. Healthcare costs have been rising dramatically in the last decade, because of the ageing population, and the expanded treatment possibilities in particular. To deliver the high quality, and safe care that patients deserve, and at the same time remain financially healthy, organizations are in need of strategies that combine quality improvement (QI) with increased efficiency. Within the current perceptions in healthcare, improvement of quality and safety are inherent to additional costs, and an increased burden to professionals. However, business strategies, like lean management, improve both quality and efficiency in health care settings according to an increasing number of scientific studies. Lean management in healthcare is a strategy that aims to improve quality, safety, and efficiency of care. It focuses on the redesign and improvement of processes towards a customer-focused ideal-state, aiming to minimize waste, defined as tasks that do not directly benefit the patient. Because lean has its origins in the automobile industry, translation of this approach to the care sector is not straightforward. Healthcare differs from the industry in that quality, or value for the patient, is not clear at the start of the value stream, and it changes throughout the clinical process to meet the changing patient needs. Despite the differences between the automobile industry and healthcare, the lean philosophy has been widely disseminated in a variety of care settings in the forgoing years. Although the overall impression is in favour of implementing lean in healthcare, counter-arguments against this philosophy are also publicized. Two major issues seem present regarding lean healthcare. First, lean healthcare is explained in a variety of ways. The majority of healthcare organizations implement lean as a set of tools guided by the idea of eliminating waste, based on what is defined as being valuable, rather than as a total system governing the whole of the organization. When lean tools are implemented without being part of a comprehensive management system combined with a supportive organizational culture and committed leaders, sure improvements will be made, but these will lack sustainability. In order to create sustainable change, interventions should simultaneously focus on the organizational structure, its culture, and the clinical processes. Second, quality and safety are difficult concepts to measure objectively. To meet these concepts complex nature, mixed methods should be used for their evaluation. A variety of effects, end-outcomes and surrogate endoutcomes, should be evaluated by means of quantitative and qualitative measures. This ensures triangulation to discover the same truth and increases understanding in measured effects. These multiple effects include outcomes affecting the patient, the employee and the organization. When positive effects 8

10 SUMMARY on all three dimensions can be achieved, sustainability of improvements will benefit. Lean management in health care seems promising, but its effects on the patient, employee, and organizational level remain unclear. Tools for measuring effects of lean in healthcare and a uniform strategy for its implementation are not yet available. Although evidence about the effects of lean management in health care settings seems promising, it is very limited. The radiotherapy setting in particular is expected to profit from lean initiatives. Radiotherapy entails the treatment of oncology patients with radiation, and treatment compromises a clear value stream, and includes repetitive processes in a high technology environment. However, little has been published about the effects of lean in radiotherapy settings. The objective in this dissertation was to perform an extensive assessment of a lean management transition in a healthcare setting, where specific lean tools were combined with an overall management approach. Effects on the level of the patient, the employee and the organization were evaluated with a special focus on quality, safety, and efficiency. Three research aims were defined: 1. To develop an instrument to improve management decision making in a healthcare setting, regarding the choice for a lean change program and increase understanding in employees views and commitment regarding the proposed lean program. 2. To develop an instrument to measure patient safety awareness and intentions for patient safety behaviour in addition to the internationally used instruments for safety climate measurement. 3. To evaluate effects of a lean transition in a healthcare setting on the patient, the employee and the organizational level. A radiotherapy institute (in the Netherlands), MAASTRO CLINIC, was followed from 2010 to 2014 in their journey towards a lean healthcare organization. In 2007, their lean journey started by initiating a stand-alone lean based project to improve quality, safety and efficiency on their treatment facilities. In 2010, the lean philosophy was up taken in the organizations long term strategy. To create sustainable improvement, MAASTRO CLINIC combined specific interventions (lean tools), which focussed on the clinical processes, with an overall (generic) strategy, which focussed on the organizations structure and culture elements. As an example of a specific lean intervention, the administrative department optimized their working environment by implementing 5S; a lean tool to create an orderly workplace and remove waste associated with clutter and disorganization. In 2013, multiple projects were initiated to improve process flow and decrease waiting times for the patients. Management was restructured from a functional to a process oriented organization in January Multidisciplinary teams were created for every care path (e.g. breast, head and 9

11 SUMMARY neck, prostate) to facilitate and improve process innovation, and increase employee commitment to change. In 2012, the chief executive officer discussed the organizations vision and aims with all employees in small group meetings, to create a shared long term vision and improve organizational culture. To address RESEARCH AIM 1, a workshop was developed as an instrument to improve management decision making in a healthcare setting. To successfully implement lean, it is vital to have a detailed understanding of employees views regarding the program, especially when the proposed changes are potentially contested. During the workshop, the participating employees listed the most important expected effects of a proposed lean change program. The majority of negative effects were expected on the short term, although the number of positive effects outnumbered the negative in the end. This might partly explain why getting the lean management philosophy into practice is difficult. High-level managers reported only a moderate to positive value of the workshop regarding their decision making process. However, they expected increased employee commitment to the lean program and improved bottom up thinking, as a result of participating in the workshop. As a result, this type of workshop has the potential to increase successful implementation, against relatively small costs. Considering the limited results in this study, no strong conclusions could be formulated. However, the potential benefits of involving employees in early stages of change were demonstrated. Although this kind of workshop should be studied in a variety of health care settings, wider implementation can be considered. Regarding RESEARCH AIM 2, a survey to measure patient safety awareness and intentions for patient safety behaviour was developed. Since quality and safety of care is for a part dependent on employee s behaviour, periodic measurement of safety behaviour could provide management with relevant information to adjust initiated improvement interventions. The objective measurement of safety behaviour is difficult and time-consuming (for instance by observing behaviour or compliance to safety tasks). Therefore, a factorial survey was developed to measure intentions for safety behaviour. The survey included twenty hypothetical scenarios about incidents, which randomly varied on; work pressure, person causing incident, whether the patient-level was reached, severity of harm, notification by the patient and support from management. After each scenario, the respondents answered one question on their safety awareness and four on their intentions for specific safety behaviour. The factorial survey results were compared to results from the Hospital Survey on Patient Safety Culture (HSOPSC), which measured patient safety climate. The factorial survey results complemented the HSOPSC results and presented additional information: e.g. the intentions to improve structurally were measured by the factorial survey, and the factors influencing intentions for safety behaviour could be determined. This could increase managements understanding in employees safety behaviour. An important drawback of the factorial survey was the lacking 10

12 SUMMARY opportunity to benchmark with other health care organizations, because of the large detail of the used case descriptions (vignettes) to make them recognizable for clinical professionals. This level of detail inevitably led to issues around the opportunity to generalize findings. Before implementation on a broader scale, the preferred level of detail in the vignettes should be studied to make generalizations and follow up for longer time periods possible, without sacrificing incidents reality. Regarding RESEARCH AIM 3, a lean transition combined with a multi-faceted organizational approach in a radiotherapy setting was evaluated. A panel of ten experts had listed the most important indicators for evaluation on three dimensions: the patient, the employee, and the organization. Since it is difficult to detect the real effect of a lean management (change) program, end-point outcomes were combined with multiple surrogate end-points like process evaluation, compliance to the intervention, culture and errors (incidents). Mixed methods were used for their evaluation. Quantitative data, from several software programs and multiple surveys, were further explored by qualitative data gained from a focus group and structured interviews. To realize safe radiotherapy treatment and be able to innovate and optimize the treatment process, first the relevant processes should be stabilized. Standard operating procedures (SOPs) were implemented, and the process was redesigned to increase its stability. In 2007, a project team of 25 (out of 60) technologists redesigned the treatment process, and developed SOPs for two instead of three technologists. As a result, the observed compliance to specific patient safety tasks increased and improvements sustained after 1.5 years. Perceived importance of tasks correlated positively with compliance, and sustainability. Regular feedback to raise employees awareness about the importance of the safety tasks is thus crucial to increase compliance, and sustain improvements. The importance of a safety culture in an organization to create quality, and safety improvement is no longer questioned. However, achieving sustainable culture improvement is not straightforward. Evidence is growing for a multifaceted approach, in which multiple safety interventions are combined. A longitudinal evaluation of safety climate, safety awareness and intentions for safety behaviour showed improvements during the lean transition in the studied setting. A workshop based on the Manchester Patient Safety Framework (MaPSaF) did not detect any changes in safety culture. The HSOPSC (climate survey) and the factorial survey (awareness and intentions for safety behaviour) presented improvements. The intention to report incidents that did not reach the patient-level, and the number of reported incidents from the incident reporting system (IRS) decreased. However, employees experienced sustained safety awareness, and an increased intention to structurally improve. Therefore, the decreased number of reported incidents was explained as a decrease of the number of incidents actually happening, owing to increased problem solving and equipment improvements. The robustness of the measured data can be 11

13 SUMMARY questioned, since most conclusions were based on surveys and qualitative data. To conclude carefully, the patient safety culture improved after combining specific lean management initiatives with a management restructure to facilitate continuous improvement. From July 2011 to March 2013, seven accelerators (treatment equipment), and treatment planning software were replaced by an integrated system with six accelerators, as a part of the generic interventions to create improvements in quality, safety, and efficiency. The equipment transition was not a specific lean intervention, but it supported the lean philosophy by understating the importance of adapting technology to the human, and organizational processes, and not the contrary. The number of incident reports, related to the accelerator, decreased significantly from 41 (2010) to 14 (2013) reports per month, whereas the number of delivered fractions per accelerator increased by 20%. The proportion of technical coded root causes assigned to the reported incidents decreased, whereas the proportion of organizational coded causes remained stable. Furthermore, the down time (time equipment cannot be used for treatment due to technical breakdowns) of the accelerators decreased with 46% from 5.4% of available treatment hours in 2010 to 2.9% in The quantitative results were further explored by structured interviews with professionals. As the number of reported incidents depends on the employees willingness to report, one could question whether employees reported less incidents or the number of incidents actually happening decreased. Interviewees did not experience a decreased willingness to report. Instead, they experienced improved treatment processes, explained by the technical transition, and because management increasingly focussed on improvement. When the long term patient benefits of a process redesign based on lean principles could be presented, commitment of clinical professionals, and healthcare management to lean management initiatives might increase, and sustainability might improve. A mathematical model (Markov model) was used to calculate the long term cost-effectiveness of a logistic redesign of the diagnostic, and preparation process of radiotherapy for patients with head and neck cancer (HNC). The redesign included two main organizational changes: 1) performing tumour biopsy under local instead of full anaesthesia, and 2) performing a diagnostic CT-PET before radiotherapy, instead of an additional CT- PET during the preparation for radiotherapy treatment. The redesigned care process led to reduced waiting times and proved cost-effective on the long-term. The incremental QALYs varied between 0.13 to 0.66, with zero to small additional costs for the redesign. Although the model outcomes were based on many assumptions, the presented redesign resulted in significantly better longterm patient outcomes and cost savings for patients with HNC. Health care professionals might be less attracted to process redesign as compared to adapting new technologies. However, the benefits for patients, and the cost reduction for healthcare in this study were clear cut, and cannot be discarded. In 2013, 15 specific lean projects were initiated to improve patient flow, and decrease waiting times. These projects were preceded by generic interventions 12

14 SUMMARY to increase commitment and create shared visions. The percentage of patients, of which the waiting time exceeded the Dutch national norms and objectives for waiting time, had significantly decreased. Patient safety, employee satisfaction, and the percentage of employee absenteeism had also improved. The improvements in waiting times in this study were moderate, but these could be a potential motivator to struggle forward, and strive for continuous improvement. Although no strong conclusions can be drawn, the combination between the lean initiatives, the increased focus on continuous improvement, creating commitment to change, and facilitating organizational structures, was possibly the success factor of this multi-facetted approach. The large ambiguity of objectives, and the uncertain cause-effects relations in care processes, might be influencing concepts for sustainable improvement. After the lean transition, survey results presented enhanced predictability of processes, improved standardization and improved shared visions. Based on structured interviews with twelve professionals, lean management positively affected the existing ambiguity of objectives, and reduced uncertainties of cause-effect relations in the clinical processes. As a result, decision making benefitted from lean, and the chance to realize sustainable quality improvement increased. Because the presented results are prone to some methodological limitations regarding measurement, and generalization, conclusions should be interpreted with care. However, all included measurements presented improvement. Combining the specific lean tools with a generic management approach seemed successful to creating sustainable improvement in quality, safety, and efficiency in this health care setting. The specific set of interventions should be adapted to the specific problems, and needs of every care organization. However, to ensure sustainability, all three dimensions; organizational structure, culture, and processes should be addressed simultaneously. The combination of quantitative, and qualitative measures resulted in an indepth analysis of the evaluated outcomes. The structured interviews performed in the last year of evaluation, made interpretation of quantitative outcomes possible. To explore results throughout the evaluation period, and enable the initiated interventions or the measurement protocol to be revised, future projects should incorporate in-depth interviews periodically, and in an earlier stage of the project. The initiated specific lean projects were only a first step towards continuous improvement. The question remains if these first successes can motivate the clinical professionals to realize substantial and sustainable improvements in process flow, and quality of care. The multi-facetted approach seems successful to create quality improvement, but still has to prove its success in striving for perfection. Although some improvements have been made, the organizations challenge is to not only to sustain the realized improvements, but also to raise 13

15 SUMMARY them to a higher level, and ideally deliver the perfect care for all patients. Only when continuous improvement is internalized by all professionals, and has become one of the basic values in the professional status, sustainable quality improvement has become a reality. Although many professionals share the opinion that quality of care should be improved, creating sustainable change is a continuous struggle, and asks determination, and perseverance to a long term vision. As safe high quality care is a privilege for all patients, health care professionals, management, and policy makers should do all it takes to make this a reality. Lean management seems a promising strategy, helping healthcare management to create the road to improvement. 14

16 HOOFDSTUK S Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour Samenvatting

17 SAMENVATTING Kwaliteit, veiligheid en efficiëntie zijn basisvoorwaarden geworden voor goede zorg. De vraag naar kwalitatief hoogwaardige geïndividualiseerde en geïntegreerde zorg is gegroeid. Ondanks de vele verbeterinitiatieven, zijn er slechts weinig voorbeelden waar kwaliteit en veiligheid duurzaam werden verbeterd. De kwaliteit en veiligheid van de zorg zijn nog verre van optimaal en verbetering is een noodzaak. Naast de toegenomen vraag naar verbeterde kwaliteit, zijn de kosten voor gezondheidszorg exponentieel toegenomen. Dit komt door de vergrijzing, de groeiende incidentie van chronische ziekten, de hogere eisen die mondigere patiënten stellen, maar vooral ook door de ontwikkeling van nieuwe diagnostische en therapeutische technieken. Om betaalbare, hoog kwalitatieve zorg voor iedereen te kunnen leveren, zijn strategieën noodzakelijk die kwaliteitsverbetering van de zorg combineren met het verhogen van de efficiëntie. Binnen de huidige inzichten in de gezondheidszorg zijn de verbetering van kwaliteit en patiëntveiligheid inherent aan extra kosten en hogere werklasten voor professionals. Echter wordt in toenemende mate bewezen dat bepaalde verbeterstrategieën, zoals lean management, resulteren in een verbeterde kwaliteit van zorg en een verhoogde efficiëntie. Lean management, toegepast als bedrijfsfilosofie in de gezondheidszorg, streeft naar maximale kwaliteit en veiligheid van zorg tegen de laagste kosten met tevreden medewerkers. Door de patiënt centraal te stellen en de zorgprocessen te herinrichten en verbeteren, worden overbodige activiteiten, gedefinieerd als niet direct waarde toevoegend voor de patiënt, geminimaliseerd. De lean filosofie heeft zijn oorsprong in de automobielindustrie. Een één op één vertaling van deze filosofie naar de gezondheidszorg is niet mogelijk. Een belangrijk verschil tussen de gezondheidszorg en de industrie is dat het concept kwaliteit niet duidelijk gedefinieerd kan worden bij aanvang van het proces. Kwaliteit van zorg wordt gaandeweg het zorgproces gezamenlijk met de patiënt vorm gegeven. Vervolgens is kwaliteit niet statisch, maar wordt dit gedurende het zorgproces aangepast aan de veranderende noden van de patiënt. Ondanks de vele verschillen tussen de sectoren, wordt lean de laatste jaren veelvuldig toegepast binnen verschillende gezondheidszorgsettings. Globaal gezien is men positief over de implementatie van lean. Echter worden ook negatieve geluiden gepubliceerd. Wanneer men de literatuur omtrent lean beschouwd, worden twee problemen gesignaleerd. Ten eerste wordt lean in de gezondheidszorg op verschillende manieren geïnterpreteerd en geïmplementeerd. Binnen het merendeel van de gezondheidszorgorganisaties wordt lean geïmplementeerd als een combinatie van instrumenten die gericht zijn op het elimineren van overbodige activiteiten. Wanneer deze instrumenten niet worden opgenomen in een overkoepelend managementsysteem met een ondersteunende cultuur en betrokken en toegewijde leidinggevenden, zullen hoogstwaarschijnlijk verbeteringen zichtbaar zijn op de korte termijn. Deze zullen echter niet leiden tot een duurzame kwaliteitsverbetering. Wanneer men op lange termijn de 16

18 SAMENVATTING kwaliteit, veiligheid en efficiëntie van zorg wil verbeteren, zal men tegelijkertijd moeten werken aan de managementstructuur van de organisatie, de heersende organisatiecultuur en de specifieke zorgprocessen. Ten tweede, zijn kwaliteit en patiëntveiligheid moeilijk te meten concepten. Om aan de complexiteit van deze concepten tegemoet te komen, dienen verschillende methoden met elkaar gecombineerd ter evaluatie. Met behulp van zowel kwantitatieve als kwalitatieve methoden dienen meerdere effecten te worden opgevolgd. Door triangulatie van meetgegevens wordt de werkelijkheid zo goed mogelijk benaderd en worden de gemeten effecten beter begrepen. De te evalueren effecten omvatten resultaten die de patiënt, de medewerker en de organisatie aangaan. Wanneer men op alle drie deze dimensies positieve effecten/verandering kan realiseren, vergroot men de kans op een duurzame kwaliteitsverbetering van zorg. Lean management lijkt veelbelovend om de gezondheidszorg te verbeteren. Echter zijn de effecten van deze verbeterstrategie op de patiënt, de werknemer en organisatie nog onvoldoende beschreven. Toepassing van lean management binnen de radiotherapiesector lijkt veelbelovend. Radiotherapie omvat de behandeling van kankerpatiënten met behulp van straling. Deze behandeling bestaat uit een duidelijke stroom van repetitieve processtappen in een hoogtechnologische zorgomgeving. Desondanks is in de wetenschappelijke literatuur nog zeer weinig beschreven over lean management toepassingen in de radiotherapiesector. Het doel van dit proefschrift was om binnen een gezondheidszorgorganisatie de effecten van een lean transitie uitgebreid te evalueren. De lean transitie omvatte de implementatie van specifieke lean instrumenten die werden ondersteund door een overkoepelend managementsysteem. Binnen een tijdsbestek van vier jaren werden de mogelijke effecten geëvalueerd op het niveau van de patiënt, de werknemer en de organisatie, met specifieke aandacht voor de kwaliteit, veiligheid en efficiëntie van zorg. Drie subdoelen werden gedefinieerd: 1. Een instrument ontwikkelen om de managementbesluitvorming in een gezondheidszorginstelling te verbeteren, inzake de keuze voor een lean veranderprogramma, door management meer inzicht te geven in de gepercipieerde opvattingen van medewerkers. 2. Een instrument ontwikkelen om het patiëntveiligheidsbewustzijn en de intenties tot patiëntveiligheidsgedrag te evalueren in aanvulling op de veiligheidscultuur. 3. De effecten van een lean transitie in een gezondheidszorgorganisatie evalueren vanuit het perspectief van de patiënt, de medewerker en de organisatie. Een radiotherapeutisch instituut (in Nederland), MAASTRO CLINIC, werd van 2010 tot 2014 gevolgd tijdens op hun reis naar een lean organisatie. In 2007 is het instituut met lean management gestart door een losstaand project te initiëren 17

19 SAMENVATTING om de kwaliteit, veiligheid en efficiëntie te verbeteren in het radiotherapiebehandelproces. In 2010 werd de lean filosofie in de lange termijn strategie van de organisatie opgenomen. Een veelzijdige veranderstrategie werd ingezet om de kans op succesvolle implementatie te maximaliseren. Specifieke lean interventies (instrumenten) gericht op procesverbetering werden gecombineerd met een generieke aanpak van de structuur en cultuur binnen de organisatie. Een voorbeeld van een specifieke lean interventie was de optimalisatie van de administratieve afdeling door hun werkomgeving en de werkprocessen met behulp van de 5S-methodiek te verbeteren: een lean instrument gericht op het creëren van een goed georganiseerde en opgeruimde werkomgeving door het verwijderen van overbodige materialen. In 2013 werden meerdere projecten uitgezet om de stroom van processtappen in de organisatie te verbeteren. De managementstructuur werd gereorganiseerd van een functionele naar een meer proces georiënteerde organisatie in januari Multidisciplinaire teams werden opgericht om procesinnovatie te faciliteren en betrokkenheid van de medewerkers te verhogen. In 2012 bediscussieerden de leden van de raad van bestuur hun visie met alle medewerkers tijdens kleine bijeenkomsten, om een brede acceptatie van doelen en een gedeelde visie te realiseren. Om SUBDOEL 1 te realiseren werd een workshop ontwikkeld als een instrument om de managementbesluitvorming in een gezondheidszorginstelling te verbeteren. Een goed inzicht in de opvattingen en percepties van medewerkers ten aanzien van een voorgesteld veranderprogramma is essentieel voor het laten slagen van het veranderprogramma. Dit is nog eens extra belangrijk wanneer de ideeën over de effecten van het voorgestelde veranderprogramma divers zijn. Tijdens de workshops brainstormden verschillende klinische professionals en managers over de belangrijkste effecten die zij verwachtten van het voorgestelde lean programma. Negatieve effecten werden voornamelijk gerapporteerd op de korte termijn, terwijl het aantal positieve effecten het aantal negatieve overtroffen op de lange termijn. Dit verklaart mogelijk waarom de lean filosofie in theorie eenvoudig is, maar in de klinische praktijk moeilijk te realiseren blijkt. De topmanagers van de betrokken organisatie waren slechts matig tot licht positief inzake de relevantie van de workshops ten aanzien van hun besluitvorming. Echter, verwachtten ze een toegenomen betrokkenheid van medewerkers voor het lean programma en een verbeterd bottom-up proces ten gevolge van de workshops. Aangezien dit positief wordt geassocieerd met het succesvol implementeren van veranderprogramma's, leidt deze workshop mogelijk tot succesvollere implementatie van lean, tegen relatief lage kosten. Sterke conclusies konden echter niet worden geformuleerd, aangezien het onderzoek zeer kleinschalig was. Echter werden wel de voordelen geschetst van het betrekken van medewerkers in een vroeg stadium van een verbeterinitiatief. Ondanks dat men dit instrument op grotere schaal dient te testen, kan een verdere opschaling worden overwogen. 18

20 SAMENVATTING Om SUBDOEL 2 te verwezenlijken werd binnen MAASTRO CLINIC een vragenlijst ontwikkeld die het patiëntveiligheidsbewustzijn en de intenties tot patiëntveiligheidsgedrag evalueerde. Aangezien de kwaliteit en veiligheid van zorg voornamelijk afhangt van het gedrag van de individuele zorgprofessional, kan het periodiek evalueren van dit veiligheidsgedrag managers waardevolle informatie opleveren om de verbeterinterventies af te stemmen om de specifieke noden van de organisatie. Het objectief evalueren van veiligheidsgedrag is echter moeilijk en tijdrovend. Om die reden werd besloten een vragenlijst te ontwikkelen die door middel van de vignettechniek de intenties tot veiligheidsgedrag meet. De vragenlijst bestond uit twintig hypothetische beschrijvingen (vignetten) over incidenten die willekeurig varieerden ten aanzien van; werkdruk, de veroorzaker van het incident, of het incident de patiënt bereikte, de ernst van de mogelijke schade, of de patiënt het incident had opgemerkt en of management patiëntveiligheidsactiviteiten ondersteunde. Na elke beschrijving werden één vraag gesteld over het veiligheidsbewustzijn en vier over de intenties tot gedrag. Resultaten van de vignetvragenlijst werden naast die van de Hospital Survey on Patient Safety Culture (HSOPSC: vragenlijst die patiëntveiligheidsklimaat meet) gelegd. De resultaten van de twee vragenlijsten ondersteunden elkaar. De vignetvragenlijst presenteerde aanvullende informatie, zoals bijvoorbeeld de intentie tot structureel verbeteren die onder de zorgprofessionals werd gemeten en het detecteren van de meest invloedrijke factoren inzake de intentie tot veiligheidsgedrag. Een groot nadeel van de vignetvragenlijst was dat door het grote detailniveau van de vignetten, het onmogelijkheid was om tussen organisaties te vergelijken. Dit detail was noodzakelijk om de vignetten zo herkenbaar mogelijk te maken voor professionals en de realiteit zo goed mogelijk te kunnen benaderen. Wanneer implementatie op grotere schaal wordt overwogen, dient men het detailniveau van de vignetten te bestuderen, om een goede balans te vinden tussen de mogelijkheid tot generaliseren van resultaten en tegelijk realistische voorbeelden te behouden. Inzake SUBDOEL 3 werd een lean transitie, welke gecombineerd werd met een overkoepelende managementstrategie, geëvalueerd in een radiotherapeutisch instituut (MAASTRO CLINIC). Tien experts hadden de meest belangrijke meetindicatoren inzake de patiënt, de medewerker en de organisatie samengevat. Gezien het meten van het daadwerkelijke effect van een lean interventie moeilijk is, werden einduitkomsten gecombineerd met surrogaat uitkomsten, zoals procesevaluatie, naleving van de interventie, cultuur en veiligheidsincidenten. Kwantitatieve resultaten vanuit softwareprogramma s en verschillende vragenlijsten werden met behulp van kwalitatieve interviews met zorgprofessionals geïnterpreteerd en geëxploreerd. Om een veilige radiotherapiebehandeling te garanderen en het behandelproces te kunnen optimaliseren, dienen alle relevante processen stabiel te zijn. Een stabiel proces geeft hetzelfde resultaat, dezelfde kwaliteit, onafhankelijk van 19

21 SAMENVATTING behandelaar of het moment van de dag. In 2007, werd het radiotherapeutisch behandelproces herontworpen door een project team van 25 (van de 60) radiotherapeutisch laboranten. Standaard operationele procedures (SOP's) voor twee in plaats van drie laboranten werden uitgewerkt en gefaseerd geïmplementeerd. Dit herontworpen proces verhoogde de naleving op specifieke patiëntveiligheidstaken die 1.5 jaar later had stand gehouden. Het belang dat medewerkers hechtten aan taken bleek positief gecorreleerd met de mate van duurzame naleving. Om de naleving van procedures te verbeteren is het zodoende van cruciaal belang om medewerkers bewuster te maken over het nut en de achtergronden van de taken of procedures door regelmatig geven van feedback. Het belang van een goede cultuur om kwaliteit en patiëntveiligheid te verbeteren is onbetwistbaar. Echter hoe men een goede patiëntveiligheidscultuur kan realiseren is niet eenvoudig. De overtuiging groeit dat meerdere interventies met elkaar gecombineerd dienen te worden, gericht op verschillende dimensies van kwaliteit en veiligheid. De patiëntveiligheidscultuur/klimaat, het veiligheidsbewustzijn en de intenties tot veiligheidsgedrag lieten gedurende een periode van drie opvolgende jaren een verbetering zien na een lean transitie in de bestudeerde organisatie. Een workshop gebaseerd op de Manchester Patient Safety Framework (MaPSaF) liet geen verandering in patiëntveiligheidscultuur zien. De HSOPSC (klimaatvragenlijst) en de vignetvragenlijst (veiligheidsbewustzijn en intenties tot veiligheidsgedrag) lieten daarentegen wel verbeteringen zien. De intentie tot melden van incidenten die de patiënt niet bereiken en het aantal gemelde incidenten in het incidentmeldingsysteem daalde. De medewerkers rapporteerden echter een stabiel veiligheidsbewustzijn en een verhoogde intentie om structureel te verbeteren. Zodoende werd de daling van het aantal gerapporteerde veiligheidsincidenten door medewerkers toegeschreven aan een daling van het aantal incidenten die daadwerkelijk gebeurd waren, mogelijk als gevolg van een toegenomen probleemoplossend vermogen onder de medewerkers en verbeterde apparatuur. Conclusies dienen met enige voorzichtigheid te worden geformuleerd, aangezien de resultaten gebaseerd zijn op vragenlijsten en interviews die objectiviteit niet kunnen garanderen. Desondanks leek het combineren van specifieke lean management initiatieven met een overkoepelende generieke managementstrategie een succesvolle aanpak ter verbetering van de patiëntveiligheidscultuur in een zorgorganisatie. Als onderdeel van de generieke interventies om kwaliteit, patiëntveiligheid en efficiëntie te verbeteren werden van juli 2011 tot maart 2013 alle zeven behandeltoestellen (versnellers) en het software systeem voor het maken van de bestralingsplannen vervangen door een geïntegreerd systeem met zes versnellers. Ondanks dat dit geen specifieke lean interventie was, werd de lean filosofie hierdoor wel ondersteund. Het belang om de apparatuur goed op de mens en de organisatorische processen af te stemmen werd benadrukt, in plaats van andersom. Het aantal gerapporteerde incidenten gerelateerd aan de 20

22 SAMENVATTING behandeltoestellen daalde significant van 41 (2010) naar 14 (2013) meldingen per maand, terwijl het aantal geleverde fracties per versneller steeg met 20%. Bovendien, daalde het percentage technisch gecodeerde oorzaken, terwijl het percentage organisatorisch gecodeerde oorzaken stabiel bleef. De downtime (niet operationele tijd) van de behandeltoestellen ten gevolge van technische storingen daalde van 5.4% van de beschikbare behandeluren in 2010 naar 2.9% in Met behulp van gestructureerde interviews met klinische professionals werden de kwantitatieve resultaten geïnterpreteerd. Men kan zich afvragen of de afname van gemelde incidenten verklaard wordt door een afname van het daadwerkelijke aantal voorgevallen incidenten of doordat medewerkers de incidenten minder hebben gemeld. De geïnterviewde professionals rapporteerden geen afgenomen meldingsbereidheid. Integendeel, zij percipieerden verbeterde behandelprocessen, die gedeeltelijk werden toegeschreven aan de transitie naar de nieuwe apparatuur en gedeeltelijk aan een toegenomen management focus op continue verbeteren. Door het visualiseren van de lange termijn effecten van een herontworpen zorgproces op basis van lean principes, worden zorgprofessionals en managers mogelijk meer betrokken en gemotiveerd om de lean filosofie in hun praktijk toe te passen. Met behulp van een economische model (Markov model) werd de kosten-effectiviteit van een procesoptimalisatie op de lange termijn berekend van het diagnostische traject en het radiotherapie-voorbereidingstraject bij patiënten met hoofdhalskanker. Deze procesoptimalisatie omvatte twee belangrijke organisatorische veranderingen; respectievelijk de biopsie van het tumorweefsel onder lokale in plaats van volledige narcose, en het uitvoeren van een CT-PET in bestralingshouding tijdens het diagnostische traject in plaats van een extra CT-PET tijdens de voorbereidingstraject van radiotherapie. De herontworpen zorgprocessen leidde tot een verkorting van de wachttijden en bleken kosteneffectief op de lange termijn. Het effect, uitgedrukt als het aantal additionele, voor kwaliteit gecorrigeerde, levensjaren (QALY's) varieerde tussen 0.13 en 0.66 tegen slechts geringe extra kosten voor de procesoptimalisatie. Rekening houdend met de vele aannames in het model, resulteerde de procesoptimalisatie in lange termijn verbeteringen gecombineerd met een kostbesparing voor de patiënt (zorgbetaler). Hoewel het herontwerpen van zorgprocessen minder aantrekkelijk is voor professionals in de gezondheidszorg in vergelijking met de implementatie van nieuwe technologieën, laat deze studie duidelijk positieve effecten zien voor patiënten met hoofdhalskanker, welke niet ontkent kunnen worden. In 2013 werden 15 specifieke lean projecten opgestart om de doorstroming van patiënten te verbeteren en wachttijden te verkorten. Deze projecten werden voorafgegaan door generieke interventies om betrokkenheid onder de medewerkers te verhogen en een gedeelde visie te creëren. Het percentage van de patiënten, waarvan de wachttijd de Nederlandse nationale normen en doelstellingen overschreed, was significant gedaald. Er werden geen negatieve 21

23 SAMENVATTING neveneffecten voor patiënten en medewerkers geconstateerd. Integendeel, de veiligheid van patiënten, de werknemertevredenheid en het percentage ziekteverzuim verbeterden eveneens. Hoewel de wachttijden slechts kleine verbeteringen lieten zien, motiveren deze resultaten mogelijk de medewerkers om te streven naar continue verbetering. Door de methodologische beperkingen kunnen geen sterke conclusies worden geformuleerd. Echter leek de toegepaste combinatie van specifieke lean initiatieven met een generieke aanpak succesvol. De generieke aanpak omvatte het creëren van een organisatiestructuur die innovatie ondersteunt, het verhogen van betrokkenheid onder medewerkers met toegenomen aandacht voor continu verbeteren. De grote mate van ambiguïteit van doelstellingen en de onzekerheid van oorzaakgevolg relaties in de zorgprocessen zijn mogelijk invloedrijke concepten inzake het creëren van duurzame kwaliteitsverbetering in de zorg. Gebaseerd op resultaten uit vragenlijsten verbeterden de voorspelbaarheid van zorgprocessen, de standaardisatie en het creëren van een gezamenlijk doel na de lean transitie. Tijdens twaalf interviews gaven de professionals aan dat na de lean transitie de ambiguïteit van doelstellingen afnam en de onzekerheid van oorzaakgevolg relaties in de zorgprocessen verminderde. Als gevolg verbeterde mogelijk de organisatorische besluitvorming wat de kans op duurzame kwaliteitsverbetering in de zorg ten goede komt. De geformuleerde conclusies dienen met enige voorzichtigheid geïnterpreteerd te worden, gezien de methodologische beperkingen van de verzamelde resultaten. Desondanks geven alle verzamelde effecten een verbetering weer. Het combineren van de generieke aanpak met specifieke lean instrumenten lijkt succesvol in het realiseren van verbetering in kwaliteit, patiëntveiligheid en efficiëntie binnen deze zorginstelling. De specifieke lean instrumenten dienen aangepast te worden aan de specifieke problemen en noden van elke zorginstelling. Om echter duurzame verbetering te verwezenlijken dient men simultaan te werken aan alle drie de concepten: organisatiestructuur, cultuur en de processen. De combinatie van verschillende kwantitatieve en kwalitatieve gegevens hebben geleid tot een diepgaande evaluatie van de effecten. Interpretatie van kwantitatieve effecten werd mogelijk door in het laatste jaar van de evaluatie de kwantitatieve methoden aan te vullen met gestructureerde interviews. Wanneer in toekomstig onderzoek interviews periodiek worden uitgevoerd gedurende de evaluatieperiode, creëert men de mogelijkheid om tijdens het implementatietraject verbeterinterventies naar wens aan te passen en eventueel het meetprotocol bij te sturen. De geïnitieerde lean interventies waren enkel eerste stappen op de weg naar continue verbetering. Of deze eerste successen klinische professionals motiveren tot substantiële en duurzame verbetering in de stroom van processen en kwaliteit van zorg moet zich nog uitwijzen. De gecombineerde aanpak lijkt de 22

24 SAMENVATTING kwaliteit van zorg te verbeteren, maar moet zich nog bewijzen in de zoektocht naar perfectie. De uitdaging van deze zorginstelling ligt niet enkel in het behouden van de verbeteringen, maar in het ontwikkelen tot een zorginstelling welke de perfecte zorg voor alle patiënten levert. Enkel wanneer klinische professionals continu verbeteren hebben opgenomen in hun professionele waarden is duurzame verbetering een feit. De kwaliteit van de zorg moet verbeteren, dat is duidelijk voor de meeste professionals. Echter blijft veranderen moeilijk en levert dit een voortdurende strijd tussen verschillende belangen op. Het vraagt vastberadenheid, doorzettingsvermogen en een focus op de lange termijn, om de dagelijkse praktijk in de zorg duurzaam te verbeteren. Echter, veilige gezondheidszorg van hoge kwaliteit is een voorrecht voor alle patiënten. Daarom, zijn professionals in de gezondheidszorg, het management en beleidsmakers moreel verplicht alles in hun mogelijkheden te doen om dit te realiseren. Lean management lijkt een veelbelovende strategie om de weg naar continu verbeteren in de gezondheidszorg te creëren. 23

25 SAMENVATTING 24

26 1 Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour CHAPTER Introduction and outline

27 CHAPTER 1 BACKGROUND Healthcare is increasingly more focused towards quality, safety, and efficiency. Individualized high quality care, which meets the patients needs, is demanded. Quality and safety of care have not reached a satisfactory level. 1 Many healthcare organizations have been actively improving the quality and safety of their delivered care over the past two decades. Subsequently, a massive increase of published papers concerning patient safety followed. 2 Although some improvements have been accomplished, many quality and safety initiatives fail to realize sustainable improvement, even though the implemented improvement program appeared to be successful at first. Despite the numerous improvement efforts, there is still room to further improve quality and safety of care. 1,3-6 In addition to increased quality of care, efficiency of care processes should be improved to bring healthcare costs to a minimum. Healthcare costs have been exponentially rising because of the ageing population, and the expanded treatment possibilities in particular Since resources are scarce, healthcare management has to make difficult spending decisions. To deliver high quality and safe care, and at the same time remain financially healthy, organizations need strategies to combine quality improvement (QI), and increase efficiency more stringent than ever before. Within the current perceptions, improvement of quality and safety are inherent to additional costs, and an increased burden for professionals. However, an increasing number of studies show that business strategies, like lean management, improve both quality and efficiency in health care at the same time. 6,11-18 Lean management Lean management is a holistic philosophy aiming to create quality and safety combined with efficiency and satisfied employees. Lean has been originally developed by the Toyota Motor Company. Because of its successes, this business strategy was rapidly diffused within the global car manufacturing 19 and has been adopted by other manufacturing and service industries. Within the lean management philosophy the customer (i.e. patient) is the central starting point. All processes and process steps should add value in the costumers perspective. Process steps that do not directly add value for the customer, defined as waste, should be minimized through the regular redesign of processes. The philosophy encloses a long-term vision aiming for continuous improvement and engaging all employees for improvement. Within the lean philosophy, a combination of tools is used to create improvement. Examples of these lean tools include; value stream mapping which visualizes the entire process with all its bottlenecks: Kaizen which implies daily (small) improvements involving all employees; and the five why s are used to develop greater understanding of problems and solve their root causes instead of only 26

28 INTRODUCTION AND OUTLINE addressing the symptoms. 20 Liker visualized the lean philosophy as the construction of a house (Figure 1) 20 ; the tools positioned in the pillars, the fundament includes standardization of processes and a thorough understanding of the long term philosophy. In addition, employees should be committed to change and share the organizations vision and common goals. The quest for perfection is the ultimate goal of lean management. Through continuously improving processes, the best quality and safety to the lowest costs with the shortest process time and a high morale among employees should be realized. Figure 1. The Toyota Production System. Lean believers argue that lean is beneficial for employees, owing to increased influence on their own working conditions. On the contrary, critics state that lean would result in monotonous and repetitive work, with negative effects on job satisfaction. Although this discussion is still ongoing 21-23, an increasing number of healthcare organizations are implementing lean principles as quality improvement approach and the literature regarding lean healthcare has been rapidly growing in the last two decades. Lean healthcare As lean management, has its origins in the automobile industry, it is a production system. To improve outcomes, lean change concepts, and tools are designed to reduce waste, and variation in systems where the concept of a value stream applies. In healthcare, many processes reveal comparisons with a production process, from doctor visits to medication administration to laboratory and radiology processes. However, it should be recognized that the lean approach has to be translated and adapted to the specific problems and needs in 27

29 CHAPTER 1 health care processes. 24 Healthcare differs from the industry in that quality, or value for the patient, is not clear at the start of the value stream. In the automobile industry, the customer chooses between the many possible product variations up front. Quality of the end-result can be clearly defined before entering the value stream. In healthcare, however, value for the patient and the required end-result are progressively defined and adjusted to the changing patient needs along the care process. Quality in healthcare is not a static concept, but it depends on uncontrollable factors outside the process, since patients themselves are intrinsic producers of their own outcomes. 24 In addition, the definition of quality of a care process is hampered by the complex relation between relevant stakeholders (e.g. patients, health care insurers, politics, etc). 25 Despite the differences between the automobile industry and healthcare, the lean philosophy has been widely disseminated in healthcare in the forgoing years to improve quality and efficiency of care in a variety of care settings. As a result the available literature regarding lean healthcare has increased in recent years. Some leading health care organizations were successful in adopting lean as their model for improvement and management. 24 For example, the Virginia Mason Medical Center found that translating the concept of jidoka (the continuous inspection, and intermitting the process when a defect is detected) into the realm of medication errors and adverse events presented a host of definitional problems. The Patient Safety Alert was designed: when an error was detected which could endanger patient safety, a signal was declared activating a set of procedures for rectifying the error without actually halting patient care. Similarly, the flow station was invented to help eliminate waste and improve flow in the hospital workplace where, the providers, not the patients, are constantly on the move. 26 At ThedaCare, improvement teams (including clinical professionals) are continually reminded to consider the patient first. Although this seems evident, value in a care process, defined in terms of the patient, is an emotionally loaded exercise. Not only do physicians embrace different methods and measures, there is often a good deal of ego invested in those methods. 27 The relationship between QI approaches and Lean is often questioned, as well as the approach that should be used to improve an organizations care system. However, QI approaches, like the Institute for Healthcare Improvement approach based on the application of Deming s System of Profound Knowledge 28, and lean are complementary ways of approaching improvement. Therefore, it is not necessary to choose one over the other as a guide to action. Furthermore, integrating perspectives and lessons from the two approaches has the potential to strengthen both Lean and IHI-QI

30 INTRODUCTION AND OUTLINE Although the overall impressions are in favour of implementing lean in healthcare 6,14,16,17, counter-arguments against this philosophy are also publicized. Two major issues seem present regarding lean healthcare: 1) the extensive variety of strategies for implementing lean management 2) the difficulties to objectively measure effects of lean interventions Regarding the first issue, lean in healthcare is explained in a variety of ways. The majority of healthcare organizations implement lean as a set of tools guided by the idea of eliminating waste based on what is defined as being valuable, rather than as a total system governing the whole of the organization. 6,29 When lean is implemented solely as an approach providing some standards for processes, creating flow, decreasing interruptions, and potential for errors (lean tools) without being part of a comprehensive management system with a supportive organizational culture and committed leaders, sure improvements will be made, but these will lack sustainability. 6,17,18,29 In congruence with the quality and safety improvement literature, the outcomes for the patient are the result of clinical as well as management processes (Figure 2). Management processes will generally affect quality and safety (clinical) outcomes through their effect on intervening variables, such as morale, culture or absenteeism. 30 interventions interventions context of change effect/outcome patient organisational structure generative interventions culture behaviour quality safety efficiency specific interventions processes Figure 2. The model for creating sustainable change. 29

31 CHAPTER 1 The clinical and management processes need different strategies to create sustainable change. To realize quality and safety improvement in clinical processes, specific interventions should be initiated. The tools used in the lean approach can be considered as specific interventions addressing specific problems in clinical processes. However, to realize sustainable improvements, these specific interventions should be combined with generative interventions to improve the management processes. Although these processes do not directly result in patient benefits, they are required to facilitate the specific interventions by creating a supportive culture, and committed leaders and clinical professionals. 1,30,31 Therefore, a system-level approach is advised to create sustainable change, in which lean tools (specific interventions) are combined with an overall strategy to create a culture of improvement, and guide leaders and clinical professionals in their change process. 6,17,18,29,30 Leaders should be trained to become managers of the system and facilitate quality improvement instead of being problem solvers. 6 Clinical professionals should be trained in teamwork and changing the professional (hierarchal) culture among doctors and between doctors and nurses. 32 Changing professionals behaviour and breaking professional routines are, next to use of human factor proof equipment, important aspects to create sustainably improved care. Individuals behaviour takes place in the context of organizational structure, culture, and processes. In order to create sustainable behavioural change, interventions aimed towards these three aspects should be integrated (Figure 2). 33 To change the organizational structure and the culture dimensions of care organizations, generative QI interventions are developed, while specific interventions are directed to improve the clinical processes. Regarding the second issue on the difficulties to objectively measure the effects of lean, the majority of literature is supportive of lean in healthcare. However, evidence is limited and results should be interpreted with caution. 17 Few studies have included effects related to outcomes for the patient or the employee A major problem with evaluating end-points regarding patient or employee outcomes is the difficulty of separating the interventions real effect from the noise, since the end-point is influenced by many other factors than the implemented changes. When only end-points are used to evaluate improvement interventions, one risks to conclude no improvement was realized when this is not true. Therefore, it is advised to combine end-point outcomes with surrogate end-points like process evaluation, compliance to the intervention, culture and errors (incidents). 31 In addition, quality and safety are difficult to measure objectively since these depend on technical as well as social practices. 37,38 No one size fits all methodology for the measurement of quality and safety is available. 39 To meet the complex nature of the concepts under evaluation, change strategies, like lean management, in healthcare should be evaluated using a mixed methodology Multiple effects should be evaluated by means of quantitative and qualitative measures. This ensures triangulation to discover 30

32 INTRODUCTION AND OUTLINE the same truth and increases understanding in measured effects. 39 These multiple effects include outcomes affecting the patient, the employee and the organization (Figure 2). When positive effects on all three dimensions can be achieved, sustainability of improvements will benefit. Despite the many problems of evaluating effects of lean initiatives, one should strive to evaluate accurately, and on a periodical base to facilitate the learning process of the organization. Effect evaluation creates the opportunity to celebrate achieved successes, which enhance employee commitment and sustainability of the introduced change. 6 Through measurement, deviations of aimed results can be detected and interventions can be readjusted accordingly, visualized in Figure 2 by the arrows returning from effects to interventions. This backward loop emphasizes that improvement is a continuous process with no end-point. This backward loop is also one of the core elements in the Model for Improvement, including the Plan-Do-Study-Act (PDSA) cycle. 43 The PDSA cycle can complement the lean approach as a valuable tool to realize continuous improvement, since the constant (periodic) evaluation of outcomes result in adjusted interventions. The recurring improvement loops should facilitate a gradual improvement of quality outcomes (Figure 3). 24 A review on the application of the PDSA method revealed that only 14% of included quality improvement studies reported the use of data at frequent intervals for evaluation. 44 This could be an important factor for the disappointing results of quality improvement initiatives. Evaluation of daily processes on the work floor is incorporated in lean management as Genchi Genbutsu (go and see) and is one of leans key principles. 20 Although the evaluation of effects of lean is difficult, it should be incorporated in the overall management system to support the learning culture and enhance employees commitment to quality improvement. Figure 3. Model for continuous improvement. Gradual improvement is based on learning from positive and negative results. The ineffective interventions should be revised or abandoned, and the effective further developed. 31

33 CHAPTER 1 Lean in radiotherapy Radiotherapy is a treatment that involves the use of high-energy radiation and is commonly used to treat cancer. Almost half of all people with cancer have radiotherapy as part of their treatment plan. Demand for radiotherapy has been growing, and quality standards and costs have been rising. 7-9,45 Although radiotherapy presents a long history with quality assurance, patients are still confronted with treatment errors. 46,47 Lean management might help to further improve quality, safety and efficiency in radiotherapy. However, a literature search in Pubmed revealed only four relevant articles of which one is included as chapter 5 in this dissertation. Aasebø et al. presented decreased workup time from a mean of 64 days to 16 days, and the median time from diagnosis to surgery was reduced from 26.5 days to 15 days after introducing a Lean project. 14 Kim et al. presented an increased percentage (43% to 95%) of patients A Google search revealed one additional article, which again presented positive results of implementing lean in a radiotherapy setting 50, and three Dutch books that incorporated in total five chapters describing results for radiotherapy institutes in the Netherlands. The difficulties to realize sustainable improvement, holding on to a long term ideal 51, increasing employee commitment by participating in multidisciplinary improvement teams 52, aiming for continuous improvement 53, and the changing role of leaders 54 were addressed, including the importance of connecting the objectives of the organisation and the individual professionals. 55 However, the chapters did not present objective effects of lean in radiotherapy. treated for bone and brain metastases within one day. 48,49 In conclusion, evidence regarding the effects of lean management in health care settings seems promising, however, it is very limited. 17,34,36,56 The radiotherapy setting in particular is expected to profit from lean initiatives, as radiotherapy treatment includes repetitive processes in a high technologic environment. However, very little scientific literature has been published about the effects of lean in radiotherapy. 32

34 INTRODUCTION AND OUTLINE DISSERTATION OBJECTIVE The objective in this dissertation is to perform an extensive assessment of a lean management transition in a healthcare setting, where specific lean tools were combined with an overall management approach. Effects on the level of the patient, the employee and the organization (Figure 2) were evaluated with a special focus on quality, safety, and efficiency. This objective was subdivided into three research aims: 1. To develop an instrument to improve management decision making in a healthcare setting, regarding the choice for a lean change program and increase understanding in employees views and commitment regarding the proposed lean program. 2. To develop an instrument to measure patient safety awareness and intentions for patient safety behaviour in addition to the internationally used instruments for safety climate measurement. 3. To evaluate effects of a lean transition in a healthcare setting on the patient, the employee and the organizational level. DISSERTATION OUTLINE To successfully implement a lean management program, managers should share a realistic view of the programs effects on their employees daily practice. 57 In addition, when employees are involved in the early stages of change, the implementation of a lean program could benefit from improved management decision making 58,59 and improved employee commitment. 60 To improve the organizations management decision making regarding the implementation of a proposed lean program and increase managers understanding in the related employees views, an instrument (workshop) was developed (research aim 1) and results are presented in chapter 3. Since the evaluation of effects is profound, and quality and safety of care is for a part dependent on employee s behaviour, we developed a survey to measure patient safety awareness and intentions for safety behaviour during the lean transition. In chapter 4, the results of this survey are presented (research aim 2) and related to those of an internationally used survey to measure safety climate. The following chapters all consider research aim 3, and together present an extensive assessment of the effects of the lean transition supported by the organizational changes. Chapter 5 presents the compliance results after redesigning and standardizing the radiotherapy treatment process over a three year period. This chapter describes one the fundaments of the lean transition, since standardization is one of the basic requirements to build on. In chapter 6, effects of the lean transition on patient safety culture, in specific the safety climate, safety awareness and 33

35 CHAPTER 1 intentions for safety behaviour are presented. Chapter 7 presents the specific results on patient safety incidents when technology was better synchronized and was increasingly human factor proof. Although this is not a direct lean intervention, it is part of the overall management philosophy to create improved outcomes for patients and employees. It supports the lean philosophy by understating the importance to adapt technology to the human and organizational processes and not the contrary. Chapter 8 considers the costeffectiveness of a redesigned process regarding the diagnostics and preparation process for patients with head and neck cancer. Presenting the long term benefits for the patient of a process redesign based on lean principles, might increase clinical professionals commitment to lean management initiatives. To commit healthcare managers to these lean initiatives, and ensure sustainability of results, quality improvements will have to be combined with efficiency gains or cost reductions. Chapter 9 covers a broad evaluation of the lean transition and presents results on waiting times, and on outcomes for patients and employees. In chapter 10, the employees perceptions of the lean transition are presented, based on structured interviews with professionals, complemented by survey results. In particular two concepts, which might influence the realization of sustainable quality improvement, were evaluated: ambiguity of objectives, and uncertainty of cause-effects relations in care processes. Chapter 11 presents a general discussion, the limitations of this research, and the concluding remarks and future perspectives. 34

36 INTRODUCTION AND OUTLINE REFERENCES 1. Dixon-Woods M, Baker R, Charles K, et al. Culture and behaviour in the English National Health Service: overview of lessons from a large multimethod study. BMJ Qual Saf 2014; 23: Lilford R, Stirling S, Maillard N. Citation classics in patient safety research: an invitation to contribute to an online bibliography. Qual Saf Health Care 2006; 15: Beer M, Nohria N. Cracking the code of change. Harvard Business Review 2000; 78: Brackett T, Comer L, Whichello R. Do lean practices lead to more time at the bedside? J Healthc Qual 2013; 35: Glasgow JM, Scott-Caziewell JR, Kaboli PJ. Guiding inpatient quality improvement: a systematic review of Lean and Six Sigma. Jt Comm J Qual Patient Saf 2010; 36: Kaplan GS, Patterson SH, Ching JM, Blackmore CC. Why Lean doesn't work for everyone. BMJ Qual Saf Orszag PR, Ellis P. The challenge of rising health care costs--a view from the Congressional Budget Office. N Engl J Med 2007; 357: Slotman BJ, Vos PH. Planning of radiotherapy capacity and productivity. Radiotherapy and Oncology 2013; 106: Van de Werf E, Verstraete J, Lievens Y. The cost of radiotherapy in a decade of technology evolution. Radiotherapy and Oncology 2012; 102: Sullivan R, Peppercorn J, Sikora K, et al. Delivering affordable cancer care in high-income countries. The Lancet Oncology 2011; 12: Endsley S, Magill MK, Godfrey MM. Creating a lean practice. Fam Pract Manag 2006; 13: McCarthy M. Can car manufacturing techniques reform health care? Lancet 2006; 367: Rutledge J, Xu M, Simpson J. Application of the Toyota Production System improves core laboratory operations. Am J Clin Pathol 2010; 133: Aasebo U, Strom HH, Postmyr M. The Lean method as a clinical pathway facilitator in patients with lung cancer. Clin Respir J 2012; 6: Kaplan GS, Patterson SH. Seeking perfection in healthcare. A case study in adopting Toyota Production System methods. Healthc Exec 2008; 23: 16-8, Mason SE, Nicolay CR, Darzi A. The use of Lean and Six Sigma methodologies in surgery: A systematic review. Surgeon McIntosh B, Sheppy B, Cohen I. Illusion or delusion--lean management in the health sector. Int J Health Care Qual Assur 2014; 27: Wood D. A prescription for lean healthcare. Healthc Q 2014; 17: Womack JP, Jones DT, Roos D. The Machine that Changed the World. Free Press, New York, Liker JK. The Toyota Way. McGrawHill, New York,

37 CHAPTER Conti R, Angelis J, Cooper C, Faragher B, Gill C. The effects of lean production on worker job stress. International Journal of Operations & Production Management 2006; 26: Hasle P, Bojesen A, Jensen PL, Bramming P. Lean and the working environment: a review of the literature. International Journal of Operations & Production Management 2012; 32: Schouteten R, Benders J. Lean Production Assessed by Karasek s Job Demand Job Control Model. Economic and Industrial Democracy 2004; 25: Scoville R, Little K. Comparing Lean and Quality Improvement. Cambridge: Institute for Healthcare Improvement; Abernethy MA, Chua WF, Grafton J, Mahama H. Accounting and Control in Health Care: Behavioural, Organisational, Sociological and Critical Perspectives. In: Chapman CS, Hopwood AG, Shields MD, editors. Handbook of Management Accounting Research. Oxford: Elsevier; p Kenney C. Transforming Health Care: Virginia Mason Medical Center's Pursuit of the Perfect Patient Experience. Productivity Press, Taylor & Francis Group, New York, Toussaint JS, Gerard R. On the Mend: Revolutionizing Healthcare to Save Lives and Transform the Industry. MA: Lean Enterprise Institute, Cambridge, Demming WE. The New Economics for Industry, Government and Education. MIT Press, Cambridge, Al-Balushi S, Sohal AS, Singh PJ, Al Hajri A, Al Farsi YM, Al Abri R. Readiness factors for lean implementation in healthcare settings--a literature review. J Health Organ Manag 2014; 28: Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 1. Conceptualising and developing interventions. Qual Saf Health Care 2008; 17: Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 3. End points and measurement. Qual Saf Health Care 2008; 17: Drotz E, Poksinska B. Lean in healthcare from employees perspectives. Journal of Health Organization and Management 2014; 28: Guldenmund FW. The use of questionnaires in safety culture research an evaluation. Safety Science 2007; 45: Vest JR, Gamm LD. A critical review of the research literature on Six Sigma, Lean and StuderGroup's Hardwiring Excellence in the United States: the need to demonstrate and communicate the effectiveness of transformation strategies in healthcare. Implement Sci 2009; 4:

38 INTRODUCTION AND OUTLINE 35. Nicolay CR, Purkayastha S, Greenhalgh A, et al. Systematic review of the application of quality improvement methodologies from the manufacturing industry to surgical healthcare. Br J Surg DelliFraine JL, Langabeer JR, 2nd, Nembhard IM. Assessing the evidence of Six Sigma and Lean in the health care industry. Qual Manag Health Care 2010; 19: Dixon-Woods M, Leslie M, Bion J, Tarrant C. What Counts? An Ethnographic Study of Infection Data Reported to a Patient Safety Program. Milbank Quarterly 2012; 90: Vincent C, Burnett S, Carthey J. The measurement and monitoring of safety. The Health Foundation, London, Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 4. One size does not fit all. Qual Saf Health Care 2008; 17: Cooper Ph.D MD. Towards a model of safety culture. Safety Science 2000; 36: Kaboli PJ, Mosher HJ. Using balanced metrics and mixed methods to better understand QI interventions. BMJ Quality & Safety 2014; 23: Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Quality & Safety 2013; 22: Langley GJ, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. Jossey-Bass, San Francisco, Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic of the application of the plan do study act method to improve quality in healthcare. BMJ Quality & Safety Williams MV, Summers ET, Drinkwater K, Barrett A. Radiotherapy dose fractionation, access and waiting times in the countries of the UK in Clinical oncology Royal College of Radiologists Great Britain 2007; 19: Clark BG, Brown RJ, Ploquin JL, Kind AL, Grimard L. The management of treatment error through incident learning. Radiotherapy and Oncology 2010; 95: Shafiq J, Barton M, Noble D, Lemer C, Donaldson LJ. An international review of patient safety measures in radiotherapy practice. Radiotherapy and Oncology 2009; 92: Kim CS, Hayman JA, Billi JE, Lash K, Lawrence TS. The application of lean thinking to the care of patients with bone and brain metastasis with radiation therapy. J Oncol Pract 2007; 3: Kim CS, Spahlinger DA, Kin JM, Coffey RJ, Billi JE. Implementation of lean thinking: one health system's journey. Jt Comm J Qual Patient Saf 2009; 35:

39 CHAPTER Trilling L, Pellet B, Delacroix S, Colella-Fleury H, Marcon E. Improving care efficiency in a radiotherapy center using Lean philosophy: A case study of the proton therapy center of Institut Curie 2014; Orsay. Health Care Management (WHCM), 2010 IEEE Workshop p Backes H, Jacobs M. Lean in MAASTRO Clinic: gestaag richting 'true North'. In: Benders J, Rouppe van der Voort M, Berden B, editors. Lean denken en doen in de zorg. Boom Lemma uitgevers, Den Haag, p Naber T, Belonje-Reijenga R. Met lean naar 'sneller, beter, leuker' werk in Instituut Verbeeten. In: Benders J, Rouppe van der Voort M, Berden B, editors. Lean denken en doen in de zorg. Boom Lemma uitgevers, Den Haag, p Simons P, Backes H. Op weg naar een continu lerende organisatie. In: Rouppe van der Voort M, Benders J, editors. Lean in de zorg. Boom Lemma uitgevers, Den Haag, p Stoffer M, van Zon T. De kleine dingen die het doen. In: Rouppe van der Voort M, Benders J, editors. Lean in de zorg. Boom Lemma uitgevers, Den Haag, p Leenheer S, Naber T. 'Meer dan resultaten alleen': de lean reis van Instituut Verbeeten. In: Rouppe van der Voort M, Benders J, editors. Verder met lean in de zorg; naar de kern van de zaak. Boom Lemma uitgevers, Den Haag, Nicolay CR, Purkayastha S, Greenhalgh A, et al. Systematic review of the application of quality improvement methodologies from the manufacturing industry to surgical healthcare. British Journal of Surgery 2012; 99: Morténius H, Marklund B, Palm L, Björkelund C, Baigi A. Implementation of innovative attitudes and behaviour in primary health care by means of strategic communication: a 7-year follow-up. Journal of Evaluation in Clinical Practice 2012; 18: Collier N, Fishwick F, Floyd SW. Managerial Involvement and Perceptions of Strategy Process. Long Range Planning 2004; 37: Wagner JA, Leana CR, Locke EA, Schweiger DM. Cognitive and motivational frameworks in US research on participation: A meta-analysis of primary effects. Journal of Organizational Behavior 1997; 18: Khatri N, Halbesleben JRB, Petroski GF, Meyer W. Relationship between management philosophy and clinical outcomes. Health Care Management Review April/June 2007; 32:

40 CHAPTER 2 Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour Research methodologies

41 CHAPTER 2 Lean management in healthcare is a holistic philosophy aiming to create the ultimate quality and safety of care combined with efficient processes and satisfied clinical professionals. 1 To evaluate a lean transition in a healthcare setting, a radiotherapy institute in the Netherlands (MAASTRO CLINIC) was followed in their journey towards a lean healthcare organization. In October and November of 2010, a panel of ten experts listed the most important indicators for evaluation, by use of the Delphi-technique. 2 The experts were selected on a pragmatic base. Seven of twelve experts were working at MAASTRO CLINIC; the chief execute officer, one radiation oncologist, one clinical physicist, two radiation technologists, the lean manager, and the safety manager. Three experts were from the Hasselt University of which two were safety experts and one was an expert in business economics. Multiple indicators covered outcomes for patients, employees, and the organization. Since the end-points of quality and safety of care are difficult to evaluate accurately 3, multiple surrogate endpoints were selected. Specific issues were addressed as patient safety incidents, patient safety culture and awareness, intentions for safety behaviour, compliance to procedures, process (waiting) times, patient and employee satisfaction, employee absenteeism, efficiency and economic issues like costbenefit analyses. SETTING MAASTRO CLINIC is a radiotherapy institute in the Netherlands, where on average oncology patients receive radiotherapy treatment every year. This corresponds to on average 200 oncology patients receiving treatment every day. The radiotherapy process is characterized by a complex multidisciplinary treatment facility using highly complex technology. A team of 19 radiotherapy oncologists, eight clinical physicists and 70 radiation technologists (2014) was committed to deliver high qualitative individualized radiotherapy treatment for all patients. Before 2011, three multi-disciplinary units operated the clinic. In January 2011, the three units were restructured in two operational units. Each unit was responsible for the treatment of specific oncology sites (e.g. breast-, lung-, prostate cancer) and included radiation oncologists and technologists. The clinical physicists were not divided in the units. Since 2003, the organization has been engaged in improving patient safety. Their safety management system was the first to be certified from all healthcare organizations in the Netherlands (2008). Prospective as well as retrospective risk analyses are performed. To avoid adverse events from happening, prospective risk analyses are performed before implementing new technology or new treatment options using the Dutch version of the Healthcare Failure Mode and Effect Analysis (HFMEA). 4,5 Regarding the retrospective risk analysis, safety 40

42 RESEARCH METHODOLOGIES incidents are voluntarily reported in an incident reporting system. All reported incidents are analyzed by trained safety professionals. The root causes are coded in human, organizational and technical codes following the Prevention and Recovery Information System for Monitoring and Analysis (PRISMA) technique 6,7 and recorded in a software program. Results are periodically reported to management and employees. LEAN AT MAASTRO CLINIC In 2007, MAASTRO CLINIC started their Lean journey by initiating a stand alone lean based project to improve quality, safety and efficiency in radiotherapy treatment. In 2010, the lean philosophy was up taken in the organizations long term strategy. Since a system-level approach is advised to create sustainable improvement, MAASTRO CLINIC combined specific interventions (lean tools) focused at the organizations processes with an overall strategy aimed at the organizations structure and culture elements, see Figure interventions interventions context of change effect/outcome patient organisational structure generative interventions culture behaviour quality safety efficiency specific interventions processes Figure 1. The model for creating sustainable change. Although not all initiated implementations were not fully documented, Table 1 presents an overview of the most important interventions. In January 2011, management was restructured from a functional oriented to a process oriented 41

43 CHAPTER 2 Table 1. Initiated implementations in MAASTRO Year Initiated interventions General Specific Structure Culture Process Information sharing about lean philosophy to Process redesign on accelerators; SOP design for two instead of three technologists Training Lean Six sigma (green belt)for 16 clinical employees and 3 operational managers - VSM* and process redesign for overall treatment process of patients with lung cancer Lean field trip to Seattle for 5 managers, including 2 oncologists - Training lean in healthcare for 2 managers - Restructure from functional to process oriented management - Initiation of multidisciplinary teams for care paths - Patient flow incorporated in management indicators - Multidisciplinary meetings increasingly structured - Formation of team of professionals to intermediate between management and clinics - Implementation of software system for management of protocols/standard Operation Procedures - Whiteboard on treatment planning department to visualize the process flow - Improvement of quality and release time of new software - Improvement of test before implementing new applications - Redesign of procedures for quality control of accelerators (treatment equipment) - 5S for work environment at administrative department - Process redesign at administrative department - Digitalization of entire patient process 42

44 RESEARCH METHODOLOGIES Year Initiated interventions General Specific Structure Culture Process Lean field trip to Aravind eye hospital in India for 4 managers - 5-day Lean symposium and field trip in Mineapolis - Symposium lean in healthcare for 6 clinical professionals - Black belt training for 1 professional - Road shows: CEO discussed organizations vision and aims in small group meetings - Replacement of accelerators (treatment equipment) and treatment planning software by an integrated system - Design software/application for automatic table movement (shift) by project team - Digitalization of whiteboard on treatment planning department to visualize the process flow Improve patient registration: multidisciplinary meeting mail phone - Visualize administrative work flow to improve clarity for all professionals - Reduce time needed for initial consultation with oncologist - Reduce process times for palliative patients in specific - Combine patient s appointment for initial intake and CT - Improve check processes for the preparation phase - Improve scheduling process for treatment planning and visualize priorities - Improve process times and stability for preparation phase of treatment - Design an interface for combining data from white board (which visualizes the to-do-list of processes) with staffing - Redesign management dashboard and automate reports to make work load prediction possible - Explore a work flow management system and improve or replace scheduling system - Explore the use of smart-segmentation for treatment planning of head and neck cancer - Make all relevant applications and software compatible to the new work flow 43

45 CHAPTER 2 organization. Two managers, one clinical and one operational, were in charge of all disciplines and multidisciplinary teams were initiated for the different care paths (e.g. breast, head and neck, prostate) to facilitate and improve process innovation and increase employee commitment to change. In the years between 2010 and 2012 the entire management and multiple professionals (oncologists, physicists, radiation technologists and administrative employees) received an intensive lean management training. In 2012, the chief executive officer discussed the organizations vision and aims with all employees in small group meetings (road shows), in order to create a shared long term vision. To reduce persistent safety issues and be able to treat patients conform the best possible quality standards, all treatment equipment (accelerators) and treatment planning software were replaced by a new integrated system from 2011 to In addition, the entire patient process was digitalized and specialized software to manage clinical protocols and standard procedures was implemented. From 2011 to 2012, the administrative department used several lean tools to improve their working process. Their working environment was optimized by implementing 5S; a lean tool to create an orderly workplace and remove waste associated with clutter and disorganization. In 2013, multiple projects were initiated to improve process flow and decrease waiting times for the patients. A large group of multidisciplinary professionals were involved in these projects. RESEARCH METHODOLOGIES Regarding RESEARCH AIM 1 (chapter 3), no tools were found in the literature to improve management decision making and increase understanding in employees views and commitment regarding the implementation of a lean program. An instrument (workshop) was developed, piloted and performed with two groups of clinical professionals combined with management. Eight top managers in the organization evaluated the outcomes. Regarding RESEARCH AIM 2 (chapter 4), a factorial survey was developed to measure patient safety awareness and intentions for safety behaviour. Clinical staff of MAASTRO CLINIC was presented a survey with twenty scenarios about incidents followed by five questions regarding their safety awareness and intentions for safety behaviour. Results were compared to the results from the Hospital Survey on Patient Safety Culture (HSOPSC). Regarding RESEARCH AIM 3, the effects of the lean transition were evaluated at MAASTRO CLINIC. A panel of ten internal and external experts listed the most important indicators for evaluation on three dimensions: the patient, the employee, and the organization (table 2). Since it is difficult to detect the real effect of a change program like lean management, one should combine endpoint outcomes with surrogate end-points like process evaluation, compliance to 44

46 RESEARCH METHODOLOGIES the intervention, culture and errors (incidents). 3,13 The measurement of endoutcomes for oncology patients is probably even more difficult than for patients in general. Mortality among oncology patients is high caused by disease progression, and effects of radiotherapy errors usually reveal themselves on the long term. Therefore, separating the real effect of a lean intervention and noise is extremely difficult in a radiotherapy setting. As a result, surrogate outcomes were combined with patient satisfaction as the only end-outcome for the patient (table 2). Table 2. List of indicators defined by panel of ten experts Indicator Patient Employee Organization Chapter 1 Incident reports 5, 6, 7, 9 2 Safety culture/ climate 4, 6, 9 3 Intentions for safety behaviour 4, 6, 9 4 Improvement culture (intentions) 4, 6, 9, 10 5 Patient satisfaction 9 6 Employee satisfaction 3, 9 7 Compliance to specific processes 5 8 Waiting times for patients 8, 9 9 Process times 9 10 Downtime treatment equipment 7 11 Disease absenteeism 9 12 Efficiency (cost-benefits) 3, 5, 8 13 Transparancy of organization 3, Level of standardization 10 To meet the complex nature of the listed indicators 14,15, mixed methods were used for their evaluation Quantitative and qualitative measures were combined to increase understanding in the measured effects. For the chapters 5, 6, 7 and 9, the mixed method contained quantitative data from several software programs and multiple surveys. These quantitative results were further explored by qualitative data gained from a focus group and structured interviews with twelve professionals from all disciplines throughout the organization. The interviews were audio recorded, fully typed and coded using specialized software (NVIVO). Inductive reasoning was used for coding, to explore quantitative results, detect trends and formulate conclusions how to interpret the quantitative results. In chapter 8, an economic (Markov) model was constructed to evaluate long term cost-benefits of a redesigned care process for patient with head and neck 45

47 CHAPTER 2 cancer. Probabilistic sensitivity analyses were performed to account for the uncertainties of the used model parameters. In the Markov model QALY s were incorporated as end-point outcomes on the patient level, which were related to the activity based costs. To explore the employees perceptions of the lean transition in chapter 10, twelve structured interviews with professionals from all disciplines in the organization were complemented by survey results. Interviews were analyzed using a deductive approach, starting from predefined categories. The predefined categories were based on the Thompson and Tuden model for organizational decision making. 19 In particular ambiguity of objectives, uncertainty of causeeffects relations in the care processes and information asymmetry were explored. The research methodologies used for the separate projects are explained in detail in the methods sections of the following chapters. 46

48 RESEARCH METHODOLOGIES REFERENCES 1. Liker JK. The Toyota Way. McGrawHill, New York, Linstone HA, Turoff M. The Delphi method : techniques and applications. Addison-Wesley Pub. Co., Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 3. End points and measurement. Qual Saf Health Care 2008; 17: DeRosier J, Stalhandske E, Bagian JP, Nudell T. Using Health Care Failure Mode and Effect Analysis: The VA National Center for Patient Safety's Prospective Risk Analysis System. The Joint Commission Journal on Quality Improvement 2002; 27: Habraken MM, Van der Schaaf TW, Leistikow IP, Reijnders-Thijssen PM. Prospective risk analysis of health care processes: a systematic evaluation of the use of HFMEA in Dutch health care. Ergonomics 2009; 52: Van der Schaaf TW, Lucas DA, Hale AR. Near-miss reporting as a safety tool. Butterworth and Heinemann, Oxford, van Everdingen JJE, Smorenburg SM, Schellekens W, Cucic S. Patient Safety Toolbox: instruments for improving safety in health care organisations. Houten; Al-Balushi S, Sohal AS, Singh PJ, Al Hajri A, Al Farsi YM, Al Abri R. Readiness factors for lean implementation in healthcare settings--a literature review. J Health Organ Manag 2014; 28: Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 1. Conceptualising and developing interventions. Qual Saf Health Care 2008; 17: Kaplan GS, Patterson SH, Ching JM, Blackmore CC. Why Lean doesn't work for everyone. BMJ Qual Saf McIntosh B, Sheppy B, Cohen I. Illusion or delusion--lean management in the health sector. Int J Health Care Qual Assur 2014; 27: Wood D. A prescription for lean healthcare. Healthc Q 2014; 17: Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 4. One size does not fit all. Qual Saf Health Care 2008; 17: Dixon-Woods M, Leslie M, Bion J, Tarrant C. What Counts? An Ethnographic Study of Infection Data Reported to a Patient Safety Program. Milbank Quarterly 2012; 90: Vincent C, Burnett S, Carthey J. The measurement and monitoring of safety. London; Cooper Ph.D MD. Towards a model of safety culture. Safety Science 2000; 36: Kaboli PJ, Mosher HJ. Using balanced metrics and mixed methods to better understand QI interventions. BMJ Quality & Safety 2014; 23:

49 CHAPTER Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Quality & Safety 2013; 22: Thompson JD, Tuden A. Strategies, Structures and Processes of Organizational Decision. In: Thompso JD, editor. Comparative Studies in Administration. Pittsburgh, Pittsburgh University Press;

50 CHAPTER 3 Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour Workshops as a useful tool to better understand care professionals views on a lean change program International Journal of Health Care Quality Assurance. 2014, in press Pascale Simons, Jos Benders, Wim Marneffe, Madelon Pijls-Johannesma, and Dominique Vandijck.

51 CHAPTER 3 ABSTRACT Purpose For change programs to succeed, it is vital to have a detailed understanding of employees views regarding the program, especially when the proposed changes are potentially contested. Gaining insight into employee perceptions helps managers to decide how to proceed. We conducted two workshops in a radiotherapy institute to assess the benefits and drawbacks, as well as their underlying causes, of a proposed lean change program. Managers views on the workshops usefulness were charted. Design/Methodology/Approach Two workshops were organized in which employees predicted positive and negative effects of a lean program. The workshops combined a structured brainstorm (KJ-technique) and an evaluation of the expected effects. Eight top managers judged the workshops value on supporting decision making. Findings Fifteen employees participated in the workshops. Participants from workshop 2 reported more expected effects (27 effects; 18 positive) than from workshop 1 (14 effects; 6 positive). However, when effects were categorized, similar results were shown. Three from eight managers scored the results relevant for decision making and four neutral. Seven managers recommended future use of the instrument. Increased employee involvement and bottom-up thinking combined with relatively low costs were appreciated most. Practical implications The workshop could serve as a simple instrument to improve decision making and enhance successful implementation of change programs, as it was expected to enhance employees involvement and was relatively easy to conduct and cheap. Originality/Value The workshop increased insight into employee views, facilitating adaptive actions by healthcare organization managers. 50

52 WORKSHOPS TO INCREASE UNDERSTANDING IN CARE PROFESSIONALS VIEWS INTRODUCTION To realize quality improvements in healthcare, many change programs are implemented. However, generally, about 70 percent of all change initiatives are claimed to fail. 1 Top managers in healthcare face the problem that they are obliged to choose between the overwhelming set of Quality improvement (QI) initiatives, without having a clear idea about the benefits for their specific situation and the chances for successful implementation. 2 However, they are expected to make well-considered choices, because resources are limited and employee trust in management should not be harmed. Low trust in management, for instance owing to accumulated failed changes, tend to undermine successful implementation of QI programs. 3-5 Lean healthcare is a QI initiative that receives considerable attention lately. However, besides promising results, negative aspects of this improvement strategy are also publicized. Critics claim that lean results in monotonous and repetitive work, with negative effects on job satisfaction. Furthermore, lean is used for downsizing. 6 The phrase lean and mean exemplifies this position. Lean advocates, however, argue that lean is beneficial for employees, particularly because they can influence their own work and working conditions by suggesting improvements. This controversy may be called classic, as the discussion continues for over two decades. 7-9 Thus, it is clear that the effects of lean for employees are often contested. Obviously, the different positions resonate in any local context in which lean is to be implemented and influence employees views towards this implementation. Hence, for any lean implementation to succeed it is vital that management is aware of employee opinions and share a realistic view on the effects on their employees daily practice. 10 When employees are involved in an early stage of QI, not only program implementation would benefit from the possible adjustment to the specific organizational needs, but also from improved management decision making 11,12 and improved employees commitment. 13 To provide healthcare managers more opportunities to judge the possibility to successfully implement a (lean) change program, practical tools should become available to improve understanding of employee views regarding the specific program. These tools should consume little time and present management valuable data/information for the decision making process. We hypothesized that management decision making would benefit from a workshop, where employees predict positive and negative effects of a lean change program. Employee brainstorming and discussion were expected to raise their awareness about program pros and cons, and provide management increased understanding of employees underlying assumptions and their perceived effects for daily practice. The quality of decision making and the program s implementation should benefit. We investigated the value of a workshop from a 51

53 CHAPTER 3 managers perspective to determine to what extent management decision making would be supported regarding a lean change program implementation. METHODS Setting This study was performed at an independent radiotherapy institute in the Netherlands, where yearly patients with cancer are treated with radiotherapy. Radiotherapy is characterised as a high-technology healthcare process, where many professionals have to interact as a team to guarantee optimal care. The top managers decided to implement the lean management philosophy throughout the entire organisation in 2010, to improve care quality. They aimed to improve processes towards a patient-focused ideal state and improve continuously. Goals of lean management are to provide the best possible quality and safety for the patient, with the shortest lead time, to the lowest cost and with a high morale for employees. 14 Lean has proven effective in the car manufacturing business 15 and healthcare managers can also benefit from this approach Instrument A workshop was developed to support management decision making. A more detailed view on the lean change program positive and negative effects should be provided, from the perspectives of the different professions within the organization. The workshop had two parts, combining two techniques. During the first part, a group of employees must reach consensus on the most important effects of a proposed change program using the KJ-technique (Kawakita Jiro). 19 The KJ-technique, also called an affinity diagram, is a structured brainstorm that allows groups to quickly reach consensus on priorities of subjective and qualitative data. During the second part, the most important effects perceived by the employees were listed based on the healthcare failure mode and effect analysis (HFMEA), an instrument to determine the potential risks and its related causes before incidents actually occur. 20,21 Every effect was reported: the impact on the short or long term, the level of impact on a scale from one to five (one for smallest impact, five for largest impact) and if the impact was considered positive or negative. Additionally, employees discussed to what degree every cause contributed to the actual effect (scale 1-10, one: smallest, ten: largest contribution) and reported on how easy or difficult they expected the causes could be measured and/or changed using an ordinal scale with four levels. The planned time frame was two hours per workshop, 70 minutes for the first and 50 minutes for the second part. Every participant received a summary of the proposed change program, before the start, which contained multiple change elements, based on lean management 52

54 WORKSHOPS TO INCREASE UNDERSTANDING IN CARE PROFESSIONALS VIEWS principles, which were proposed to the management prior to the workshop. The participants were asked to carefully read the summary and list all positive and negative effects they expected from the change (lean) program. Three dimensions/categories were prioritized: patient, organization and employee, by dividing ten points. During the first twenty minutes, participants were asked to write (positive/negative) effects on sticky notes and stick them on a wall, making them visible for all participants. Discussion was not allowed, because premature discussion often focuses on irrelevant items to the focus question, which can be considered as waste. The sticky notes of other participants were a possible trigger for new ideas about effects. The following fifty minutes (part one) were used to group the effects, give group names, prioritize groups and finally discuss which groups of effects were perceived as the most important. In the second part, employees discussed and reported the root causes and some additional information for every effect listed in the most important groups. Procedure To test whether the workshop results would support management decision making and increase their understanding of employees views regarding the lean change program we focused on two questions: 1. Does the workshop provide valuable information for management decision making? (a) Do multiple workshops present comparable results? (b) How does management rate the value of the workshop regarding decision making and what effects do they expect from the workshop? (c) How does management experience the workshops cost-benefit ratio? 2. What are the actual costs of the workshop? To answer the questions two separate workshops were organized. For every workshop, ten to fifteen employees were invited to participate, based on a compromise between involving as many employees as possible and the invested time for the organization. Both workshops were performed by the same workshop leader. To prevent a selection bias, employees were randomly selected from the different professions (in the organization), including clinical staff (radiotherapist-oncologists, radiation technologists and physicists), administrative employees and managers. The final results of the two workshops were presented to the institute s top managers. The presentation was followed by a survey to determine how the managers valued the results from the workshop. The questionnaire contained nine questions: four with a Likert-scale, two yes/no questions, one multiple choice and two open ended. Workshop costs were calculated by multiplying employee time investment with the average wage per profession involved. The time investment of the project leader was not taken into account, because the leaders invested time was not representative for 53

55 CHAPTER 3 following workshops to organize, since these were first experiments. The instrument was pilot tested with two employees. Based on these results, a distinction between positive and negative impact of the effects was added for every dimension: patient, employee and organization. The overall group judgment on the impact of effects was left out and the planned time frame for the workshop was slightly adjusted. Analyses Since participants were asked to freely brainstorm the perceived (group) effects, the researcher categorized and counted these to compare the workshops. Average impacts were calculated for all effects on the dimensions/categories: patient, organization and employee. The overall mean impact of effects was determined for the dimensions separately by multiplying the average number the effects were reported by the impact on all three dimensions. To correct potential difference in importance, the overall mean impact of effects was multiplied by the mean relative importance for each dimension. RESULTS The workshop was performed twice in Fifteen employees participated in workshop one (august) and workshop two (September, 2011). Participants brainstormed about the effects of a lean change program, formulated by management. Owing to practical constraints, workshop one counted eight participants, representing all approached professions. However, no radiotherapist-oncologists and management participated in workshop two, counting seven participants. 1a. Comparability The results from both workshops were compared to determine whether they did not strongly depend on the opinions of specific individuals in the teams. Workshop one resulted in 14 potential effects of the lean program and workshop two in 27 (Table 1). Total effects between the workshops differed almost exclusively on effects with a positive impact. Workshop two participants reported more extreme values for the impact of the effects on the patient, the organization and the employees. On the level of root causes, workshop two showed more causes than workshop one (workshop one: 31 and workshop two: 73), especially regarding the causes related to positive effects (workshop one: 18, workshop two: 48). Besides this difference, the two workshops were similar. Participants from both workshops expressed positive and negative effects for employee satisfaction, quality and patient safety. Both workshops identified exclusive positive effects for employee involvement from the lean program and exclusive negative effects for the challenges in the work. Standardization was mentioned most often, both positive and negative related causes. Visualization, 54

56 WORKSHOPS TO INCREASE UNDERSTANDING IN CARE PROFESSIONALS VIEWS more extensive testing, creating flow, clear responsibilities, open communication and increased employee involvement were mentioned as positive related causes in both workshops. Lost autonomy and decreased time to innovate, due to increased bureaucracy, were mentioned as causes for negative effects of the lean change program. 1b. Managers judgment about the workshop results Eight top managers judged the relevance of the workshop for decision making neutral (four) to positive (three), (Table 2). Five from eight managers would recommend the workshop to other departments/organizations and seven would use this technique to improve decision making in the future. Six from eight managers expected increased commitment from employees and the creation of a supportive structure for change (QI) programs as a result from the workshops. Increased support for management decisions and bottom-up thinking were also reported as positive results (question eight). Five from eight managers expected the workshop to be most useful in situations when management decision making is complex, like in organizational issues and when cause-effect relations are unclear (question nine). The results from the workshops provide management increased insight into effects and their causes, as seen by the employees. Regarding the second part of the workshop, the managers appreciated most to what degree a cause contributed to the effect and if the effect showed impact on the detected differences for the short or long term effects (question six, Table 2). Participants recorded a negative impact for almost all short term effects, while most long term effects were perceived positively (Table 1). Most negative short term effects were related to organization culture with consequences for the employees; e.g., the sacrifice in autonomy and agitation on the working floor, due to resistance to change. Examples of positive long term effects for employees were increased respect/trust between employees and increased feedback. 1c. Managers judgment about the cost-benefit ratio of the workshops Five of the eight top managers scored positive on the cost/benefit ratio of the workshops (question four, Table 2). 2. Workshop costs The first part of the workshop covered a brainstorm based on the KJ-technique, which was well structured and the planned 70 minutes were sufficient to reach consensus on the most important groups of effects. However, the second part, investigating the effects using techniques from the HFMEA, was experienced as less structured due to considerable discussion between participants. The valuable discussions had to be interrupted after the planned 50 minutes. Extra sessions, with only a select group of the participants, were planned to finish the workshop second part. On average, 23 employee hours were invested per 55

57 CHAPTER 3 Table 1. Workshop on the level of the effects (for the most important groups). Group of effects Effects N Impact a ST / LT N Impact a ST/LT Number Patient Organization Employee Patient Organization Employee Quality and patient safety ST LT ST/LT ST 4 Work satisfaction LT ST/LT ST/LT ST 6 Employee commitment ST/LT LT 2 Employee frustration ST/LT Culture/ Consequences Challenge in work ST/LT LT 2 for employees Possibilities for LT innovation Open communication LT 1 Overview on work LT 4 processes ST 2 Efficient operational management LT ST 1 56

58 WORKSHOPS TO INCREASE UNDERSTANDING IN CARE PROFESSIONALS VIEWS Other Lack of compliance ST No dependency on treatment location ST/LT 2 Sufficient supply of medical goods ST 1 Increased access of information ST/LT 1 2 Total number of effects Total number of positive effects Total number of negative effects N: Number a The impact on the three dimensions presents the mean scores per dimension on a 1-5 scale; 1: very small impact to 5: very large impact ST: Short term LT: Long term -: not mentioned 57

59 CHAPTER 3 Table II: Results on how managers perceived the workshops. Questions Absolutely not Absolutely Do results of the workshops reflect your expectations on effects of the lean program? To what degree are the results from the workshop relevant for decision making? Do the results from the workshops support the formulation of an action plan? To what extent is the workshop significant, related to costs/benefits? What elements seem relevant in the second part of the workshop? No, N (%) Yes, N (%) Impact per dimensions; patient, organization and employee 5 (63) 3 (38) Contribution of the cause to the effect 3 (38) 5 (63) Effect on the short or the long term 3 (38) 5 (63) Possibility to measure change 7 (88) 1 (13) Possibility to change cause/effect 5 (63) 3 (38) 6 Would you recommend this type of workshop to other departments/organizations? 3 (38) 5 (63) 7 Would you recommend this type of workshop for future management decision making related to change programs? 1 (13) 7 (88) Note: the number in the columns for questions 1-4 represent the number of times the score (at the top of the column) was reported N: Number Survey-questions 8 and 9 were open ended questions and results are not included in the table 58 58

60 WORKSHOPS TO INCREASE UNDERSTANDING IN CARE PROFESSIONALS VIEWS workshop (workshop one: 21 and workshop two: 24). When participants average wages were incorporated (corrected for professions), the costs for workshop one were 1330 and for workshop two 715. The costs varied because of the different professions participating in the workshops (excluding out-ofpocket expenses). Workshop one included two radiotherapists-oncologists and a manager. In workshop two, no radiotherapists-oncologists or a manager participated, resulting in lower costs. Costs were minimized because only a select group of the earlier participants completed the workshop second part with extra planned sessions. DISCUSSION We present positive results on the value of a workshop, as an instrument to support management decision making. Top managers rated the workshop neutral to positive regarding its value for supporting decision making. The participating employees perceived the lean change program as neutral, since they reported negative as well as positive effects of the lean program. Therefore, the managers decided to continue the implementation of the lean program. However, management prioritized projects aimed to improve work flow above projects with efficiency gains, to meet the expressed concerns of employees. Not all managers rated the workshops as relevant for decision making, possibly because the decision for implementation did not change. In open-ended questions, the managers reported the increase of employee commitment and improved bottom up thinking as most valuable, due to participation in the workshops. Since participants brainstormed about potential effects of the proposed change program and interacted with other professionals, including management, they had the opportunity to develop greater understanding regarding the lean change program. 22 The extensive list of positive and negative effects and their causes showed an active participation in the workshops. This inter-active method of information sharing could indeed result in increased employee commitment, which could be regarded as a positive side-effect. This might again be associated with an increased success factor of implementing change. 23 Employees might take more responsibility for the decisions and feel more motivated to successfully implement the lean program, because they had the chance to interact with management and possibly influence decisions. Another positive side-effect of the workshops was creating opportunity for managers to communicate their visions to prevent confusion and misunderstanding among employees. This enhances employees trust in managers 24, which is also important for successful QI. 3 Although employee involvement has shown positive effects on management decision making and the implementation of change, few healthcare organization managers actually 59

61 CHAPTER 3 involve employees in their management decisions. 25,26 The key explanation appears that involving employees is difficult in practice. To give employees a chance to be involved, managers are obliged to relinquish some control and allocate more time for the decision making process. Participative decision making is also more time consuming, because all employee opinions have to be taken into account. 22 In this study, managers reported improved employee commitment owing to workshop participation, but they only expressed a moderate positive value on their decision making process. They might also experience difficulties relinquishing control, which complicates real employee participation in decision making. The cost/benefit ratio of the workshop was rated positively by the managers. To put the costs of the workshops (average cost of 1,023: 715 plus 1,331 / two) in perspective, a previous study presented an estimated cost of minimal 23,655 for the introduction of an incident reporting system (part of a patient safety management system) and a minimum annual running cost of 37, Costs of the workshop can be reduced if only the items contribution of cause to effect and long or short term effect are taken into account and the first and the second workshop part are separated in two different sessions. This creates the opportunity for the workshop leader, to better prepare the second part of the workshop and use employee time more efficiently. The comparison between the two workshops was complicated, because they were based on structured discussions leading to qualitative results. However, when effects and causes were categorized, many similarities were present. Nevertheless, workshop two resulted in more information than workshop one, partly as a result of a learning curve for the workshop leader and partly owing to different participating professions. Since the results of the workshops did not strongly differ, we concluded that they produced reliable results when the categorized effects were grouped. However the actual reported effects and causes differed. The top managers judged the relevance of results for decision making neutral to positive. However, at the time the workshops were held, management had already decided to implement the lean change program. Only the time frame and sequential actions were still open for discussion. Since managers had to judge the value of the workshop, regarding its support for their decision making, in a hypothetical environment, their judgments might be distorted. Interestingly, although in the end, more positive than negative effects were reported, many negative effects emerged especially for employees in the short term. This might be one reason why the philosophy of lean management is not difficult in theory, but getting it into practice is. Employees should strongly trust management and the expected long term benefits for the patient should be clear, to overcome the expected negative short term effects. Setting clear goals, creating commitment from employees, involving them in the decision making process and improving 60

62 WORKSHOPS TO INCREASE UNDERSTANDING IN CARE PROFESSIONALS VIEWS bottom up thinking have shown positive results on successful implementation of change/qi programs. 5,22,23 Besides the negative short-term effects, participants also reported positive long term effects for employees, organization culture and for the patients (quality and safety). Employee brainstorms and discussions about the potential effects of the lean change program might have raised their awareness about its pros and cons in the long run. This could result in more commitment to the lean change program. However, the question remains, whether employees are willing to accept the conflicting interests of their own negative effects on the short run, against the positive effects for patients on the long run. Conclusion Although top managers, included in this study, reported only a moderate to positive value of the workshop regarding their decision making, they expected increased employee commitment for the lean change program and improved bottom up thinking. Since employee commitment is positively associated with the successful implementing QI programs 23, this type of workshop has the potential to increase successful implementation, against relatively low costs. Therefore, it could be used as a simple instrument to create a supportive foundation for (culture) change and improve management decision making, especially for complex decision making when cause-effect relations are unclear. However, further research should study this type of workshops impact on the actual success factor for implementation and should study whether results can be generalized to other healthcare settings. 61

63 CHAPTER 3 REFERENCES 1. Beer M, Nohria N. Cracking the code of change. Harvard Business Review 2000; 78: Walshe K. Pseudoinnovation: the development and spread of healthcare quality improvement methodologies. International Journal for Quality in Health Care 2009; 21: Morgan DE, Zeffane R. Employee involvement, organizational change and trust in management. International Journal of Human Resource Management 2003; 14: Andersson LM. Employee Cynicism: An Examination Using a Contract Violation Framework. Human Relations 1996; 49: Reichers AE, Wanous JP, Austin JT. Understanding and managing cynicism about organizational change. Academy of Management Executive 1997; 11: Benders J, Van Bijsterveld M. Leaning on lean: the reception of a management fashion in Germany. New Technology, Work and Employment 2000; 15: Conti R, Angelis J, Cooper C, Faragher B, Gill C. The effects of lean production on worker job stress. International Journal of Operations & Production Management 2006; 26: Hasle P, Bojesen A, Jensen PL, Bramming P. Lean and the working environment: a review of the literature. International Journal of Operations & Production Management 2012; 32: Schouteten R, Benders J. Lean Production Assessed by Karasek s Job Demand Job Control Model. Economic and Industrial Democracy 2004; 25: Morténius H, Marklund B, Palm L, Björkelund C, Baigi A. Implementation of innovative attitudes and behaviour in primary health care by means of strategic communication: a 7-year follow-up. Journal of Evaluation in Clinical Practice 2012; 18: Collier N, Fishwick F, Floyd SW. Managerial Involvement and Perceptions of Strategy Process. Long Range Planning 2004; 37: Wagner JA, Leana CR, Locke EA, Schweiger DM. Cognitive and motivational frameworks in US research on participation: A meta-analysis of primary effects. Journal of Organizational Behavior 1997; 18: Khatri N, Halbesleben JRB, Petroski GF, Meyer W. Relationship between management philosophy and clinical outcomes. Health Care Management Review April/June 2007; 32: Liker JK. The Toyota Way. McGrawHill, New York, Womack J, Jones D, Roos D. The Machine that Changed the World. Macmillan, New York, Endsley S, Magill MK, Godfrey MM. Creating a lean practice. Fam Pract Manag 2006; 13:

64 WORKSHOPS TO INCREASE UNDERSTANDING IN CARE PROFESSIONALS VIEWS 17. McCarthy M. Can car manufacturing techniques reform health care? Lancet 2006; 367: Rutledge J, Xu M, Simpson J. Application of the Toyota Production System improves core laboratory operations. Am J Clin Pathol 2010; 133: Spool JM. The KJ-Technique:A Group Process for Establishing Priorities. 2004, [cited January, 2011]; Van der Schaaf TW, Lucas DA, Hale AR. Near-miss reporting as a safety tool Butterworth and Heinemann, Oxford, van Everdingen JJE, Smorenburg SM, Schellekens W, Cucic S. Patient Safety Toolbox: instruments for improving safety in health care organisations. Houten, Brown M, Cregan C. Organizational change cynicism: The role of employee involvement. Human Resource Management 2008; 47: Hung DY, Rundall TG, Cohen DJ, Tallia AF, Crabtree BF. Productivity and turnover in PCPs - The role of staff participation in decision-making. Medical Care 2006; 44: Engström AK, Axelsson R. The double spiral of change experiences of privatization in a Swedish hospital. The International Journal of Health Planning and Management 2010; 25: Bonias D, Leggat SG, Bartram T. Encouraging participation in health system reform: is clinical engagement a useful concept for policy and management? Australian Health Review 2012; 36: Hickey JV, Casner-Lotto J. How to get true employee participation. Training & Development 1998; 52: Baatenburg de Jong M. Costs and benefits of incident reporting based on PRISMA, Eindhoven University of Technology,

65 CHAPTER 3 64

66 CHAPTER 4 Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour A factorial survey on intentions for safety behaviour providing opportunities to improve safety Submitted Pascale Simons, Ruud Houben, Petra Reijnders, Madelon Pijls-Johannesma, Wim Marneffe, Annemie Vlayen, Johan Hellings, and Dominique Vandijck.

67 CHAPTER 4 ABSTRACT Purpose To realize safe, high quality treatment, employees should behave according to patient safety standards. Periodic measurement of safety behaviour could provide management relevant information to adjust the implementation of interventions and maximize improvement. Therefore, we constructed a factorial survey measuring safety awareness/behaviour. Method Cross-sectional results of the factorial survey were compared with results from the Hospital Survey On Patient Safety Culture, distributed in MAASTRO radiotherapy in 2010/2011. Respondents were presented twenty scenarios about incidents, randomly varying on; work pressure, person causing incident, whether patient-level was reached, severity of harm, notification by patient and management support. After each scenario, questions were asked about safety awareness and behaviour. Chi-square and multi-level regression were used for analysis. Results Response rates were 64% (N=54) for the culture survey and 62% (N=52) for the factorial survey on intentions. The culture survey reflected positive opinions regarding non-punitive response and incident reporting, in accordance with high scores (factorial survey) on safety awareness (9.0, scale:1-10) and reporting intentions (8.7). Whether or not an incident reached the patient-level predicted safety awareness and intentions for safety behaviour (β=-1.3/-3.08) most strongly. Severity of harm showed little additional effects (β=-0.24/-0.42). Conclusions The factorial survey presented practical information on safety awareness and intentions for behaviour. Therefore, it created additional opportunities for improving safety interventions. Since behaviour is expected to change before values, one could hypothesize that factorial surveys would be more sensitive to change than culture surveys. Longitudinal research should further study the surveys sensitivity to measure changes. 66

68 FACTORIAL SURVEY ON SAFETY BEHAVIOUR INTRODUCTION Patient safety is one of the main elements for high quality care. Although radiotherapy presents a long history with quality assurance, treatment errors still reach the patient-level. 1,2 Many strategies for organizational change promise improved care. However, how can we evaluate whether these strategies are effective regarding patient safety? Since patient safety measures are the result of technical as well as social practices, they are difficult to define precisely. 3,4 Whenever radiotherapy treatment harms the patient, the effects are not always directly visible. Since the patients disease can also progress, cause-effect relations of treatment failures and harm are not always evident. This lack of a visible cause-effect relation further troubles the objective measurement of patient safety within radiotherapy. Widely used methods to retrospectively evaluate patient safety include incident reporting systems and Root Cause Analyses. 1,2,5-8 However, these methods are based on voluntary reporting and therefore do not measure safety completely objective. 9 Patient safety culture is also measured in many organizations, since one expects culture to be one of the key requirements to improve safety behaviour among employees. 10 Most frequently applied measurements of culture are the Safety Attitudes Questionnaire (SAQ) 11 and the Hospital Survey on Patient Safety Culture (HSOPSC) 12, both presenting good psychometric properties Periodic measurement of culture change is advised every 2-3 years, since changing culture needs time. 14,15 A measurement time frame of two to three years is inappropriate for management to evaluate and guide improvement strategies. One could question, whether we could not directly measure safety behaviour or intentions for behaviour. According to Schein s model, organizational culture is represented by three subsequent levels: artefacts & creations (behaviour), values and basic assumptions. 16 Each level becomes more difficult to articulate and change. Culture instruments often focus on the values (second level) regarding safety, whereas behaviour (first level) could be more easily changed and measured. Guldenmund stated that cultural change is not essential for behaviour to change. Culture is only one of three major forces (together with structure and processes) in organizations operating on employees behaviour. 17 To realize safe, high quality radiotherapy treatment, employees should behave according to patient safety standards. Periodic measurement of safety behaviour could, therefore, provide management with relevant information to adjust interventions during implementation and maximize safety improvement. Therefore, we used a factorial survey (FS) design to ask employees about their safety awareness and their intentions for safety behaviour. We aimed to create 67

69 CHAPTER 4 enhanced possibilities for management to maximize patient safety. Results on intentions for safety behaviour (factorial survey) are presented and related to results on safety culture (HSOPSC). METHODS Settings This study was performed at MAASTRO clinic, an independent radiotherapy institute in the Netherlands, where yearly patients are treated with radiotherapy. Two organizational units covered daily practice, each consisting of the three main disciplines; (radiotherapy-) oncologists, (radiation) technologists and (clinical) physicists. The organization has been actively involved in safety improvement since Instruments Results from the HSOPSC and a factorial survey on patient safety actions, designed for MAASTRO clinic, were collected. The HSOPSC results were put in perspective, by comparing the results to the published national data from the Dutch speaking (Flemish) hospitals in Belgium. Hospital Survey on Patient Safety Culture (HSOPSC) The HSOPSC is a validated questionnaire on safety culture used in many countries over the world. 12 The Flemish (Belgium) version of the survey was used in this study. 15,18 Four overall safety outcomes, ten dimensions of culture and the respondents general background were measured by 53 questions, of which 43 used a 5-point scale; strongly disagree /disagree /neutral /agree /strongly agree. Factorial Survey on Patient Safety Actions A factorial survey design, traditionally used to analyse human judgment, was used in MAASTRO clinic to measure employees safety awareness and intentions for safety behaviour. In factorial surveys, respondents are presented questions based on detailed hypothetical descriptions (eg. vignettes), in which several variables, expected to influence judgement, are systematically varied. Every vignette represents a combination of variables (dimensions), which vary on different levels (values). All possible combinations of the variable levels present the vignette universe By having respondents judge a randomly selected sample from the vignette universe (all possible combinations), it is possible to include a large number of dimensions and levels in the design. This enhances the resemblance between the hypothetical vignettes and reality. 20 After every vignette, the respondents are asked questions on intended behaviour. This design should better suit the analysis of context and conditions affecting human 68

70 FACTORIAL SURVEY ON SAFETY BEHAVIOUR judgements than traditional surveys, which directly ask respondents about their judgements. 20,22 This technique would also be less subject to social desirability bias, because respondents are probably not fully attentive to the manipulation of the vignettes variables. 21 The MAASTRO survey included vignettes based on six variables (Table 1), each varying on two or three levels: impact of work pressure, person causing the incident, whether or not the patient-level was reached, severity of (potential) harm, if the incident was noticed by the patient and if management was supportive on safety. Two baseline stories were described in cooperation with two oncologists and three technologists, to detect potential problems of identification with the hypothesized vignettes. One story was based on the work procedure of a doctor (an example is included in Table 1) and another on a technologists. The vignette universe (all possible combinations) of this study was determined by multiplying the levels of each variable. This resulted in =144 different vignettes for each story and 2 144=288 different vignettes in total. The oncologists vignette (Table 1) described a hypothetical story about a female treated for a lung tumour. She had to start with treatment as soon as possible. A few days after her first treatment, radiation had to be adjusted, because chemotherapy was added. The technologists vignette described a story about a male patient being wrongly positioned for treatment of a head & neck tumour. Both stories systematically varied with regard to the six variables. After every vignette, the respondent was asked whether he/she would; 1. experience the incident as an error, 2. discuss it with colleagues, 3. report it in the incident reporting system (IRS), 4. take action in order to prevent future incidents and 5. inform the patient about the incident. Seven additional items measured specific respondents characteristics including profession, work unit, age, gender, work experience and whether the respondent had been previously involved in incidents causing serious harm and/or had experienced harm of a medical mistake in personal life. In 2010 two pilot tests were completed. Two oncologists in training and three technologists participated. The piloted survey took minutes to complete and presented little variation in scores. Based on the results of the pilot test, the yes/no-answer categories were replaced by a Likert scale from 1-10 to maximize sensitivity. To reduce the completion time, the numbers of vignettes were reduced from thirty to twenty. 69

71 CHAPTER 4 Table 1. Example of one out of 144 possible vignettes for the story based on the oncologists work procedure. Base story: A female patient had to start treatment for a lung tumour as soon as possible. A few days after treatment start, you received the results from the CT-PET. She had lymphatic metastases and together with specialists from another department, you decided to treat her with additional chemotherapy. Treatment dose should be adjusted, as the originally prescribed dose would harm the patient when combined with chemo. Case sample Variable Levels You received the results from the CT-PET yourself and decided, in consultation with the internist, to let her start with chemotherapy. You reported the treatment change in her medical record and scheduled her appointments for chemo. However, you forgot to communicate this with the planners and the technologists to adjust her appointments for radiation her treatment plan (dose). Person causing the incident self/colleague own unit/colleague other unit The day after you saw the patient for her weekly consult, before her radiation. You noticed the comment in her medical record and checked whether or not all necessary changes were executed. Nobody knew anything about any changes in her treatment schedule. Her treatment plan was changed right away and all checks were completed. The patient did not receive a wrong radiation treatment. Whenever the treatment plan was not changed in time and the patient had received a wrong radiation, harm for the patient was limited. The tumor was not located near any critical organs. Patient reached by the incident Severity of (potential) harm Patient not reached (near miss)/patient reached (miss) without harm/miss with harm No harm/serious harm to patient The patient had not noticed any changes and possible mistakes regarding her treatment. Incident noticed by patient You had plenty of time, since the next scheduled patient did not show. Impact of work pressure Your supervisor thinks patient safety is nonsense. Every individual is responsible for their own actions and mistakes are the result of lacking personal qualities. Last month, your colleague had made a mistake, and she was blamed for this. Management support for PS Noticed/not noticed by patient Low/high time pressure Highly supportive management/punish ment by management Would you: Absolutely not Absolutely yes 1. experience the incident as an error? 2. discuss it with colleagues? 3. report it in the Incident Reporting System? 4. take action in order to prevent future incidents? 5. inform the patient about the incident? 70

72 FACTORIAL SURVEY ON SAFETY BEHAVIOUR Data Collection All clinical staff (oncologists, technologists, physicists) of MAASTRO clinic (91 in total) received the Flemish (Belgium) version of the HSOPSC electronically. Reminders were sent after one week. The respondents were offered to respond anonymously, to maximize the response rate. Three months later, the same group of professionals received the FS. The two surveys were not distributed simultaneously, because this could impede a high response rate for one of the surveys. Since every respondent received twenty randomly selected vignettes, a maximum of 91 20=1820 vignettes could be collected when response would be 100%. Every vignette would occur five times in the group of technologists and three times in the groups of oncologists and physicists. Surveys were checked for duplicate vignettes, disseminated by personal mail and could be completed in an excel sheet. Personal reminders were sent after one week. The nonresponders were approached for their reasons for not responding, to detect possible confounding variables. Analyses Data from the HSOPSC was analysed in accordance with the guidelines of the Agency for Healthcare Research and Quality (AHRQ). 27 Frequencies of response were calculated for each survey item. The two most positive response categories were combined (strongly agree/agree). Frequencies were calculated for each dimension and chi-square tests were used to test differences between respondents characteristics (e.g. respondents profession, gender etc.) For the FS, average scores on the five questions were calculated. Because factorial surveys present hierarchically nested data 28, multi level regression analysis was performed, including two levels: vignettes and respondents. The Linear Mixed Models procedure from SPSS 19.0 was used. The five questions were inserted as dependent variables in the model, the six vignette variables and the respondents characteristics as independent variables. For every model (five), the best covariance structure was chosen using restricted Maximum Likelihood (REML) estimation methods. The models were reduced using a topdown procedure and the Maximum Likelihood (ML) estimation method. The six vignette variables, respondents discipline/ unit/ previous involvement and personal experience with harmful incidents, always remained in the model. Additional characteristics and interaction terms with significant effects, were added for each of the five models. The two different base stories were included as a variable in the analyses as well. The five regression models are presented in the appendix. 71

73 CHAPTER 4 RESULTS HSOPSC results In December 2010, 54 (64%) questionnaires were returned, of which 39 (72%) responded anonymously (no difference per profession). The majority of respondents was represented by technologists (N=41), followed by doctors (N=8) and physicists (N=5). The global patient safety was perceived as very good by the majority of respondents (62%) and as acceptable by 38%. In accordance with the cut-off scores from the AHRQ guidelines 27, non-punitive response to incidents (Table 2) was perceived as a strength (86% positive), but staffing (36% positive), management support (40% positive), teamwork across units (34% positive) and handover and transitions (32% positive) were areas for improvement. Technologists reflected less positive opinions than oncologists and physicists. The two organizational units also presented differences in perceived safety culture (Table 2). When these results were compared to the national Belgian benchmark (Figure 1), non-punitive response to incidents and the frequency of events in this study reported more positive reflections than the benchmark results, while the remaining items indicated a less positive reflection for the studied organization. FS Results In March 2011, 52 (62%) questionnaires were returned. Lack of time was mentioned as the main reason (67%) for not responding. The majority of respondents (N=34, 65%) were technologists. Results were equally distributed over the two units. Four respondents (8%) experienced harm from medical mistakes in personal life, 26 (52%) reported they had been involved in incidents causing serious harm to a patient. In total, 52 (respondents) 20 (vignettes) = 1040 vignettes were received. On average the vignettes were represented 3.6 times, although, two of 288 vignettes were not. Mean scores on the five questions varied between on a scale from 1-10 (Table 3). The smallest variation was detected for experiencing incidents as errors and discussing them with colleagues. Compound symmetry heterogeneous was used as the best fitting covariance structure for all questions, except question 4, which best fitted with compound symmetry correlation metric. The type of incident and the severity of harm were the only variables significantly affecting all five safety related questions. Whether or not an incident reached the patient-level presented largest effects on safety awareness (β= ) and intentions for behaviour (Table 4). The additional effect of harm on safety awareness was smaller (incidents without harm - incidents with harm: β= ). Incidents that did not reach the patient were experienced less as an error when one had not caused the incident themselves. The intention 72

74 FACTORIAL SURVEY ON SAFETY BEHAVIOUR Table 2. HSOPSC overall results, separated for profession and organizational unit. Table presents absolute number of respondents (percentage of total) with positive scores on ten dimensions of patient safety culture and two overall outcomes. Patient Safety Culture Dimensions D1 D2 Supervisor/manager expectations and actions promoting safety Organizational learning continuous improvement Overall Profession P- Units Technologist Oncologist Physicist value Unit 1 Unit 2 Not applicable 138 (64%) 94 (57%) 18 (90%) 26 (81%) (50%) 67 (67%) 32 (73%) (64%) 76 (62%) 11 (73%) 17 (71%) (54%) 53 (71%) 22 (67%) D3 Teamwork within units 152 (70%) 117 (71%) 14 (70%) 21 (66%) (70%) 71 (71%) 30 (68%) P- value D4 Communication openness 117 (72%) 83 (67%) 15 (100%) 19 (79%) (63%) 53 (71%) 30 (91%) D5 Feedback and communication about error 97 (60%) 64 (52%) 13 (87%) 20 (83%) (40%) 52 (69%) 25 (76%) D6 Non-punitive response to error 140 (86%) 103 (84%) 15 (100%) 22 (92%) (83%) 65 (88%) 29 (88%) D7 Staffing 78 (36%) 54 (33%) 7 (35%) 17 (53%) (31%) 34 (34%) 20 (45%) D8 Management support for patient safety 65 (40%) 36 (30%) 11 (73%) 18 (75%) < (40%) 28 (38%) 17 (52%) D9 Teamwork across units 73 (34%) 55 (34%) 6 (30%) 12 (38%) (25%) 40 (40%) 15 (34%) D10 Handoffs and transitions 70 (32%) 56 (34%) 0 (0%) 14 (44%) < (33%) 41 (41%) 5 (11%) O1 Overall perceptions of patient safety 122 (57%) 81(50%) 16 (80%) 25 (78%) (45%) 60 (61%) 28 (64%) O2 Frequency of events reported 106 (65%) 79 (64%) 8 (53%) 19 (79%) (60%) 54 (72%) 20 (61%) Bold results show significant differences. The column not applicable almost entirely presents the physics, since they work for both units. 73

75 CHAPTER 4 Figure 1. The positive dimensional scores (percentage of respondents with positive scores) for the ten dimensions and the two overall outcomes for the Hospital Survey On Patient Safety Culture (HSOPSC). The mean positive results of MAASTRO CLINIC are presented as dashed lines. The box-plots present the results from all acute Flemish (in Belgium) hospitals [6] and cover the 25 th to 75 th percentile. The bold lines present the hospitals median scores and the whiskers indicate the spread of results. This figure only presents an indication of differences, because the scores cannot be directly compared. Safety Dimensions Dim1 Dim2 Dim3 Dim4 Dim5 Dim6 Dim7 Dim8 Dim9 Dim10 Supervisor/manager expectations and actions promoting safety Organizational learning continuous improvement Teamwork within units Communication openness Feedback and communication about error Non-punitive response to error Staffing Outcome dimensions Dimo1 Dimo2 Management support for patient safety Teamwork across units Handoffs and transitions Overall perceptions of patient safety Frequency of events reported 74

76 FACTORIAL SURVEY ON SAFETY BEHAVIOUR Table 3. Average scores on the factorial survey PS awareness (Q1) and behaviour (Q2-Q5) show high average scores with large standard deviations. Repeating questions following each vignette (dependent variables) Mean Standard Deviation Q1: Do you experience this incident as an error? Q2: Do you discuss this incident with your colleagues? Q3: Do you report this incident in the IRS? Q4: Do you take action in order to prevent future incidents? Q5: Do you inform the patient about this incident? Scores are presented on a scale from 1-10: 1 presents lowest score and 10 the most positive score. to discuss an incident with a colleague increased, when the incident had reached the patient and even more when it had caused harm. The intention to inform patients increased, when one had caused the incident themselves and when the patient took notice of the incident, especially for incidents that did not reach the patient or did not actually harm a patient. When respondents had been involved in a serious incident, their intention to inform patients increased less for incidents that did not reach the patient-level to incidents causing harm. Safety awareness did not differ between respondents professions, Table 4-5. However, technologists and oncologists presented less intentions to report. Physicists presented higher intentions to discuss an incident and to take actions for prevention. If a physicist caused the incident themselves, his or her intention to discuss incidents with colleagues decreased, (β= 5.57). Oncologists intention to discuss was smallest and decreased even further when they had not caused the incident themselves (β= -3.47). The technologists vignettes were perceived more as errors, but resulted in less safety related actions than the oncologists vignettes (Table 4). Especially technologists and physicists presented less intention to report for the technologists vignettes (Table 5), while technologists and oncologists reported less intention to take action to prevent incidents for the technologists vignettes. Respondents from unit 2 presented less intentions to discuss an incident with colleagues (β= -5.08) than respondents from unit 1. Female respondents presented higher intentions to take action than the males. 75

77 CHAPTER 4 Table 4. Results from the Factorial Survey on Patient Safety Actions. β1 for the independent variables and respondents characteristics. Effects Categories Q1: Experience the incident as an error? (β; p-value) Type of incident Q2: Discuss incident with colleagues? (β; p-value) Q3: Report incident in IRS? (β; p-value) Q4: Take action to prevent future incidents? (β; p-value) Q5: Inform the patient about incident? (β; p-value) Near miss a -1.71; < ; < ; < ; < ; <0.001 Miss without harm a -0.19; ; ; ; ; Severity of harm No harm b -0.28; < ; < ; ; < ; <0.001 Person who caused the incident Colleague own unit c -0.27; ; ; ; ; <0.001 Colleague other unit c -0.63; < ; ; ; ; <0.001 Noticed by patient Not noticed d -0.04; ; ; ; ; <0.001 Work pressure Low work pressure e -0.10; ; ; ; ; 0.63 Management support Highly supportive management f -0.01; ; ; ; ; 0.97 Profession Radiotherapist-oncologist g -0.28; ; < ; ; ; 0.26 Physicist g -0.24; ; < ; ; ; 0.15 Unit Unit two h 0.15; ; < ; ; ; 0.31 Previous involvement in harmful incident Yes i -0.22; : < ; ; ; 0.01 Previous experience in personal life* Yes j -0.77; ; ; ; ; <0.001 Gender Male k NA 0.35; ; ; 0.13 NA 76

78 FACTORIAL SURVEY ON SAFETY BEHAVIOUR Work experience 1/5 years l NA NA NA -1.19; 0.40 NA 6/10 years l NA NA NA 1.69; 0.11 NA Age < 30 years m NA -1.61; <0.001 NA -5.61; <0.001 NA years m NA -0.52; 0.30 NA -2.92; NA years m NA -1.36: 0.03 NA -1.60; 0.08 NA Difference between Technologists story n 0.56; < ; ; ; ; <0.001 two base stories Bold results show significant values of β1 * Results should be interpreted with caution, because these are based on the data of only four respondents NA: not applicable a reference category: miss with harm b reference category: serious harm c reference category: caused by themselves d reference category: noticed by patient e reference category: high work pressure f reference category: punishment/ no support from management g reference category: radiation technologist f reference category: unit one i reference category: no previous involvement j reference category: no serious harm in personal life k reference category: female l reference category: >10 years m reference category: > 50 years n reference category: oncologists story 77

79 CHAPTER 4 Table 5. Results from the Factorial Survey on Patient Safety Actions showing the β1 s with p-values for all relevant interaction-effects. Interaction - effects Categories Q1: Experience the incident as an error? (β; p-value) Person who caused the incident * type of incident Person who caused the incident * profession Person who caused the incident * gender Type of incident * gender Type of incident * noticed by patient Type of incident * previous involvement Type of incident * previous experience in personal life λ Q2: Discuss incident with colleagues? (β; p-value) Q3: Report incident in IRS? (β; p-value) Q4: Take action to prevent future incidents? (β; p-value) Q5: Inform the patient about incident? (β; p-value) Colleague own unit * miss no harm a 0.18; 0.43 NA 0.52; 0.09 NA 0.62; 0.08 Colleague own unit * miss with harm a 0.30; 0.18 NA 0.04; 0.14 NA 0.81; 0.02 Colleague other unit * miss no harm a 0.61; NA 0.04; 0.89 NA 0.73; 0.05 Colleague other unit * miss with harm a 0.59; NA 0.55; 0.07 NA 1.31; <0.001 Colleague own unit * oncologist b NA -0.35; 0.13 NA NA NA Colleague own unit * physicist b NA 0.40; 0.11 NA NA NA Colleague other unit * oncologist b NA -0.45; NA NA NA Colleague other unit * physicist b NA 0.16; 0.52 NA NA NA Colleague own unit * male c NA 0.11; 0.58 NA NA NA Colleague other unit * male c NA 0.53; 0.01 NA NA NA Near miss * male d NA 0.44; ; NA NA Miss no harm * male d NA 0.13; ; 0.04 NA NA Near miss * not noticed e NA NA NA NA -1.42; <0.001 Miss no harm * not noticed e NA NA NA NA 0.01; 0.98 Near miss * previously involved f NA NA NA NA 0.76; 0.01 Miss no harm * previously involved f NA NA NA NA 0.51; 0.07 Near miss * experience in personal life g NA NA NA NA 1.33; 0.02 Miss no harm * experience in personal life g NA NA NA NA 0.65; 0.18 Previous involvement * age Profession * previous experience in personal life λ Involved * <30 years h NA -5.05; <0.001 NA NA NA Involved * years h NA -4.80; <0.001 NA NA NA Involved * years h NA -4.59; NA NA NA Oncologist * experience in personal life i NA NA -5.99; <0.001 NA -1.31; 0.22 Physicist * experience in personal life i NA NA 0,20; 0,82 NA -4.33; <

80 FACTORIAL SURVEY ON SAFETY BEHAVIOUR Severity of harm * previous experience No harm * experience in personal life j 0.83; NA NA NA NA in personal life λ Work experience * 1-5 years experience * experience in previous experience personal life k NA NA NA -7.50; NA in personal life λ 6-10 years experience * experience in personal life k NA NA NA -2.15; 0.31 NA Management support * profession Highly supportive management * oncologist l NA NA NA 0.02; 0.93 NA Highly supportive management * physicist l NA NA NA -0.59; 0.04 NA Gender * unit Male * unit 2 m NA NA NA -3.24; 0.01 NA Profession * difference two base stories Bold results show significant values of β1 Oncologist * technologists story n NA NA 0.73; : 0.08 NA physicist * technologists story n NA NA -0.30; ; 0.03 NA NA: not applicable: not in regression equation because non-significant interaction for dependent variable λ Results should be interpreted with caution, because these are based on the data of only four respondents a reference category: self * near miss b reference category: self * technologist c reference category: self * female d reference category: miss with harm * female e reference category: miss with harm * noticed by patient f reference category: miss with harm * no previous involvement g reference category: miss with harm * no previous experience in personal life h reference category: no previous involvement * >50 years i reference category: technologist * no previous experience in personal life j reference category: miss with harm * no previous experience in personal life k reference category: > 10 years work experience * no previous experience in personal life l reference category: punishment/no support from management * technologist m reference category: female * unit 1 n reference category: technologist * oncologists story 79

81 CHAPTER 4 DISCUSSION A survey (FS), based on a factorial survey design, aimed to measure safety awareness and behaviour, was constructed, pilot tested and disseminated at a radiotherapy institute in March Results were compared to the HSOPSC. FS versus HSOPSC Safety awareness (9.0, 1-10 scale) and intention to report (8.7) scored very high with the FS, conform the relative high scores on non-punitive response to incidents and frequency of events by the HSOPSC, when compared to the Belgian benchmark. The HSOPSC revealed surprisingly less positive results on the overall perception of safety. Despite the many years of active promotion of safety in the organization, four out of ten safety culture dimensions reported scores beneath the AHRQ cut-off score (50%) and five scored less positive than the Belgian benchmark. 15 This could be an effect of safety issues being more critically judged by respondents with a strong safety awareness. However, the moderate response rate to both surveys could also indicate that employees still have not internalized safety, despite the ongoing effort on safety promotion. This was also confirmed by the relative high percentage of anonymous responses (72%) regarding the HSOPSC. Although results from the HSOPSC were below expectation, participants revealed positive safety behaviour ( 7.3) with the FS. This could indicate that safety culture may not be the only factor influencing safety behaviour as stated in Guldenmund s model. Other forces like the organization structure and the actual processes affect safety behaviour as well. 17 Both instruments presented differences between professions. Technologists perceived overall patient safety, communication openness, feedback about error, management support and expectations (HSOPSC) less positive than the other professions. In line with these results, technologists expressed less intention to report and act to prevent future incidents (FS). However, oncologists expressed positive perceptions on the above dimensions of the HSOPSC, but expressed equal (low) intentions (FS) to report an incident and act to prevent as technologists. This again reflects that patient safety culture is only one of the forces influencing safety behaviour. Respondents from unit 2 perceived a more positive culture, reported more trust in incident reporting, analyzing and taking action (HSOPSC) and might have expressed less intention to discuss incidents with colleagues (FS), while they (females unit 2) expressed higher intentions to take action (FS) than respondents from unit 1. Differences in management styles and opinion leaders attitudes regarding patient safety could possibly explain the differences between the units results. 29,30 Surprisingly, safety awareness and behaviour were not affected by support of the immediate superior (FS). The dimension on the 80

82 FACTORIAL SURVEY ON SAFETY BEHAVIOUR supervisor/management expectations and actions from the HSOPSC in this study revealed less positive scores than for the Belgian benchmark. Hospital leadership is seen as critical to an organization s safety culture. 23,31,32 However, little has been published on the effects from supportive leadership of immediate superiors. The employees in this study, did not take the supervisors opinion into account. They reported high intentions to act on safety, despite the limited support they experienced from their immediate superiors. However, the results on safety intentions related to management support might be distorted. The vignettes described a hypothetical situation in which management support varied. Respondents might not identify with this situation well and might hold on to their perceived reality. Additional information gained from the FS The FS was more sensitive than the HSOPSC regarding differences between incidents that reached the patient-level or did not. The HSOPSC was not able to detect these differences, since no related questions were incorporated. Harm to a patient and the severity of harm presented small effects on safety behaviour. To encourage safety behaviour, employees awareness should be raised about the possibility to act upon near misses (patient not reached), and as a result avoid actual misses (patient is reached) from happening. Safety awareness, intention to report incidents and the intention to inform patients increased, when a near miss (not reach the patient) was caused by the respondent themselves, probably because one felt responsible. When the incident had actually reached the patient-level (misses), motivation for action might increase over the lack of experienced responsibility. Management could, therefore, emphasise team responsibility for patient safety instead of promoting individual actions. In contrast to our expectations, respondents which had been previously involved in harmful incidents reported less intentions to inform patients about misses. They might have experienced conflicting cognitions, 33 being partly responsible for harming the patient while they instead felt obliged to help, certainly not hurt them. Management could discuss these results with employees to intercept irrational perceptions and improve open communication to colleagues and patients. Significant differences were reported for the two base stories. Although professionals perceived the technologists vignettes more as errors than the oncologists, they led to less safety related actions, especially for technologists and physicists. These professionals possibly recognized the hypothetical vignettes as every day possibilities and inherent to their daily practice. Physicists reported higher intentions to take action for the technologists vignettes, probably because these are in their capacity to change. The differences between the two base stories could partly be explained by a different severity for the patient, and could partly show the problems for respondents to judge hypothetical situations, less adapted to their profession and daily judgments. 81

83 CHAPTER 4 Ideally, safety improvement interventions should be evaluated periodically by the measurement of the professionals safety behaviour. However, objective measurement of behaviour is difficult. Observations are costly and timeconsuming and surveys on behaviour are subject to bias, especially when the behaviour concerns controversial issues. The factorial survey method should be less subject to social desirability bias, because respondents are probably not fully attentive to the manipulation of the influencing factors. 22 However, a limitation is that not actual behaviour is questioned, but the intentions for behaviour. Although intentions are strongly related to behaviour, one should also believe that one can successfully perform the behaviour to produce the desired outcomes. 34 As a result improved intentions do not automatically improve behaviour. The presented results on the respondents safety intentions, therefore, do not equal their actual safety behaviour in reality. A benefit of the factorial survey technique is that in addition to intentions for behaviour the related influencing factors could be evaluated. Because people are not always aware of the influencing factors on their judgements, they experience difficulty in explicating these when directly asked about them. 22 An increased understanding of the factors influencing intentions for patient safety behaviour is essential for management to maximize safety improvement interventions. The FS was specifically adapted to practice in the MAASTRO clinic, resulting in very recognizable cases for professionals. However, given the large detail in case description, one should be cautious before generalising this exact survey to other settings. The survey should be adapted to the specific conditions of every organization. The preferred level of detail in the vignettes should be studied to make generalizations and follow up for longer time periods possible, without sacrificing incidents reality. Conclusion Improvement strategies aim to enhance patient safety behaviour, which should therefore not be excluded for measurement. The FS seems a promising instrument to measure safety awareness and intentions for safety behaviour. It complemented the HSOPSC results and presented additional information, which increases managements understanding in employees safety behaviour. This creates opportunities to manage specific improvement actions. Since safety behaviour is expected to change before values, one could hypothesize that the FS would be more sensitive to change than traditional safety culture surveys. This could be an important advantage of the FS. Further research should study the survey s sensitivity to measure change. 82

84 FACTORIAL SURVEY ON SAFETY BEHAVIOUR REFERENCES 1. Clark BG, Brown RJ, Ploquin JL, Kind AL, Grimard L. The management of radiation treatment error through incident learning. Radiother Oncol 2010; 95: Shafiq J, Barton M, Noble D, Lemer C, Donaldson LJ. An international review of patient safety measures in radiotherapy practice. Radiother Oncol 2009; 92: Dixon-Woods M, Leslie M, Bion J, Tarrant C. What Counts? An Ethnographic Study of Infection Data Reported to a Patient Safety Program. Milbank Quarterly 2012; 90: Vincent C, Burnett S, Carthey J. The measurement and monitoring of safety. London; Reason J. Human error: models and management. BMJ 2000; 320: Rex JH, Turnbull JE, Allen SJ, Vande Voorde K, Luther K. Systematic root cause analysis of adverse drug events in a tertiary referral hospital. Jt Comm J Qual Improv 2000; 26: Van der Schaaf TW, Lucas DA, Hale AR. Near-miss reporting as a safety tool. Butterworth and Heinemann, Oxford, van Everdingen JJE, Smorenburg SM, Schellekens W, Cucic S. Patient Safety Toolbox: instruments for improving safety in health care organisations. Houten; Capuzzo M, Nawfal I, Campi M, Valpondi V, Verri M, Alvisi R. Reporting of unintended events in an intensive care unit: comparison between staff and observer. BMC Emerg Med 2005; 5: Nieva VF, Sorra J. Safety culture assessment: a tool for improving patient safety in healthcare organizations. Qual Saf Health Care 2003; 12 Suppl 2: ii Sexton JB, Helmreich RL, Neilands TB, et al. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 2006; 6: Blegen MA, Gearhart S, O'Brien R, Sehgal NL, Alldredge BK. AHRQ's hospital survey on patient safety culture: psychometric analyses. J Patient Saf 2009; 5: Etchegaray JM, Thomas EJ. Comparing two safety culture surveys: Safety Attitudes Questionnaire and Hospital Survey on Patient Safety. BMJ Quality & Safety 2012; 21: Hellings J, Schrooten W, Klazinga NS, Vleugels A. Improving patient safety culture. Int J Health Care Qual Assur 2010; 23: Vlayen A, Hellings J, Claes N, Peleman H, Schrooten W. A nationwide Hospital Survey on Patient Safety Culture in Belgian hospitals: setting priorities at the launch of a 5-year patient safety plan. BMJ Quality & Safety 2012; 21:

85 CHAPTER Schein EH. Coming to a New Awareness of Organizational Culture. Sloan Management Review 1984; Guldenmund FW. The use of questionnaires in safety culture research an evaluation. Safety Science 2007; 45: Hellings J, Schrooten W, Klazinga N, Vleugels A. Challenging patient safety culture: survey results. Int J Health Care Qual Assur 2007; 20: Jasso G. Factorial Survey Methods for Studying Beliefs and Judgments. Sociological Methods & Research 2006; 34: Rossi PH, Anderson AB. An Introduction. In: Rossi PH, Nock SL, editors. Measuring Social Judgments: The Factorial Survey Approach. Beverly Hills: Sage Publications; p Wallander L. 25 years of factorial surveys in sociology: A review. Social Science Research 2009; 38: Alexander CS, Becker HJ. The use of vignettes in survey research. Public Opinion Quarterly 1978; 42: Sammer CE, Lykens K, Singh KP, Mains DA, Lackan NA. What is patient safety culture? A review of the literature. J Nurs Scholarsh 2010; 42: Kaldjian LC, Jones EW, Wu BJ, Forman-Hoffman VL, Levi BH, Rosenthal GE. Reporting medical errors to improve patient safety: a survey of physicians in teaching hospitals. Arch Intern Med 2008; 168: Espin S, Levinson W, Regehr G, Baker GR, Lingard L. Error or "act of God"? A study of patients' and operating room team members' perceptions of error definition, reporting, and disclosure. Surgery 2006; 139: Uribe CL, Schweikhart SB, Pathak DS, Dow M, Marsh GB. Perceived barriers to medical-error reporting: an exploratory investigation. J Healthc Manag 2002; 47: Sorra JS, Nieva VF. Hospital Survey on Patient Safety Culture. Rockville: Agency for Healthcare Research and Quality Hox JJ, Kreft IGG, Hermkens PLJ. The Analysis of Factorial Surveys. Sociological Methods & Research 1991; 19: Barling J, Loughlin C, Kelloway EK. Development and test of a model linking safety-specific transformational leadership and occupational safety. Journal of Applied Psychology 2002; 87: Flin R, Yule S. Leadership for safety: industrial experience. Qual Saf Health Care 2004; 13: ii Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Qual Saf 2013; 22: Richardson A, Storr J. Patient safety: a literative review on the impact of nursing empowerment, leadership and collaboration. International Nursing Review 2010; 57: Festinger L. A Theory of Cognitive Dissonance. Stanford University Press, Stanford,

86 FACTORIAL SURVEY ON SAFETY BEHAVIOUR 34. Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes 1991; 50:

87 CHAPTER 4 Appendix. The five regression models for the questions included in the factorial survey. The six vignette variables, respondents discipline/ unit/ previous involvement and personal experience with harmful incidents, always remained in the model. Additional characteristics and interaction terms with significant effects, were added for each of the five models. Q1. experience the incident as an error? β0 + β1work pressure + β2person caused + β3type + β4severity + β5noticed by patient + β6management support + β7profession + β8unit + β9previous involvement + β10personal experience + β11base stories + β12person caused*type + β13severity*personal experience Q2. discuss incident with colleagues? β0 + β1work pressure + β2person caused + β3type + β4severity + β5noticed by patient + β6management support + β7profession + β8unit + β9previous involvement + β10personal experience + β11gender + β12age professional + β13base stories + β14person caused*profession + β15person caused*gender + β16type*gender + β17unit*age + β18previous involvement*age Q3. report incident in the Incident Reporting System? β0 + β1work pressure + β2person caused + β3type + β4severity + β5noticed by patient + β6management support + β7profession + β8unit + β9previous involvement + β10personal experience + β11gender + β12base stories + β13base stories*profession + β14person caused*type + β15type*profession + β16type*gender + β17profession*personal experience + β18unit*gender Q4. take action in order to prevent future incidents? β0 + β1work pressure + β2person caused + β3type + β4severity + β5noticed by patient + β6management support + β7professional expertise + β8profession + β9unit + β10previous involvement + β11personal experience + β12gender + β13age professional + β14base stories + β15base stories*profession + β16management support*profession + β17professional expertise*unit + β18professional expertise*personal experience + β19unit*gender Q5. inform the patient about the incident? β0 + β1work pressure + β2person caused + β3type + β4severity + β5noticed by patient + β6management support + β7profession + β8unit + β9previous involvement + β10personal experience + β11base stories + β12person caused*type + β13person caused*noticed by patient + β14person caused*profession + β15type*noticed by patient + β16type*previous involvement + β17type*personal experience + β18profession*personal experience 86

88 CHAPTER 5 Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour Does compliance to patient safety tasks improve and sustain when radiotherapy treatment processes are standardized? European Journal of Oncology Nursing, 2014, in press Pascale Simons, Ruud Houben, Jos Benders, Madelon Pijls-Johannesma, Dominique Vandijck, Wim Marneffe, Huub Backes, and Siebren Groothuis.

89 CHAPTER 5 ABSTRACT Purpose To realize safe radiotherapy treatment, processes must be stabilized. Standard operating procedures (SOP s) were expected to stabilize the treatment process and perceived task importance would increase sustainability in compliance. This paper presents the effects on compliance to safety related tasks of a process redesign based on lean principles. Method Compliance to patient safety tasks was measured by video recording of actual radiation treatment, before (T 0 ), directly after (T 1 ) and 1.5 years after (T 2 ) a process redesign. Additionally, technologists were surveyed on perceived task importance and reported incidents were collected for three half-year periods between 2007 and Results Compliance to four out of eleven tasks increased at T 1, of which improvements on three sustained (T 2 ). Perceived importance of tasks strongly correlated (0.82) to compliance rates at T 2. The two tasks, perceived as least important, presented low base-line compliance, improved (T 1 ), but relapsed at T 2. The reported near misses (patient-level not reached) on accelerators increased (P<0.001) from 144 (2007) to 535 (2009), while the reported misses (patientlevel reached) remained constant. Conclusions Compliance increased after introducing SOP s and improvements sustained after 1.5 years, indicating increased stability. Perceived importance of tasks correlated positively to compliance and sustainability. Raising the perception of task importance is thus crucial to increase compliance. The redesign resulted in increased willingness to report incidents, creating opportunities for patient safety improvement in radiotherapy treatment. 88

90 COMPLIANCE TO PATIENT SAFETY TASKS INTRODUCTION The risk for patients being harmed from radiotherapy treatment is relatively low, when compared to other medical specialties. 1 Although radiotherapy presents a long history with quality assurance, treatment errors still reach the patient-level. 2,3 A considerable percentage of errors in radiotherapy are due to errors in treatment set-up and delivery. 4 To assure quality and safety, protocols and procedures are mentioned as important elements and their absence or inadequacy are associated with errors. 5,6 Therefore, clearly stated protocols are one of the required aspects for a quality assurance system in radiotherapy. 7 However, even when procedures are in place, these are not always followed. Employees could have forgotten about the procedures or were possibly not aware of the consequences of non-compliance. 8 Besides increasing quality standards, demand for radiotherapy has been growing and health care cost rising To realize sustainable quality improvements, quality and efficiency should be combined. Lean management is a strategy to improve both quality and efficiency. Lean was originally developed within the Toyota Motor Company, and later diffused within the global car manufacturing industry. 12 This has inspired lean health care Processes are redesigned and improved towards a customer-focused ideal state by reducing waste. Waste includes tasks that do not directly benefit the patient and can be minimised through the regular redesign of processes. 16 However, to create a solid base for continuous (quality) improvement, first unwanted variation should be reduced and processes should be stabilized, for instance by working with standard operating procedures (SOP s). SOP s are detailed written instructions outlining the tasks needed to complete a job. SOP s are clearly essential for quality improvement, but the required level of detail has not yet been established. To balance the simplicity and completeness of SOP s is challenging. 17 Furthermore, SOP s can only stabilize a process, when employees actually work according to them. Even when SOP s are available, questions remain about employees compliance and how these SOP s affect the quality of care or patient safety. Effects of lean interventions on the quality or patient safety are rarely quantified 18-20, partly because patient safety measures are difficult to define and measure and are the result of technical as well as social practices. 21,22 Widely used methods to retrospectively evaluate patient safety include incident reporting systems and root cause analyses. 2,3,23-25 Although these techniques give organizations many improvement opportunities, they are based on voluntary reporting and probably not all errors will be reported. Therefore, these retrospective data do not report safety objectively. 26 More objective methods include observations to determine the process reliability. 27,28 Therefore, we observed compliance to patient safety tasks on a linear accelerator, to detect an effect of a lean based process redesign of the radiotherapy treatment process on patient safety. 89

91 CHAPTER 5 We hypothesized that the redesign of a treatment process, with the implementation of detailed SOP s, would stabilize the process, resulting in improved compliance. Furthermore, we hypothesized that tasks perceived as more important by employees would present higher compliance rates and increased sustainability of improvements. METHODS Setting This study was performed in the MAASTRO CLINIC, a radiotherapy department in the Netherlands, where on average 200 oncology patients receive radiation therapy every day. In 2007, three multi-disciplinary units operated in the clinic. Each site specific unit (e.g. breast cancer, lung cancer) consisted of radiationoncologists, technologists and physicists. Redesign of the radiation treatment process For many years, the linear accelerators were operated by three technologists. One operated the accelerator from the treatment console, while the other two positioned the patient on the treatment table ( original treatment process in Figure 1). General medical protocols per treatment site and technical guidelines for daily practice were available. However, these allowed considerable variation in task completion. No procedure for structured communication existed, responsibilities were not explicitly assigned to individual technologists, and process steps were not executed in a fixed order (i.e. only the outcome of the process was fixed, not the process itself). In January 2007, a project team of 25 (out of the 60) technologists redesigned the process for the accelerator. Waste within the process was identified (indicated by the curved brackets in Figure 1) by visualizing the process in a flow chart. Employees discussed the added value of all separate steps/tasks. Remarkably, little waste could be defined within the tasks. However, the sequence of the tasks and their assignment to individual technologists were rearranged ( redesign of treatment process in Figure 1). The project team decided to design and implement a SOP for two technologists. The SOP described the task sequence and responsibility in great detail. Two units implemented the SOP in October 2007 for breast cancer patients (Figure 2) to start with. Implementation gradually expanded and since 2010, all patients are treated by two technologists following site specific SOP s. Compliance measurement To detect increased process stability and sustainability of improvements, compliance to procedures was observed in daily practice. Observations were recorded before the process redesign (T 0 ), directly following implementation (T 1 ) 90

92 COMPLIANCE TO PATIENT SAFETY TASKS Figure 1. Flow chart of the irradiation treatment process presenting the process steps in a chronological order from left to right. The original process describes the process for three technologists (Techn. 1-3), where only medical protocols per treatment site and technical guidelines for daily practice were available. The flow chart presents many process steps in between technologist two and three, corresponding the lack of clear responsibilities. The chronological order of process steps varied between treatments. The curved brackets indicate waste for the individual technologists. A project group of 25 technologists redesigned the process to a future state for two technologists (redesign of the treatment process in the flow chart). In the process redesign, the chronological order of process steps was defined (left to right in flow chart) and tasks were explicitly assigned to individual technologists. 91

93 CHAPTER 5 and 1.5 years after (T 2 ). A fixed camera system was installed on one of the accelerators in January The installation was approved by the medical staff and the works council. Employees and patients anonymity was guaranteed to protect their privacy. Patients were informed in their changing rooms about the possible observations. To increase reliability of observations, only about 1% of the patient treatments were actually recorded and technologists were unaware when observations took place. Furthermore, only treatments for breast cancer were observed to ensure the reliability of comparisons among T 0, T 1 and T 2. To examine the variability of the radiotherapy treatment, a detailed process description was essential. The project team had already visualised the treatment procedure of breast cancer patients (Figure 1). Since this is only a representation of the desired situation, daily practice was evaluated by comparing this flow chart to observations of actual treatments. Two observers with more than 10 years experience in radiotherapy treatment, including the first author of this paper, determined compliance using a previously developed score list. 28 Five random radiation technologists selected 11 tasks from the score list, which should be important for patient safety (Table 1). Seven technologists specialised in patient safety and a patient safety manager, separately selected the most crucial tasks for patient safety out of the 11 from Table 1. This resulted in five most crucial tasks for patient safety (see Table 1). Compliance was measured at three moments: at T 0, where every patient was treated by three technologists and only technical guidelines were present; at T 1 directly after implementing the process redesign; and at T years after implementation. At T 1 and T 2, patients were treated by two technologists following the SOP (Figure 2). Task compliance was measured by dividing the number of times the task was performed by the total number of times the task should be performed. Survey on perceived task importance A survey was developed on the perceived importance of tasks, to test our hypothesis that a task perceived as more important by employees would result in higher compliance rates and increased sustainability. Employees from both units were asked to rate how important they perceived nine tasks on a scale from 1-10 (1: not important and 10 extremely important). The survey was distributed between T 1 and T 2 and a Spearman correlation was calculated for perceived importance and compliance to tasks at T 1 and T 2. Incident reporting system To detect possible effects of the process redesign on patient safety, data from incident reporting were gathered. The MAASTRO CLINIC has been using an incident reporting system (IRS) since Data were gathered from January to September for 2007, 2008 and The number of reported incidents (adjusted for the number of patients treated) was identified for the total organization and for accelerators in specific. The reported incidents were divided 92

94 COMPLIANCE TO PATIENT SAFETY TASKS into misses (incidents reaching patient-level) and near misses (incidents not reaching patient-level), since the proportion of reported near misses from the total would give an indication about the employees willingness to report. Irradiation of Breast cancer Technologist 1 Technologist 2 Log in Check patient information Call for patient by intercom Get digital patient information Activate portal imaging Take patient information along Get patient from change room and check ID photo Name patient by name for ID-check Guide patient inside treatment room Inform about patients side-effects Check birth date on patient information Inform patient about portal imaging Help patient on treatment table Check patient position with digital photograph Table movement by hand if necessary Position patient Restore ink lines on patients skin (initiative) Stick markers on patient skin Position table on high laser Zero table settings Execute shift movement of table Rotate table if necessary Inform colleague about possibility bolus Hand over bolus to colleague if necessary Walk to operating room Consultation between technologists Start irradiation Observe patient during treatment Position next field if necessary Observe patient and equipment (Observation partly executed by colleague to check patient information of next patient) Call next patient by intercom Turn key back (stop irradiation) Check all fields being treated Get digital information next patient Activate portal imaging Walk to treatment room Clean treatment tools/markers if necessary Position treatment tools Log in Check presence of patient in digital system Search patient information Walk to treatment room Position treatment tools Ask patient for ID: birth date + check with digital information Help patient on treatment table Dim lights Position table on lower laser Position table Stick markers on patient skin Position gantry Position flatpanel for portal imaging Check shift Check field projection on skin with markers doubtful => compare with printed skinview of treatment plan/ consult colleague Stick bolus to patients skin if necessary Accept treatment in treatment room Lights on Take patient information a long Inform patient about leaving the treatment room Release treatment room Complete form for portal imaging Store patient information Check presence next patient in digital system Search patient information next patient Partly observe patient Walk to treatment room Remove markers from patients skin Position gantry in 0 degrees Help patient from table Guide patient to change room Take patient information along Get patient from change room and check ID photo Name patient by name for ID-check Guide patient inside treatment room And so on Technologist 1 and 2 are altered! Treatment site specific actions C H R O N O L O G I C A L O R D E R Figure 2. Standard Operating Procedure (SOP) of patients irradiated for breast cancer. The two columns represent the two technologists. Tasks are described in great detail and presented in a chronological order. (Tasks specific for breast cancer treatments are highlighted in red to increase technologists awareness to the differences per site-specific SOP. The tasks highlighted in blue present the tasks gradation to the patient treated next.) Analysis Results were adjusted for missing values and not observable scores. The overall compliance was calculated using the mean of the proportion of compliance for all tasks. T-tests were used to determine differences in overall compliances. Chisquare tests were performed on the frequencies of compliance and noncompliance per task. Results on perceived task importance were presented in means and a non-parametric test for independent samples (Mann-Whitney U test) was performed to compare both units. Correlations with compliance were analyzed by Spearman coefficients. Z-tests for two proportions were calculated for the IRS results. P-values 0.05 were considered statistically significant. 93

95 CHAPTER 5 RESULTS Compliance measurements In total, 167 recordings of treatments were videotaped and analysed. At T 0 (February-March 2007) 56 treatments were recorded, at T 1 (September-October 2007) 56 and at T 2 (February-March 2009) 55. Four tasks (5, 8, 9 and 10) showed high compliance rates at T 0, leaving little or no room for improvement (Table 1). These high compliance rates sustained (95% or more) throughout all measurements. Four tasks (1, 3, 4 and 11) improved from T 0 to T 1 (Table 1). Two tasks (6 and 7) with moderate compliance rates did not improve. Compliance to task 2 (check special actions) improved from 74% to 94% (T 2 ). Compliance to task 4 (call patient by name) partly relapsed from 76% to 56%. Tasks 1 and 3 also indicated a (not significant) partial relapse. At T 2, 1,5 years after implementing the SOP s, compliance to four tasks had improved. Besides the four tasks with initial high compliance rates, compliance to task 7 (communicate about ink lines) did not change, remaining moderate with 51% at T 2. Compliance to task 6 (check position of patients hands) decreased from 59% to 49%. Overall compliance and the mean compliance to the crucial tasks for patient safety did not significantly differ between measures. Compliance to tasks differed little between the units, only task 4 (calling patient by name) showed differences. Compliance improved from T 0 (unit 1: 19%, unit 2: 43%, P=0.17) to T 1 for both units, showing significant higher compliance for unit 2 at T 1 (unit 1: 69%, unit 2: 81%, P=0.001). However, improvement sustained for unit 1, but partially relapsed for unit 2 at T 2 (unit 1: 68%, unit 2: 43%, P=0.21), resulting in no difference in the end. Perceived importance of tasks In January 2008 (between compliance measurement T 1 and T 2 ), 23 surveys (52% response) on the perceived importance of tasks were returned, equally distributed over both units. Employees scored the perceived importance of recorded tasks between on a scale from 1-10 (Table 2). They perceived task 9 (cross-check of shift movement) as most important (9.8) followed by task 2 (check of special actions: 8.7), task 10 (projection of treatment field on the patient s skin: 8.7) and task 5 (check of patient s date of birth: 8.6). These tasks also presented high compliance rates varying from 94% to 100% at T 2. Unit 2 perceived task 4 (importance of the check of patient s identification photo) more important than unit 1. Tasks 1 and 4 were perceived as least important (7.4) from the nine proposed and showed initial low compliance, increase at T 1 and partial relapse at T 2. The correlation between perceived importance and compliance to tasks was 0.50 (P=0.17) at T 1 and 0.82 (P=0.006) at T 2. 94

96 COMPLIANCE TO PATIENT SAFETY TASKS Table 1. Compliance to the most important tasks for patient safety analysed by two observers on video recordings T0 (2007, N=56), T1 (2007, N=56) and T2 (2009, N=55) Key tasks to patient safety on the linear accelerator % performed P 1 T0 T1 T2 T0-T1 T1-T2 T0-T Read communication form before calling patient by intercom < Check special actions Check identification photo before getting patient out of change room < Call patient by name < Ask patient s date of birth constant Check position of patient s hands using patient information Communicate verbally about quality of ink lines on patient Execute SHIFT movement (movement of table by hand following constant previously determined numbers) 9. 2 Cross check of SHIFT movement Check projection of treatment field on patient s skin constant Communicate verbally about bolus (tool only needed in <0.00 < particular patients) 1 Mean overall compliance Mean compliance crucial patient safety tasks P-values based on chi-square tests 2 the five most crucial tasks for patient safety 3 significant change due to a different number of not observable scores at T0 and T1 4 due to many missing values, reported numbers were not representative for reality 5 P-values based on independent t-tests, means calculated per item 95

97 CHAPTER 5 Data from the incident reporting system The number of reported incidents increased from 886 in 2007 to 1250 in 2009 (Table 3). This increase was almost entirely based on incidents related to the linear accelerators (193 in 2007 to 586 in 2009, P<0.001). The reported incidents were differentiated into misses (incidents reaching patient-level) and near misses (incidents not reaching patient-level). The number of reported misses remained stable over the years. However, the number of reported near misses increased 10% (P<0.001) from 2007 to 2008 and another 17% from 2008 to 2009 (P<0.001) for the treatment process (accelerators) in specific, while remaining constant for the rest of the organization. The large increase (>40%) in overall reported incidents was almost exclusively the result of increased reporting of near misses on the linear accelerator. DISCUSSION The redesign of a treatment process and the implementation of SOP s resulted in improved compliance rates to specific patient safety tasks. Improvement sustained 1.5 years after the process redesign for most tasks, as hypothesized. Only compliance to task 4 (call patient by name) partly relapsed. Overall compliance to the 11 patient safety tasks did not differ between measurements, which could be partly explained by a large variation in observations and partly because compliance to four tasks showed little or no room for improvement due to high compliance rates at base line measurement. Overall compliance (T 0 : 62%, T 1 and T 2 : 77%) was below our expectation, since only tasks, important for patient safety, were observed. However, results conformed to compliance percentages on healthcare performance elsewhere. 29,30 Perceived importance of tasks associated with sustainability of change Perceived importance of tasks was associated with higher levels of compliance and increased sustainability of improvements. At T 1, perceived importance showed only moderate correlation to compliance. At T 2, correlation increased, probably because employees had been confronted with the tasks presenting a discrepancy between perceived importance and low compliance rates. To further increase employees awareness, non-compliance to these tasks was related to error making by presenting harmful incidents from practice. Confirming our hypothesis, compliance to the task perceived as least important (4: call patient by name), presented only partly sustainable improvements (T 0 : 29%; T 1 : 76%; T 2 : 56%). When analyzed in more detail, our hypothesis was not confirmed for unit 1. Although employees from unit 1 perceived this task as less important than unit 2, they presented sustainable improvements at T 2. However, employees from unit 1 presented lower initial compliance rates (T 0 ). At T 2, these 96

98 COMPLIANCE TO PATIENT SAFETY TASKS Table 2. Results from 23 surveys on the importance of tasks perceived by the employees. No results for tasks 7 and 8 could be presented (NA), because these were not included in the survey. Key tasks to patient safety on the linear accelerator Perceived Importance (N=23) Overall SD Unit 1 Unit 2 P-value 1 Mean 1. 2 Read communication form before calling patient by intercom Check special actions Check identification photo before getting patient out of change room Call patient by name Ask patient s date of birth Check position of patient s hands using patient information Communicate verbally about quality of ink lines on patient NA NA NA NA NA 8. 2 Execute SHIFT movement (movement of table by hand following previously determined numbers) NA NA NA NA NA 9. 2 Cross check of SHIFT movement Check projection of treatment field on patient s skin Communicate verbally about bolus (tool only needed in particular patients) P-values based on a Mann-Whitney U test, a non-parametric test for independent samples 2 the five most crucial tasks for patient safety NA: Not Applicable 97

99 CHAPTER 5 differences between units had diminished. The motivation of employees from unit 1 may have increased, due to the confrontation of the higher compliance rates of unit 2. Since the perceived importance of tasks highly correlated with compliance at T 2, this could indeed play an important role in the sustainability of changes. When employees are more aware about the importance of tasks, they could be increasingly motivated to adhere to procedures. However, results also show that other factors must be present which influence compliance. Change of perceived importance could not be detected, since results were based on one crosssectional measurement. Interestingly, an important mismatch was present between the safety experts, also working in daily practice, and the remaining employees. Task 1 (read communication form before calling patient) was mentioned as one of the most crucial patient safety tasks by the seven safety experts. However, employees perceived this task as least important (equal to task 4) and compliance appeared moderate at all three measures. The safety experts were possibly more aware of safety hazards and the potential causes, because they have been confronted with the reported incidents in the organization. Therefore, compliance to this task could be improved by raising employees awareness regarding its importance. Increased willingness to report on the accelerators The employees presented a large willingness to report, since only a small percentage of the reported incidents (5%, 49 of the 886 in the first half of 2007) actually reached the patient-level. 3 During the implementation of the process redesign, the number of reported near misses (patient-level not reached) on the accelerator increased from 144 (2007) to 535 (2009). Since the number of incidents that actually reached the patient-level (misses) and potentially harm the patient remained constant, we ascribed the increased reporting to an increased willingness to report. This creates increased opportunities for safety improvement before patients are actually harmed. More open communication due to employees participation in the redesign, but also process deviations being more easy to detect when processes are stabilized, could explain the increase in the reporting of near misses. 16 Although the initial willingness to report was high, it still significantly improved after the process redesign. This corresponds to the earlier presented problems regarding the objectivity of data from voluntary reporting systems. The results from the IRS must be interpreted with caution, since these were based on patients treated for any treatment site by all three units and the observations only included treatment for patients with breast cancer by units 1 and 2. 98

100 COMPLIANCE TO PATIENT SAFETY TASKS Table 3. Data from the Incident Reporting System at MAASTRO CLINIC from three time periods. Data Incident Reporting System (IRS) Jan-Sep 07 Jan-Sep 08 Jan-Sep 09 P ( 07-08) P ( 08-09) Total reported incidents NA NA Patients treated NA NA Total reported incidents / patients treated 48% 47% 69% < Reported incidents on accelerators (/ total incidents) 193 (22%) 279 (31%) 586 (47%) < < Total reported incidents minus accelerators NA NA Miss 2 (/ total incidents) 49 (6%) 47 (5%) 51 (4%) Miss 2 (/ incidents on accelerators) 49 (25%) 47 (17%) 51 (9%) < Near miss 3 total organization (/ total incidents) 837 (94%) 858 (95%) 1199 (96%) near miss 3 on accelerators (/ incidents on accelerators) 144 (75%) 232 (83%) 535 (91%) < < Ratio of miss 2 / near miss 3 on accelerators < P-value based on z-test for two proportions 2 miss: incident that reached the patient 3 near miss: incident that did not reach the patient 4 P-value based on z-test for a ratio NA: Not Applicable (cq not relevant) 99

101 CHAPTER 5 Positive side-effects of the process redesign The lean process redesign focussed on standardizing the treatment processes, resulting in more process stability. Furthermore, since lean aims to create continuous improvement, the project team was stimulated to standardize, but also to continuously improve the process. During the group discussions, the team developed a sticker on the patient s file as a reminder for a special task, which resulted in a decrease of reported incidents regarding this task. The development of SOP s did not stop with the treatment process. Since the start of the redesign, SOP s were developed throughout the entire organisation. The employees views regarding SOP s could be positively affected by their positive experience on designing SOP s and bringing them to practice. Efficiency increased as well, as all patients are being treated by two instead of three technologists. Although every employee expressed different experiences regarding the process redesign, nearly all experienced an increased sense of responsibility, clearer expectations and a better work atmosphere. Conclusion In conclusion, compliance and willingness to report were positively affected by the introduction of SOP s and effects sustained for 1.5 years. This could indicate increased process stability, potentially resulting in increased patient safety. The perceived importance of tasks was positively correlated to the level of compliance and could be an important factor for realizing sustainable improvements. To further improve or sustain increased compliance rates, we recommend regular feedback to the staff on all safety related tasks to raise employees awareness about the importance of these tasks. Further research should study the sustainability of results and study how behaviour can be institutionalized. 100

102 COMPLIANCE TO PATIENT SAFETY TASKS REFERENCES 1. Munro AJ. Hidden danger, obvious opportunity: error and risk in the management of cancer. Br J Radiol 2007; 80: Clark BG, Brown RJ, Ploquin JL, Kind AL, Grimard L. The management of radiation treatment error through incident learning. Radiotherapy and Oncology 2010; 95: Shafiq J, Barton M, Noble D, Lemer C, Donaldson LJ. An international review of patient safety measures in radiotherapy practice. Radiotherapy and Oncology 2009; 92: Valentin J. Prevention of accidental exposures to patients undergoing radiation therapy. A report of the International Commission on Radiological Protection. Ann ICRP 2000; 30: Thwaites D, Scalliet P, Leer JW, Overgaard J. Quality assurance in radiotherapy: European Society for Therapeutic Radiology and Oncology Advisory Report to the Commission of the European Union for the Europe Against Cancer Programme. Radiotherapy and Oncology 1995; 35: Pawlicki T, Dunscombe P, Mundt A, Scalliet P. Quality and Safety in Radiotherapy. Taylor & Francis Group, Boca Raton, Leer JWH, Corver R, Kraus JJAM, Togt JCvd, Buruma OJS. A quality assurance system based on ISO standards: experience in a radiotherapy department. Radiotherapy and Oncology 1995; 35: Dunscombe P. Recommendations for safer radiotherapy: what's the message? Front Oncol 2012; 2: Orszag PR, Ellis P. The challenge of rising health care costs--a view from the Congressional Budget Office. N Engl J Med 2007; 357: Slotman BJ, Vos PH. Planning of radiotherapy capacity and productivity. Radiotherapy and Oncology 2013; 106: Williams MV, Summers ET, Drinkwater K, Barrett A. Radiotherapy dose fractionation, access and waiting times in the countries of the UK in Clinical oncology Royal College of Radiologists Great Britain 2007; 19: Womack JP, Jones DT, Roos D. The Machine that Changed the World. Free Press, New York, Endsley S, Magill MK, Godfrey MM. Creating a lean practice. Fam Pract Manag 2006; 13: McCarthy M. Can car manufacturing techniques reform health care? Lancet 2006; 367: Rutledge J, Xu M, Simpson J. Application of the Toyota Production System improves core laboratory operations. Am J Clin Pathol 2010; 133: Liker JK. The Toyota Way. McGrawHill, New York, Radiology BIo, Staff RCoR, Staff BIoR, Radiologists RCo. Towards Safer Radiotherapy. Royal College of Radiologists,

103 CHAPTER Vest JR, Gamm LD. A critical review of the research literature on Six Sigma, Lean and StuderGroup's Hardwiring Excellence in the United States: the need to demonstrate and communicate the effectiveness of transformation strategies in healthcare. Implement Sci 2009; 4: Nicolay CR, Purkayastha S, Greenhalgh A, et al. Systematic review of the application of quality improvement methodologies from the manufacturing industry to surgical healthcare. Br J Surg DelliFraine JL, Langabeer JR, 2nd, Nembhard IM. Assessing the evidence of Six Sigma and Lean in the health care industry. Qual Manag Health Care 2010; 19: Dixon-Woods M, Leslie M, Bion J, Tarrant C. What Counts? An Ethnographic Study of Infection Data Reported to a Patient Safety Program. Milbank Quarterly 2012; 90: Vincent C, Burnett S, Carthey J. The measurement and monitoring of safety. London; Reason J. Human error: models and management. BMJ 2000; 320: Rex JH, Turnbull JE, Allen SJ, Vande Voorde K, Luther K. Systematic root cause analysis of adverse drug events in a tertiary referral hospital. Jt Comm J Qual Improv 2000; 26: van Everdingen JJE, Smorenburg SM, Schellekens W, Cucic S. Patient Safety Toolbox: instruments for improving safety in health care organisations. Houten; Capuzzo M, Nawfal I, Campi M, Valpondi V, Verri M, Alvisi R. Reporting of unintended events in an intensive care unit: comparison between staff and observer. BMC Emergency Medicine 2005; Oakley E, Stocker S, Staubli G, Young S. Using video recording to identify management errors in pediatric trauma resuscitation. Pediatrics 2006; 117: Simons PA, Houben RM, Backes HH, Pijls RF, Groothuis S. Compliance to technical guidelines for radiotherapy treatment in relation to patient safety. Int J Qual Health Care 2010; 22: Nolan T, Resar R, Haraden C, Griffin FA. Improving the Reliability of Health Care. IHI Innovation Series white paper. Institute for Healthcare Improvement, Boston, available on Swenne C, Alexandrén K. Surgical team members compliance with and knowledge of basic hand hygiene guidelines and intraoperative hygiene. Journal of Infection Prevention 2012; 13:

104 CHAPTER 6 Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour Does Lean management improve patient safety culture? An extensive evaluation of safety culture in a radiotherapy institute European Journal of Oncology Nursing, 2014, in press Pascale Simons, Ruud Houben, Annemie Vlayen, Johan Hellings, Madelon Pijls-Johannesma, Wim Marneffe, and Dominique Vandijck.

105 CHAPTER 6 ABSTRACT Purpose The importance of a safety culture to maximize safety is no longer questioned. However, achieving sustainable culture improvements are less evident. Evidence is growing for a multifaceted approach, where multiple safety interventions are combined. Lean management is such an integral approach to improve safety, quality and efficiency and therefore, could be expected to improve the safety culture. This paper presents the effects of lean management activities on the patient safety culture in a radiotherapy institute. Methods Patient safety culture was evaluated over a three year period using triangulation of methodologies. Two surveys were distributed three times, workshops were performed twice, data from an incident reporting system (IRS) was monitored and results were explored using structured interviews with professionals. Averages, chi-square, logistical and multi-level regression were used for analysis. Results The workshops showed no changes in safety culture, whereas the surveys showed improvements on six out of twelve dimensions of safety climate. The intention to report incidents not reaching patient-level decreased in accordance with the decreasing number of reports in the IRS. However, the intention to take action in order to prevent future incidents improved (factorial survey presented β: 1.19 with p:0.01). Conclusions Due to increased problem solving and improvements in equipment, the number of incidents decreased. Although the intention to report incidents not reaching patient-level decreased, employees experienced sustained safety awareness and an increased intention to structurally improve. The patient safety culture improved due to the lean activities, and actual patient safety outcomes might have improved as well. 104

106 DOES LEAN IMPROVE PATIENT SAFETY CULTURE? INTRODUCTION Safety has a long history in radiotherapy and oncology. 1-3 To maximize the safety of treatment, sustainable improvements in patient safety should be created. The importance of a patient safety culture for improving safety, is no longer questioned. 4 A patient safety culture is referred to as the employees shared beliefs, values and attitudes regarding patient safety in an organization, which are reflected in the daily operational clinical practice. 4,5 Although patient safety culture is an important concept for realizing safe care, the strategy how to create a safety culture and how to evaluate improvements is less evident. Safety culture is, as well as patient safety outcomes, difficult to measure objectively. 6-8 To evaluate safety culture, surveys are widely implemented. However, these surveys evaluate safety climate and provide a snapshot of the perceptions of professionals on the observable, surface-level aspects of the culture on a specific moment in time. 9 The most frequently applied surveys are the Safety Attitudes Questionnaire (SAQ) 10 and the Hospital Survey on Patient Safety Culture (HSOPSC) 11, both presenting good psychometric properties. 4,10-12 However, following the reciprocal safety culture model of Cooper and Philips which is based on the social learning theory 13, safety climate is just one of the elements affecting safety culture. In addition to safety climate, safety behaviour and situational factors like the safety management system interact bi-directionally. To meet the complex nature of this concept, triangulation of methodologies should be used for its evaluation. 7,13 If we want to evaluate whether we, as professionals, progress on safety awareness and behaviour, we should also measure exactly these. According to Schein s model of organizational culture, behaviour (first level) could be more easily changed and measured than the values/climate (second level) regarding safety. 14 Besides the rigorous measurement problems, changing patient safety culture is challenging. No uniform strategy to improve safety culture is available and sustainability of effects are unclear. Generalizability of evidence of the different strategies is limited, since effects are highly dependent on intrinsic organizational elements such as the level of executive support, the extent of implementation, the size of the organization and the personnel facilitating for change. However, evidence is growing for a multifaceted approach, where multiple safety interventions are carefully selected and combined to strengthen each other, instead of initiating isolated safety projects. 7 Lean management is a quality improvement philosophy, which uses a set of instruments and incorporates a long term vision aiming for continuous improvement. 15 This philosophy focuses on the improvement of the care processes and eliminating non-value added (waste) steps from the patients 105

107 CHAPTER 6 perspective. By putting the patient central, striving for continuous optimization of processes and engaging all employees for improvement, this philosophy aims to maximize quality and safety for the patient combined with efficiency and motivated employees. Several instruments can be used to help improve the processes; e.g. value stream mapping, which visualizes the entire patient process with all its bottlenecks, and Kaizen (Japanese for improvement), meaning daily small changes for the better involving all employees to create continuous improvement. Improved patient safety culture is one of the expected benefits of lean, because lean aims to commit employees to quality and safety improvement. We hypothesized that the patient safety culture and the intentions for safety behaviour would benefit from the undertaken lean actions. This study aimed to evaluate the patient safety culture and safety behaviour of healthcare professionals within a radiotherapy institute after undertaking lean actions for a three year period. METHODS Study Design To evaluate changes in patient safety culture and behaviour data was collected longitudinally for three years. Because of the challenging aspects of patient safety culture evaluation, a triangulation of methodologies was used for data collection. In the mixed methods study, quantitative results were collected for three time periods, once before and twice throughout the implementation of the lean based actions (Figure 1) and were combined with qualitative data. Quantitative results incorporated data from: 1. Workshops of safety climate 2. Surveys 2a. surveys on safety climate 2b. Surveys on patient safety awareness and behaviour 3. Reports from an incident reporting system, reporting behaviour The workshops and surveys evaluated the patient safety climate, as well as intentions for safety behaviour, each from a different perspective. The reports from the incident reporting system were gathered to complement the results on safety culture and behaviour, since willingness to report is one of the issues reported by the surveys. Since the evaluation of safety culture strategies would benefit from combining quantitative and qualitative methodologies 7, structured interviews were performed to increase understanding in the quantitative results. 106

108 DOES LEAN IMPROVE PATIENT SAFETY CULTURE? Figure 1. Timeline of the study. The items beneath the timeline represent the implemented actions and the items above the timeline the different methodologies and measurements for evaluation. HSOPSC: Hospital Survey on Patient Safety Culture MaPSaF: Manchester Patient Safety Framework Setting This study was performed at a radiotherapy institute in the Netherlands, where yearly oncology patients receive radiotherapy treatment. The radiotherapy process is characterized by a complex multidisciplinary treatment facility using highly complex technology. The organization has been actively promoting patient safety since Before 2011, several isolated projects, based on the lean management philosophy, were initiated to realize further improvements of the quality and efficiency for the treatment process. However, to realize sustainability in quality improvement for the total radiation process and to control for increasing costs, the organization reorganized from managing the different professions to managing multidisciplinary care pathways in January 2011 (Figure 1). This should facilitate the lean actions and in particular promote continuous improvement. To improve employee s commitment to the organizational goals, road shows were implemented. These road shows included small group sessions, in which executive management discussed the organizations strategy with all employees. In 2013, multiple lean based projects were initiated to improve the entire (flow of the) patient process, from patient registration until the start of treatment. Many professionals from all levels of the organization were engaged in these flow projects. In addition to these organizational interventions, the treatment planning system and the accelerators were replaced by new technology from 2011 to This might have interfered the previously mentioned actions. 107

109 CHAPTER 6 Quantitative Methods The quantitative methods used in this study included: 1. Workshops Within a three year period, two workshops were performed based on the Manchester Patient Safety Framework. The Manchester Patient Safety Framework (MaPSaF) is based on Westrums theoretical framework and developed for healthcare organizations to reflect on their progress in developing a mature safety culture During the workshops the participants were presented specific descriptions of the organization at five levels of safety culture maturity for nine dimensions relevant for patient safety, outlined in the first column of Table 2. The five levels of culture maturity include; 1) Pathological: organizations with a dominant negative attitude regarding safety which invest little in safety improvement, 2) Reactive: safety is only considered after an incident has occurred, 3) Bureaucratic: paper-based organizations where safety involves ticking boxes to prove to auditors that safety is considered, 4) Proactive: safety improvement is highly valued, staff are rewarded for raising patient safety issues and continuous improvement of safety is actively invested in, and 5) Generative: safety is an integral part of everything and is in the mindset of all employees throughout the organization. Following the NHS guidelines, the MaPSaF can be helpful to evaluate specific interventions, which aim to improve the safety culture. 19 Both workshops (2011 and 2013) were performed with largely the same participants. The participants were randomly selected in 2011 stratified per profession. To receive a realistic view on the organizations safety culture, employees from the relevant clinical professions were invited to participate; oncologists, technologists (specialized oncology nurses), clinical physicists, and management. For every included profession, at least two employees participated to discuss the scored levels of maturity. 2. Surveys Two different surveys were distributed three times during the three year period with approximately one year time intervals. A total of about 93 employees were invited to complete the surveys in 2011, 109 in 2012 and 114 in 2013, including oncologists, (clinical) physicists, and technologists (specialized nurses). 2a. Surveys on safety climate: Hospital Survey On Patient Safety Culture (HSOPSC) The HSOPSC is a validated questionnaire on safety climate used in many countries over the world. 11 The Flemish (Belgium) version of the survey 20,21, used in this study involved four overall safety outcomes, ten dimensions of safety climate and the respondents general background, measured by 53 questions, of which 43 used a 5-point scale; strongly disagree/ disagree/ neutral/ agree/ strongly agree. 108

110 DOES LEAN IMPROVE PATIENT SAFETY CULTURE? 2b. Surveys on patient safety awareness and behaviour: factorial survey Patient safety awareness and intentions for safety behaviour were evaluated using a factorial survey design. This design is traditionally used to analyse human judgement. Respondents are presented questions based on detailed hypothetical descriptions (eg. vignettes), in which several variables, expected to influence judgement, are systematically varied. Every vignette represents a combination of variables (dimensions), which vary on different levels (values). All possible combinations of the variable levels present the vignette universe Respondents were presented twenty scenarios about incidents, randomly varying on; work pressure, person causing the incident, whether the patientlevel was reached, severity of harm, notification by the patient and the level of management support. After each scenario, one question was asked about safety awareness and four about intentions for safety behaviour Reports from an incident reporting system (IRS) An IRS has been operational since Root causes are determined for all reported incidents by trained professionals and coded following the PRISMA 26,27 technique. The number of reported incidents are evaluated and differentiated between incidents that have reached the patient level (misses) or have not (near misses). Data from 2004 to 2013 were monitored. Qualitative Methods The qualitative methods included structured interviews to increase understanding of quantitative results. The structured interviews were performed with ten professionals and a focus group session was performed with five patient safety professionals. The interviewees were randomly selected stratified per professions: oncologists, technologists, physicists, administrative department and management. The focus group included five technologists trained in patient safety. The interviews and the focus group session were audio recorded and fully typed. Transcripts were content analysed using inductive analysis supported by NVivo-10 software. 28 Data Analysis Averages were calculated for the nine dimensions of the workshops (MaPSaF) and chi-quadrates were used to test differences between the two workshops. Frequencies of response were calculated for each survey item of the HSOPSC, conforming the guidelines of the Agency for Healthcare Research and Quality (AHRQ). 29 The two most positive response categories were combined (strongly agree/agree) and the two most negative (disagree/strongly disagree). Frequencies were calculated and chi-square tests were used to test differences between measurements for each dimension and for the separate items. Logistic regression was used to determine which dimensions influenced the overall score of patient safety. 109

111 CHAPTER 6 Average scores on the five questions in the factorial survey were calculated. Because factorial surveys present hierarchically nested data 30, multi level regression analysis was performed, including two levels: vignettes and respondents. The five questions were inserted as dependent variables in the model, the six vignette variables, the respondents characteristics and the measurements as the independent variables. RESULTS The results for the different methodologies are presented separately in the following paragraphs. Table 1 presents an overview of the results of all methodologies. 1. Workshops (MaPSaF) results Twelve employees participated in the MaPSaF workshop in January 2011 and ten in September 2013, including management, technologists, oncologists and physicists. Within two hours, nine dimensions, critical for patient safety, were discussed and graded on a five level maturity scale for patient safety culture (Table 2). All scores ranged in between level three (bureaucratic culture) and level four (proactive culture). No significant differences could be detected between measurements. However, some differences between professions could be indicated resulting in specific improvement actions formulated by the participants at the end of both workshops. 2a. Survey (HSOPSC) results on safety climate At T 1 (2010) 54 (64%) questionnaires were returned, at T 2 (2012), 53 (48%) and at T 3 (2013), 51 (47%). In 2012, staffing, teamwork across units and handoffs and transitions presented more positive scores than in 2010 (Table 3). These improvements had sustained in 2013 and several other dimensions like feedback and communication about error, experienced management support for patient safety and the overall perception of patient safety improved on positive scorings. The logistic regression resulted in three dimensions positively influencing the overall score on patient safety, corrected for the change between measurements: communication openness with an odds ratio (OR) of 5.88 (p:0.01) followed by organizational learning with OR:3.78 (p:0.03) and feedback and communication about error with OR:2.60 (p:0.04). The dimension nonpunitive response to error presented a strong inverse relation to the overall patient safety with an OR:0.29 (p:0.01). Other dimensions presented no significant influence on the global patient safety score. 110

112 DOES LEAN IMPROVE PATIENT SAFETY CULTURE? Table 1. Overview of the outcomes of different methodologies for evaluating patient safety culture between 2011 and Willingness to report incidents Intention for structural improvement Workshops (MaPSaF) HSOPSC Factorial surveys Incident reports Interviews No significant changes between 2011 and 2013 No significant changes between 2011 and 2013 The intention to report incidents that had no potential to harm the patient decreased in 2012 Non-punitive response to error worsened in 2013, especially regarding mistakes being kept in a personnel file Discussion about ways to prevent errors from happening again improved in 2013 The intention to report decreased slightly for near misses a The intentions to prevent future incidents strongly improved in 2013, especially for near misses a The number of reported near misses a decreased with 50% (2009: 1746 and 2013: 870), but the number of misses b also decreased with 46% (2009: 89 and 2013: 48) The decrease of reports was not explained by a decreased willingness to report, but by the transition to new equipment and the improved innovation speed among employees - The intention to solve problems increased Management support No significant changes between 2011 and 2013 Support from hospital management and the direct supervisors improved in 2013 No significant effect of management support was detected on safety awareness and patient safety actions - Management support and the facilitation for improvement actions improved Team work No significant changes between 2011 and 2013 Teamwork as well as the handoffs and transitions between units improved in 2012 and sustained in 2013 a near misses: incidents that do not reach the patient-level b misses: incidents that have reached the patient-level The ntentions to report increased in 2012 and 2013 for incidents caused by the other unit

113 CHAPTER 6 Table 2. Results for the Manchester Patient Safety Framework (MaPSaF) in 2011 (N=12) and 2013 (N=10). Scores are presented on a scale from one to five corresponding to 1: Pathological, 2: Reactive, 3: Bureaucratic, 4: Proactive and 5: Generative. The average scores the participants reported for the overall organization showed no statistically significant differences between 2011 and 2013 on a scale with five categories. However, the average scores between the participating professions differed. Patient Safety Culture Dimensions Overall Technologist Oncologist Physicist Management Priority and responsibility to patient safety Record, evaluate and learn from incidents Resources regarding patient safety Communication about safety Team working Personnel management and safety issues Qualified staff and patient safety Compliance and feedback Continuous improvement

114 DOES LEAN IMPROVE PATIENT SAFETY CULTURE? 2b. Survey (factorial) results on safety awareness and behaviour At T 1 (2011) 52 (62%) questionnaires were returned, at T 2 (2012) 43 (43%) and at T 3 (2013) 30 (30%). Averages on the five questions related to safety awareness and intention for safety behaviour scored between 6.9 and 9.0 (Figure 2) on a scale from one to ten. From 2011 to 2012, the intention to discuss with colleagues decreased, but recovered in 2013 (Table 4). The intention to take action in order to prevent future incidents improved in Figure 2. Results from the factorial survey for 2011, 2012 and Average scores on the five questions related to patient safety awareness and intentions for safety behavior presented on a scale from one to ten. All results decreased slightly (significant) from 2011 to The intention to take action for preventing future incidents improved strongly from 2012 to Multiple other variables influenced the main effects. Especially the type of incident presented strong interaction effects (Figure 3). Incidents that did not reach the patient-level (near misses) were experienced less as a mistake than the ones that did reach the patient-level (misses) in 2012 and 2013 compared to However, the intention towards taking action to prevent future incidents (structural improvement), strongly improved in 2013, especially for the near misses. Where in 2011, respondents reported smaller intentions to report when a colleague from a different unit was responsible for the incident, in 2012 and 2013 this difference was leveled out. The intention to inform a patient about the incident decreased in 2012 and 2013 for near misses. Figure 3. Factorial survey results for incidents that reach the patient (misses) or not (near misses). The intention to structurally improve increased strongly in 2013, especially for near misses. 113

115 CHAPTER 6 Table 3. HSOPSC results for all items separately and categorized in the safety dimensions. Only positive response categories (strongly agree/agree) are presented. Scores are displayed as percentages of total responses and P-values <0.05 are presented in bold. Dimensions of HSOPSC T D1: Supervisor/manager expectations and actions promoting PS My supervisor/manager says a good word when he/she sees a job done according to established patient safety procedures My supervisor/manager seriously considers staff suggestions for improving patient safety R Whenever pressure builds up, my supervisor/manager wants us to work faster, even if it means taking shortcuts R My supervisor/manager overlooks patient safety problems that happen over and over D2: Organizational learning/ continuous improvement We are actively doing things to improve patient safety Mistakes have led to positive changes here After we make changes to improve patient safety, we evaluate their effectiveness D3: Teamwork within units People support one another in this unit When a lot of work needs to be done quickly, we work together as a team to get the work done In this unit, people treat each other with respect When one area in this unit gets really busy, others help out D4: Communication openness Staff will freely speak up if they see something that may negatively affect patient care Staff feel free to question the decisions or actions of those with more authority R Staff are afraid to ask questions when something does not seem right D5: Feedback and communication about error We are given feedback about changes put into place based on event reports We are informed about errors that happen in this unit In this unit, we discuss ways to prevent errors from happening again D6: Non-punitive response to error R Staff feel like their mistakes are held against them R When an event is reported, it feels like the person is being written up, not the problem R Staff worry that mistakes they make are kept in their personnel file T T P T2-T1 P T3-T2 P T3-T1 114

116 DOES LEAN IMPROVE PATIENT SAFETY CULTURE? D7: Staffing < We have enough staff to handle the workload < < R Staff in this unit work longer hours than is best for patient care R We use more agency/temporary staff than is best for patient care R We work in "crisis mode" trying to do too much, too quickly D8: Management support for PS < Hospital management provides a work climate that promotes patient safety The actions of hospital management show that patient safety is a top priority R Hospital management seems interested in patient safety only after an adverse event happens D9: Teamwork across units < * < There is good cooperation among hospital units that need to work together Hospital units work well together to provide the best care for patients R Hospital units do not coordinate well with each other R It is often unpleasant to work with staff from other hospital units D10: Handoffs and transitions < R Things "fall between the cracks" when transferring patients from one unit to another R Important patient care information is often lost during shift changes R Problems often occur in the exchange of information across hospital units < R Shift changes are problematic for patients in this hospital O1: Overall perceptions of PS Patient safety is never sacrified to get more work done Our procedures and systems are good at preventing errors from happening R It is just by chance that more serious mistakes don't happen around here R We have patient safety problems in this unit O2: Frequency of reported events When a mistake is made, but is caught and corrected before affecting the patient, how often is this reported? When a mistake is made, but has no potential to harm the patient, how often is this reported? When a mistake is made that could harm the patient, but does not, how often is this reported? Positive scores: negative formulations were recoded P-values<0.05 are bold 115

117 CHAPTER 6 3. Reports from IRS From 2004 to 2009, the number of reports increased from 510 to 1835 reports on yearly basis (Figure 4). However, the number of reported incidents that reached patient-level (misses) decreased with 27% from 2004 (N=122) to 2009 (N=89). Since the proportion of misses/near misses decreased, the management concluded the willingness to report had increased from 2004 to From 2009 the number of reported near misses decreased with 50% from 1746 to 870 in However, the number of reported misses kept decreasing with 46% from 89 in 2009 to 48 in Of all reported incidents, on average 30% was related to the linear accelerator process. Since in 2011 all accelerators were replaced, the decreased number of reports could partly be explained by the transition to new equipment. The number of treatments had increased over the years. However, when the number of reported incidents were corrected for the increased number of treatments, the curve (Figure 4) was not significantly different. Figure 4. Results from the Incident Reporting System for 2004 until The number of reported incidents reaching the patient-level (misses) decreased only slightly from 122 in 2004 to 48 in The number of reported incidents that did not reach the patient-level (near misses) increased from 2004 to 2009 followed by a decrease in reports. 116

118 DOES LEAN IMPROVE PATIENT SAFETY CULTURE? Table 4. Factorial survey results presented in β scores with corresponding P-values between brackets. The not statistically significant interaction effects (NSS) are not presented in the table, since these were left out of the final multi level model. Results were based on 52 questionnaires in 2011 (T1), 43 in 2012 (T2) and 30 in 2013 (T3). Questions 1: experience this incident as a mistake? β (p) 2: discuss this incident with your colleagues? β (p) 3: report this incident in the IRS? β (p) 4: take action in order to prevent future incidents? β (p) 5: inform the patient about this incident? β (p) measurement T2 a 0.20 (0.91) (0.002) 0.43 (0.12) 0.36 (0.36) 0.58 (0.08) T3 a 0.18 (0.35) (0.93) 0.52 (0.08) 1.19 (0.01) 0.21 (0.58) interaction measurement * type of T2 * near miss B (0.002) 0.18 (0.28) (0.21) (0.33) (0.001) effects incident T3 * near miss B (<0.001) (<0.001) (0.003) (0.002) (<0.001) T2 * miss no harm B 0.10 (0.49) (0.89) 0.35 (0.10) (0.14) (0.52) a reference category: T1 T3 * miss no harm B 0.03 (0.83) 0.01 (0.93) 0.07 (0.73) (0.65) (0.89) measurement * profession T2 * technologist c 0.55 (0.19) 0.53 (0.03) (0.21) (<0.001) NSS meaurement * person who caused the incident T3 * technologist c (0.27) 0.75 (0.02) 0.02 (0.95) (0.22) NSS T2 * oncologist c 0.66 (0.14) 0.60 (0.03) (0.85) 1.08 (0.01) NSS T3 * oncologist c (0.001) (0.93) (0.001) (0.22) NSS T2 * themselves d NSS (0.03) (0.12) NSS (0.44) T3 * themselves d NSS (0.003) (0.01) NSS (0.17) T2 * colleague own unit d NSS (0.02) (0.23) NSS (0.59) T3 * colleague own unit d NSS (0.001) (0.03) NSS (0.004) measurement * unit T2 * unit 1 e NSS 0.61 (0.001) NSS 0.96 (0.001) (0.41) T3 * unit 1 e NSS (0.008) NSS 1.31 (<0.001) (0.01) measurement * previous T2 * previously involved f NSS 0.61 (0.002) NSS 0.12 (0.62) NSS involvement in harmful incident T3 * previously involved f NSS (0.06) NSS (<0.001) NSS measurement * gender T2 * male g NSS 0.79 (<0.001) NSS NSS NSS T3 * male g NSS 0.65 (0.002) NSS NSS NSS measurement * work T2 * < 1 year h NSS NSS NSS (<0.001) NSS experience T3 * < 1 year h NSS NSS NSS (<0.001) NSS T2 * 1-5 years h NSS NSS NSS (0.05) NSS T3 * 1-5 years h NSS NSS NSS (<0.001) NSS T2 * 6-10 years h NSS NSS NSS (<0.001) NSS T3 * 6-10 years h NSS NSS NSS (<0.001) NSS f reference category: T1 * no previous involvement in harmful incident b reference category: T1 * miss with harm g reference category: T1 * female c reference category: T1 * clinical physicist h reference category: T1 * > 10 years work experience d reference category: T1 * colleague other unit NSS: not statistically significant scores e reference category: T1 * unit 2 117

119 CHAPTER 6 Interviews Ten professionals, including management, oncologists and technologists, were interviewed. First explanations regarding the decreased number of reports in the IRS were explored. This was followed by an exploration of professionals views on the experienced patient safety culture, especially regarding the decreased intention to report and the increased intention to structurally improve. Most interviewees explained the decrease of reports by the transition of the accelerator and the planning equipment during 2011 and Another frequently mentioned cause was the improved innovation speed. The interviewees experienced an increased intention to solve problems and an increased support and facilitation for improvement actions by management. They experienced increased opportunities to discuss and solve problems in a multidisciplinary setting, due to the care path teams which increased the success factor of problem solving. They also expressed that, due to an enhanced focus on process improvement together with multiple professions, the way they perceived their work processes and the bottle necks changed. System thinking was promoted. The safety professionals from the focus group reported a decrease of the number of reports from 2010 to 2013, but an increase of the quality of reports. However, a critical note was placed regarding the willingness to report. During the focus group, the professionals discussed whether the reduction in number of reports could be a result of the reduced time professionals experienced to report an incident. They did not reach a consensus. DISCUSSION The safety culture decreased little in 2012 (factorial survey results), after the reorganization of management structures in the beginning of The HSOPSC presented small improvements regarding staffing, teamwork across units and transitions and handoffs. During this period several new staff members were employed. Furthermore, three organizational units were rearranged into two larger units and the treatment process was no longer assigned to one of the units, but to a team of technologists coming from both units. This could have positively influenced teamwork across the units resulting in improved transitions of patients. However, scores on non-punitive response to error and frequency of reported events decreased. This decrease could be the result of the change in management style, the omission of safety as a standard item in team meetings and the uncertainty a rigorous reorganization of this kind brings about among employees. However, this decrease proved temporary, since in 2013 almost all surveyed items indicated an improvement when compared to 2010/2011. The MaPSaF workshop did not present any significant changes in the safety culture, possibly because the initial level of maturity in 2010 was already moderate to high. The HSOPSC presented significant improvement on six out of twelve dimensions. Two dimensions decreased during the studied years; non- 118

120 DOES LEAN IMPROVE PATIENT SAFETY CULTURE? punitive response to error and the frequency of reported events. When studying this into detail, this decreased reporting was only relevant for incidents which could not potentially harm the patient. This decreased willingness to report corresponded to the decreasing number of reports on near misses in the IRS during these years. However, the proportion of reported misses/near misses in the IRS remained constant indicating a sustained willingness to report. The factorial survey results also presented only a small decrease in intention to report a near miss. This decrease was minimal and could, therefore, not explain the decrease in reported events in the IRS. During the interviews several professionals were shown the results and asked for their explanation on the decreased reports. All interviewees declared that they could not believe that the willingness to report had decreased, since they believed safety awareness was high and they experienced little barriers to report. They expressed that safety got a great deal of attention from management as well as from the safety professionals among employees. Instead, they experienced an increased commitment to safety issues from employees since The interviewees presented two possibly important determinants for the decreased reports. First, in 2011 en 2012 all accelerator and planning equipment was replaced. Before this transition the equipment had different suppliers and many work-arounds were in place leading to an immense source of possible process deviations. These deviations decreased drastically after replacement of the equipment process. This could partly explain the decreased reports in the IRS and could be perceived as a positive effect, since not only the number of reported incidents decreased, but in fact the number of actual incidents could have decreased. Second, the interviewees expressed an increased intention to improve, because improvement was increasingly facilitated and supported by management. Within lean management open communication and ringi (shared) decision making are highly valued to create the foundation for continuous improvement. To improve shared decision making, a forum was created where employees could discuss problems and possible solutions in a multidisciplinary setting, the care path teams. Responsibilities were clearly defined in the care path teams, which increased clarity who to notify in case problems were perceived. An increased focus on improvement in team meetings was also expressed as a positive factor. The increase in experienced attention from management by the interviewees was confirmed by the strong increase of experienced management support from 2012 to 2013 based on the HSOPSC results. The increase of experienced willingness to improve was confirmed by the drastic increase in intention to take action in order to prevent future incidents based on the results from the factorial survey. This increased intention to structurally improve could have resulted in improved solving of problems. This also corresponded to the increased positive scores (67% in 2010 to 88% in 2013) on the item In this unit, we discuss ways to prevent errors from happening again from the HSOPSC. Besides increased facilitation, a change of perception was expressed due to the initiated flow projects in These lean based projects, aimed to improve 119

121 CHAPTER 6 flow, increased detailed insight in the processes, increased mutual understanding between professions and increased system thinking due to the focus on the processes. Interviewees from the safety team expressed that the quantity of reports could have decreased, however, the quality of the content of reports increased in their perspective. Because of the many mediating factors influencing the reporting behaviour of professionals, the intention to report cannot be used as a relevant indicator for safety culture. A limitation of this study is that the measured improvements in the patient safety culture could not directly be ascribed to the undertaken lean activities alone. The organizational restructure, the management activities to improve employees commitment and the replacement of the treatment planning system and the accelerators were also partly responsible for the measured improvements. Another interfering aspect could be that this radiotherapy institution already had a long history with safety improvement. Therefore, employees in this organization might be more amenable to quality improvement initiatives than the average health care professional. However, this paper presented an improved patient safety culture, due to combining lean initiatives with an organizational restructure to facilitate the lean philosophy, which cannot be ignored. To test if this approach is also successful for health care organizations in general, similar approaches should be studied in other health care settings. Conclusion Based on the results from the triangulation of methodologies, patient safety culture improved significantly due to the undertaken lean activities and the reorganization to managing care pathways instead of professions in order to facilitate continuous improvement. Although the intention to report decreased for near misses without a possibility to harm a patient, employees experienced a sustained safety awareness, improved quality of reports and a strong increase in creating structural improvements. Due to improvements in equipment and increased problem solving, the actual number of incidents could have decreased. The decreased number of reported incidents was, therefore, not explained by decreased safety awareness or decreased willingness to report, but by improved treatment processes. Possibly not only the patient safety culture had improved due to the lean activities, but the actual patient safety outcomes might have improved as well. 120

122 DOES LEAN IMPROVE PATIENT SAFETY CULTURE? REFERENCES 1. Clark BG, Brown RJ, Ploquin JL, Kind AL, Grimard L. The management of radiation treatment error through incident learning. Radiother Oncol 2010; 95: Pawlicki T, Dunscombe P, Mundt A, Scalliet P. Quality and Safety in Radiotherapy. Taylor & Francis Group, Boca Raton, Shafiq J, Barton M, Noble D, Lemer C, Donaldson LJ. An international review of patient safety measures in radiotherapy practice. Radiother Oncol 2009; 92: Nieva VF, Sorra J. Safety culture assessment: a tool for improving patient safety in healthcare organizations. Qual Saf Health Care 2003; 12 Suppl 2: ii Schein EH. Organizational Culture and Leadership. John Wiley & Sons, San Francisco, Dixon-Woods M, Leslie M, Bion J, Tarrant C. What Counts? An Ethnographic Study of Infection Data Reported to a Patient Safety Program. Milbank Quarterly 2012; 90: Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Quality & Safety 2013; 22: Vincent C, Burnett S, Carthey J. The measurement and monitoring of safety. London; Zohar D. Thirty years of safety climate research: Reflections and future directions. Accident Analysis & Prevention 2010; 42: Sexton JB, Helmreich RL, Neilands TB, et al. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 2006; 6: Blegen MA, Gearhart S, O'Brien R, Sehgal NL, Alldredge BK. AHRQ's hospital survey on patient safety culture: psychometric analyses. J Patient Saf 2009; 5: Etchegaray JM, Thomas EJ. Comparing two safety culture surveys: Safety Attitudes Questionnaire and Hospital Survey on Patient Safety. BMJ Quality & Safety 2012; 21: Cooper Ph.D MD. Towards a model of safety culture. Safety Science 2000; 36: Schein EH. Coming to a New Awareness of Organizational Culture. Sloan Management Review 1984; Liker JK. The Toyota Way. McGrawHill, New York, Ashcroft DM, Morecroft C, Parker D, Noyce PR. Safety culture assessment in community pharmacy: development, face validity, and feasibility of the Manchester Patient Safety Assessment Framework. Qual Saf Health Care 2005; 14:

123 CHAPTER Law MP, Zimmerman R, Baker GR, Smith T. Assessment of safety culture maturity in a hospital setting. Healthc Q 2010; 13 Spec No: Mannion R, Konteh FH, Davies HT. Assessing organisational culture for quality and safety improvement: a national survey of tools and tool use. Qual Saf Health Care 2009; 18: Manchester Patient Safety Framework (MaPSaF) - Facilitator guidance retrieved , Hellings J, Schrooten W, Klazinga N, Vleugels A. Challenging patient safety culture: survey results. Int J Health Care Qual Assur 2007; 20: Vlayen A, Hellings J, Claes N, Peleman H, Schrooten W. A nationwide Hospital Survey on Patient Safety Culture in Belgian hospitals: setting priorities at the launch of a 5-year patient safety plan. BMJ Quality & Safety 2012; 21: Jasso G. Factorial Survey Methods for Studying Beliefs and Judgments. Sociological Methods & Research 2006; 34: Rossi PH, Anderson AB. An Introduction. In: Rossi PH, Nock SL, editors. Measuring Social Judgments: The Factorial Survey Approach. Beverly Hills: Sage Publications; p Wallander L. 25 years of factorial surveys in sociology: A review. Social Science Research 2009; 38: Simons P, Houben R, Reijnders P, et al. A Factorial Survey on Safety Behaviour Providing Opportunities to Improve Safe Radiotherapy Treatment. "Submitted for publication". 26. Van der Schaaf TW, Lucas DA, Hale AR. Near-miss reporting as a safety tool. Butterworth and Heinemann, Oxford, van Everdingen JJE, Smorenburg SM, Schellekens W, Cucic S. Patient Safety Toolbox: instruments for improving safety in health care organisations. Houten, Thomas DR. A General Inductive Approach for Analyzing Qualitative Evaluation Data. American Journal of Evaluation 2006; 27: Sorra JS, Nieva VF. Hospital Survey on Patient Safety Culture. Agency for Healthcare Research and Quality, Rockville, Hox JJ, Kreft IGG, Hermkens PLJ. The Analysis of Factorial Surveys. Sociological Methods & Research 1991; 19:

124 CHAPTER 7 Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour Safer radiotherapy treatment due to synchronized equipment a mixed methods study Submitted Pascale Simons, Jochen Bergs, Madelon Pijls-Johannesma, Huub Backes, Wim Marneffe, and Dominique Vandijck.

125 CHAPTER 7 ABSTRACT Introduction To realize individualized safe radiotherapy of high quality, not only multiple professionals have to work together as a team, but all different treatment processes and equipment have to be connected correctly. We hypothesized that the treatment process would result in less safety events, when an integrated system is used incorporating the planning system with all necessary treatment equipment. Methods To evaluate patient safety on the accelerators, the number of reported safety incidents and the root causes were recorded from 2010 to An Arima was performed and results from interviews provided an exploration of quantitative results. The down time was reported by the percentage of hours an accelerator was down against the available treatment hours. Results In 2010, 33% of the reported incidents were related to the accelerators (average 41/month). This decreased to 20% of reports in 2013 (14/month), while the number of delivered fractions per accelerator increased by 20% (643/month in 2010 to 807/month in 2013). The proportion of causes coded as technical decreased from 24% in 2010 to 18% in Interviewees explained the decrease of reports by the transition of equipment, not by a decreased safety awareness or a decreased willingness to report. The down time decreased with 86% from 5.4% of available treatment hours in 2010 to 2.9% in Conclusions The number of reported incidents related to the accelerators decreased with the new equipment. Not only all necessary equipment was connected as an integral system, the equipment might also provide more opportunities to better match the radiation process for the particular organization. The down time of the accelerators decreased, and might also have positively influenced safety events from happening. Since safety awareness did not decrease, the transition to the new equipment positively affected patient safety by a decreased risk for serious safety events. 124

126 SAFER RADIOTHERAPY TREATMENT INTRODUCTION In radiotherapy, quality and safety are of high priority. 1,2 Individualized patient care has led to many possibilities for radiation treatment resulting in a high innovation rate of treatment protocols. To realize individualized safe and high quality care, not only multiple professionals have to work together as a team, but also, all different treatment processes and equipment have to be interconnected correctly. Since the possibilities of treatment modalities have increased as well as the controlling mechanisms (e.g. portal imaging or dose guided radiotherapy) 3 system synchronization has become a major challenge. As a result, several unfortunate safety incidents have taken place. 4-6 Due to the high cost of the accelerators (treatment equipment) and the oligopolistic market (three major suppliers worldwide), one is inclined to regard this equipment as an established fact. Therefore, accelerator equipment is not easily adapted to the particular demands of individual organizations. Rather, professionals and the processes are expected to adjust to the equipment. This results in suboptimal processes and many workarounds for professionals. From a quality and safety perspective this is far from ideal. Following human factors engineering, systems (technical and organizational) should be designed to support human performance. This discipline studies the relationships between 7,8 humans and technology. Following their perspectives, training of professionals to compensate for a poor designed system should be avoided. Instead, technologies, processes and tools should be designed to better support the clinical professionals to deliver safe care. 9 This approach is reinforced by the lean management philosophy. Lean is a quality improvement strategy aimed to redesign processes towards a customer-focused ideal state by reducing all nonvalue added steps (waste) in a process. 10 As for the human factor science, the lean philosophy focuses on optimizing the work processes to support the professionals and not the other way round. However, to create sustainable quality and safety improvement, specific improvement interventions based on lean principles should be supported by a system-level approach. This includes organizations structural and cultural elements, but also reliable (human factor proof) equipment supporting the clinical professionals in their daily work processes Previous literature reveals that a considerable percentage of errors in radiotherapy stems from errors in treatment set-up and delivery. 16 Within the studied radiotherapy institute, the number of reported incidents from January to July 2010, conformed the literature results, as a large percentage of reports was related to the accelerator process (35%: 291 of the 839 incidents). Besides these safety related problems, the accelerators presented a large amount of down time, whereas the accelerators had only been operational since This again caused many process disruptions leading to frustrated employees, which potentially increased the chance for safety incidents to occur. In addition, the 125

127 CHAPTER 7 accelerators did not meet the organizations demand regarding treatment opportunities (e.g. use of cone beam). To create more stabile and safe processes combined with high quality treatment, management decided to replace all treatment and treatment planning related equipment, including seven accelerators, portal imaging devices, and treatment planning software. We hypothesized that if an integrated system would be used, reliably connecting all required treatment equipment, and technical down time of accelerators would decrease, the irradiation treatment process would become safer. METHODS A radiotherapy institute in the Netherlands (MAASTRO clinic) has been actively promoting safety since 2003 and started their Lean journey in In 2010, the lean philosophy was up taken in the organizations long term strategy. Specific lean interventions focused at the clinical processes were combined with an overall strategy aimed at the organizations structure and culture elements. In January 2011, the organizational structure of the institute was reorganized to support the lean philosophy and facilitate continuous improvement. Instead of managing the different clinical professions, care paths were created to stimulate communication between professions, and to increase process innovation. An incident reporting system was in place, and results were periodically reported to management and employees. Healthcare failure modes and effects analysis (HFMEA) 17,18 were performed for all mayor clinical processes, and processes on the accelerator were redesigned by the professionals (lean project) to improve quality and patient safety. However, a large percentage of reported incidents related to the accelerator process remained unsolved. In addition, the accelerators experienced a large amount of down time despite of an experienced team of engineers being permanently available in the organization, and lacked the opportunity to connect additional technical systems as the planning system and portal imaging reliably. In February 2011, the chief executive officer, therefore, publically announced that in the years to come all equipment was being replaced by an integral system from a different supplier. From July 2011 to March 2013, all new treatment planning software, six of the most advanced accelerators at the time, and portal imaging were installed and tested. In addition, all clinical professionals received intensive training and all required clinical processes were adjusted and optimized. Incident Reports To evaluate whether the transition to the new equipment resulted in safer radiotherapy treatment, data from the incident reporting system was recorded monthly as an indicator for patient safety. Trained safety professionals determined the root causes of all reported incidents. These root causes were coded in human, organizational and technical codes following the Prevention and 126

128 SAFER RADIOTHERAPY TREATMENT Recovery Information System for Monitoring and Analysis (PRISMA) technique 19,20 and were recorded in a software program. The safety professionals followed periodic trainings and a yearly inter reliability test was performed to maximize reliability of analysis. Results on the number of incidents and the distribution of coded root causes were periodically reported to management and employees. Interrupted time series analysis was used (ARIMA models) to detect whether or not the transition of equipment had a significant effect on the distribution of the incident reports. By using an ARIMA model the intervention effect can be 21 differentiated from possible underlying secular trends. The monthly distribution of reported incidents related to the accelerator process was presented separately. To correct for the differing number of treatment fractions per month, the number of reported incidents was divided by the number of delivered treatment fractions. This ARIMA model presents the number of reported incidents per delivered treatment fraction. Interviews To meet the complex nature of safety measurement, the quantitative results from the incident reporting system were combined with qualitative measures to increase understanding of data from the incident reporting system Structured interviews were performed with ten professionals and a focus group session was performed with five patient safety professionals. The interviews and the focus group session were audio recorded, fully typed and analyzed using specialized software (NVIVO). Inductive reasoning was used for coding, to detect trends and interpret the quantitative results. 21 Down time accelerators The amount of down time was recorded for the seven former and the six new accelerator. The down time was reported per month and per accelerator as the proportion of hours with technical failure divided by the available treatment hours. The available treatment hours were set on 14 hours per day per accelerator, as patients were treated from eight in the morning to ten in the evening. RESULTS Incident reports The number of reported incidents decreased from 2010 to 2013 (Figure 1) for the entire organization. The number of incidents related to the accelerator process decreased most. Where in 2010, 33% of all reported incidents were related to the accelerators (average 41 reports per month) in 2013 this had decreased to 20% (average 14 reports per month). In November 2011, the first patient was treated on a new accelerator, and since April 2013, the old accelerators were no longer used for patient treatment. The number of reports 127

129 CHAPTER Reports entire organization Accelerator reports Total minus accelerator reports Figure 1. The number of reported incidents for the entire organization. varied strongly per month for every accelerator (Figure 2). The first months after taking a new accelerator into use, a relative large number of incidents was reported. After a few months, the number of reports decreased and stabilized Transition period Linac1 Linac2 Linac3 Linac5 Linac6 Linac7 Linac8 Evian Heraklion Oslo Porto Berlijn Amsterdam Figure 2. The number of reported incidents /number of fractions for every separate accelerator per month. Linac 1 to 8 are the old accelerators. The dashed lines present the new accelerators. 128

130 SAFER RADIOTHERAPY TREATMENT The ARIMA model resulted in a significant decrease of accelerator related reports (Figure 3), owing to the transition of equipment. Although the number of reported incidents that were not related to the accelerator process also decreased in the studied years, the ARIMA model detected no significant effect of the transition in equipment. The transition period of the accelerators (February 2011 to March 2013) had been left out of the analysis. In April 2013, incidents were reported less per delivered fraction than before the transition (Table 1) or 52 less reports per 1000 delivered fractions (p=0.030). However, in October 2013 the effect was no longer significant on a 5% level (p=0.061). Figure 3. A time series analysis (ARIMA model) of the reported accelerator incidents. Reported incidents decreased significantly from January 2010 until February 2011(before transition) compared to April to December 2013 (after transition). In consistence with the decreasing number of incidents, the number of root causes also decreased from an average 150 causes in 2010 to 38 in 2013 (Figure 4). The proportion of causes coded as technical decreased slightly from 24% in 2010 to 18% in Whereas the proportion of human causes increased from 27% in 2010 to 32% in 2013, the organizational causes remained stabile (49% in 2010 and 50% in 2013). The absolute number of technical causes in 2013 was small with an average of seven technical coded root causes per month. In 2010 on average 15% of accelerator incidents were related to the portal imaging process, varying between 10-50% per month. This process included taking the portal image; verifying patients actual position with the planned position and potentially correcting patients treatment position. The absolute number of incidents related to portal imaging decreased from on average nine 129

131 CHAPTER 7 Table 1. Results from time series analysis (ARIMA) for the reported accelerator incidents. Effect sizes from the ARIMA model for the number of reported incidents related to the accelerators from January 2010 until February 2011 compared to April to December Effect P value Time period -0,442 0,805 Interact -0,398 0,919 Phase month 1 (April 2013) -52,098 0,030 Phase month 2 (May 2013) -52,496 0,025 Phase month 3 (June 2013) -52,893 0,023 Phase month 4 (July 2013) -53,291 0,026 Phase month 5 (August 2013) -53,689 0,033 Phase month 6 (September 2013) -54,087 0,044 Phase month 7 (October 2013) -54,485 0,061 Phase month 8 (November 2013) -54,882 0,082 Phase month 9 (December 2013) -55,280 0,107 P<0.05 are presented in bold Human causes Organisational causes Technical causes Transition period Figure 4. Root causes of reported accelerator incidents. All accelerator incidents were analyzed and root causes were coded in human, organizational and technical codes. Technical coded causes decreased more than human or organizational coded causes. 130

132 SAFER RADIOTHERAPY TREATMENT reported incidents per month in 2010 to two per month in In 2010, 17% of the causes for the portal imaging incidents were reported as technical, which decreased to 5% in In 2010, seven accelerators delivered on average 4500 radiotherapy fractions per month (643 fractions per month per accelerator). After the transition, these six new accelerators delivered on average 4842 fractions per month resulting in 807 fractions per accelerator per month. The number of fractions delivered per accelerator increased by 20%. The number of required professionals per accelerator remained equal for the former and the new accelerators. Interviews Ten professionals, including management, oncologists and technologists, were interviewed and a focus group session with five safety professionals was performed. The decreased number of reports in the IRS was explored. Most interviewees explained the decrease of reports by the transition of the accelerators and the treatment planning software. The safety professionals reported a decreased number of reports, but an increased quality of reports. In addition, the interviewees expressed that employees experienced an increased willingness to improve. This could be the result of management increasingly facilitating and supporting improvements after the organizations restructure. Interviewees did not explain the decreased number of reported incidents by decreased safety awareness or decreased willingness to report, but by improved treatment processes, which was congruent with results on the safety culture in the studied organization. 27 Down time accelerators The percentage down time of the accelerators decreased from an average 5.4% per month in 2010 to 2.9% per month in 2013 for the new accelerators (Figure 5). The down time for the new accelerators remained high, which was explained by start-up problems which still had to be solved. These accelerators were the first in their generation and down time was expected to decrease further in 2014, when a new upgrade would be installed. 131

133 CHAPTER 7 Figure 5. The down time for accelerators. The down time (time the equipment cannot be used for treatment due to a technical breakdown) varied strong per month per accelerator. However, the overall down time of the new accelerators in 2013 (dashed line) was smaller with less spread than for the old accelerators in DISCUSSION In 2010, a radiotherapy institute decided to replace all accelerators and the treatment planning software in order to deliver higher quality and safer treatment for future cancer patients. They experienced a large amount of accelerator down time, and a unreliable connection between accelerators, portal imaging, and treatment planning software. As a result clinical professionals had to deal with a lot of rescheduling and many workarounds in the treatment process. We hypothesized that the treatment process would become safer, when an integrated system was used connecting all necessary equipment reliably. In 2011 and 2012, the treatment planning software and seven accelerators were replaced by the most advanced equipment at the time, including six accelerators. As expected, the number of incident reports decreased, in particular the accelerator related reports. Although the number of delivered fractions per accelerator increased by 20% (643/month in 2010 to 807/month in 2013), the number of reported incidents related to the accelerator decreased significantly from 41 reports per month in 2010 to 14 in Not only the number of reports decreased, but also the proportion of technical coded root causes assigned to the reported incidents. Where the proportion organizational coded causes remained stable, the proportion of technical causes decreased, which could indicate improved equipment. This was even more prominent for the portal imaging process, which was experienced as a problematic process and represented 10-50% of accelerator incidents per month in Besides the organizational causes for portal imaging reports (average 75 per month; 69% of all causes), technique was also a prominent cause in 2010 (average 29 per month; 17%). In 2013 the technical causes of portal imaging reports were 132

134 SAFER RADIOTHERAPY TREATMENT reduced even further to on average 1 cause per month (5% of all causes) with the new equipment, whereas organizational causes reduced less to on average 19 causes per month (84% of all). The equipment transition reduced the number of reported incidents significantly, as hypothesized. One could question the reliability of the number of incident reports as an outcome indicator for patient safety. Although voluntary incident reporting systems prove valuable in safety improvement 28, probably not all safety incidents are reported, which endangers the objectivity of results. 29 To increase reliability of the results from the incident reporting system, results were discussed among clinical professionals during structured interviews. The interviewees did not experience decreased safety awareness or a decreased willingness to report, which was in congruence with results on improved patient safety culture in the organization. 27 Instead, the interviewees experienced improved treatment processes, partly as a result of the technical transition and partly because of an increased management focus on continuous improvement. The reorganisation of management structure might have also affected the number of reported incidents. In addition, due to the focus on standardizing processes based on the lean philosophy, employees were sparked to directly adjust the procedures and work processes to the new equipment. This could also have affected the number of incidents in a positive way. Although all these issues might have played an important role in decreasing the number of incidents, interviewees expressed the transition to the new equipment as the strongest influencing factor. The technical down time decreased with 46% from 5.4% of available treatment hours in 2010 to 2.9% in Although down time improved, 2.9% was still not satisfactory. However, the technicians expected the down time to decrease further in 2014, since the new accelerators were the first in their generation and start-up problems would be solved with the coming upgrades. The decreased down time was expected to result in less rescheduling of patients, and less disruptions of work processes for employees, which in turn could decrease the risk on safety incidents. Conclusion In conclusion, replacing all treatment equipment and treatment planning software by a new integral system, connecting all necessary equipment reliably, resulted in a decreased number of reported accelerator related incidents. Less technical causes were assigned to the remaining incident reports. Down time of the accelerators decreased which might have also prevented safety events from happening. Since safety awareness did not decrease, we concluded that the transition to the new equipment positively affected patient safety by a decreased number of incidents resulting in a decreased risk for serious safety incidents and harm to patients. 133

135 CHAPTER 7 REFERENCES 1. Clark BG, Brown RJ, Ploquin JL, Kind AL, Grimard L. The management of radiation treatment error through incident learning. Radiotherapy and Oncology 2010; 95: Shafiq J, Barton M, Noble D, Lemer C, Donaldson LJ. An international review of patient safety measures in radiotherapy practice. Radiotherapy and Oncology 2009; 92: Moran JM, Fraass BA. Introduction: Quality, Technology, and Outcomes in Radiation Oncology. Seminars in Radiation Oncology 2012; 22: Bissonnette J-P, Medlam G. Trend analysis of radiation therapy incidents over seven years. Radiotherapy and Oncology 2010; 96: Derreumaux S, Etard C, Huet C, et al. Lessons from recent accidents in radiation therapy in France. Radiat Prot Dosimetry 2008; 131: Williams MV. Radiotherapy near misses, incidents and errors: radiotherapy incident at Glasgow. Clin Oncol (R Coll Radiol) 2007; 19: Chan AJ, Islam MK, Rosewall T, Jaffray DA, Easty AC, Cafazzo JA. The use of human factors methods to identify and mitigate safety issues in radiation therapy. Radiother Oncol 2010; 97: Rivera AJ, Karsh BT. Human factors and systems engineering approach to patient safety for radiotherapy. Int J Radiat Oncol Biol Phys 2008; 71: S Russ AL, Fairbanks RJ, Karsh B-T, Militello LG, Saleem JJ, Wears RL. The science of human factors: separating fact from fiction. BMJ Quality & Safety 2013; 22: Liker JK. The Toyota Way. McGrawHill, 2004, New York. 11. Al-Balushi S, Sohal AS, Singh PJ, Al Hajri A, Al Farsi YM, Al Abri R. Readiness factors for lean implementation in healthcare settings--a literature review. J Health Organ Manag 2014; 28: Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 1. Conceptualising and developing interventions. Qual Saf Health Care 2008; 17: Kaplan GS, Patterson SH, Ching JM, Blackmore CC. Why Lean doesn't work for everyone. BMJ Qual Saf McIntosh B, Sheppy B, Cohen I. Illusion or delusion--lean management in the health sector. Int J Health Care Qual Assur 2014; 27: Wood D. A prescription for lean healthcare. Healthc Q 2014; 17: Valentin J. Prevention of accidental exposures to patients undergoing radiation therapy. A report of the International Commission on Radiological Protection. Ann ICRP 2000; 30: DeRosier J, Stalhandske E, Bagian JP, Nudell T. Using Health Care Failure Mode and Effect Analysis: The VA National Center for Patient Safety's 134

136 SAFER RADIOTHERAPY TREATMENT Prospective Risk Analysis System. The Joint Commission Journal on Quality Improvement 2002; 27: Habraken MM, Van der Schaaf TW, Leistikow IP, Reijnders-Thijssen PM. Prospective risk analysis of health care processes: a systematic evaluation of the use of HFMEA in Dutch health care. Ergonomics 2009; 52: Van der Schaaf TW, Lucas DA, Hale AR. Near-miss reporting as a safety tool. Butterworth and Heinemann, Oxford, van Everdingen JJE, Smorenburg SM, Schellekens W, Cucic S. Patient Safety Toolbox: instruments for improving safety in health care organisations. Houten, Thomas DR. A General Inductive Approach for Analyzing Qualitative Evaluation Data. American Journal of Evaluation 2006; 27: Dixon-Woods M, Leslie M, Bion J, Tarrant C. What Counts? An Ethnographic Study of Infection Data Reported to a Patient Safety Program. Milbank Quarterly 2012; 90: Vincent C, Burnett S, Carthey J. The measurement and monitoring of safety. The Health Foundation, London, Cooper Ph.D MD. Towards a model of safety culture. Safety Science 2000; 36: Kaboli PJ, Mosher HJ. Using balanced metrics and mixed methods to better understand QI interventions. BMJ Quality & Safety 2014; 23: Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Quality & Safety 2013; 22: Simons PAM, Houben R, Vlayen A, et al. Does lean management improve patient safety culture? An extensive evaluation of safety culture in a radiotherapy institute. European Journal of Oncology Nursing. 28. Leape LL. Reporting of Adverse Events. New England Journal of Medicine 2002; 347: Capuzzo M, Nawfal I, Campi M, Valpondi V, Verri M, Alvisi R. Reporting of unintended events in an intensive care unit: comparison between staff and observer. BMC Emerg Med 2005; 5:

137 CHAPTER 7 136

138 CHAPTER 8 Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour Cost-effectiveness of reduced waiting time for head and neck cancer patients due to a lean process redesign Submitted Pascale Simons, Bram Ramaekers, Frank Hoebers, Kenneth W. Kross, Madelon Pijls-Johannesma, Wim Marneffe, and Dominique Vandijck.

139 CHAPTER 8 ABSTRACT Introduction Compared to new technologies, the redesign of care processes is generally considered less attractive to improve patient outcomes. Nevertheless, it might result in better patient outcomes, without further increasing costs. As early initiation of treatment is of vital importance for patients with head and neck cancer (HNC), these care processes were redesigned. This study aims to assess patient outcomes and cost-effectiveness of this redesign. Methods An economic (Markov) model was constructed to evaluate the biopsy process of suspicious lesion under local instead of general anaesthesia, and combining the CT-PET s for diagnostics and radiotherapy planning. Patients treated for HNC were included in the model stratified by disease location (larynx, oropharynx, hypopharynx and oral cavity) and stage (I-II and III-IV). Probabilistic sensitivity analyses were performed. Results Time gains for the included patient groups varied from 5-22 gained days, resulting in gained QALYs between 0.13 to The new work flow (NWF) was cost-effective for all included patient groups, when the informal ceiling ratio for high burden diseases in the Netherlands ( 80,000) was used. For patients treated for tumours located at the larynx and oral cavity, the NWF resulted in additional QALYs and decreased costs compared to the regular work flow. Extrapolation of individual net monetary benefits resulted in an organizational benefit of 14,071,232, and a national benefit of 5,299,676,846. Conclusion The redesigned care process reduced waiting time for treatment of HNC patients, and proved cost-effective. Since care improved, this redesign of processes should be implemented on a wider scale. 138

140 COST-EFFECTIVENESS OF REDESIGNED CARE PROCESS INTRODUCTION Optimal treatment for every individual patient has always been the main goal in medicine. However, the increased number of costly new treatment options combined with the ageing population result in dramatic increased healthcare costs. 1,2 Since resources are scarce, decisions have to be made on which treatment options should be made available for society, and included in healthcare insurance packages. Cost-effectiveness analyses can support these difficult decisions by providing relevant information including the long term costs, and benefits for patients, and the healthcare sector. 3 A redesign intervention of care processes might optimize quality, and efficiency of care, although often considered less attractive by medical professionals as compared to adapting new technologies. Time to treatment or waiting time might be shortened resulting in better patient outcomes without driving costs to a maximum. For oncology patients in general, and for patients with head and neck cancer (HNC) in particular, waiting time is significantly associated with patient outcome. Since HNC tumours have a fast doubling time, long waiting times cause tumour progression and negatively impact local tumour control, and survival rates. 4 Based on theoretical evidence, delay of radiotherapy (treatment with irradiation) may affect the outcomes of treatment by permitting proliferation of clonogenic cells, leading to decreased probabilities of local control, which has been confirmed by retrospective observational studies. 5 Chen et al. showed in a systematic review that the risk of local recurrence (relative risk (RR): 1.15 per month waiting time) and mortality (RR: 1.16 per month waiting time) increased for HNC with increased waiting time for radiotherapy. 4 Waaijer et al. estimated an average control loss of 16-19% due to tumour progression for a mean waiting time of 56 days, 6 potentially resulting in increased mortality. 7 The probability and severity of treatment complications might also increase, because larger volumes have to be irradiated, causing a potential decrease of quality of life for patients. 8 Therefore, optimisation of care processes to minimize waiting time is important to improve outcomes for patients with HNC. The treatment of HNC patients needs optimal collaboration between several disciplines (head, and neck (HN) surgeons, radiation oncologists, medical oncologists, pathologists, and radiologists, etc.) to improve medical decisionmaking, which could hamper short waiting times. In an aim to optimize waiting times, we redesigned the care process, and evaluated its benefits in terms of patient outcome, and cost-effectiveness. 139

141 CHAPTER 8 METHODS Markov model description A cost-benefit analysis was performed for patients with HNC staged I-II, and III- IV located in the oropharynx, larynx, hypopharynx, and/or oral cavity. Patients were treated at the Maastricht University Medical Centre (MUMC+), and at MAASTRO CLINIC, in the joint Multidisciplinary Head and Neck Board, including HN surgeons as well as radiation oncologists. These particular patient groups were considered, because of their expected benefits on patient outcomes 4, and the possibilities of redesigning the workflow for these patients in particular. Results were stratified for the tumour sites, and stages, since differences in improvements of waiting time was expected, but also the prognoses, and quality of life of these patients groups differed. A Markov model was used to analyse costs, and benefits of a logistic process redesign of the diagnostic, and where relevant the preparation phase of radiotherapy. This redesign included two main organizational changes: 1. performing a diagnostic tumour biopsy, and evaluation of the upper aero-gastrointestinal tract under local instead of general anaesthesia (time to treatment shortened with 17 days) 2. performing a diagnostic CT-PET in radiation treatment position using immobilization mask before radiotherapy instead of an additional CT-PET during preparation for radiotherapy (time to treatment shortened with 5 days) The rationale for the first change was that the examination under general anaesthesia required available time in the operation theatre and this proved to be a rate-limiting step in the diagnostic process of HNC patients. To increase efficiency of the diagnostic process, and decrease waiting times for patients, new methods were evaluated to diagnose HNC without losing quality of investigation. Since the development of the chip-on tip cameras, imaging of the lungs has improved drastically, and is increasingly performed under local anaesthesia. Because of these positive results, a similar pilot was performed for the HNC patients. The accuracy of this diagnostic tumour biopsy for HNC using local anesthesia is still under evaluation. The rationale for the second change was that staging CT-PET is not standard in the diagnostics of HNC patients. Usually loco-regional staging is performed by CT, and/or MRI of the HN region in accordance with the national guidelines for clinical practice. 9 Screening for distant metastases is usually done by a conventional chest X-ray (for low risk patients), or a CT-chest for high risk patients. For radiation treatment planning, CT-PET in treatment position of the 140

142 COST-EFFECTIVENESS OF REDESIGNED CARE PROCESS HN, and upper thoracic area is often performed to facilitate the delineation of the Gross Tumour Volume. 10,11 Due to the etiological factors associated with HNC (i.e. nicotine, and alcohol abuse), these patients are also at risk of secondary tumours, e.g. lung cancer, and/or oesophageal cancer. Performing a CT-PET for radiation treatment planning, therefore, increases the detection of second primary tumours, and/or metastases, which had not been identified in conventional staging. This would lead to additional investigational procedures, and delay of the start of treatment. By performing a diagnostic PET-CT of the HN area, and the chest in radiation treatment position, optimal staging, including screening for second primaries, or metastases, and preparation for state of the art radiation treatment planning is combined. The original/regular process flow (RWF) was compared to three new process flows (NWF); (1) tumour biopsy (to define tumour status) under local anesthesia. Since local anesthesia can be provided outside of the operation quarter, scheduling is independent of surgery schedules, and delays; (2) a diagnostic CT-PET used before radiotherapy; and (3) a combination of 2, and 3 (see Table 1). Only patients with tumours located at the larynx, and oral cavity were considered for a biopsy under local anesthesia, since the other tumours cannot be assessed optimal in local anesthesia because palpation forms a big part of this assessment. Most patients with tumours of the oral cavity are treated with surgery as primary treatment. Postoperative radiation therapy is given based on indications derived from the pathology report. Therefore, these patients were not considered for a CT-PET in radiation treatment position. Table 1. The gained working days due to the new workflow related to regular workflow. Results were separated per treatment site, and stage. Tumour Site Stage Local Anaesthesia Diagnostic CT-PET Gained Waiting time Larynx stage I-II Yes Yes 22 stage III-IV Yes Yes 22 Oropharynx stage I-II No Yes 5 stage III-IV No Yes 5 Hypopharynx stage I-II No Yes 5 stage III-IV No Yes 5 Oral cavity stage I-II Yes No 17 stage III-IV Yes No 17 The current Markov model used in this study included four different health states: progression free survival, local/regional recurrence of the tumour, 141

143 CHAPTER 8 metastases, and death (see Figure 1). A cycle time of 6-months, and a life time horizon was used to incorporate most long term effects. Figure 1. Markov model with half year cycle time and a life time span. Model input parameters Actual treatment, and diagnoses was equal for the different modalities. However, waiting time differed, because of deviating process designs. Input parameters of the model are presented in Table 2. Data from a pilot study in the studied organization was used to indicate time gains for each process flow, and were expressed in the number of working days (5 days per week). A time gain of 22 working days could in reality cover 30 days of waiting time. The most conservative estimation (i.e. shortest time gain) was used for the NWF. The differing time gains were related to the relative risks of 1.16 for death, and metastases, and 1.15 for local/regional recurrence for every month waiting time. 4 Therefore, each tumour location resulted in different transition probabilities between the defined health states. The relative risks for gained time were assumed to be linearly related to the amount of time gained. To estimate survival, local/regional recurrence, and metastases for oropharynx, and larynx tumours, parametric survival models (exponential and Weibull) were estimated stratified for tumour site, and stage. Estimates were based on a data retrieved from medical files (period ) of 2096 HNC patients (938 with tumours of the larynx stage I-II, and 620 stage III-IV, and for the oropharynx 83 with stage I-II, and 455 stage III-IV). 12 Hazard ratios reported by Datema et al. 142

144 COST-EFFECTIVENESS OF REDESIGNED CARE PROCESS were used to convert transition probabilities for oropharynx tumours to tumours located at the hypopharynx and the oral cavity. 13 These relative risks were assumed to be equal for survival, local/regional recurrence, and distant metastases. Toxicity was assumed to be equal for NWF and RWF. Effects and costs Utility scores were combined with life expectancy to calculate quality adjusted life years (QALYs). Utility scores provide a single index value for health related quality of life ranging from 0 (death) to 1 (optimal health), and were estimated based on the Euroqol-5D questionnaire. 14 The disutility for the local/regional recurrence, and metastases health states were assumed to be site, and disease stage independent. Costs were estimated by the costs per healthcare activity in the studied organisations in 2013 multiplied by the required healthcare activities, based on the opinions of clinical experts (Appendix). Costs which potentially differed between the two logistical processes were mainly focused on since only these costs could influence the incremental cost-benefit analysis. Three dimensions of costs were included in the Markov model: 1. Event costs: costs for all the expected health care activities in need incurred when the health state of a patient changed (e.g. a recurrence). 2. Follow up (health state): all follow up costs for the first five years after treatment until the next event. From the sixth year onward, follow up costs were assumed to be zero. 3. Intervention costs: the intervention cost were partly covered by the costs associated with the implementation of the redesign (NWF), which is based on the purchase cost for the endoscope (Pentax chip on tip) with unit (estimated at 50,000), and spread over 1,000 patients to be diagnosed ( patients expected in five years). The intervention costs also covered the costs for unnecessary activities, which included the costs for patients whose biopsy under local anaesthesia failed, and an additional biopsy under full anaesthesia had to be performed. They also included the costs for patients who received a diagnostic CT-PET in radiation therapy position with mask, but in the end appeared to have no indication for radiotherapy. Only the costs for the health care activities were included in this study. To account for inflation rates, future QALYs, and costs were discounted by rates of 1.5% and 4% respectively

145 CHAPTER 8 Table 2. Input parameters of Markov model. Parameter Estimate Se / 95%CI Utility score for progression free state Laynx, Stage I-II Beta Laynx, Stage III-IV Beta Oropharynx, Stage I-II Beta Oropharynx, Stage III-IV Beta Hypopharynx, Stage I-II Beta Hypopharynx, Stage III-IV Beta Oral cavity, Stage I-II Beta Oral cavity, Stage III-IV Beta Distribution Source Disutility local/regional recurrence, and distant metastasis versus progression free Beta 14 Parameters parametric survival model* Larynx, Stage I-II Mortality (base) Exponential Mortality (tumor_site_specific) Exponential Mortality (log_scale) Exponential Local/regional recurrence (base) Weibull Local/regional recurrence (tumor_site_specific) Weibull Local/regional recurrence (log_scale) Weibull Distant metastasis (base) Exponential Distant metastasis (tumor_site_specific) Exponential Distant metastasis (log_scale) Exponential Larynx, Stage III-IV Mortality (base) Exponential Mortality (tumor_site_specific) Exponential Mortality (log_scale) Exponential Local/regional recurrence (base) Weibull Local/regional recurrence (tumor_site_specific) Weibull Local/regional recurrence (log_scale) Weibull Distant metastasis (base) Exponential Distant metastasis (tumor_site_specific) Exponential Distant metastasis (log_scale) Exponential Oropharynx, Stage I-II Mortality (base) Weibull

146 COST-EFFECTIVENESS OF REDESIGNED CARE PROCESS Mortality (tumor_site_specific) Weibull Mortality (log_scale) Weibull Local/regional recurrence (base) Weibull Local/regional recurrence (tumor_site_specific) Weibull Local/regional recurrence (log_scale) Weibull Distant metastasis (base) Weibull Distant metastasis (tumor_site_specific) Weibull Distant metastasis (log_scale) Weibull Oropharynx, Stage III-IV Mortality (base) Weibull Mortality (tumor_site_specific) Weibull Mortality (log_scale) Weibull Local/regional recurrence (base) Weibull Local/regional recurrence (tumor_site_specific) Weibull Local/regional recurrence (log_scale) Weibull Distant metastasis (base) Weibull Distant metastasis (tumor_site_specific) Weibull Distant metastasis (log_scale) Weibull Hazard Ratios Hypopharynx versus Oropharynx lognormal 13 Hazard Ratios Oral cavity versus Oropharynx lognormal 13 Relative Risks New Workflow versus Regular Workflow LR relative risk for local/regional recurrences when gained 1 month in treatment delay relative risk for distant metastases when gained 1 month in treatment delay relative risk for death when gained 1 month in treatment delay lognormal lognormal lognormal Abbreviations: Se: standard error, CI: confidence interval, RWF: regular work flow, NWF: new work flow * Interpretation parameters parametric survival model: S(t) = survival probability at time t = е λtα λ = event rate parameter = (base) + (tumor_site_specific) α = shape parameter indicating the time dependent deviation = 1/(log_scale)

147 CHAPTER 8 Markov model analysis The life years (LYs), QALYs and expected total costs were calculated per tumour site, and stage (eight patient groups) for the RWF as well as the NWF. The incremental cost-effectiveness ratio (ICER) was calculated by dividing the incremental (NWF minus RWF) costs by the incremental QALYs. This ICER represented the additional costs of one QALY, when the NWF was implemented for the specific tumour site. A treatment is considered cost-effective when the ICER is below the price a patient (or the society) is willing to pay for an additional QALY (ceiling ratio). A ceiling ratio of 80,000 was adopted, as this is the informal ceiling ratio for high burden diseases in the Netherlands. 16 This means that society is willing to pay 80,000 for an additional year in perfect health. Sensitivity analysis Probabilistic sensitivity analyses using Monte-Carlo simulation was performed to account for the uncertainty of the input parameters in the model. 3 The simulation incorporated 5,000 iterations. The results of these simulations were illustrated using cost-effectiveness acceptability curves (CEACs). Extrapolation of results Patient outcomes were extrapolated to the organizational, and the national level (the Netherlands). The incremental net monetary benefit (INMB) was multiplied by the number of patients treated. The INMB is a representation of the incremental gains/benefits (QALYs multiplied by the ceiling ratio) minus the incremental costs. 17 For example, an INMB of 100 means that the total benefit of the proposed change exceeds the total costs with 100 on the long run. The INMB on the organizational level presented the number of patients treated within the studied organization (based on the annual reports of the two participating organizations) multiplied by the INMB. This organizational INMB represented the incremental benefit from the perspective of the healthcare payers and did not represent the incremental benefit or cost for the specific organization. The INMB extrapolated to the national level for the Netherlands was calculated by multiplying the incidence numbers from the Integral Cancer Centre for the four included treatment sites 18 and the INMB. 146

148 COST-EFFECTIVENESS OF REDESIGNED CARE PROCESS Table 3. Outcomes of Markov model for the new workflow versus the regular workflow per treatment site and stage. Outcomes Larynx Oropharynx Hypopharynx Oral cavity I-II III-IV I-II III-IV I-II III-IV I-II III-IV Costs NWF Mean 7,806 15,739 13,588 16,015 13,377 15,700 5,045 20,652 95% CI 7,260-8,345 15,368-16,531 12,608-15,644 15,686-16,586 12,258-15,822 15,330-16,485 3,512-11,542 19,904-21,634 Costs RWF Mean 8,445 15,993 13,135 15,562 13,068 15,397 5,742 20,846 95% CI 7,883-8,963 15,664-16,672 12,154-15,227 15,254-16,135 11,938-15,532 15,035-16,216 4,070-8,438 20,107-21,749 QALYs NWF Mean % CI QALYs RWF Mean % CI Incremental Mean QALYs 95% CI Incremental Mean Costs 95% CI ICER /QALY dominant 29 3,777 2,875 2,909 2,119 dominant dominant INMB per Mean 53,703 51,916 11,260 11,380 10,430 10,408 44,309 43,782 patient 95% CI 9, ,627 7, ,569 1,506-2,463 1,829-24,240 1,500-23,359 1,660-22,906 7,690-95,812 7,767-92,909 # patients in a organization Organizational INMB 4,779,543 4,620, , , , ,973 1,595,137 1,576,153 # patients in the b Netherlands National INMB 1,696,737,754 1,640,287, ,579, ,827,976 25,928,035 25,873, ,232, ,209,016 NWF: new work flow RWF: regular work flow CI: Credibility Interval QALY: quality adjusted life year SD: standard deviation ICER: incremental cost-effectiveness ratio INMB: Incremental Net Monetary Benefit calculated for ceiling ratio: 80,000 a the total number of patients per treatment site based on the organizational annual report of The total equally divided between stages I-II and III-IV b numbers based on incidence numbers of Integral Cancer Centre in the Netherlands in 2012: 147

149 CHAPTER 8 RESULTS Different time gains were realized for the eight included patient groups (Table 1) varying from five to 22 gained days in waiting time. These resulted in the additional incremental QALYs varying between 0.13 to 0.66 (Table 3). The new care process (NWF) for patients with tumours located at the larynx, and the oral cavity was less expensive than the RWF, which resulted in negative incremental costs. The incremental costs for the NWF for oropharynx and hypopharynx tumours was small, varying from 304 to 453. As a result, ICERs presented a cost of approximately 2,875 to 3,777 for an additional gained QALY for patients with oropharynx tumours, and 2,119 to 2,909 for patients with hypopharynx tumours (Table 3 and visualized in Figure 2, and 3). The NWF was cost-effective for all studied treatment sites, using a ceiling ratio of 80,000. Summation of the organizational INMB per treatment site, and stage (Table 3), using a ceiling ratio of , resulted in an overall INMB of 14,1 million for the healthcare payer if the studied organization would implement the process redesign for all their patients treated for the included cancer sites. Extrapolation to the national level of the Netherlands, resulted in an overall INMB of 5,299,7 million for the healthcare payer. DISCUSSION The NWF was cost-effective for all included HNC patients. The NWF dominated the RWF for patients treated for cancer of the larynx, and the oral cavity, because of less costs, and gained QALYs. The NWF for patients with hypopharynx, and oropharynx cancers was also cost-effective with a ceiling ratio of 80,000. When society was willing to pay 2,500 for a gained QALY for patients with hypopharynx cancer, and 3,500 for oropharynx, the NWF was cost-effective. When INMB s were extrapolated to the organizational, and national level, the benefits of redesigning the workflow became very clear. An INMB of 14,1 million for the healthcare payers was calculated if the redesign was fully implemented in the studied organization and an INMB of 5,299,7 million if extrapolated to national level. However, this extrapolation of results is purely indicative. The hypothetical extrapolation to the national level is based on the assumption that the redesign of the relevant processes would be comparable for all care organizations in the Netherlands. Reality proves different. Therefore, this hypothetical extrapolation of the INMB only serves as an illustration of the potential benefits of relative small logistical process improvements. Differences in cost-effectiveness between tumour sites were partly based on the differences in implementation costs between the two organizational changes. The biopsy under local anesthesia (NWF) was less expensive than the regular biopsy under full anesthesia (RWF), while the NWF regarding the CT-PET process was more expensive than for the RWF. In addition, the redesigned biopsy 148

150 COST-EFFECTIVENESS OF REDESIGNED CARE PROCESS Figure 2. Cost-Effectiveness planes, and the Cost-Effectiveness Acceptability Curves (CEACs) of the redesigns for treatment of the larynx, and oropharynx separated per stages. The more points in the Cost-Effectiveness plane are located in the right below quadrant, the more times the new work flow (NWF) was cost-effective during the Monte-Carlo simulation. When in the CEACs, the NWF curve is presented above the regular work flow, the NWF is cost-effective for the given ceiling ratio (horizontal axis). 149

151 CHAPTER 8 Figure 3. Cost-Effectiveness planes, and the Cost-Effectiveness Acceptability Curves (CEACs) of the redesigns for treatment of the hypopharynx, and oral cavity separated per stages. The more points in the Cost-Effectiveness plane are located in the right below quadrant, the more times the new work flow (NWF) was cost-effective during the Monte-Carlo simulation. When in the CEACs, the NWF curve is presented above the regular work flow, the NWF is cost-effective for the given ceiling ratio (horizontal axis). 150

152 COST-EFFECTIVENESS OF REDESIGNED CARE PROCESS process resulted in larger effects on gained waiting time than the redesign of the CT-PET process, resulting in a larger number of gained QALYs. However, since the CEACs presented the NWF for all patient groups cost-effective at very small ceiling ratios, no additional one-way sensitivity analysis was performed to calculate the separate contributions of these uncertain parameters on the outcome. Some limitations of this study should be discussed. First, the relative risks for decreased waiting times were assumed equal for all eight sites, and stages, although these resulted in different transition probabilities between health states. Nevertheless, the effect of a decreased time to initiation of treatment could be considered more pronounced for the faster growing tumours within HNC. Second, one could question the assumption that the relative risk for decreased waiting time is linearly related to time, which directly affects the outcomes in the model. Third, the relative risks used to configure the health states transition probabilities of the patients with oropharynx tumours to hypopharynx, and oral cavity were assumed equal for survival, local/regional control, and metastases. This assumption, which could affect the outcomes, might be inconsistent with reality. Fourth, differences in QALYs between the NWF, and RWF might be underestimated since shortened time to treatment might result in smaller radiation treatment fields and, therefore, less radiation related toxicity (improved quality of life for the NWF). Fifth, time gains from the introduced workflow were based on preliminary results from a pilot study. These might not represent the time gains that are aimed for, since the organizations are still in a learning process, and might further improve the process to gain additional time. If waiting time is expected to further improve, the costeffectiveness of the NWF is likely to improve as well. Sixth, the decreased time before patients were diagnosed could result in reduced stress levels for patients, since living in uncertainty is extremely stressful, especially when cancer related. The potentially reduced stress level for the NWF was not included in the outcomes of the model. Seventh, a health care perspective was used for this model. All societal cost, and benefits regarding the loss of productivity at work were not included. Incorporating these costs might improve the costeffectiveness of the NWF, because QALYs were gained which could result in reduced loss of productivity. Positive effects of the redesigned processes have to be mentioned as well. Patient groups, which were not incorporated in the outcomes of the model, benefitted from the redesign as well. Because the biopsies of a number of patients were performed under local anesthesia, the operating room became available for other HNC patients. Patients with locally advanced tumours, who were not included in the trajectory for biopsy under local anesthesia or who were on the waiting list for surgery, experienced less waiting time before 151

153 CHAPTER 8 treatment. Another positive effect of the study was a possibly improved diagnosis, and treatment decision, because CT-PET images became available during the diagnostic process instead of afterwards, and in specific before the performed biopsy. Although health care professionals might be less attracted to process, and organizational redesign as compared to adapting new technologies, the present study demonstrated that the redesign of a care process resulted in significantly better long-term patient outcomes, and cost savings. The reduction of waiting time for treatment is shown to positively impact patient prognosis. 4,6,8 An additional problem with process redesign interventions and a more efficient use of medical technologies is the inappropriate use of financial motivators. More efficient use of technologies, and redesigned care processes, which may result in benefits for the patient, might not be rewarded by finances or even result in less payments. This is an important impediment for creating efficient, and high quality care. To support efficient, and qualitative care, organizations should be rewarded appropriately by the financers of care. This study proved that the reorganization of care processes can create qualitative, and efficient care and should be implemented on a much wider scale. In conclusion, a redesign intervention of the diagnostic process, and the radiotherapy preparation for HNC patients resulted in reduced waiting time for treatment, more QALYs, and proved to be cost-effective. 152

154 COST-EFFECTIVENESS OF REDESIGNED CARE PROCESS REFERENCES 1. Orszag PR, Ellis P. The challenge of rising health care costs--a view from the Congressional Budget Office. N Engl J Med 2007; 357: Sullivan R, Peppercorn J, Sikora K, et al. Delivering affordable cancer care in high-income countries. The Lancet Oncology 2011; 12: Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford University Press, Oxford, Chen Z, King W, Pearcey R, Kerba M, Mackillop WJ. The relationship between waiting time for radiotherapy and clinical outcomes: A systematic review of the literature. Radiotherapy and Oncology 2008; 87: Huang J, Barbera L, Brouwers M, Browman G, Mackillop WJ. Does delay in starting treatment affect the outcomes of radiotherapy? A systematic review. J Clin Oncol 2003; 21: Waaijer A, Terhaard CHJ, Dehnad H, et al. Waiting times for radiotherapy: consequences of volume increase for the TCP in oropharyngeal carcinoma. Radiotherapy and Oncology 2003; 66: Wadsley JC, Bentzen SM. Investigation of relationship between change in locoregional control and change in overall survival in randomized controlled trials of modified radiotherapy in head-and-neck cancer. Int J Radiat Oncol Biol Phys 2004; 60: Mackillop WJ, Bates JHT, O'Sullivan B, Withers HR. The effect of delay in treatment on local control by radiotherapy. International Journal of Radiation Oncology*Biology*Physics 1996; 34: Integral Cancer Centre the Netherlands. Oncoline: richtlijnen oncologische zorg. Assessed: May 2013, Castadot P, Geets X, Lee JA, Grégoire V. Adaptive functional image-guided IMRT in pharyngo-laryngeal squamous cell carcinoma: Is the gain in dose distribution worth the effort? Radiotherapy and Oncology 2011; 101: Delouya G, Igidbashian L, Houle A, et al. 18F-FDG-PET imaging in radiotherapy tumor volume delineation in treatment of head and neck cancer. Radiotherapy and Oncology 2011; 101: Egelmeer AG, Velazquez ER, de Jong JM, et al. Development and validation of a nomogram for prediction of survival and local control in laryngeal carcinoma patients treated with radiotherapy alone: a cohort study based on 994 patients. Radiother Oncol 2011; 100: Datema FR, Ferrier MB, van der Schroeff MP, Baatenburg de Jong RJ. Impact of comorbidity on short-term mortality and overall survival of head and neck cancer patients. Head Neck 2010; 32: Ramaekers BL, Joore MA, Grutters JP, et al. The impact of late treatmenttoxicity on generic health-related quality of life in head and neck cancer patients after radiotherapy. Oral Oncol 2011; 47:

155 CHAPTER Diemen. Health Care Insurance Board: Guidelines for pharmaco-economic research [in Dutch]. Health Care Insurance Board, Council for Public Health and Health Care. Sensible and sustainable care [in Dutch]. Council for Public Health and Health Care, Zoetermeer, Briggs AH, O'Brien BJ, Blackhouse G. Thinking Outside the Box: Recent Advances in the Analysis and Presentation of Uncertainty in Cost- Effectiveness Studies. Annual Review of Public Health 2002; 23: Integral Cancer Centre the Netherlands. Numbers about cancer. Assessed: May 2013, 154

156 COST-EFFECTIVENESS OF REDESIGNED CARE PROCESS Appendix. Cost input parameters of Markov model Costs Estimated value Se/ 95%CI Distribution Source RWF Event cost parameters of Markov States per Site, and Stage Larynx, Stage I-II costs of progression free 7,646 7,152-8,052 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 3,108 2,984-3,281 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,761 - Fixed MAASTRO clinic/mumc+, data on file Larynx, Stage III-IV costs of progression free 15,397 15,279-15,515 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 6,195 5,993-6,399 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,761 - Fixed MAASTRO clinic/mumc+, data on file Oropharynx, Stage I-II costs of progression free 11,889 11,830-11,948 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 10,119 9,489-10,717 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,793 - Fixed MAASTRO clinic/mumc+, data on file Oropharynx, Stage III-IV costs of progression free 15,002 14,942-15,060 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 4,972 - Fixed MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,728 - Fixed MAASTRO clinic/mumc+, data on file Hypopharynx, Stage I-II costs of progression free 11,682 11,623-11,740 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 11,226 10,831-11,318 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,728 - Fixed MAASTRO clinic/mumc+, data on file Hypopharynx, Stage III-IV costs of progression free 14,795 14,735-14,853 Beta PERT MAASTRO clinic/mumc+, data on file 155

157 CHAPTER 8 costs of local/regional recurrence 8,099 7,915-8,145 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,728 - Fixed MAASTRO clinic/mumc+, data on file Oral cavity, Stage I-II costs of progression free 4,172 3,380-5,333 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 13,799 13,000-14,956 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,761 - Fixed MAASTRO clinic/mumc+, data on file Oral cavity, Stage III-IV costs of progression free 20,148 19,540-20,776 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 7,529 7,226-7,842 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,761 - Fixed MAASTRO clinic/mumc+, data on file NWF Event cost parameters of Markov States per Site, and Stage Larynx, Stage I-II costs of progression free 7,068 6,569-7,468 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 2,523 2,401-2,696 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,761 - Fixed MAASTRO clinic/mumc+, data on file Larynx, Stage III-IV costs of progression free 15,120 14,937-15,351 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 5,612 5,409-5,816 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,761 - Fixed MAASTRO clinic/mumc+, data on file Oropharynx, Stage I-II costs of progression free 12,339 12,193-12,545 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 10,119 9,489-10,717 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,793 - Fixed MAASTRO clinic/mumc+, data on file Oropharynx, Stage III-IV costs of progression free 15,452 15,309-15,646 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 4,972 - Fixed MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,728 - Fixed MAASTRO clinic/mumc+, data on file 156

158 COST-EFFECTIVENESS OF REDESIGNED CARE PROCESS Hypopharynx, Stage I-II costs of progression free 11,988 11,841-12,192 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 11,226 10,831-11,318 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,728 - Fixed MAASTRO clinic/mumc+, data on file Hypopharynx, Stage III-IV costs of progression free 15,101 14,957-15,294 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 8,099 7,915-8,145 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,728 - Fixed MAASTRO clinic/mumc+, data on file Oral cavity, Stage I-II costs of progression free 3,588 2,797-4,751 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 13,696 12,890-14,864 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,761 - Fixed MAASTRO clinic/mumc+, data on file Oral cavity, Stage III-IV costs of progression free 20,015 19,386-20,682 Beta PERT MAASTRO clinic/mumc+, data on file costs of local/regional recurrence 6,946 6,643-7,258 Beta PERT MAASTRO clinic/mumc+, data on file costs of distant metastasis 3,761 - Fixed MAASTRO clinic/mumc+, data on file Health state costs Year 1 per Site and Stage follow up costs of progression free in the first year after treatment follow up costs of local/regional recurrence in the first year after treatment follow up costs distant metastasis in the first year after treatment Year 2 per Site and Stage follow up costs of progression free in the second year after treatment Fixed Fixed Fixed 93 - Fixed MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file 157

159 CHAPTER 8 follow up costs of local/regional recurrence in the second year after treatment follow up costs distant metastasis in the second year after treatment Year 3 per Site and Stage follow up costs of progression free in the third year after treatment follow up costs of local/regional recurrence in the third year after treatment follow up costs distant metastasis in the third year after treatment Year 4 per Site and Stage follow up costs of progression free in the fourth year after treatment follow up costs of local/regional recurrence in the fourth year after treatment follow up costs distant metastasis in the fourth year after treatment Year 5 per Site and Stage follow up costs of progression free in the fifth year after treatment follow up costs of local/regional recurrence in the fifth year after treatment follow up costs distant metastasis in the fifth year after treatment Fixed Fixed 93 - Fixed Fixed Fixed 46 - Fixed 65 - Fixed 65 - Fixed 46 - Fixed 65 - Fixed 65 - Fixed MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file MAASTRO clinic/mumc+, data on file 158

160 COST-EFFECTIVENESS OF REDESIGNED CARE PROCESS Costs NWF * 50 - Fixed Eo Costs of unnecessary actions for NWF number of unnecessary CT-PET's Beta Pilot study MAASTRO clinic cost for unnecessary diagnostic CT-PET per patient minus costs CT H&N and X-thorax * fraction of patients receiving the unnecessary CT-PETs Fixed MAASTRO clinic/mumc+, data on file number of unnecessary biopsy under local anesthesia costs for unnecessary local anesthesia: cost per surgery with local anesthesia * the fraction of patients receiving this unnecessary Beta PERT Fixed MUMC+, data on file Abbreviations: Se: standard error, CI: confidence interval, RWF: regular work flow, NWF: new work flow, Eo: expert opinion Eo a Costs of NWF were calculated by dividing the cost for purchase of equipment ( 50,000) by the expected number of patients (1000) to be treated with the equipment b cost for unnecessary diagnostic CT-PET per patient minus costs CT H&N and X-thorax * fraction of patients receiving the unnecessary CT-PETs c cost per surgery with local anesthesia * the fraction of patients receiving this unnecessary 159

161 160

162 CHAPTER 9 Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour Does a lean transition decrease waiting times and improve outcomes for patients and employees? Submitted Pascale Simons, Huub Backes, Jochen Bergs, Davy Emans, Madelon Pijls-Johannesma, Maria Jacobs, Wim Marneffe, and Dominique Vandijck.

163 CHAPTER 9 ABSTRACT Background and Purpose Treatment delays should be avoided, especially in radiotherapy, given its importance on tumour control and survival. Lean initiatives which aim to improve both quality and efficiency may, therefore, become increasingly important. We hypothesized that waiting times would decrease and patients and employees would benefit following a lean transition. Material and Methods In 2013, 15 projects were initiated to improve flow in a radiotherapy institute. Process/waiting times, patient satisfaction, safety, employee satisfaction and absenteeism were evaluated by mixed methods from 2010 to Data were extracted from databases, surveys and interviews. Time series analysis, chisquare tests, multi-level regression and t-tests were used for analysis. Results Median process times only tended to improve slightly, which was not proven significant. However, the percentage of palliative patients of which the waiting time had exceeded the Dutch national norm (10 days) improved from 27% to 16% and the percentage exceeding the national objective (7 days) improved from 25% to 17%. For curative patients, exceeding of norms (28 days) did not change (7%), but exceeding of objectives (21 days) improved from 14% to 10%. Reported safety incidents decreased with 50% between 2009 and 2013, and safety culture improved, as well as awareness and intention to solve problems. Employee satisfaction improved slightly and absenteeism decreased from 4.6% (2010 to 20120) to 3.6% in 2013 (P<0.001). Conclusions Waiting times for radiotherapy decreased slightly due to lean initiatives, without negatively affecting patient safety, employee satisfaction and absenteeism. This could be a potential motivator to strive for continuous improvement. 162

164 DOES LEAN IMPROVE PATIENT & EMPLOYEE OUTCOMES? INTRODUCTION Demand for radiotherapy has been growing, quality standards have been increasing, and health care costs rising 1-4. Patients are increasingly demanding individualized high quality care without waiting time. To meet the patients demands and remain financially healthy, the need for strategies that combine sustainable quality improvement with increased efficiency are more stringent than ever. Within radiotherapy, quality assurance has a long history and patient safety is incorporated in all processes 5,6. In the current perceptions in healthcare safety is inherent to additional activities and costs. Therefore, safety management might negatively affect waiting times. Although waiting time is unwanted in every field of healthcare, it is even more fundamental in radiotherapy, as local tumour control and survival rates are negatively affected by increased waiting time, especially for specific tumour sites (e.g. breast, head and neck cancer) 7,8. In addition, costs of radiotherapy have to be managed closely. Although, the rapid innovation of technology provides increased opportunities for treatment, it also brings about increasing costs 3. Therefore, the need for strategies to improve quality of care, waiting times and efficiency has become increasingly important within the oncology care. Lean management is a strategy that aims to improve both quality and efficiency. It was originally developed within the Toyota Motor Company, and later diffused within the global car manufacturing industry 9. This management philosophy has also been adopted in healthcare Lean focuses on the redesign and improvement of processes towards a customer-focused ideal state aiming to minimize waste, defined as tasks that do not directly benefit the patient. Waste leads to unnecessary more complex processes, which can increase costs and employees frustrations and in turn results in loss of quality and safety. Lean uses multiple methods to minimize waste like value stream management, which visualizes the entire patient process with all its bottlenecks, and Kaizen, which implies daily improvement involving all employees. It encloses a long term vision aiming for continuous improvement to create flow for the patient combined with committed employees 13. Besides promising results of lean, negative aspects of this improvement strategy are also publicized. Lean could negatively affect job satisfaction, due to monotonous and repetitive work and has been used for plain downsizing 14. However, lean advocates argue that lean is beneficial for employees, particularly because employees are increasingly involved in improvements regarding their work processes. Although this discussion is still ongoing at present 15-17, an increasing number of healthcare organizations are implementing lean principles. Although safety already has a prominent role within radiotherapy, the lean philosophy might further improve quality and efficiency and deal with the unwanted waiting times before treatment start. We hypothesized that waiting times would decrease due to a lean transition in a radiotherapy setting. Since satisfied patients and 163

165 CHAPTER 9 employees are prerequisites of sustainable high quality care and only and only few studies have included these 18-20, we included these in this effect evaluation in addition to waiting times. MATERIAL AND METHODS Setting This study was performed in the MAASTRO clinic, a radiotherapy department in the Netherlands, where on average 200 oncology patients receive radiation therapy every day. Since 2003 the organization has been engaged to improve patient safety and in 2007 they formalized all safety related interventions in a certified safety management system. To further improve quality and safety and simultaneously increase efficiency some lean based projects were initiated since In 2010, the lean philosophy was up taken in the organizations long term strategy to aim for individualized high quality care. The organization has chosen for a multifaceted strategy to maximize the chance for successful implementation, since 70 percent of implemented change initiatives in healthcare fail 21. The management structure was reorganized from a functional to a process oriented management in January 2011 (Figure 1). Multiple disciplines had one integrated management. Multidisciplinary teams were initiated for the different care paths (e.g. breast, head and neck) to facilitate and improve process innovation. Every care path team was managed by a clinical professional and were held responsible for formulating policy objectives, evaluate and communicate results to colleagues and the board. To increase commitment and create a shared vision among employees, the board implemented roadshows in 2012: board members discussed the long term vision with all employees in multiple small group meetings. Employees had the chance to interact and participate in the creation of a shared vision. From 2011 to 2012, all treatment equipment (seven accelerators) and planning software were replaced by a new integrated system with six accelerators, which also resulted in less patient safety incidents. However, during the implementation phase of the new equipment, all professionals had to receive intensive training, which asked a lot of flexibility and energy from employees and clinical time. The new planning software provided the opportunity to increase the number of intensity modulated radiotherapy treatment (IMRT) plannings, which inevitably created increased waiting times for patients. In 2013, 15 projects were initiated to improve process flow and decrease waiting times for the patients, see Appendix for a short summary of projects. Some specific examples of these flow projects were: 1) creating one-piece-flow at the administrative department. Instead of batching tasks to schedule a patient and wait until all documentation was present, employees completed all tasks for every individual patient at once. 164

166 DOES LEAN IMPROVE PATIENT & EMPLOYEE OUTCOMES? Figure 1. Timeline of project. All implemented actions are presented beneath the timeline and all different measurements above the timeline. 2) Redesigning the initial consultation with the oncologist. The consultation process was optimized by reducing waste and several oncologist s tasks were reassigned to an assistant, like completing the demographics in the patient s medical file, informing the patient about the radiation process and completing standard administrative tasks. This should result in a decreased time required for the initial consultation, which again could increase the availability of time slots for scheduling an initial consultation. 3) Creating a drop-in CT: instead of scheduling the CT after the consult with the oncologist, patients could choose to wait and have the CT immediately after the consult. In 2010, 96 clinical professionals used seven accelerators to treat on average the same number of patients (on average 250 patients per month) as in 2014 by 86 professionals using six accelerators. Process related measurements Process times were extracted from databases containing all patient schedules from January 2010 to April 2014 (Figure 1) and were based on the required time for the entire process flow; from the first multidisciplinary meeting where the patients diagnosis and the treatment options are discussed to the patient s first irradiation for all treated patients. In addition these overall process times were 165

167 CHAPTER 9 separated in six sub-processes. The median process times per month and the 25 and 75 percentiles were reported and process times were separated per care path to detect possible differences. A detailed view for the sub-processes was reported by the monthly mean process times from January 2012 to April In the Netherlands, the Dutch Association for Radiotherapy and Oncology (NVRO) has set norms and objectives regarding the waiting times for oncology patients in general and for palliative patients in particular (Table 1). 22 In addition to the medians, the monthly percentages of treated patients who exceeded these national norms and objectives were reported from January 2010 to April To statistically compare time trend before and after the initiated projects to improve flow, time series analysis (Auto-Regressive Integrated Moving Average (ARIMA) model) was used. 23,24 The non-seasonal variant of the ARIMA model was used because not enough measures were recorded. Patient related measurements Patient safety Since it is difficult to detect patient safety end-outcomes, these should be combined with surrogate end-points like process evaluation, safety culture and errors (incidents). 25,26 To increase understanding in the measured effects, quantitative and qualitative measures are combined Since patient safety end-outcomes are extremely difficult to measure in the radiotherapy setting, the number of incident reports were combined with results on patient safety culture surveys. These quantitative data were further explored by structured interviews with clinical professionals. 32 The number of reported patient safety incidents were extracted from an incident reporting system (IRS) from January 2004 to December These incidents were differentiated between: 1) misses defined as incidents that have reached the patient-level and could have potentially harmed the patient and 2) near misses defined as incidents that have not reached the patient-level, which include incidents that have been corrected in time, but also reports on process deviations that do not necessarily impact the patient directly. In addition to patient safety incidents the patient safety culture was evaluated by two surveys. The surveys were each distributed three times between 2010 and 2013 (Figure 1). A factorial survey was used to evaluate safety awareness and intentions for safety behavior. This design is traditionally used to analyze human judgment. Respondents are presented questions based on detailed hypothetical descriptions (eg. vignettes), in which several variables, expected to influence judgment, are systematically varied Respondents were presented twenty scenarios about incidents followed by one question about safety awareness and four about intentions for safety behaviour. 36 To evaluate the safety climate the validated Hospital Survey On Patient Safety Culture (HSOPSC) was used 37, covering four overall safety outcomes and ten dimensions of safety climate. 36 Chi-squares were used to analyze HSOPSC results and the factorial survey 166

168 DOES LEAN IMPROVE PATIENT & EMPLOYEE OUTCOMES? results were analyzed by multi-level regression. Interviews were fully typed and coded with specialized software (NVIVO). Inductive reasoning was used for coding, to detect trends and interpret the quantitative results. Patient satisfaction Patient satisfaction was measured by a survey four times between 2010 and However, in 2013 a different survey was used than the forgoing years. In 2013, the national consumer quality index for radiotherapy was used. 38 Between 2010 and 2012 a organization specific survey addressed six processes: journey and parking facilities, service at reception, first consult with assistant, intake with oncologist, preparation process, and radiation treatments. All patients who finished treatment in a two to three month period were sent an anonymous questionnaire, including an envelope to return by post. For the measures of 2010 to 2012, unpaired t-tests were used to test differences between the average satisfaction scores for the six patient related processes. For the measure of 2013 only descriptive analyses were used. Employee related measurements Employee satisfaction The satisfaction among employees was measured by a survey on seven dimensions: salary, professional status, relation, administration, autonomy, task requirements and interaction. 39,40 Unpaired t-tests were used to test differences between 2010 and Absenteeism The percentage of employee absenteeism has been recorded from 2010 to Pregnancy and maternity leave were not included in the percentage of absenteeism. The percentage of absenteeism was compared to the national level of employee absenteeism in healthcare. 41 Monthly average percentages were presented. RESULTS Process related outcomes The median waiting times, time between the first multidisciplinary meeting where the patients diagnosis and the treatment options are discussed to the patient s first irradiation, tended to increase in In the last half of 2013, after the initiation and implementation of the flow projects (Figure 2), the median waiting times tended to decrease slowly, and the variation between patients waiting times tended to decrease. However, this decrease proved not statistically significant. Median waiting times differed between the varying care 167

169 CHAPTER 9 paths, but for none a relevant decrease of median waiting times was presented. The results presented large variations and many outliers. Days between first multidisciplinary meeting and first irradiation Increased number IMRT plannings Flow Projects Figure 2. Median process times / month of the total administrative and preparation process. Median process times (black line) and 25- and 75-percentiles (grey lines) are presented in days over a four year period. All acute patients are treated within 24 hours and were not included in these results. The percentage of patients of who the waiting time had exceeded the Dutch national norm (28 days for curative patients or 10 days for palliative patients) and/or the national objective (21 days for curative patients or 7 days for palliative patients) increased in 2012, but decreased in the second half of 2013 (Figure 3). The decrease in percentage of palliative patients exceeding the national norms for acceptable waiting time was statistically significant based on the ARIMA results (Table 1). For the patients treated with curative intention the percentage of patients exceeding the objective (21 days) also decreased significantly. However, the decrease in percentage of curative patients exceeding the norm of 28 days did not reach the 5% level of significance. The detailed results of the waiting/process times (Figure 4) showed that for palliative patients waiting times decreased most between: the first multidisciplinary meeting and the registration of the patient; intake consult with the oncologist and the scheduling of the CT (-PET); scheduling and actual performing the CT(-PET), but also in the clinical preparation process for radiotherapy (CT (-PET) to first treatment, including the planning of treatment and all checks). Although all waiting/process times for patients treated with curative intention were larger, similar improvements in the administration process were shown. However, the waiting /process times for the actual clinical preparation process did not show relevant improvements for these patients. 168

170 DOES LEAN IMPROVE PATIENT & EMPLOYEE OUTCOMES? Table 1. ARIMA results for the percentage of patients who exceeded the national norms and objectives for waiting times in radiotherapy. Results were distinguished between patients receiving palliative treatment and the patients treated with curative intention. Acute patients were treated within 24 hours and were not included in this study. Palliative exceeding norm* (>10days) Palliative exceeding objective* (>7days) Curative exceeding norm* (>28days) Curative exceeding objective* (>21days) Effect P-value Effect P-value Effect P-value Effect P-value Time period a Interact b Phase c month 1 (July 2013) Phase c month 2 (August 2013) < Phase c month 3 (September 2013) < <0.001 Phase c month 4 (October 2013) < <0.001 Phase c month 5 (November 2013) < <0.001 Phase c month 6 (December 2013) < <0.001 Phase c month 7 (January 2014) < <0.001 Phase c month 8 (February 2014) < <0.001 Phase c month 9 (March 2014) < <0.001 Phase c month 10 (April 2014) < <0.001 * Norms and objectives of the Dutch Association for Radiotherapy and Oncology (NVRO) were used. a effect before the intervention: pre-slope b difference between pre-slope and post-slope c effect per month after the intervention 169

171 CHAPTER 9 70% Palliative patients Exceeding National norm (>10days) Increased number IMRT plannings Flow Projects 60% Exceeding National objective (>7 but <10days) 50% 40% 30% 20% 10% 0% % Curative patients Exceeding National norm (>28days) Increased number IMRT plannings Flow Projects 35% Exceeding National objective (>21 but <28days) 30% 25% 20% 15% 10% 5% 0% Figure 3. Percentage of patients of who the process time exceeded the Dutch national norms and objectives for waiting time per month. The figure on top presents the percentages for palliative patients and the figure beneath for curative patients. Since all acute patients are treated within 24 hours, these were not included in the figure. In addition to the waiting times the flow projects resulted in several improvements, which had not been quantified in detail. For instance, due to the redesign of the initial consultation with the oncologist, oncologists could better prepare, because all relevant clinical information were available at the time needed. The missing clinical information decreased to zero and patients were better informed. The drop-in CT resulted in less administration for professionals and decreased travel time for patients. 170

172 DOES LEAN IMPROVE PATIENT & EMPLOYEE OUTCOMES? 10 Palliative patients Administration Preparation Multidisciplairy meeting => Patient registration Patient registration => Schedule intake Schedule intake => Intake oncologist Intake oncologist => Schedule CT (-PET) Schedule CT (-PET) => CT (-PET) CT (-PET) => First treatment 10 Curative patients Administration Preparation Multidisciplairy meeting => Patient registration Patient registration => Schedule intake Schedule intake => Intake oncologist Intake oncologist => Schedule CT (-PET) Schedule CT (-PET) => CT (-PET) CT (-PET) => First treatment Figure 4. Mean process times (in days) for six separate process steps including waiting time. The first five represent administrative processes and the last the clinical preparation process. The figure on top presents the result for the palliative patients and the figure beneath for the remaining patients treated with curative intention. The acute patients are treated within 24 hours and were not included in these results. The dark bars represent the mean days of monthly scores after the projects aimed to improve process flow. Patient related outcomes Patient safety The annual number of reported incidents increased from 510 in 2004 to 1835 in 2009 (Figure 5). However, the number of reported incidents that reached patient-level (misses) decreased with 27%, from 122 in 2004 to 89 in The increased number of near misses was almost entirely based on an increase of accelerator related incidents. Therefore, the team of professionals trained in patient safety ascribed the increased number of reports to an increased willingness to report by clinical professionals. From 2009 the number of reported incidents decreased with 50% to 918 in The number of reported misses 171

173 CHAPTER 9 Figure 5. The number of reported incidents from 2004 to Reports were separated in incidents reaching the patient-level (misses) or not (near misses). potentially harmful for patients also decreased with 46% from 89 in 2009 to 48 in The patient safety culture improved from 2010 to 2013, based on results from the surveys and confirmed by the interviews with professionals. 36 Six out of twelve safety culture dimensions from the HSOPSC improved, but the frequency to report decreased. Based on results from the factorial survey, especially the intention to structurally improve increased in Employees experienced a sustained safety awareness, improved quality of reports and a strong increase in creating structural improvements. Therefore, the decreased number of reported incidents was not explained by decreased willingness to report, but by improved treatment processes due to improvements in equipment and increased problem solving. Patient satisfaction In 2010, 282 questionnaires were returned, in and in The mean scores on patient satisfaction on the six processes of treatment varied between 7.9 to 8.5 on a scale from one to ten, with an equal average of 8.3 for every year. In 2013, the national survey for consumer quality index for radiotherapy 38 was sent to 58 patients and 38 were returned. The respondents valued their overall satisfaction on average 9.1 on a scale from one to ten. 172

174 Employee related outcomes DOES LEAN IMPROVE PATIENT & EMPLOYEE OUTCOMES? Employee satisfaction In 2010, 75 of approximately 120 send questionnaires were returned, in , in 2013 and in The seven surveyed dimensions regarding employee satisfaction showed no changes for the oncologists (Table 2). The radiation technologists scored least positive on most dimensions, closely followed by the administrative employees. However, technologists also showed most improvements during the studied years. No changes regarding the salary or other compensating conditions were implemented. Absenteeism From January 2010 to June 2012, the percentage of absenteeism varied from 2.5% to 7.2% (4.6% on average), which was representative for the Dutch national average percentage of absenteeism in healthcare (5.1%). 41 In 2013 the average absenteeism had decreased to 2.6% (P<0.001). DISCUSSION During 2013, 15 multidisciplinary projects were initiated to increase process flow and decrease waiting times for radiotherapy treatment. These projects were preceded by several actions to improve commitment, create a shared vision and facilitate change. Care path teams were created directed by clinical leaders and top-management discussed their vision with all employees in small group sessions. After the implementation of new planning software in 2012, the number of IMRT plannings increased resulting in increased waiting times for patients to start their treatment. Since the waiting times exceeded the Dutch national norms and objectives 22, professionals experienced a strong sense of urgency to improve process times. Since the basic conditions for creating change (i.e. highly committed employees with a shared vision experiencing a strong urge to change) were met, change was facilitated by freeing professionals from their clinical responsibilities. This resulted in 15 initiated flow projects. The overall process times presented only small improvements in median process times between the first patient related action and the actual start of treatment. However, the variation in process times decreased. This resulted in a decreased percentage of patients exceeding the national norms and objectives for palliative and curative patient treatments during the second half of Process times of the administrative processes decreased for palliative as well as curative treatments. However, the time required for the clinical preparation process of radiotherapy improved for palliative patients only. For the curative treatments only small improvements in waiting time were realized. The organization of the oncologists time, to create continuous flow and not create batches with stocked patient files, was most problematic. The realized improvements might therefore 173

175 CHAPTER 9 Table 2. Employee satisfaction measured on seven dimensions for three years. Scores are presented as means on a scale from one to seven. Oncologists Clinical physicists Radiation technologists Administrative employees Received surveys Salary c b 3.9 d a d Professional status d Relation Administration a d Autonomy a d Task requirements b b 4.6 d Interaction d Overall Score b 4.6 d a p<0.05 for differences between 2010 and 2012 b p<0.05 for differences between 2012 and 2013 c p<0.05 for differences between 2013 and 2014 d p<0.05 for differences between 2010 and

176 DOES LEAN IMPROVE PATIENT & EMPLOYEE OUTCOMES? only be just a small win, but might motivate professionals to go forward and further improve their processes. Although skeptics of the lean philosophy claim that employee satisfaction would decrease and work pressure would increase, which could result in an increased level of absenteeism, this study showed no such results. The radiation technologists even experienced improved satisfaction regarding to salary, professional status, administration, autonomy and task requirements. This was probably not a direct result from the lean strategy, but from lean principles combined with an organizational model to increase employees commitment and a specific strategy to improve organizational culture. The percentages of absenteeism also represented a positive change. Again, this improvement might not be the result from the implemented lean actions, but from an intensified policy against absenteeism. However, increased levels of absenteeism were not measured. Patient safety incidents kept decreasing. One could question the reliability of the number of incident reports as an outcome indicator for patient safety, since it is based on voluntary reporting. 42,43 Therefore, these numbers were complemented by the measurement of the safety climate, employees safety awareness and intentions for safety behavior, which all presented sustainable results. The employees intentions to solve problems had significantly improved. 32 Since employees experienced increased possibilities to structurally solve problems, employee satisfaction related to professional status and autonomy might have positively been affected. The lean activities did not affect patient satisfaction. Patients were already highly satisfied in the start of the study, which did not change. One could question the importance of patient satisfaction as a relevant outcome indicator of the quality of patient care, but still it should not be disregarded since this is an important end-outcome on the patient-level. One could carefully conclude that the quality of care improved after the lean transition since the waiting times were slightly reduced, which has a direct positive impact on the prognoses of treated patients 44. In addition, the decreased waiting time can possibly reduce the level of anxiety and stress among patients, since the uncertain period is shortened for the oncology patients and their relatives. However, only moderate improvements of waiting time were measured and the lean based projects cannot be evaluated separately from the managerial changes. Only a relative short period of time after the implementation of the lean based projects was covered. Therefore, sustainability of results could not be evaluated. Although no strong conclusions can be drawn, the implemented multifaceted organizational approach combined with the lean based projects might be a successful strategy to realize improvement. Evaluation on the longer term could determine the sustainability of results and detect if the studied approach in the 175

177 CHAPTER 9 end leads to continuous improvement, since this is the ultimate goal of lean management. Conclusion In conclusion, the percentage of patients who exceeded the Dutch national norms and objectives for waiting time significantly decreased. This indicated improved waiting times, owing to the lean based actions combined with an organizational strategy to improve commitment, shared visions and facilitation for change. The improved waiting times did not result in negative side-effects for patients and employees. On the contrary, patient safety, employee satisfaction and the percentage of absenteeism had also improved. Although the improvements in waiting times were moderate, these could be a potential motivator for professionals to go forward and strive for continuous improvement. 176

178 DOES LEAN IMPROVE PATIENT & EMPLOYEE OUTCOMES? REFERENCES 1. Orszag PR, Ellis P. The challenge of rising health care costs--a view from the Congressional Budget Office. N Engl J Med 2007; 357: Slotman BJ, Vos PH. Planning of radiotherapy capacity and productivity. Radiotherapy and Oncology 2013; 106: Van de Werf E, Verstraete J, Lievens Y. The cost of radiotherapy in a decade of technology evolution. Radiotherapy and Oncology 2012; 102: Williams MV, Summers ET, Drinkwater K, Barrett A. Radiotherapy dose fractionation, access and waiting times in the countries of the UK in Clinical oncology Royal College of Radiologists Great Britain 2007; 19: Clark BG, Brown RJ, Ploquin JL, Kind AL, Grimard L. The management of radiation treatment error through incident learning. Radiotherapy and Oncology 2010; 95: Shafiq J, Barton M, Noble D, Lemer C, Donaldson LJ. An international review of patient safety measures in radiotherapy practice. Radiotherapy and Oncology 2009; 92: Mackillop WJ. Killing time: The consequences of delays in radiotherapy. Radiotherapy and Oncology 2007; 84: Mackillop WJ, Bates JHT, O'Sullivan B, Withers HR. The effect of delay in treatment on local control by radiotherapy. International Journal of Radiation Oncology*Biology*Physics 1996; 34: Womack JP, Jones DT, Roos D. The Machine that Changed the World. Free Press, New York, Endsley S, Magill MK, Godfrey MM. Creating a lean practice. Fam Pract Manag 2006; 13: McCarthy M. Can car manufacturing techniques reform health care? Lancet 2006; 367: Rutledge J, Xu M, Simpson J. Application of the Toyota Production System improves core laboratory operations. Am J Clin Pathol 2010; 133: Liker JK. The Toyota Way. McGrawHill, New York, Benders J, Van Bijsterveld M. Leaning on lean: the reception of a management fashion in Germany. New Technology, Work and Employment 2000; 15: Conti R, Angelis J, Cooper C, Faragher B, Gill C. The effects of lean production on worker job stress. International Journal of Operations & Production Management 2006; 26: Hasle P, Bojesen A, Jensen PL, Bramming P. Lean and the working environment: a review of the literature. International Journal of Operations & Production Management 2012; 32:

179 CHAPTER Schouteten R, Benders J. Lean Production Assessed by Karasek s Job Demand Job Control Model. Economic and Industrial Democracy 2004; 25: Vest JR, Gamm LD. A critical review of the research literature on Six Sigma, Lean and StuderGroup's Hardwiring Excellence in the United States: the need to demonstrate and communicate the effectiveness of transformation strategies in healthcare. Implement Sci 2009; 4: Nicolay CR, Purkayastha S, Greenhalgh A, et al. Systematic review of the application of quality improvement methodologies from the manufacturing industry to surgical healthcare. Br J Surg DelliFraine JL, Langabeer JR, 2nd, Nembhard IM. Assessing the evidence of Six Sigma and Lean in the health care industry. Qual Manag Health Care 2010; 19: Beer M, Nohria N. Cracking the code of change. Harvard Business Review 2000; 78: Nederlandse Vereniging voor Radiotherapie - Indicatoren. [Objectives and norms for waiting times in radiotherapy]. assessed April 2013, Eccles M, Grimshaw J, Campbell M, Ramsay C. Research designs for studies evaluating the effectiveness of change and improvement strategies. Qual Saf Health Care 2003; 12: Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. Acad Pediatr 2013; 13: S Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 3. End points and measurement. Qual Saf Health Care 2008; 17: Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 4. One size does not fit all. Qual Saf Health Care 2008; 17: Cooper Ph.D MD. Towards a model of safety culture. Safety Science 2000; 36: Dixon-Woods M, Leslie M, Bion J, Tarrant C. What Counts? An Ethnographic Study of Infection Data Reported to a Patient Safety Program. Milbank Quarterly 2012; 90: Kaboli PJ, Mosher HJ. Using balanced metrics and mixed methods to better understand QI interventions. BMJ Quality & Safety 2014; 23: Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Qual Saf 2013; 22: Vincent C, Burnett S, Carthey J. The measurement and monitoring of safety. London, Simons PAM, Houben R, Vlayen A, et al. Does lean management improve patient safety culture? An extensive evaluation of safety culture in a radiotherapy institute. Eur J Oncol Nurs in press. 178

180 DOES LEAN IMPROVE PATIENT & EMPLOYEE OUTCOMES? 33. Jasso G. Factorial Survey Methods for Studying Beliefs and Judgments. Sociological Methods & Research 2006; 34: Rossi PH, Anderson AB. An Introduction. In: Rossi PH, Nock SL, editors. Measuring Social Judgments: The Factorial Survey Approach. Sage Publications, Beverly Hills, p Wallander L. 25 years of factorial surveys in sociology: A review. Social Science Research 2009; 38: Simons PAM, Houben R, Vlayen A, et al. A Factorial Survey on intentions for safety behaviour providing opportunities to improve safety. in review. 37. Blegen MA, Gearhart S, O'Brien R, Sehgal NL, Alldredge BK. AHRQ's hospital survey on patient safety culture: psychometric analyses. J Patient Saf 2009; 5: Zuizewind C, Hendriks M, van der Hoek L, Plass A. CQ-index Radiotherapie. NIVEL, Utrecht, Stamps PL, Piedmont EB, Slavitt DB, Haase AM. Measurement of work satisfaction among health professionals. Med Care 1978; 16: Stamps PL, Piedmonte EB. Nurses and Work Satisfaction: An Index for Measurement. Health Administration Press Perspectives, Ann Arbor, Rijksinstituut voor Volksgezondheid en Milieu; Ministry of Health WaS. Gezondheidszorgbalans. assessed May 2013, rbeidsverzuim/. 42. Capuzzo M, Nawfal I, Campi M, Valpondi V, Verri M, Alvisi R. Reporting of unintended events in an intensive care unit: comparison between staff and observer. BMC Emerg Med 2005; 5: Leape LL. Reporting of Adverse Events. New England Journal of Medicine 2002; 347: Chen Z, King W, Pearcey R, Kerba M, Mackillop WJ. The relationship between waiting time for radiotherapy and clinical outcomes: A systematic review of the literature. Radiotherapy and Oncology 2008; 87:

181 CHAPTER 9 Appendix. Summary of the 15 lean based projects to improve overall process times and flow Improve process times from multidisciplinary meeting to Computerized Tomography (CT) 1. Improve patient registration: multidisciplinary meeting mail phone 2. Visualize administrative work flow to improve clarity for all professionals 3. Reduce time needed for initial consultation with oncologist 4. Combine patient s appointment for initial intake and CT 5. Reduce process times for palliative patients in specific Improve process times from CT to first treatment 6. Improve check processes for the preparation phase 7. Improve scheduling process for treatment planning and visualize priorities 8. Improve process times and stability for preparation phase 9. Design an interface for combining data from white board (which visualizes the to-do-list of processes) with staffing Visualize management indicators 10. Redesign management dashboard and automate reports to make work load prediction possible 11. Optimize visualization of processes in preparation phase (digital white board) Improve technical systems relevant for the clinic 12. Explore a work flow management system and improve or replace scheduling system 13. Explore the use of smart-segmentation for treatment planning of head and neck cancer 14. Gradual transition to new treatment planning software 15. Make all relevant applications and software compatible to the new work flow 180

182 CHAPTER 10 Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour Does lean in healthcare reduce ambiguity of objectives and clarify uncertainties of cause-effect relations? A mixed methods approach Submitted Pascale Simons, Jos Benders, Jochen Bergs, Wim Marneffe, Dominique Vandijck.

183 CHAPTER 10 ABSTRACT Introduction Many change initiatives in healthcare fail to create sustainable improvement. The large ambiguity of objectives and uncertain cause-effect relations in care processes are likely impediments for realizing sustainable quality improvement. We investigated if lean management would positively affect these concepts. As a result, the organization could become more amenable for quality improvement initiatives. Methods In 2014, twelve professionals from a Dutch radiotherapy institute were interviewed regarding their perceptions of an initiated lean program in their organization. A survey (25 questions), addressing the same concepts, was conducted among the interviewees in 2011 and The structured interviews were analyzed using a deductive approach. Quantitative data was analyzed using appropriate statistics. Results Interviewees expressed an improved shared vision and reduced uncertainty of cause-effect relations. Overall more positive (99) than negative effects (18) of lean were expressed. The surveys revealed enhanced process predictability and standardization, and improved shared visions. Conclusion Lean management positively affected the existing ambiguity of objectives and reduced uncertainties of cause-effect relations in clinical processes. Therefore, decision making benefitted from lean and the chance to realize sustainable quality improvements increased. Although the improved processes are part of the organizational whole, much more remains to be improved. Specifically, clinical professionals remained unsatisfied regarding the transparency of the organizational communication. 182

184 DOES LEAN REDUCE GOAL AMBIGUITY & UNCERTAINTIES OF CAUSE-EFFECTS? INTRODUCTION Quality and safety are highly valued in health care. To improve quality and safety of care, many initiatives are commenced, although, many change initiatives fail to create sustained improvement. 1-5 The causes for this high failure rate are not yet clear. Without doubt, the implementation process will play an important role in the success of change strategies. Adapting the improvement strategy to the organizations characterization of decision making might be important to successfully implement sustainable improvements. As organizational decision making is relevant to create sustainable quality improvement, it should be measured and considered when implementing quality improvement programs. Organizational decision making is, according the Thompson and Tuden (1959) framework, defined by two conditions: 1) ambiguity of objectives or the lack of a shared vision among employees and 2) uncertainty of cause-effect relations. 6 The combinations of these two conditions classify four decision making contexts (Figure 1) for which different decision making and improvement approaches should be used. An organization with high ambiguity regarding organizational objectives, combined with many uncertainties regarding the outcomes of processes is difficult to manage. Figure 1. Thompson and Tuden (1959) framework for organizational decision making. 183

185 CHAPTER 10 Decision making, and therefore creating change, in this context is largely based on inspiration (i.e. Decision by Imagination). In contrast, if ambiguity and uncertainty are low it becomes clear why things should be done or one should have to change (i.e. Decision by Computation). In this context, decision making can be facilitated by formulating standard operating procedures (SOPs), complemented with indicators to measure and monitor outcomes. 7 Although many quality improvement initiatives intervene on the computation context of decision making, most health care settings do not fit this context. First, different objectives are present, owing to a complex relation between relevant stakeholders (e.g. patients, health care insurers, politics, etc). In addition, professional groups and departments often experience conflicting goals and objectives. 7 Improvement projects often emphasize these conflicts of interest, stretching out the importance of managements role. Hence, to increase successful quality improvement, new approaches should be inclined to decrease the ambiguity of objectives and reach consensus. 6 Second, healthcare professionals are primarily focused on processes affecting their work and are highly educated and trained in specific fields of care. This often limits their individual knowledge to specific stages of the patient process resulting in information asymmetry, which in turn can jeopardize mutual understanding and respect. 8 Third, cause-effect relations are not always clear in health care. It is often difficult to predict with certainty the outcomes of the actions taken. This may result in an excuse culture where professionals are not willing to take responsibility for their actions. 8 Although many care organizations have to deal with conflicting objectives and unclear cause effect relations, most initiated improvements strategies do not consider these complex problems. This might explain why such a high percentage of improvement initiatives in health care fail. To realize sustainable quality and safety improvement in health care, evidence for multi-faceted approaches is growing. 9 Lean management is an example of a multi-faceted approach that aims to improve quality, safety, and efficiency. It was originally developed within the Toyota Motor Company and has also been adopted in healthcare Lean primarily focuses on the improvement of processes to a customer (i.e. patient) focused ideal state. 13 However, it also defines some basic requirements to realize improvement, like standardizing processes and visual management. These form the fundaments of the Toyota Production System (Figure 2). These concepts are expected to decrease the level of uncertainty of process outcomes by making processes more transparent and stable. Common goals and Ringi (shared) decision making are central elements in the system, which address the ambiguity of objectives. They aim to increase employee commitment to the common goals, decrease information asymmetry by open communication, and increase mutual understanding between colleagues. Lean management is an approach that strives to clarify goal 184

186 DOES LEAN REDUCE GOAL AMBIGUITY & UNCERTAINTIES OF CAUSE-EFFECTS? Figure 2. The Toyota Production System ambiguity and increase insight into the cause-effect relationships. But does it succeed? We evaluated the effects of a lean-inspired change program in a radiotherapy institute on goal ambiguity and insight into the causality of its care processes to provide an answer to this question. METHODS Quantitative survey results were combined with structured interviews to evaluate the organizations characterization of decision making, in terms of goal ambiguity and clarity about causal relationships, during a lean transition. In 2011 and 2014, twelve employees throughout the organization completed a survey. In 2014, these professionals were interviewed regarding their perceptions on the related concepts. Every interview was preceded by a survey to prepare the interviewee to the possibly unfamiliar concepts addressed in the interview and to detect quantitative comparisons and trends between the interviewees which could confirm interview results. 185

187 CHAPTER 10 Setting This study was performed at the MAASTRO clinic, a radiotherapy institute in the Netherlands. On average, 200 oncology patients are treated each day in an outpatient setting. Despite the organizations small scale (250 employees in February 2014), its goals for clinical performance and research are ambitious. Since 2003, the organization has been engaged in improving patient safety. Since 2007, several alone standing lean based projects were initiated and in 2011 the lean philosophy was formally adopted as the long term strategy to realize individualized high quality care for all patients. The organization was restructured in order to facilitate and improve process innovation. The management structure was reorganized from functional to process oriented (value streams). Multidisciplinary teams were initiated for the different care paths (e.g. breast, head and neck), led by a clinical professional. To increase employee commitment and create a shared vision, board members discussed the long term vision with all employees in multiple small group meetings during From 2011 to 2012, all treatment equipment (accelerators) and planning software were replaced by a new integrated system and the patient process was fully digitalized. To improve the clinical processes, 15 projects were initiated to increase flow and decrease waiting times for the patients in Survey The participating interviewees completed a survey in January 2011 and before they were interviewed in The questionnaire contained 25 questions addressing the concepts: goal ambiguity, information asymmetry, and uncertainty of cause-effect relations in work processes. The perceived effects, and future expectations of the lean philosophy complemented these concepts. A Likert-scale from one to seven, (one equals the most negative perception and seven the most positive) was used and paired t-tests were used to test differences between the scores for both measurements. Interviews Twelve professionals were interviewed in January 2014, including four managers (one as practicing clinical physicist), three oncologists, two radiation technologists (oncology nurses), one clinical physicist, one administrative employee, and one employee responsible for the treatment equipment. The interviewer, and main author of this paper, had ten years work experience as clinical professional in the studied organization, and three years as patient safety officer. The interviews were structured by 16 questions (Appendix) addressing the concepts included in the surveys: ambiguity of objectives was addressed by four questions (4-7), information asymmetry by three (1-3), uncertainty of cause-effect relations by two (8, and 9), the perceived effects of lean management by four (12-15) and the future expectations by three questions (10, 11 and 16). All interviews were audio recorded and transcribed verbatim. Following a deductive approach, each transcript was analyzed and 186

188 DOES LEAN REDUCE GOAL AMBIGUITY & UNCERTAINTIES OF CAUSE-EFFECTS? coded starting from predefined categories (based on the Thompson and Tuden framework), using specialized software (NVIVO). The predefined categories were further complemented during the coding process. RESULTS Surveys Eleven of the twelve interviewees completed the survey in 2011 and The overall average scores did not differ between measures (4.3 in 2011 versus 4.6 in 2014; P = 0.11). The paired t-test for the separate questions (Table 1) revealed a significant improvement in perceived predictability of processes (question 12), in standardization of processes (question 14) and in shared visions among different professions / disciplines (question 21). The evaluation of projects (question 9.1) and the perceived compliance to procedures (question 16) improved on a 90% reliability level. Interviews The interviews took on average 60 minutes. To put the perceptions of the interviewees in perspective, Table 2 presents an overview of the number of times the related concepts were categorized within the interviews. Almost all interviewed employees expressed up front that lean management was no longer practiced. However, they all expressed several positive and negative changes that more or less could be ascribed to lean management during their interviews. At the end of the interviews they all drew their own conclusion that they were practicing more lean than they had thought. I actually think a success is that even though we cannot use the term lean any more, we perform things lean. It just happens. I'm talking about continuous improvement, training skills. There is a huge difference compared to a few years ago: the methods to train, the involvement of employees, the development of SOPs, the people themselves. When we implemented the new equipment, employees would not treat patients if they had no SOPs. Five years ago, employees would have glared at you if you had just mentioned SOPs. Most interviewees associated negative feelings to the word lean. Lean was mostly associated with efficiency in the negative sense of the word (i.e. cost reduction). Especially clinical professionals appear to interpret efficiency negatively. They focus on the patient and quality of care, making them less engaged to reduce costs and increase efficiency for the organization. Colleagues of mine have many different incorrect thoughts about lean, which do not position lean positive. This is particular related to the concept of efficiency. Efficiency is obviously quickly interpreted as doing this fast." 187

189 CHAPTER 10 Table 1. Survey results for 2011 and just before the interviews in Presented scores are the mean scores on a seven point Likert scale: one equals the most negative and seven the most positive scores. Questions 1 To what extent do you perceive difficulties regarding the knowledge / information transfer? 2 To what extent has the implementation of lean principles affected the knowledge / information transfer? 3 To what extent does the person with the substantive knowledge about a process have the control over this process? 4 To which extent has the implementation of lean principles affected the process experts' control over his processes? 5 To what extent is information in general transparent in the organization? 6 To what extent do the organizations' indicators fit what you think is important for your work? Mean scores 2011 N= N=11 P- value To what extent do the organizations' objectives fit your own objectives? To what extent do you experience conflicts between things you and the organization find important? To what extent are innovative projects evaluated before the project start? To what extent are innovative projects evaluated during a project? To what extent are innovative projects evaluated after project completion? To what extent are you aware of your colleagues' practice within the organization To what extent do you have insight in the performance of other units? To what extent do processes follow the planned trajectory? To what extent is the outcome / result of your action unsure / unclear? To what extent are processes standardized? To what extent are procedures complied to? To what extent is compliance to procedures evaluated? To what extent do you know your professional responsibilities? To what extent are professional responsibilities documented? To what extent do you receive feedback on your responsibilities? To what extent do your close colleagues shared the same vision and objectives? 21 To what extent do the different professions / disciplines in your unit share the same vision? 22 To what extent do you think the introduction of lean management have been clearly communicated? 23 To what extent do you believe in benefits from lean management for your organization? 24 To what extent do you think has the lean implementation positively affected the clarity of your work? 25 To what extent do you think that the level of quality care that is provided has become more clear for the organization, due to the lean implementation? P-values are based on a paired t-test

190 DOES LEAN REDUCE GOAL AMBIGUITY & UNCERTAINTIES OF CAUSE-EFFECTS? Table 2. Overview of the number of times the related concepts were mentioned Categorized concepts Number of quotes Attitude towards lean positive 14 negative 24 Perceived effects of lean positive 99 negative 18 neutral 1 Ambiguity of objectives reduced 21 Shared vision among colleagues increased 24 Lack of shared vision among colleagues 29 Conflicts perceived in organization 67 Conflicting objectives: research and clinical practice 18 Standardization 55 positively perceived 37 negatively perceived 10 neutral 8 Information asymmetry in clinical process decreased 53 sustained 21 Organizational information asymmetry 6 decreased sustained 19 Employees' commitment 58 improved commitment 32 Uncertainty of cause-effect relations decreased 20 Although clinical professionals appeared to perceive efficiency as negative, they did not express a negative association to the actual content of lean management. Some clinical professionals questioned the appropriateness of the lean philosophy for their work. In addition, managers beliefs regarding the benefits of lean were not unanimous. They expressed a more negative attitude towards lean than clinical professionals. The interviewed clinical professionals appeared to have broader understanding of the lean concept than several interviewed managers. The cultural aspects of lean were increasingly understood by clinical professionals, like creating a shared vision and increasing employee involvement to improvement. Management noticed a negative attitude against lean in the organization. They decided to replace the term lean with flow to prevent unnecessary discussions about its terminology and focus on the content of the lean initiations. 189

191 CHAPTER 10 "If you look at the content of lean, I think everyone can become enthusiastic about this. Because that is about how can I do my own work better tomorrow. But if we all keep calling these actions by name, then all professionals feel obliged to have an opinion about lean. Then you have a discussion about the theory, which is of no good." Effects ascribed to lean management Together the twelve interviewees mentioned 118 times an effect explicitly ascribed to lean. The majority of the quotes (99 of 118) were positive, the remaining quotes yielded negative associations (18 of 118) or referred to a neutral effect (1 of 118). Most positive effects were referred to decreased information asymmetry. Furthermore, processes had become increasingly transparent, owing to visual management and the increased possibility to discuss issues and information in a multidisciplinary setting. As a result information and knowledge had become increasingly clear. 190 "In every project, a technologist, a doctor, and an administrative employee are represented. One of the professionals is in the lead who is also responsible for communicating the discussed items to all relevant colleagues." Most mentioned concepts were the creation of a culture of continuous improvement, increased efficiency and employee commitment, and standardization of processes. Interviewees also expressed improved teamwork and increased process performance. Employee satisfaction improved and uncertainty of cause-effect relations decreased resulting from increased transparency of the clinical processes, improved agreement about outcomes, and more clearly defined responsibilities. "They start to feel ownership, to feel that they realized the improvements and to have fun in sustaining their improvements. These are the first signs of sustainable change." Process improvement requires discussions between professionals, which is time consuming. This temporarily increased their workload, which was mentioned as a negative effect of lean. Lack of visibility of effects resulting from the lean intervention was negatively perceived. The most important negative effect expressed was that beneficial results from lean redesigns were not returned to the employees in question. You have to consider that the time gained, is returned to the people as unscheduled time. The time gains should not again be used to schedule more patients in less time.

192 DOES LEAN REDUCE GOAL AMBIGUITY & UNCERTAINTIES OF CAUSE-EFFECTS? Ambiguity of objectives The ambiguity of objectives in the studied organization is a major issue. Although eight of twelve interviewees expressed reduced ambiguity (21 quotes) and experienced increased shared visions among colleagues (24 quotes), six interviewees expressed issues (29 quotes) related to a lacking shared vision. Several causes for ambiguity were addressed: benefits of actions being invisible, conflicting goals between research and clinical practice, no actual participation of professionals in management decision making, and unrealistic objectives. "I think it's very important that we provide good patient care. I think it's less important that we perform research on the highest level and I actually think that's not realistic either. Of course we can contribute to science, but we cannot compare ourselves with Stanford or Paris. These are institutions where 1,000 people work in a radiotherapy centre. That doesn t seem realistic." The interviewees were asked about the possible conflicts they experienced in their organization. Although nearly all interviewees perceived the word conflict as too negative, they frequently (67 quotes) mentioned a perceived tension. Most quotes were associated with the conflicting objectives between research and clinical practice (18). Since the studied organization was not an academic institution, and therefore did not receive funding for research, most research had to be financed by the clinical practice. This frustrated employees, in particular professionals that did not participate in research activities. The extensive focus on research demands sacrifices from the clinic, because finances are withdrawn from the clinic to invest in research." Our financial resources are according to a peripheral clinic. In addition we want research and they do believe that s what we all want. Well, I m not one of them all. I do not necessarily have to perform research." The tension between standardization of processes, respect for individuals, and the acknowledgement of exceptions was a concern. The pressure to standardize everything is automatically present in our organization. If standards are met, everyone is happy and satisfied. That brings about the risk of not stopping the process if needed. That s something you should watch. Standard procedures should not be blindly followed." Although ambiguity of objectives was still present in the organization, improvements were experienced among the interviewees (21 quotes). These improvements were mainly associated to increased involvement in management decision making owing to improved transparency. This resulted in improved understanding and trust in management. Interviewees also perceived improved 191

193 CHAPTER 10 multidisciplinary communication, because multidisciplinary teams were created and instead of focusing on individual professional actions the focus shifted to the overall clinical process. People increasingly understand the defined policies, because they increasingly see the contexts of these policies. The multidisciplinary care path teams resulted in working together with colleagues who share goals. I think this is a success. Employees who were not involved in research activities, (seven of twelve interviewees) perceived conflicting goals between research and the clinic more often compared to employees who participated in research activities. They also expressed a less positive perception regarding a shared vision (Figure 3) and were less satisfied about the communication of organizational information (organizational information asymmetry sustained and transparency even worsened). On the contrary, they perceived improved employees commitment and improved flow of clinical data. Employees who expressed an improved shared vision also perceived a decreased clinical information asymmetry, improved flow of clinical data, increased culture of improvement, increased problem solving, improved team performance, and increased clarity of responsibilities. Number of references per interviewee 10 Involved in research (N=5) 9 Not involved in research (N=7) Figure 3. The number of quotes per interviewee separated for interviewees involvement in research. The number of quotes per interviewee were calculated by dividing the number of quotes for the group by the number of interviewees represented in the group. 192

194 DOES LEAN REDUCE GOAL AMBIGUITY & UNCERTAINTIES OF CAUSE-EFFECTS? Uncertainty of cause effect relations The level of uncertainty in oncology increases, because of increased treatment possibilities and the progressive use of complex technology. However, nine interviewees clearly expressed that the uncertainty of the cause effect relations in their work processes had decreased (20 quotes) in recent years. "I think that an administrative employee, responsible for the patients entire schedule, will work with more pleasure. Things are increasingly clear, because the scheduling process is visualized on a board. Previously, I had no overview and was always in doubt if I had done things right. And now I can judge this up front." Since we use specialized software to manage our standard procedures, constraints of a treatment planning have been increasingly documented. We used to discuss and approve all treatment plans in a doctors meeting. Because constraints have become increasingly clear, only treatment plans are discussed for which constraints could not be met. Perceived uncertainties decreased which could result from increased process standardization. Standardization had been mentioned 55 times during the interviews, 37 perceived as positive and ten as negative. As negative aspects, the interviewees experienced a pressure to comply to standard procedures and a psychological barrier to stop the process if needed. The level of detail of standardization is also difficult to determine. If processes are standardized too much, innovativeness could be jeopardized, since the organization might become bureaucratic. Standardization was experienced more negative among clinical professionals than within the members of management. "Sometimes it s just too procedural. I think standardization is good, but you should not overdo it." Although the interviewees expressed some concerns about the level of standardization that should be aimed for, they were all positive about the realized standardization. It has resulted in decreased clinical information asymmetry. Professionals knowledge became more transparent for other disciplines and the stability of process outcomes increased. If I want to know how we should treat patients with prostate cancer over here, I can see this very quickly. This was completely different four years ago. What is expected from me has been increasingly standardized. This also decreases the risk of making mistakes. Information asymmetry for clinical processes decreased (53 quotes). Improvements were addressed to the digitalization of the patient process, the 193

195 CHAPTER 10 improved (multidisciplinary) meetings and the implementation of specialized software to manage standard clinical procedures and knowledge. As a result, transparency strongly increased and communication between the different disciplines improved. We have never worked more professional than we are doing right now. We make medical, technical and logistical judgments. We are very satisfied about our performance right now. I think all those lean projects have resulted in an increased documentation of the flow. This brings clarity for everyone. Although improvements have been expressed, information asymmetry remained an area of concern (21 quotes). Communication between professions remained unclear at certain points and although knowledge was increasingly available, employees had to read and adhere to many procedures. Some people are afraid that they are no longer in control if implicit knowledge is made explicit. So actually, our software to manage our procedures has brought about improvement. Accessibility has increased, but now the problem has shifted towards the users. They have to adhere to the procedures, which is just as important. We have to give each other feedback on this issue. Interviewees who expressed decreased clinical information asymmetry, also expressed decreased ambiguity on cause-effect relations, an arising culture for improvement and improved team performance. Information asymmetry, on the level of organizational information, remained high (19 quotes on sustained versus 6 on improved asymmetry). Management expressed an increased transparency of organizational information, while clinical professionals did not share this perception. Although most interviewees experienced that management increasingly aimed to improve transparency, they were still unsatisfied with the organizational communication. The stimulus to share information in this organization is huge. Transparency is aimed for. However, it is not as transparent as one thinks. Some things I just don t understand at all. The clinical process flow also improved, mainly because of the digitalization, process improvements brought about by multidisciplinary teams, and the visualization of the administrative processes and the preparation process of radiotherapy treatment. The evaluation of projects and clinical practices also improved, but still remained 194

196 DOES LEAN REDUCE GOAL AMBIGUITY & UNCERTAINTIES OF CAUSE-EFFECTS? an area for improvement. Although interviewees expressed increasingly structured evaluation, it was still largely dependent on the involved employees. The increased attention for evaluating processes resulted in a higher demand for process data. The organization focused more on innovation than on sustainability of results. Project follow-up was perceived as problematic. However, an overall positive trend towards improved evaluation was expressed. I think that last year, we aimed to evaluate implementations more structurally to see whether the desired results had been achieved. Then again, you sometimes miss a baseline measurement. I think we do not evaluate enough. When a project reaches an end, we think we are finished. Follow-up and evaluating the projects benefits is rarely performed. DISCUSSION Following the introduction of a lean management initiative, perceived ambiguity of objectives and the uncertainties of cause-effect relations decreased in the studied organization. Perceptions of the twelve interviewed employees were affirmed by the survey results. Ambiguity of objectives remained present in the organization, although improvements were experienced. Improvements were mainly associated to the increased involvement in management decision making. This resulted from improved transparency, improved multidisciplinary communication, and an increased focus on the overall clinical process instead of focusing on the individual actions per profession. Employee commitment improved and employees felt increasingly responsible to deliver high quality care. They felt more involved in process improvements and multidisciplinary communication improved. The lack of a shared vision became especially apparent in the conflicting objectives of clinical and research activities. The lack of transparency concerning the financial distribution between clinic and research activities could be one of the major causes in this conflict. In addition, if one does not return the benefits from lean to the clinic, but invest these in research activities, the conflict might be further reinforced. The uncertainty of the cause-effect relations in the clinical work processes had decreased dramatically. Clinical activities became more transparent and responsibilities were defined more clearly. This was probably the result of the increased examination of clinical processes by multidisciplinary teams, followed by a standardization of activities (deploying SOPs) to sustain improvements. 14 Clinical information asymmetry decreased, because the operational processes improved and became more clear. Although the improved processes are obviously part of the organizational whole, much more remains to be improved. 195

197 CHAPTER 10 Specifically, clinical professionals remained unsatisfied regarding the transparency of the organizational communication. Organizational information asymmetry did not improve, although interviewees recognized that management tried to improve transparency. As the ambiguity of objectives and the uncertainty of cause-effect relations had both improved in the studied organization, we concluded that the organizational characterization of decision making, conform the Thompson and Tuden framework, shifted from the Imagination in the direction of the Computation context. This trend could enhance successful implementation of quality improvement initiatives, for instance by implementing standard procedures and the use of a set of predefined indicators for evaluation. The results of this study have to be appreciated given certain limitations. First, the presented results cannot entirely be ascribed to the lean transition. The restructure of the organization, to facilitate and improve process innovation, undoubtedly affected the outcomes. Second, since the presented results were based on the perceptions of twelve individuals, generalizing these results to the entire organization should be done with care. This might limit the value of these results, as these might not be as objective as quantitative data. However, this paper presented a clear impression of the differing perceptions of employees throughout the studied organization. Since individuals behaviour is dependent on the individuals subjective perceptions of the situation, 15 the perceptions of employees play an important role in quality improvement. Therefore, in order to make high quality care a reality, management in healthcare is obliged to have a deep understanding in the perceptions of employees. In conclusion, the lean transition positively affected the perceived ambiguity of objectives and reduced the perceived uncertainties of cause-effect relations in the clinical processes. Therefore, decision making benefitted from the lean transition and the chance to realize sustainable quality improvements increased. 196

198 DOES LEAN REDUCE GOAL AMBIGUITY & UNCERTAINTIES OF CAUSE-EFFECTS? REFERENCES 1. Beer M, Nohria N. Cracking the code of change. Harvard Business Review 2000; 78: Dixon-Woods M, Baker R, Charles K, et al. Culture and behaviour in the English National Health Service: overview of lessons from a large multimethod study. BMJ Qual Saf 2014; 23: Brackett T, Comer L, Whichello R. Do lean practices lead to more time at the bedside? J Healthc Qual 2013; 35: Glasgow JM, Scott-Caziewell JR, Kaboli PJ. Guiding inpatient quality improvement: a systematic review of Lean and Six Sigma. Jt Comm J Qual Patient Saf 2010; 36: Kaplan GS, Patterson SH, Ching JM, Blackmore CC. Why Lean doesn't work for everyone. BMJ Qual Saf Thompson JD, Tuden A. Strategies, Structures and Processes of Organizational Decision. In: Thompso JD, editor. Comparative Studies in Administration. Pittsburgh: Pittsburgh University Press; Abernethy MA, Chua WF, Grafton J, Mahama H. Accounting and Control in Health Care: Behavioural, Organisational, Sociological and Critical Perspectives. In: Chapman CS, Hopwood AG, Shields MD, editors. Handbook of Management Accounting Research. Oxford: Elsevier; p Vergauwen P, Van der Schaaf T, Van Mierlo J. A Sustainability and Accountability Approach to Patient Safety Management in Health Organizations. Accountancy & Bedrijfskunde 2010: Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Quality & Safety 2013; 22: Endsley S, Magill MK, Godfrey MM. Creating a lean practice. Fam Pract Manag 2006; 13: McCarthy M. Can car manufacturing techniques reform health care? Lancet 2006; 367: Rutledge J, Xu M, Simpson J. Application of the Toyota Production System improves core laboratory operations. Am J Clin Pathol 2010; 133: Liker JK. The Toyota Way. McGrawHill, New York, Simons PA, Houben R, Benders J, et al. Does compliance to patient safety tasks improve and sustain when radiotherapy treatment processes are standardized? Eur J Oncol Nurs 2014; 18: Merton RK. The Thomas Theorem and The Matthew Effect. Social Forces 1995; 74:

199 CHAPTER 10 Appendix. Questions to structure interviews. 1. How does the information flow through the organization at this moment? Who has what information and where do information transfers take place? - for the clinical (patient) processes - for the organizational trajectory 2. How can the flow of information be improved? 3. Which stimuli do you perceive to share or not share information? 4. Which elements or indicators do you think are of high priority to the organization? 5. Do these indicators match your own priorities? 6. Which indicators don t you use and why not? 7. Where do you experience conflicts between your own priorities and the organizations? Please state the three most important. 8. Are implemented decisions discussed up front and evaluated? Are results evaluated structurally, so one can learn from it? 9. Are specific moments organized during the process to correct processes that possibly deviate from the expected trajectory? 10. What goals do you think the organization aims for by implementing lean management? 11. Do you feel the relations between patient safety and lean management are clearly communicated? 12. Have you perceived any positive and/or negative effects due to lean? If yes, please state which effects you perceived. 13. Do you think that the lean implementation has affected the uncertainties in your work? If yes, in a positive or negative way? 14. What do you think the lean implementation has changed in the flow of information and knowledge? 15. Do you think the objectives of the organization have changed due to the lean implementation? 16. Do you expect any more effects of lean in the coming years? If yes, please state which effects you expect. 198

200 CHAPTER 11 Health QALY FLOW Satisfaction travel times QUALITY OF LIFE Client Care End-point Happiness Error Waiting times Long term benefits Life Compliance to safety tasks Service OUTCOMES CULTURE FLOW process Satisfaction TRUST Sustainability Communication Safety climate process Incidents SUSTAINABILITY autonomy Absenteeism Quality Safety Transparency standardization flow Shared vision Efficiency process times participation SAFETY AWARENESS Waiting times Equipment COMMITMENT SOP Technology TRANSPARANCY Communication Improvement culture Cost reductionimprovement Culture Frustration education Safety climate Down time FUN COMPLIANCE motivation HUMAN FACTOR equipment Clear responsibilities management decision making Problem solving Cost-benefits Safety behaviour General discussion

201 CHAPTER 11 The overall objective of this dissertation was to perform an extensive assessment of a lean management transition in a healthcare setting. Three research aims were formulated to consider this objective. This chapter summarizes the main findings on these three aims with their limitations and implications for practice. Finally, some concluding remarks and future perspectives will be considered. SUMMARY OF MAIN CONCLUSIONS To address RESEARCH AIM 1, regarding the development of an instrument to improve management decision making in a healthcare setting, an instrument was presented in chapter 3. The instrument was implemented in the form of a workshop, aiming to improve management decision making and increase understanding in employees views regarding the implementation of a lean management program. To successfully implement lean, it is vital to have a detailed understanding of employees views regarding the program 1, especially when the proposed changes are potentially contested. Gaining insight into employees perceptions helps management to decide how to proceed. Two workshops were conducted in a radiotherapy institute to assess the benefits and drawbacks of a proposed lean change program, as well as their underlying causes. The workshops covered a brainstorm to list the proposed programs most important expected effects, based on the KJ-technique (Kawakita Jiro) 2 and a deepening of the listed effects regarding their root causes. Employees commitment to participate was high, which resulted in a long list of expected positive and negative effects of the lean program on a variety of aspects. Interestingly, the majority of negative effects were expected on the short term, although the number of positive effects outnumbered the negative at the end. This might partly explain why getting the lean management philosophy into practice is difficult. High-level management reported only a moderate to positive value of the workshop regarding their decision making process. However, they expected increased employee commitment to the lean program and improved bottom up thinking, as a result of participating in the workshop. As a result, this type of workshop has the potential to increase successful implementation 3, against relatively small costs. The workshop could serve as an instrument to create a supportive foundation for change and improve management decision making, especially for complex decision making when cause-effect relations are unclear. However, the number of participants was small and the workshop had only been performed twice. Considering the limited results in this study, no strong conclusions could be formulated. However, the potential benefits of involving employees in early stages of change were demonstrated. Although this kind of workshop should be studied in a variety of health care settings, wider implementation can be considered. 200

202 GENERAL DISCUSSION Regarding RESEARCH AIM 2, a survey to measure patient safety awareness and intentions for patient safety behaviour was developed. Since quality and safety of care is for a part dependent on employee s behaviour, periodic measurement of safety behaviour could provide management relevant information to adjust the implementation of improvement. The objective measurement of safety behaviour is difficult and time-consuming (for instance by observing behaviour or compliance to safety tasks). Therefore, a factorial survey on safety awareness and intentions for safety behaviour was developed as a supplementary instrument to safety culture measurement. Chapter 4 present the results of this factorial survey compared to results from the Hospital Survey On Patient Safety Culture (HSOPSC). The HSOPSC is a validated survey measuring patient safety climate. 4 Both surveys were distributed to the clinical staff of MAASTRO CLINIC at the end of 2010 and the beginning of The factorial survey included twenty hypothetical scenarios about incidents, randomly varying on; work pressure, person causing incident, whether the patient-level was reached, severity of harm, notification by the patient and support from management. After each scenario, the respondents answered one question on their safety awareness and four on their intentions for specific safety behaviour. The factorial survey results complemented the HSOPSC results and presented additional information; the intentions to structurally improve were measured by the factorial survey and the factors influencing intentions for safety behaviour could be determined using the factorial survey. This increases managements understanding in employees safety behaviour, bringing about increased opportunities to manage specific improvement actions. Furthermore, the factorial survey on intentions for safety behaviour could be more sensitive to change than the traditional safety culture surveys, because behaviour is expected to change before values do. An important drawback of the factorial survey was the lacking opportunity to benchmark with other health care organizations. To make the used case descriptions (vignettes) recognizable for clinical professionals, they were described in large detail. This level of detail inevitably led to issues around generalizability. Another limitation of the used factorial survey was the measurement of intentions instead of behaviour itself. People will act according their intentions, given a sufficient degree of perceived behavioural control. Although the intention is not equal to behaviour, it is assumed to be the immediate antecedent of behaviour. 5 Before implementing the factorial survey technique on a broader scale, the preferred level of detail in the vignettes should be studied to make generalizations and follow up for longer time periods possible, without sacrificing incidents reality. RESEARCH AIM 3, regarding the effect evaluation of a lean transition in a radiotherapy setting is considered in chapters 5 to 10. Effects on the patient, the employee, and the organizational level were evaluated. Chapter 5 presented the behavioural effects of a lean management redesign of irradiation treatment process. To realize safe radiotherapy treatment and be 201

203 CHAPTER 11 able to innovate and optimize the treatment process, first the relevant processes should be stabilized. 6 The treatment process was expected to become more stable, when standard operating procedures (SOP s) were implemented. Behaviour was measured as observed compliance to patient safety tasks. Perceived importance of tasks was expected to increase sustainability in compliance rates. As expected, compliance to specific tasks increased after introducing SOPs and improvements sustained after 1.5 years. Perceived importance of tasks correlated positively to compliance and sustainability. Regular feedback to raise employees awareness about the importance of the safety tasks is thus crucial to increase compliance and sustain improvements. In addition, after the redesign was implemented, an increased willingness to report incidents was measured among radiation technologists. De detailed discussions between the technologists might have increased their awareness regarding the potential safety issues in the treatment process. The technologists perception of reality might have been broadened and bottlenecks in the process might have been identified, because of considering the arguments of colleagues. Additionally, they could have become more engaged to realize safe treatment of high quality. The increased number of reported incidents directly created opportunities for specific improvement actions. In chapter 6, the effects of the lean transition on the patient safety culture were presented. The importance of a safety culture in an organization to create quality, and safety improvement is no longer questioned. 7 However, achieving sustainable culture improvement is not straightforward. Evidence is growing for a multifaceted approach, in which multiple safety interventions are combined. 8 Lean management is a holistic approach to improve safety, quality and efficiency 6 and combined with an organizational restructure was expected to improve the safety culture. Safety awareness and the intentions for safety related behaviour in the studied organization were evaluated by a factorial survey three times from 2010 to Safety climate was evaluated three times from 2010 to 2013 by the HSOPSC 4 and two times (2011 and 2013) by a workshop based on the Manchester Patient Safety Framework (MaPSaF) These results were complemented by the number of reported safety incidents from the incident reporting system (IRS) and further explored by structured interviews with professionals. The workshops (MaPSaF) did not detect any changes in safety culture, but the HSOPSC, and the factorial survey presented improvements. The intention to report incidents that did not reach the patient-level, and the number of reported incidents decreased. However, employees experienced sustained safety awareness, and an increased intention to structurally improve. Therefore, the decreased number of reported incidents was explained as a decrease of the number of incidents actually happening, owing to increased problem solving, and technical improvements in equipment. From 2012 to 2013, the intention to take action in order to prevent future incidents strongly improved (factorial survey), which was confirmed by results from the interviews. Interviewees expressed that management increasingly facilitated, and supported improvement, 202

204 GENERAL DISCUSSION responsibilities were more clearly defined, and a multidisciplinary forum was created for employees to discuss problems and potential solutions. Improvements in patient safety culture were measured after implementing the lean transition. However, this measured improvement could not be fully ascribed to lean, since organizational changes might have strongly influenced the safety culture as well. Presumably the culture is affected by the combination of both strategies. This might imply that combining lean management initiatives to a management restructure to facilitate the lean principles is a successful strategy for improving patient safety culture. Although positive effects were presented in this chapter, the robustness of the measured data has to be questioned, since most conclusions were based on survey and qualitative data. Since the measurement of safety culture is difficult, multiple methods were combined to discover the underlying truth and increase understanding in the results Chapter 7 covered the patient safety effects, when seven accelerators (treatment equipment) and the treatment planning software were replaced by an integrated system including six accelerators from July 2011 to March The accelerators before 2011, presented many down time because of technical failure, were not equipped to deliver the latest state of the art high quality care and presented many human factor 16,17 problems. Although replacing the equipment was not a specific lean intervention, it was part of the generative interventions to create improvements in quality, safety and efficiency. It supported the lean philosophy by understating the importance to adapt technology to the human and organizational processes and not the contrary. In a mixed methods approach, quantitative measures were combined with qualitative interviews. The number of reported safety incidents with their root causes and the accelerator down time were recorded between Structured interviews were performed with ten professionals and a focus group session with five patient safety professionals in January As expected, the number of incident reports related to the accelerator decreased significantly from 41 (2010) to 14 (2013) reports per month, although the number of delivered fractions per accelerator increased by 20%. In addition, the proportion of technical coded root causes assigned to the reported incidents decreased, whereas the proportion organizational coded causes remained stable. Furthermore, the down time decreased with 86% from 5.4% of available treatment hours in 2010 to 2.9% in One could question whether the number of incidents actually decreased or employees reported fewer incidents. The interviewees expressed that they did not experience a decreased willingness to report. Instead, they experienced improved treatment processes, explained by the technical transition, and because management increasingly focussed on improvement. Employees felt increasingly encouraged to improve procedures and processes, because of the increased focus on standardizing processes based on the lean philosophy. Interviewees were convinced that the improvements in technical equipment were most strongly related to the decrease of the number of reported incidents on the accelerators. Therefore, one could carefully conclude 203

205 CHAPTER 11 that the transition to the new equipment positively affected patient safety by a decreased number of incidents resulting in a decreased risk for serious safety incidents and harm to patients. In chapter 8 the cost-effectiveness of a redesigned process regarding the diagnostics and preparation process for patients with head and neck cancer (HNC) were considered. Commitment of clinical professionals, and healthcare management to lean management initiatives might increase, and sustainability might be ensured, when the long term benefits for the patient are presented of a process redesign based on lean principles. A mathematical model (Markov model) was used to calculate the cost-effectiveness of the logistic redesign on the long term. The redesign included two main organizational changes: 1) performing tumour biopsy under local instead of full anaesthesia and 2) performing a diagnostic CT-PET before radiotherapy instead of an additional CT- PET during the preparation for radiotherapy treatment. The redesigned care process led to reduced waiting times and proved cost-effective. The incremental QALYs varied between 0.13 to 0.66, with none to small additional costs for the redesign. The benefits of redesigning the workflow became very clear when incremental net monetary benefits (INMB) were extrapolated to the organizational and national level. If ,- was adopted as the price a patient (or society) is willing to pay for an additional QALY (ceiling ratio) 18, and all patients, who met the inclusion criteria, in the studied organization were included during one year, a total NMB for all patients (or the payers of healthcare) of ,- was calculated. When was hypothesized that this process redesign could be realized for all patients in the Netherlands, who met the inclusion criteria, a national NMB of ,- could be realized. This is based on the unrealistic assumption that the diagnostic processes, redesign opportunities and costs in other hospitals would equal the studied organization. Although the model outcomes are based on many assumptions, the presented redesign of the care process resulted in significantly better long-term patient outcomes and cost savings for patients with HNC. This specific group of patients was selected for this study, because increased waiting time for particularly these patients is significantly associated with more negative patient outcomes. Although results may be less strong for other patient groups, it will still be worthwhile to optimize the flow of care processes. Since the effort of redesigning the care process is mostly hindered by the lack of communication and team work between different disciplines and care departments, exactly these are the greatest challenges to overcome. Although health care professionals might be less attracted to process and organizational redesign as to adapting new technologies, the benefits for patients and costs of healthcare in this study were clear cut and cannot be discarded. Therefore, these kind of redesign interventions should be implemented on a much wider scale. Chapter 9 described the effects of the lean transition regarding waiting times and outcomes for patients and employees. In 2013, 15 lean based projects were 204

206 GENERAL DISCUSSION initiated to improve patient flow and decrease waiting times. These specific projects were preceded by generative interventions to increase commitment and create shared visions. Mixed methods were used to evaluate process times, patient satisfaction, safety, employee satisfaction, and absenteeism. Administrative data were extracted from databases and combined with survey and qualitative data. The percentage of patients of which the waiting time exceeded the Dutch national norms, and objectives for waiting time 19 significantly decreased, and no negative side-effects for patients, and employees were measured. On the contrary, patient safety, employee satisfaction and the percentage of employee absenteeism had also improved. Although the improvements in waiting times were moderate, these could be a potential motivator to struggle forward and strive for continuous improvement. Measured effects may not be the direct result of the 15 initiated projects to increase flow, but can be explained by the multifaceted organizational approach. The flow projects were preceded by several managerial actions to improve commitment, create a shared vision and facilitate change. Care path teams were created in lead of clinicians and the overall vision was discussed by top-management and employees in small group sessions. Although no strong conclusions can be drawn, the combination between the increased focus on continuous process improvement (lean principles), creating commitment to change and the facilitation of organizational structures, was possibly the success factor of this multi-facetted approach. Many quality and safety initiatives fail to realize sustainable improvement and there remains room for further improvement The large ambiguity of objectives, and the uncertain cause-effects relations in care processes, might be influencing concepts to realize sustainable improvement. Lean management is an approach that strives to clarify goal ambiguity, and increase insight into the cause-effect relationships. But does it succeed? In chapter 10, the effects of a lean-inspired change program in a radiotherapy institute were presented on goal ambiguity and insight into the causality of its care processes. Twelve professionals throughout the organization were interviewed in The interviewees completed a survey in 2011, and before the interview in. The survey included 25 questions addressing the above concepts. The survey results revealed an enhanced predictability of processes, improved standardization and improved shared visions. Interviewees expressed an improved shared vision and reduced uncertainty of cause-effect relations. More positive (99) than negative effects (18) of lean were expressed. Positive effects mentioned most were: an improved shared vision, reduced uncertainty of cause-effect relations, and increasingly committed employees. Negative effects were also expressed: a temporary increase of workload because the needed discussions to redesign processes takes time, not returning the benefits of lean to the employees in question, and innovativeness being jeopardized by standardization. Since lean management positively affected the existing ambiguity of objectives, and reduced uncertainties of cause-effect relations in clinical processes, the chance 205

207 CHAPTER 11 for its successful implementation increased, and sustainability could benefit. Remarkably, most employees did not realize that managements approach to improve processes, and increase problem solving were based on lean management principles. They did also not relate the two concepts; lean and flow. Not assigning the name lean to the undertaken actions should not be a problem, because care organizations should not focus on becoming a lean organization, but on delivering the best possible care to the lowest cost with satisfied, and motivated personnel. The lean philosophy is not a goal in itself, but is a strategy to reach the set goals. METHODOLOGICAL CONSIDERATIONS Some limitations of this dissertation should be considered. First, as with all quality improvement research we faced the dilemma of the methodological difficulties for objective measurement of effects. End-point outcomes were combined with surrogate end-points like process evaluation, compliance to the intervention, culture and errors (incidents), to detect the real effect. 12,25 Mixed methods were used for the evaluation of the effects, to meet the complex nature of the indicators Although quantitative measures were combined with qualitative, still only a very limited list of effects were considered. End-points for quality and safety were hardly included, because in the field of oncology a large amount of noise is incorporated in these outcomes, which troubles the objective evaluation of improvement initiatives. Besides the limited scope of evaluated effects in the study, the methods for evaluation were not always objective. Many results were based on survey data and qualitative interviews which are inherent to reliability issues. Although conclusions should be carefully formulated, because of these measurement limitations, all measured effects complemented each other and directed in a positive evaluation of the lean transition. Second, the causality of effects is hampered. In the studied radiotherapy institute, the lean transition was combined with generative organizational interventions to increase employee commitment, and facilitate lean initiatives by promoting a culture for improvement, and creating structures to enhance clinical innovation. The measured effect were probably the result of combining these interventions. The effects could not be directly linked to one of the interventions. Therefore, we could not conclude that the lean transition in itself resulted in positive outcomes for the patient, employee, and the organization. One could question whether these generative organizational interventions are not embedded in the lean philosophy. In congruence with the quality and safety improvement literature, to create sustainable improvements a system-level approach is advised, in which lean tools (specific interventions) are combined with an overall strategy to create a culture of improvement, and guide leaders 206

208 GENERAL DISCUSSION and clinical professionals in their change process. 24,26-29 Although the majority of health care organizations implement lean as a set of tools guided by the idea of eliminating waste 24,26, Liker incorporated shared decision making and creating common goals in the lean management philosophy. 6 This could be a failure of translating the lean philosophy to healthcare. In conclusion, positive effects were measured in the studied organization, which acknowledged the overall organizational interventions combined with the specific lean tools as a successful approach to create improvement in this health care setting. Third, only one health care organization, MAASTRO CLINIC, was studied. This organization was specialized in treating oncology patients by radiotherapy in an outpatient setting. Because the diversity of treatment options and the involved diagnostics were limited, creating a shared vision among clinical professionals could be less problematic. Although the relevant forces in a hospital setting might be increasingly diverse and intense, the studied organization presented similar problems as expected other health care settings. However, the studied organization already had a long history with patient safety improvement. As a result, safety awareness among employees was already high and a strong nonpunitive culture was present in the organization at the beginning of the evaluation period. This could have enhanced the successful implementation of a change program like lean management in the studied organization, when compared to other health care settings. A benefit of studying this single case was the extensiveness of the effect assessment, and the combination of multiple measures, creating a deep understanding of several change related issues. Although one could question if the findings in this dissertation can be generalized to other health care settings, the described multi-facetted approach presented clear benefits for the patients, the employees as well as the organization. Since there is no one fits all approach to improve quality in health care, we should not try to generalize findings and advise one approach as the right one. Every health care organization should evaluate their specific problems and should select relevant interventions and tools to combine their own recipe for quality improvement. The important lesson should be that independent of the selected interventions, specific interventions should be combined with generative interventions to simultaneously create changes on the structural, cultural and process dimensions in an organization. Final conclusion Regarding RESEARCH AIM 1, the developed workshop where employees listed the expected benefits and drawbacks of a proposed lean change program, as well as their underlying causes, was appreciated by the managers, because they expected it to improve employee commitment to the lean program, and increase bottom up thinking. However, the top managers reported only a moderate to positive value of the workshop regarding their decision making process. Although results were limited, this kind of workshop could serve as an 207

209 CHAPTER 11 instrument to create a supportive foundation for change and improve management decision making, especially for complex decision making when cause-effect relations are unclear, against relatively small costs. Regarding RESEARCH AIM 2, the developed factorial survey on patient safety awareness and intentions for safety behaviour presented a valuable instrument to complement surveys on safety culture/ climate. It presented additional information for management which increases managements understanding in employees safety behaviour, and brings about increased opportunities to manage specific improvement actions. Regarding RESEARCH AIM 3, the overall conclusion is that the lean transition combined with a multi-faceted organizational approach in the studied health care setting resulted in positive effects for the patient, the employee, and the organization. A global overview of measured results is presented in Figure 1. Organizational culture improved Patient safety culture Employee satisfaction Improvement culture Efficiency Redesign diagnostic/preparation process HNC Waste reduction & flow optimization Redesign radiotherapy treatment process Waiting times decreased Absenteeism decreased patient quality safety efficiency Patients remained satisfied Patient safety improved Compliance to patient safety tasks Less reported safety incidents Safety culture/climate Problem solving increased Human factor proof equipment Patient flow improved Waiting times slightly decreased Travel time decreased Diagnostic process shortened QALYs for HNC patients increased Employees increasingly satisfied Process flow improved Patient safety culture improved Safety climate Intention to structurally improve Perceived importance of safety tasks Employee commitment improved Involved management decision making Increased transparency Improved trust in management Figure 1. Summary of measured effects of the lean transition combined with a multifaceted organizational approach in MAASTRO clinic between 2010 and Although critics claim that lean management would have negative effects on employees, no such results were measured in the studied organization. On the 208

210 GENERAL DISCUSSION contrary, employees satisfaction increased and absenteeism due to disease decreased during the studied years. Although, all interviewed employees expressed a positive trend in the organization in recent years, they also expressed some concerns on the level of standardization and the risk that efficiency might overrule quality of care. Because the presented results are prone to some methodological limitations regarding measurement and generalization, conclusions should be carefully interpreted. However, all included measurements presented improvement, and combining the generative approach with the specific lean tools proved a successful approach to create sustainable improvement in quality, safety and efficiency in studied health care organization. The specific set of interventions should be adapted to the specific problems and needs of every care organization. 30 However, to realize sustainable improved care, all three aspects; organizational structure, culture, and processes should be addressed simultaneously. Although many professionals share the opinion that quality of care should be improved, creating sustainable change is a continuous struggle and asks determination and perseverance to a long term vision. However, high quality and safe care is a privilege for all patients. Therefore, health care professionals, management and policy makers should do all it takes to make this a reality. FUTURE PERSPECTIVES The multi-faceted improvement approach, combining the lean transition with an generative organizational strategy, positively affected outcomes for the patient, the employee, and the organisation. A three year period was evaluated from 2010 to Although the flow projects showed moderate process improvements, the sustainability of these results is not yet guaranteed. These flow projects were only a first step towards continuous process improvement. The question remains if these first successes can motivate the clinical professionals to realize radical change, and create substantial and sustainable improvements in process flow, and quality of care. The multi-facetted approach seems a successful approach to create quality improvement, but still has to prove itself in being successful to strive for perfection. Continuous improvement has not yet been internalized by the majority of employees. Although some improvements have been made, the organizations challenge is to not only sustain the realized improvements, but also to raise them to a higher level, and ideally deliver the perfect care for all patients. When continuous improvement is up taken as one of the basic values in the professional status, sustainable improvement has become a reality. This dissertation only described the first steps and effects of the implementation of continuous improvement in a 209

211 CHAPTER 11 radiotherapy setting, and perseverance to the long term goals will be the crucial element for success. The combination of quantitative, and qualitative measures resulted in an indepth analysis of the evaluated outcomes. 12,14,15 By means of structured interviews, quantitative results were explored in the last year of evaluation. These interviews made interpretation of quantitative outcomes possible. In future research, these interviews should not only be performed at the end of the evaluation period, but should be performed periodical to explore results throughout the evaluation period. This might provide evidence of intervention problems in an early stage and may lead to revisions of the intervention or the measurement protocol to insure correct evaluation. 12 The lean management philosophy proved a valuable approach to simultaneously improve quality of care, patient safety, efficiency, and motivated employees. However, the implementation strategy for is crucial to realize actual positive changes. Therefore, specific lean interventions should be supported by a generative intervention where managerial structures are organized to facilitate innovation and a culture of improvement with open communication is realized. The important elements influencing the implementation of quality improvement programs are not yet cleared out. One could question whether the implementation strategy might be even more important in creating improvements than the quality improvement program is itself. Further research should study which elements are most important for the implementation of quality improvement programs and which factors influence the sustainability of improvements. 210

212 GENERAL DISCUSSION REFERENCES 1. Andersson LM. Employee Cynicism: An Examination Using a Contract Violation Framework. Human Relations 1996; 49: Spool JM. The KJ-Technique:A Group Process for Establishing Priorities. 2004, Assessed: January 2011, 3. Hung DY, Rundall TG, Cohen DJ, Tallia AF, Crabtree BF. Productivity and turnover in PCPs - The role of staff participation in decision-making. Medical Care 2006; 44: Blegen MA, Gearhart S, O'Brien R, Sehgal NL, Alldredge BK. AHRQ's hospital survey on patient safety culture: psychometric analyses. J Patient Saf 2009; 5: Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes 1991; 50: Liker JK. The Toyota Way. McGrawHill, New York, Nieva VF, Sorra J. Safety culture assessment: a tool for improving patient safety in healthcare organizations. Qual Saf Health Care 2003; 12 Suppl 2: ii Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Qual Saf 2013; 22: Ashcroft DM, Morecroft C, Parker D, Noyce PR. Safety culture assessment in community pharmacy: development, face validity, and feasibility of the Manchester Patient Safety Assessment Framework. Qual Saf Health Care 2005; 14: Law MP, Zimmerman R, Baker GR, Smith T. Assessment of safety culture maturity in a hospital setting. Healthc Q 2010; 13 Spec No: Mannion R, Konteh FH, Davies HT. Assessing organisational culture for quality and safety improvement: a national survey of tools and tool use. Qual Saf Health Care 2009; 18: Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 4. One size does not fit all. Qual Saf Health Care 2008; 17: Cooper Ph.D MD. Towards a model of safety culture. Safety Science 2000; 36: Kaboli PJ, Mosher HJ. Using balanced metrics and mixed methods to better understand QI interventions. BMJ Quality & Safety 2014; 23: Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Quality & Safety 2013; 22: Chan AJ, Islam MK, Rosewall T, Jaffray DA, Easty AC, Cafazzo JA. The use of human factors methods to identify and mitigate safety issues in radiation therapy. Radiother Oncol 2010; 97:

213 CHAPTER Rivera AJ, Karsh BT. Human factors and systems engineering approach to patient safety for radiotherapy. Int J Radiat Oncol Biol Phys 2008; 71: S Council for Public Health and Health Care. Sensible and sustainable care [in Dutch]. Council for Public Health and Health Care, Zoetermeer, Nederlandse Vereniging voor Radiotherapie. Nederlandse Vereniging voor Radiotherapie - Indicatoren. Assessed: May 2013, Beer M, Nohria N. Cracking the code of change. Harvard Business Review 2000; 78: Brackett T, Comer L, Whichello R. Do lean practices lead to more time at the bedside? J Healthc Qual 2013; 35: Dixon-Woods M, Baker R, Charles K, et al. Culture and behaviour in the English National Health Service: overview of lessons from a large multimethod study. BMJ Qual Saf 2014; 23: Glasgow JM, Scott-Caziewell JR, Kaboli PJ. Guiding inpatient quality improvement: a systematic review of Lean and Six Sigma. Jt Comm J Qual Patient Saf 2010; 36: Kaplan GS, Patterson SH, Ching JM, Blackmore CC. Why Lean doesn't work for everyone. BMJ Qual Saf Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 3. End points and measurement. Qual Saf Health Care 2008; 17: Al-Balushi S, Sohal AS, Singh PJ, Al Hajri A, Al Farsi YM, Al Abri R. Readiness factors for lean implementation in healthcare settings--a literature review. J Health Organ Manag 2014; 28: Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 1. Conceptualising and developing interventions. Qual Saf Health Care 2008; 17: McIntosh B, Sheppy B, Cohen I. Illusion or delusion--lean management in the health sector. Int J Health Care Qual Assur 2014; 27: Wood D. A prescription for lean healthcare. Healthc Q 2014; 17: Mazzocato P, Thor J, Bäckman U, et al. Complexity complicates lean: lessons from seven emergency services. Journal of Health Organization and Management 2014; 28:

214 HOOFDSTUK D Dankwoord

215 DANKWOORD Het is zover. Het boekje is af. In 2010 ging ik een nieuwe uitdaging aan en vertrok ik naar het buitenland. Dat de reis eindigde in Diepenbeek, slechts 28 km van mijn geboorteplaats, was vooraf niet verwacht. Nou, dit moest wel lukken na het bereizen van verre oorden. Van een cultuurshock heb ik eigenlijk nooit veel last gehad. Echter bleek de buitenlandse uitdaging al snel een feit. Wat had ik me hierop verkeken. Slechts 28 km van huis vandaan en toch voelde ik me aan de andere kant van de wereld. Het begon al de allereerste werkdag. Ik ging me aanmelden als promovendus, maar niemand wist waar ik het over had. Ik probeerde me als student te registreren, maar zelfs dit wilde niet lukken. Ik kon de toetsen niet vinden en kreeg mijn mailadres niet ingevuld. Ik begreep de mensen niet. Ik dacht dat we dezelfde taal spraken, maar blijkbaar toch niet. Wat gebruiken ze allemaal moeilijke woorden. En na redelijk wat van de wereld gezien te hebben en al meer dan twintig jaar werkervaring bij me te dragen voelde ik me plots heel klein worden. Tja, welkom in België. Ik wilde een uitdaging en die leek ik te krijgen ook. Na vier jaren hard werk aan mijn doctoraat, heb ik geprobeerd me zo veel mogelijk te conformeren aan de gewenste sociale gedragsnormen. Thuis werd gezegd: Je bent al een echte Belg, maar mijn collega s zullen het tegendeel beamen. Ondanks mijn vele inspanningen en pogingen me aan te passen ben ik hier slechts gedeeltelijk in geslaagd. Ik bleef de directe, overenthousiaste en soms luidruchtige (gelach) Hollander. Tja, is dit iets cultureels of meer iets persoonlijks? Ook in deze kwestie lijkt cultuur niet de enige factor die uiteindelijk het effect, het gedrag, beïnvloedt. Mijn collega s hebben zich dan ook moeten open stellen en ik moet toegeven dat dit hun aardig is gelukt. Ondanks dat ze soms schrokken van mijn directe en eerlijke reacties, werd ik toch geaccepteerd en hebben zich de nodige hilarische taferelen afgespeeld. Ik wil hiervoor al mijn Belgische collega s hartelijk bedanken. Buiten deze culturele ontdekkingstocht bleek ook het doctoraatsproject enkele verrassingen te voorzien. Ondanks dat er wat tegenslagen zijn geweest is het uiteindelijk goed gekomen en ben ik trots op het resultaat. Mijn zoontje Raf had het al snel bij het goede eind. Hij heeft ongeveer 1,5 jaar geleden een tekening voor me gemaakt, die sindsdien op mijn kast hangt. Hij tekende een schatkaart, zoals door hem beschreven, met bruggen en rotsblokken, en een weg naar de schat. Hij had het al begrepen, ik had er nog even voor nodig. Hij tekende de weg naar mijn doctoraat. Het is geen rechte weg naar het doel, maar een weg vol kronkels, aftakkingen en hindernissen. Maar als je het pad blijft volgen en onuitputtelijk doorgaat met je zoektocht, zul je de schat vinden. En kijk, na 4 jaar denken, doen, analyseren, schrijven, aanpassen, aanpassen en nog eens aanpassen, kortom hard werken, hebben we het doel gehaald: het boekje is klaar! Maar is dit nu het einddoel? Of is dit weer de start van een nieuwe uitdagende weg? Kortom, de afgelopen vier jaren heb ik enorm veel geleerd. Daar hebben mij heel veel mensen bij geholpen. Bij deze een lean dankwoord: iedereen enorm bedankt! 214

216 DANKWOORD Dat werkt natuurlijk niet. Ondanks dat ik heilig overtuigd ben van de voordelen van lean management, moet je weten waar je dit toepast. En soms, zoals in dit geval, moet je toegeven dat er een andere, betere manier is. Ik wil graag iedereen die op welke manier dan ook heeft bijgedragen aan het bereiken van deze schat bedanken voor hun aandeel. Of dit nu het aanleveren van data betrof, het invullen van mijn eindeloze vragenlijsten, het bediscussiëren van de resultaten of het zorgen voor de gewenste afleiding tijdens stress momenten, heel erg bedankt voor jullie bijdrage. Uiteraard wil ik mijn promotor Prof. dr. Dominique Vandijck bedanken. Bedankt voor de prettige samenwerking, je nuchtere kijk op ons project en het geven van de vrijheid om het project vorm te geven en bij te sturen indien nodig. Ook mijn commissieleden Madelon en Wim wil ik bedanken voor hun kritische blik en hun waardevolle bijdrage in het vormgeven van het onderzoek en het schrijven van de artikels. De leden van mijn doctoraatsjury en alle co-autheurs van de opgenomen artikelen wil ik bedanken voor de opbouwende kritiek en de tijd die men heeft genomen om bij te dragen en mij te helpen de schat te bereiken. In het bijzonder wil ik ook Jos Benders bedanken voor het stellen van de vele moeilijke vragen die mij tot denken en verbeteren hebben aanzetten. Bram, enorm bedankt voor je vriendelijke en geduldige uitleg. Zonder jou was dat Markov model er nooit gekomen en ik sta er nog steeds versteld van dat me dat (met de nodige ondersteuning) is gelukt. En het was nog leuk en gezellig ook! Dank aan al mijn collega s uit het Diepenbeekse. Mijn patiëntveiligheidscollega s van de faculteit Geneeskunde en niet te vergeten de collega s van PEC (publieke economie), bedankt dat jullie mij als Hollander tussen de bescheiden Belgen hebben geaccepteerd! Bijzondere dank aan Jochen, mijn roomie. Jij bent het lopende bewijs dat ik het soms gewoon fout heb. Waar ik je eerst in het hokje van arrogante onvriendelijke personen had geplaatst, had ik het echt volledig mis. Jij hebt mijn doctoraat fun gemaakt. Buiten je bijdrage als statisticus en verschrikkelijke criticus, heb je voor plezier en de gewenste sparring gezorgd, elke dag opnieuw. Ik wil je hiervoor enorm bedanken. Helaas is mijn andere roomie Ozhan met een nieuwe uitdaging gestart en heeft hij ons verlaten. Ook jouw nuchtere kijk op de zaken heeft bijgedragen en voor de nodige hilariteit in ons kantoor of tijdens de middagwandelingtjes (gaan we lopen.ooohh nee wandelen ) gezorgd. Mijn collega s bij MAASTRO clinic wil ik ook enorm bedanken. Het zegt genoeg dat ik na vier jaar geleden officieel vertokken te zijn, ik me nog steeds een collega voel. Huub, bedankt voor je steun gedurende het project, je filosofische ideeën en de soms heftige discussies die we gevoerd hebben.heerlijk. Petra en Rob, heel erg bedankt voor jullie patiëntveiligheidsideeën en de gezellige babbels. Ook Maria wil ik bedanken voor de waardevolle discussies. En niet te vergeten wil ik Ruud en Denis enorm bedanken. Ruud, je hebt me enorm geholpen met de benodigde statistiek en het realiseren van de vignetvragenlijst, een Excel uitdaging op en top. Denis, enorm bedankt voor het nemen van de tijd 215

217 DANKWOORD om mijn vragen met betrekking tot meldingsdata elke keer weer serieus te nemen en met de gevraagde data op de proppen te komen. We hebben hier heel wat uurtjes aan besteed. Ook Davy wil ik bedanken voor de vriendelijke en enthousiaste uitleg over de lean activiteiten die ondernomen zijn. Ondanks dat ik elke keer weer zonder afspraak met vragen en ideeën op de proppen kwam, werd er altijd tijd voor me gemaakt! Natuurlijk wil ik alle collega s van MAASTRO bedanken die mijn vragenlijsten hebben ingevuld, aan interviews of workshops hebben deel genomen en voor de gezellige babbels. Jullie zijn een enorm leuke innovatieve organisatie waar heel veel competente en vriendelijke mensen werken met een hart voor de mens. Allemaal bedankt! Ik wil eveneens dr. Tjerk van der Schaaf bedanken. Ondanks dat hij in het tweede deel van mijn doctoraatsproject niet meer betrokken was, speelde hij als promotor in het eerste deel een belangrijke rol. Ik wil hem bedanken voor zijn vooruitstrevende ideeën en zijn vertouwen in mij om dit traject mee vorm te geven. Bedankt voor de prettige samenwerking. Niet te vergeten zijn natuurlijk mijn familie en al mijn vrienden. Ondanks dat jullie soms dachten dat ik jullie vergeten had, is dit verre van de waarheid. Gelukkig bof ik met jullie allemaal en accepteren ook jullie mij zoals ik nu eenmaal ben.enthousiast, altijd bezig, vrolijk, gevoelig, maar vooral met een niet te stoppen drive. En ook al zijn onze momenten soms te spaarzaam, blijft het gezellig en genieten we van deze samen-momenten zonder boze gezichten. Ook jullie wil ik bedanken voor jullie steun en het verzorgen van afleiding. En dan nog een aantal personen die heel kort bij mijn hart staan. Mam, heel erg bedankt voor het altijd klaar staan, ook voor de kindjes. Maar mam, nog meer bedankt voor je voorbeeld. Jij, als geen ander, hebt me laten zien wat volharding is, niet bij de pakken neerzitten en genieten van het leven ondanks de vele tegenslagen. Ook al denk je soms dat ik veel op pap lijk en minder op jou, dat is niet zo. Die vrolijkheid, het gekke, het enthousiaste en de enorme wil om ergens voor te gaan, heb ik allemaal van jou! Ik ben enorm trots om jouw dochter te zijn en hoop dat je me vergeeft dat we niet zo veel tijd samen doorbrengen. Appie, hoe moet ik jou bedanken? Jouw rol in dit avontuur was onontbeerlijk. We hebben inhoudelijk gediscussieerd, maar ook heb je me gesteund in het vinden van mijn weg. Niet enkel door het opvangen van de kids en het huis als dit nodig was, maar ook als gesprekspartner of gewoon eens lekker los te gaan op de dansvloer. Ik ben je enorm dankbaar en hoop nog heel lang met je te mogen genieten van de komende avonturen. Raf en Kim, mijn allerliefste kindjes. Ondanks dat ik zo veel mogelijk quality time met jullie heb willen doorbrengen, realiseer ik me dat dit zeker niet altijd gelukt is. Jullie zijn mij het dierbaarst van alles op deze wereld. Ik wil jullie enorm bedanken voor mij soms mama vrijaf tijd te geven. Maar vooral wil ik 216

218 DANKWOORD jullie bedanken voor de spiegel die jullie me voorhielden als ik gestrest was en voor de benodigde afleiding die jullie hebben verzorgd. Als laatste, maar zeker niet minste, wil ik mijn vader bedanken. Pap, ondanks dat je er niet meer bij kunt zijn, wil ik je bedanken voor alles wat je me hebt geleerd. We hebben in november 2006 onverwacht afscheid van je moeten nemen. Je aneurysma scheurde toen ik op vakantie was. Wonder boven wonder wist je dit te overleven. Helaas moesten we na een korte herstelperiode alsnog onverwacht afscheid nemen. Een gedeelte van je darm was afgestorven en je bent uiteindelijk aan een buikholteontsteking (sepsis) overleden. Medische fout of niet? We weten het niet en eerlijk gezegd, het boeide niet. Het resultaat veranderde niet en bleef desastreus. Nog altijd doet deze gedachte pijn. Gelukkig echter blijf je in mijn hart en blijf je mijn grote voorbeeld. Ik hoop dan ook ten zeerste dat dit doctoraatsproefschrift niet het eindstation is van mijn bijdrage aan kwaliteitsverbetering in de zorg. Zo graag zou ik willen voorkomen dat mensen slachtoffer worden van gebreken in het zorgsysteem. Ter nagedachtenis van mijn vader draag ik dit proefschrift op aan jou pap, Martin Simons, voor altijd in mijn hart. 217

219 DANKWOORD 218

220 CHAPTER C Curriculum Vitae and Publications

221 CURRICULUM VITAE Pascale Simons was born on September 25, 1975, in Maastricht, the Netherlands. After completing secondary school (HAVO) at the Stedelijk Scholengemeenschap Maastricht, she started an apprenticeship for x-ray technician at the former Maasland Hospital (at current Orbis medisch en zorgconcern) in 1993 and graduated in In 1997 she left to travel through Australia for six months to broaden her horizon. In the same year, she started an apprenticeship for radiation technologist at the former Radiotherapeutisch Instituut Limburg (at current MAASTRO CLINIC) in Heerlen and graduated in From 1999 she worked at MAASTRO clinic as a radiation technologist. In 2001, she combined her jog as radiation technologist with a study Health education and Health promotion at the faculty of Health Sciences of Maastricht University. She graduated as master in Health Sciences in In 2007, she changed jobs partly to patient safety officer and partly to researcher related to lean management projects. As a patient safety officer she was responsible for the implementation of a patient safety management system in the organization, in which retrospective and prospective risk analysis were combined and supported by a management structure. As a researcher she was involved in a project, project 3-2-1, based on lean management principles to improve quality, safety and efficiency of radiation treatment. From 2010 until current, she worked as a PhD student at the Faculty of Business Economics at Hasselt University in Belgium. During these years, she followed MAASTRO CLINIC in her transition to a lean organization. She evaluated multiple outcomes on the level of the patient, the employee, and the organization with a specific focus on quality and safety of care. Currently, Pascale Simons is looking for opportunities and new challenges to meet her drive to improve quality, safety, and efficiency in health care. 220

222 PUBLICATIONS Publications international (refereed) journals - Simons PA, Houben RM, Backes HH, Pijls RF, Groothuis S. Compliance to technical guidelines for radiotherapy treatment in relation to patient safety. Int J Qual Health Care 2010; 22: Simons PAM, Houben R, Benders J, Pijls-Johannesma M, Vandijck D, Marneffe W, Backes H, Groothuis S. Does compliance to patient safety tasks improve and sustain when radiotherapy treatment processes are standardized? European Journal of Oncology Nursing 2014; 18: Simons PAM, Houben R, Vlayen A, et al. Does lean management improve patient safety culture? An extensive evaluation of safety culture in a radiotherapy institute. European Journal of Oncology Nursing. In Press, Corrected Proof, Available online 27 September Simons PAM, Benders J, Marneffe W, Pijls-Johannesma M, Vandijck D. Workshops as a Useful Tool to better Understand Care Professionals Views on a Lean Change Program. International Journal of Health Care Quality Assurance, Corrected Proof. - Bergs J, Hellings J, Cleemput I, Simons P, Zurel Ö, Vertriest S, Vandijck D. Surgical Safety Checklists: an update. Acta Chirurgica Belgica. 2014;114: Book Chapters - Pascale Simons, Huub Backes Op weg naar een continu lerende organisatie You can t improve what you don t control. In: Lean in de Zorg (2012) Ed. Rouppe van der Voort M, Benders J. Boom Lemma: Den Haag. (ISBN: ) Conferences Oral presentations international conferences - P.A.M. Simons, M. Pijls-Johannesma, G. I.J.M. Kempen, P. Lambin, D. de Ruysscher. What is the best method to evaluate the Quality of Life during curative radiotherapy? Annual ESTRO meeting 2006, 8-12 October, Leipzig, Germany. 221

223 PUBLICATIONS Oral presentations national conferences - Oktober 2014, Lean management and patient safety culture at UHasselt research conference patient safety and safety culture, Federal Civil Service Public health, Brussel, Belgium - January 2012, book presentation: Op weg naar een continu lerende organisatie You can t improve what you don t control in: Lean in de Zorg (2012) at the national conference: Lean in healthcare, Tilburg, the Netherlands - March 2008, workshop Lean: Project 3-2-1, a Lean application at the national conference: Lean in healthcare, Tilburg, the Netherlands - November 2007, Incident reporting at MAASTRO clinic at the regional conference on patient safety in Limburg, Maastricht, the Netherlans Poster presentations international conferences - April 2014: Pascale Simons, Katrien Spaas, Luc Pannekoeke, Madelon Pijls-Johannesma, Wim Marneffe and Dominique Vandijck. Weekly Brainstorm Sessions Enhance Bottom-up Innovation and Employees Commitment to Improve Radiotherapy Care at the International Forum on Quality & Safety in Healthcare, Paris, France - April 2013: Pascale Simons, Ruud Houben, Madelon Pijls, Wim Marneffe and Dominique Vandijck. Evaluating a Factorial Survey on Patient Safety Actions to Measure Change in Relation to Patient Safety Culture at the International Forum on Quality & Safety in Healthcare, London, England Poster presentations national conferences - November 2009: Reijnders P, Simons P. Communication of patient safety in MAASTRO clinic at the regional conference of patient safety in Limburg, Thorn, the Netherlands 222