SCMS ARV Bridging Assistance Fund Final Report, Chapter 8 Chapter 8. Incident Management While service delivery is planned to achieve high performance, unplanned incidents will invariably occur during program delivery. Incident reporting and management are a critical part of the supply chain. When incidents occur, the reaction must enable rapid response, optimal resolution, effective analysis to determine root cause, and swift mitigation to avoid repeated issues. From program outset, SCMS implemented an industry standard incident management system to track and resolve incidents. This included following an incident resolution SOP, as well as creating incident lists and incident reports. Communication was fundamental in resolving incidents, from acknowledging issues to agreeing on resolution and establishing steps to avoid future recurrence. Incidents enabled the project to monitor benchmarks for performance at every step of the supply chain. The incident management process is an opportunity to drive continual process improvement in all domains of operation through identifying urgent issues, establishing expectation benchmarks and consistently monitoring and trending incidents to avoid repeat occurrences. Process In this section, incidents that occur during program implementation are identified and resolved, and the root cause assessed to avoid future occurrence. Incidents can occur throughout the supply chain, and the appropriate response varies depending on the details of the particular incident. The process described below provides a general outline. The overall goal of incident management is to identify, manage, resolve and document incidents for continuous improvement. Step 1. Notification of incident and initial review The incident occurs and SCMS is informed of the situation. Preliminary investigation is performed, and a deadline is set for written documentation of the incident. Responsible: Table 8-1 summarizes the party responsible for initiating the incident management process, depending on the stage in the supply chain process when the incident occurs. "#"
SCMS ARV Bridging Assistance Fund Final Report, Chapter 8 Table 8-1. Parties responsible for initiating incident management "#$%"&'&()* +(,#%(-* +(.&()#"/* 0%),#%(-* 1&23#(24,454)/* 6#"*"&3#")4(7* Action: "#$%&#'(')* &)+* "#$%&#'(')* &)+*$#,-.,-.,-.$#/'0$* 1) Discovery of an unforeseen or out-of-the-ordinary event. 2) Report provided to SCMS within 48 hours. 3) SCMS contacts the person who discovered the incident and obtains an oral summary of the incident and issues involved. 4) SCMS requests the reporting party to document the incident in the standard format and sets a deadline for the report to be received. 5) The initial report is logged in the incident register for follow-up. Table 8-2. Incident register Step 1 outcome: Initial report received and incident management process initiated. "#$
Step 2. Review of incident report and management The aim of this step is to resolve the incident promptly and effectively. Responsible: SCMS field office (FO), SCMS procurement, LDC and depot/facility Action: 1) SCMS FO receives the incident report and reviews the documented information to establish the facts. Figure 8-1. Incident description form Incident description form Document date: 27/10/2010 Detailed description of event, product condition, analytical findings, etc: PO 6903-0 A pre alert was sent to Eastern Cape (Mthatha) for 3 504 units of Zidovudine 300 mg on 22 September. 15 boxes of 216 units were sent to the depot however the box states 175 units. On some of the boxes the 175 was scratched out and replaced with a handwritten number of 216, however on most of the boxes this was not the case. The LDC ref for Picking and Packing 85005415 showed that the units were not repacked by them, the box originated from the manufacturer. The box dimensions are also different to the LDC. The manufacturer box dimensions are 31x44x33 and the LDC box dimensions were 31X45X30. The same incident occurred at the Free State Province for the Freestate Province where they claimed to have received 779 units and 1722 units short for Lamivudine 150 mg. This incident was discovered by Neo (MSH ARV monitor) on 6 Oct 2010 Date discovered: 8 October 2010 Discovered by whom: Provincial depot manager Location/Site discovered: Eastern Cape (Mthatha) List of impacted batches, lots, etc: (include # of units if applicable) Batch ZN3010026A 15 of 216 units Manufacturer information (Mfr, location, etc):aurobindo Immediate corrective actions taken (i.e. Product placed in quarantine, place in appropriate storage, wiped down, etc): Load incident on CRM to notify PSA Requested the Depot manager to open a box that stated 175 units on the box and count the number of units Person Responsible for Action: Rohini Shookan Date for completion of action: 26 October 2010 "#%
Any Suggested Corrective Actions: SCMS ARV Bridging Assistance Fund Final Report, Chapter 8 Person Responsible for Action: Date for completion of action: PSA to investigate incident with Aurobindo and advise on corrective actions going forward R.S 29 October 2010 Incident ready for closure: No Closed By: - 2) SCMS FO contacts the responsible stakeholder (submitting the report) to present the incident facts and discuss the requirements to resolve the incident. 3) SCMS FO notifies all other stakeholders about the report result and the actions agreed on to resolve the incident. 4) SCMS FO awaits required feedback from the responsible stakeholder and ensures that the resolution steps are completed effectively. 5) If feedback or resolution is not achieved as expected, SCMS FO reinitiates contact with the responsible stakeholder and identifies actions needed to close out the incident. Step 2 outcome: Resolution process specified and initiated. Step 3. Incident closure Responsible: SCMS FO Actions: 1) Once satisfied that all required actions have been completed, SCMS completes the closure section of the report, as shown in Figure 8-2. "#&
SCMS ARV Bridging Assistance Fund Final Report, Chapter 8 Figure 8-2. Incident report closure section Incident ready for closure: Yes Closed by: Brandon Copley Incident raised with procurement unit, and forwarded to the vendors through an email communication with Glen Vincent. Date of closure: 14/09/2011 2) SCMS files the incident report on the LDC shared drive and updates the incident register. Step 3 outcome: Incident and resolution documented and filed. Process consideration While the previous process description provides a high-level discussion of a generic incident management process, every type of incident involves a different mix of responsible stakeholders, and the correct remedial actions depend on which stage of the supply chain is affected by the incident. As a result, the program drew a number of detailed workflows that defined specific actions and actors to be involved in the resolution of incidents. An example of a workflow for incident management is provided in Figure 8-3. "##
Figure 8-3. Sample workflow for incident management "#$%&'()&"(#%*(&"'+%#',-."(/%&()#0($)1)2"'(+%'*(&"3",4,#2()*(5678(567(*%(,#4"'*,2)*"()#$(*9"&"( '9%:.$(#%*(-"($%:-*()'(*%(*9"(&%%*(3)'"('9%:.$(*9"(4"#$%&(-"(/%:#$(*%(-"(&"'+%#',-." :;< :29>$%01$&%9> D1*0-256$ 1%7%8*+ BC$-($ 1*-*'720*$ '*+&(0+234*$ &%'-6 /*01('$ 1%7%8*?, ;%014208$ 1%7%8*$ &'(9*++ "(7&4*-*$ 20921*0-$'*&('-$ 2094=1208$ &29-='*+ @EF$ 1*+-'=9-2(0$ &'(9*++ "4(+*$(55$"@# "# "#$'%2+*+$ 20921*0-$(0$ A'2(0$"@#, "#$ %&&'()*+? :'(9*++$ 7%0%8*1$-($ *0+='*$ '*&4%9*7*0-$('$ "'$0(-* :C$'%2+*+$ 20921*0-+$G2-H$ /*01('+ /*01('+$-($3*$ 0(-252*1$-($ 20+&*9- "(052'7* 1$)*01('$ 20921*0- :C? :;<$-($ 7%0%8*$ 1%7%8*+,? /*01('$ -($ '*&4%9* <%-*$%01$-27*$ (5$'*&4%9*7*0-$ +-(9>$%''%08*1, /*01('$ 9%49=4%-*+$ 9'*12-$0(-*$ )%4=*,(-*.$/*01('+$%'*$0(-$'*+&(0+234*$5('$%06$1%7%8*+$&(+-$'*9*2&-$%-$:;<$=04*++$)*01('$%8'**+$&(+-$20+&*9-2(0 Challenges Incidents and unexpected occurrences are inevitable in any project or program. If they can be managed effectively to closure and used to train resources on necessary changes in behavior or actions, improvements and efficiency will result. Product incidents in the program required multiple stakeholders to be involved to ensure resolution, with short timelines to correct in order to prevent undue negative impact on the supply chain. Example: Incorrect package inserts. One incident that caused delays to the program closeout was the incorrect package inserts accompanying the Nevirapine 200mg tablets from Cipla. Unfortunately, the effort to correct this incident was manually intensive and caused up to two months of delays in distributing the product to provincial depots. Product that had already been delivered was picked up by the vendor, and the correct package inserts were inserted at the vendor s warehouse. In other instances, the repackaging occurred before shipment, causing delays in the inbound process. In both instances, resolution required "#'
cooperation and communication across vendors, procurement unit, SCMS field office, and the LDC. Example: Syringes packed separately. A further incident that caused delays was the separate packaging of syringes in round 5 for the Lamivudine solution. SCMS was notified of this variance only when the first round 5 order of Lamivudine solution was ready. The vendor provided the syringes separately, and the LDC packaged the syringes by province according to the Ministry of Health allocations. This was still not ideal, as it put strain on the whole supply chain in the pick-and-pack processes, and separately packed syringes consumed additional space. Despite these disadvantages, this was the best resolution that would avoid delaying product availability and risking a stockout, as supplies were very limited. Inbound incidents occurred most often when vendors were new to the process and still learning SCMS SOPs. Example: Hetero open truck. Upon their first SCMS delivery to the LDC, Hetero (vendor) delivered the product on the back of an open vehicle from the airport, with only netting over the pallets. This is an unacceptable practice for pharmaceutical transport, as it allows exposure to the weather (rain) and potential shrinkage. This incident was treated very seriously such that two incident resolution processes were followed: that of the South Africa field office and that of the SCMS global project. The immediate response was to photograph the truck and report the incident to the PMO. The open-truck incident was also reported to Hetero management and SCMS met with the Hetero South Africa team to walk through the shipping instructions and explain the consequences of further incidents. These actions ensured all shipments that followed were conducted with correct procedures. Figure 8-4. Documentation of the open truck delivery "#(
While the incidents on the ARV donation program were few in light of the quantities procured, warehoused and distributed, it was essential to take the time to report and manage all incidents to obtain fast resolution and avoid recurrences. SCMS established tracking mechanisms from the start of the ARV program with clear benchmarked milestones, e.g., 24 hours to GRS the receipt of product into the warehouse, 10 days for sampling and release of product. In many instances the stakeholders, while agreeing to the benchmark, did not track this requirement, and SCMS had to drive the process to provide feedback on service. In each incident, SCMS requested nonconformance reports that at times were completed not to the required quality documentation standards expected. SCMS worked with stakeholders in editing the incident report to ensure that quality reporting and sufficient detail and follow-up were achieved. When working in the supply chain with multiple partners, clear handover points of responsibility were required to determine responsibility for where an issue may have occurred. Reporting timelines in particular need to be tightly controlled. Incidents were not always reported within 48 hours, which was the timeframe established in the program. This timeframe allowed for issues to be raised with the stakeholder handing over to the receiving stakeholder. Late reporting created challenges when following up with the parties to establish where the issue occurred and who had the responsibility for fixing it. An example of this challenge was receiving commodities from a local applicant holder with a stock count difference. While damages that occurred during the inbound process were the responsibility of the local applicant holder, if the receipting process at the LDC did not report damages in 48 hours, then issues arose. To resolve this, SCMS agreed with the LDC that after 48 hours, the LDC would be responsible for the incident. Lessons learned Rigorous incident management creates process improvements and enhanced service delivery. Example: Incident outbreaks. A program of this scale has times when incidents increase noticeably. Introducing new stakeholders to unfamiliar processes increases the risk of errors. SCMS also noted that as the volumes increased, greater attention was needed to track and follow up on expected outcomes and to ensure any derailed matter was quickly resolved. For a period of time, with the large quantities being delivered to KwaZulu-Natal, the vehicles arrived later than scheduled times at the depots. When investigated, the root cause was identified as an extended vehicle loading period. Hence, vehicles left late for their destination. This was resolved by including additional time for packing and ensuring vehicles left the LDC by a certain time to arrive on time. During round 5, the project moved over 8 million units to 10 depots, equating to about 125 shipments per month (six per day) "#)
versus a third of that volume in round 1. The increase requires constant recalibration of planning to consistently achieve deadlines. Including incident management as a routine practice in the supply chain ensures continuous status review and steady progress toward consistent improvement. By recording incidents on established tracking systems and through consistently monitoring the status of incidents at weekly meetings with the procurement team and the LDC, each incident was managed through to closure. Where any issue remained unsolved or was delayed for an unreasonable period, SCMS escalated the matter to senior management for assistance to find a solution. With the high level of focus and expectation set on incident management by SCMS, stakeholders understood the expected service levels and responded accordingly. Another advantage of tracking incident trends was that outcomes provided insight into the cause of the issue and this could be used in future planning. For example, at the beginning of the program, sampling periods were adhered to in different timelines by different applicant holders. This resulted in a review of the process for each local applicant holder to identify their testing process and then refine timelines and handoff points. This aided in future activity planning. Identifying one responsible individual for incident management within each stakeholder organization was essential for tracking all the relevant details and closing incidents. Typically, each organization had multiple individuals assisting in different areas of responsibility. Accountability needed to be with one individual to drive the incident process. For example, in the sampling process, the extracted samples are delivered to the applicant holder, who delivers the sample to the selected laboratory. Then, the reverse process happens to report the outcome of the testing. SCMS worked with the applicant holder to ensure delivery of the sample and receipt of the sample release only. The in-between processes were the responsibility of the applicant holder, but each step potentially impacted the outcome and hence where issues arose, the full process was assessed to determine relevant resolution. For the LDC support activities, SCMS required root cause analysis to be managed by reviewing all impacting elements of an incident. For example, if an incorrect quantity was reported at a depot, the incident review needed to examine each step in the process, such as picking slip generation, picking activity, and dispatch loading through to receipt of proof of delivery by the depot to identify all points where the error was caused or should have been corrected. "#*
SCMS ARV Bridging Assistance Fund Final Report, Chapter 8 Outcomes For a program of this size and complexity, the number of incidents was minor, as a result of setting the standard upfront and then consistently measuring various program components. Clear procedures and consistent monitoring maintained a high level of focus on incidents and held stakeholders responsible for achieving established benchmarks and delivering expected outcomes. Table 8-3 indicates key steps in incident management, and provides a tracking tool for the incident resolution process. This tool, along with the incident register, allowed the program to see incidents through to completion and to assess the overall process of tracking and resolving incidents. Table 8-3. Key steps in incident management "#$%"&'()*#%++ ",-.'/0'1234.156"#$#%&'()*+,-.//01234./560275 +5.78'57'0/99/:'5/' 2/-79.;',1'1234.15 #/9/<;'=>;..1?' 2/-79.5.@'*;,1A?'31' 7;/A;.88',14').4'85399' 5/'B.'4/1.C,>?,@48):);8#:8 -)89(9:#89);)( A&*');)**&A*&'&;8#89B& <;:9=&;88) CD)=9':)B&*&=8D&,>?, 9;:9=&;8 +'E9;8989#FGH&'89);'8) H;=&*'8#;=8D&9;:9=&;8 #;=%#8D&*9;989#F(9;=9;%',&8#=#8&()*8D& <;:9=&;8I&A)*88)J& :)$AF&8&=#;==9#*9'&K >);8#:8*&'A);'9JF& '8#E&D)F=&*#;=9;()*$ )(8D&'98H#89);#;= *&GH9*&$&;8'8):F)'& 8D&9;:9=&;8 I&B9&C)*:D#'&:);8#:8 A&*');()*8D&<;:9=&;8 I&A)*8 I&7:);8#:8'8#E&D)F=&* 8)*&#''&''#;L *&GH9*&$&;88):F)'& 8D&9;:9=&;8#;= (9;#F9'&*&GH9*&$&;8' +C#98:)$AF&89);(*)$ 8D&'8#E&D)F=&*)(* (&&=J#:E(*)$8D& #((&:8&=:F9&;8 >F)'&8D& 9;:9=&;8 *&A)*8C98D A)'989B& )H8:)$&' @9F&8D& I&A)*8 At the end of the program, the total number of incidents was tallied across the different supply chain components. Table 8-4 tallies the number of incidents by supply chain area as well as by type of incident. Table 8-4. Number of incidents in incident register "#$&22* 4($4-&()* 84'&54(&* 4($4-&()* 9%:()4)/* 4($4-&()* 9%:54)/* 4($4-&()* * +(,#%(-*;:(:7&'&()* <* =* >>*?@* +(.&()#"/*;:(:7&'&()* <* >* A* B* 0%),#%(-*;:(:7&'&()* <*?*?* =* "'+
Process incidents, where SOPs were not followed, timeline incidents including late and early deliveries, and quantity incidents involving delivery of either extra quantities or short supply, were relatively rare based on the data, although may not have been comprehensively reported. Quality incidents were carefully tracked however, particularly at the point of receiving product into the warehouse, as this marked the point of handoff from the vendor to the LDC. This explains in part why the number of quality incidents arising during the inbound process is relatively high. Although quality incidents made up the majority of incidents that were tracked, the number of incidents is not reflective on the overall impact on the program. The project gathered detailed data on all incidents that caused product loss, thereby impacting availability of product for the program. Through close monitoring and driving of expectations, overall the project managed a minimal level of 0.05%, or 11,360 units being short supplied or damaged during the supply chain. These very low quantities are the result of good procurement, freighting, warehousing and distribution management through the supply chain. Particularly when shipment mode moved from air to ocean, a significant improvement in quality was noted. Table 8-5 presents the short supplied or damaged quantities throughout the program. At no phase of the process were damages higher than the tolerance threshold. It is also worth noting that of the product with reduced quantities, the majority of reduction in available units (71.7%) was due to vendor management or quality issues and not to handling issues in the SCMS supply chain. Although overall minimal compared to the quantities procured, the one factor that caused product unavailability most often was reduced supply. The figure would have been higher had the program not proactively tracked received quantities and requested vendors to make up the differences on subsequent shipments. In cases where short supply could not be rectified with product replacement, vendors issued a credit note, ensuring that the project did not pay for goods that were never received. "'"
Table 8-5. Program product loss summary Vendor Inbound Inventory Outbound Product Description Ordered Quantity Short/ Over Supply Quality Damages Transport Damages Warehouse Damages Outbound Damages Total Short/ Damaged Abacavir 20mg/ml Oral Sol 460,884-424 0-68 0 0-492 -0.11% Abacavir 300mg Tabs 47,600-1 0-6 -2 0-9 -0.02% Efavirenz 200mg Tabs 130,000 0 0 0 0 0 0 0.00% Efavirenz 50mg Tabs 552,000 0 0 0 0 0 0 0.00% Efavirenz 600mg Tabs 5,064,611 186 0-930 0-192 -936-0.02% Lamivudine 10mg/ml Oral Sol 543,274-384 0-402 0 0-786 -0.14% Lamivudine 150mg Tabs 6,603,127-2,727-1,622-142 -29-18 -4,538-0.07% Lamzid 150mg /300mg Tabs 100,000 4 0 0 0 0 4 0.00% Nevirapine 200mg Tabs 2,522,527-2,174 0-519 0 0-2,693-0.11% Nevirapine10mg/ml Oral Sol 241,480-317 0-89 -31 0-437 -0.18% Tenofovir DF 300mg Tabs 4,459,000-154 0-374 -348 0-876 -0.02% Zidovudine 300mg Tabs 1,186,139-730 0-137 0 0-867 -0.07% % of total 21,910,642-6,721-1,622-2,667-410 -210-11,630-0.05% Reporting and monitoring Key Performance Indicator: Incident reports filed on time Interim Tracking Indicators: Initial report received, resolution process initiated, report close Key Outcome Statistics: Number of incidents, number of incidents resolved, level of damages for the program Data Sources: Incident forms, incident register "'$