9 TROUBLE AND COMPLAINT HANDLING

Transcription

1 9 TROUBLE AND COMPLAINT HANDLING The offeror shall describe its organization, resources, strategies, practices, policies, processes, procedures, tools, systems, reports and any other relevant capabilities to provide the Government with a high degree of confidence that the offeror has sound, effective, and adequate management, technical, and operational capabilities that meet Government trouble and complaint handling requirements. (L ) 9.1 ORGANIZATION (L ) The Sprint commitment to GSA: The 24 x 7 Business Service Assurance Center applies Sprint Make life easier for Agencies that understanding of the Agencies budget, provision and pay for Sprint critical missions and infrastructure to provide worldclass trouble resolution handling. services. Sprint will provide a responsive trouble resolution process for GSA and Continuous, single point of contact for all trouble and Government customers. Sprint complaint reporting Online visibility to status and understands that there will be instances reporting via the Sprint Networx when a service outage, service Web Portal Expeditious trouble resolution degradation, or other issue requires the through well-defined processes and escalation procedures customer to open a trouble ticket or complaint. Located in Reston, VA, with a live back up site located in Atlanta, GA, the Business Service Assurance Center (BSAC) is the single point of contact for users wishing to report a trouble or complaint. The BSAC is a focal point for customer advocacy, with a primary mission to solve customer problems. Advocacy results in problem resolution and requires a combination of staff, experience, technology, and enabling processes. In addition to the BSAC, the Network Service Management organization provides an overlay level of support to the Government for all troubles or issues that require management outside of standard processes, escalation, or other focused attention. These organizations are already in place today, supporting FTS2001 customers, and will be there day one of the Networx program. Figure shows the management structure for the BSAC and Network Service Management organizations. Page 207 March 5, 2007

2 Figure BSAC Management Structure The BSAC and Network Service Management teams have years of Government experience, bringing a strong understanding of Agency needs. 9.2 RESOURCES (L ) The goal of the BSAC is to manage Sprint BSAC has a proven track issue resolution expeditiously while record in exceeding performance requirements. providing regular, timely updates to the FTS2001 time to answer SLA requirement: 18 seconds 95 Government. Pledging a commitment to percent of the time deliver excellence to our customers starts Actual BSAC performance Jan 04 Jun 05: 18 seconds 97.5 with our people. Meeting customer percent of the time expectations and providing quality customer support begins simply: one person at a time. Technology cannot over-shadow a single person s commitment to deliver excellence. To that end, the Sprint BSAC will be an integral part of the service Sprint provides the Government in support of the Networx Program. Page 208 March 5, 2007

3 Sprint staffs the BSAC and Network Service Management organizations with highly experienced, fully capable personnel familiar with the requirements of Government users. Recognizing that the needs of the Government are sometimes unique or beyond those of our commercial customers, Sprint provides the support teams specific training tailored to meeting the needs of the Government. This formal training program is mandatory for all members of the BSAC and, requires a passing score to achieve a position on the team. Government experience and training that focuses on the needs of the Government customer result in a center that is prepared today to meet all the needs of the Networx users. The primary BSAC is located in the Sprint Government support center in Reston, VA and focuses on the Government user. The back up center located in Atlanta, GA has trained individuals to support the Government and has done so for both FTS2000/FTS2001. This back up center approach provides the Government the assurance that Sprint can support Agencies during disasters such as snowstorms and other weather events, citywide or local power outages, building evacuations, equipment failures, acts of terrorism, and other disruptions. Sprint tests its back up processes and procedures periodically to ensure preparedness. BSAC personnel exceed customer expectations through proactive communications Government customers such as the FBI, U.S. and expeditious repair and confirmation of Attorneys, Department of Veterans Affairs, and the restoration. The BSAC staff is comprised of GSA have cited BSAC personnel for providing professionals who know Sprint products and exceptional support. services and are experienced with the handling of sensitive but unclassified (SBU) and priority issues, such as DOJ d security requirements and Telecommunications Service Priority (TSP) services. Page 209 March 5, 2007

4 Each BSAC employee is required to complete a ten-week classroom training program successfully before becoming an active help desk representative interfacing with customers. Each Agency will be assigned a Network Service Manager (NSM). The NSM performs the following: Oversees service assurance for each Agency, acting in a proactive manner monitoring service levels and compliance to processes Oversees escalations, progress, and communication if a trouble or complaint requires management outside the defined process Acts to provide support to Agencies on service assurance processes, training, inventory, and Time to Restore service level compliance Functions as a single point of contact to the Government for service assurance issues that go beyond the standard repair process Schedules routine meetings with the primary points of contact within the Agency to establish an ongoing and open communication channel. 9.3 STRATEGIES (L ) Service assurance is a top priority for Sprint. The BSAC provides focused trouble management and complaint handling by serving as the Government s advocate and assuming ownership of the repair process. We provide: A single point of contact 24 x 7 x 365 Trouble ticket management and ownership to resolution Reporting via the Sprint Networx Web Portal (NWP) Expeditious trouble resolutions driven by aggressive, self-imposed escalation procedures Tickets kept open until the customer verifies service restoration or issue resolution with a Sprint employee directly. (J.9 ID 10351) Page 210 March 5, 2007

5 Sprint provides the Government secure online real-time access to trouble and complaint entry and reports. Utilizing discretionary access controls, users only have access to the functions and information for which they are authorized. The Sprint NWP enables users to view status updates as frequently as they wish. (J.9 ID 10364) Figure shows a sample of the trouble reporting home page. While internally our trouble and complaint handling systems are not integrated, the Networx user will be able to view status of either through a single interface. (J.9 ID 10357) These fundamental strategies enable Sprint to focus on and be accountable for trouble resolution, giving customers the assurance that their issue is safe. To assess our success, Sprint will monitor our performance against the Time to Restore SLA and report on performance monthly to the GSA PMO. XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 9.4 PRACTICES (L ) The BSAC monitors and manages the flow of all trouble reports, both domestic and international, to provide timely resolution of troubles. The BSAC adheres to some basic practices to meet this goal. Activities that help Sprint be successful include, but are not limited to: The BSAC is supported by several online tools within its trouble reporting system to provide visibility of the ticket flow and to track status from the time of ticket generation to closure. Page 211 March 5, 2007

6 NUM-199_2 Figure Sprint Online Trouble Reporting System Online access to trouble ticket information enables the authorized user to view status notes so they can monitor progress to issue resolution. The trouble reporting system identifies chronic problems. If a service has had multiple trouble tickets opened against it within a one-month period, the system automatically flags it as a potential chronic problem so that our technicians can take appropriate action. Tickets are updated regularly with information on repair activities. Updates include notes from the fix agencies working to repair the trouble. Technicians note troubleshooting activity, test results, trouble identification and isolation, and final disposition. Page 212 March 5, 2007

7 Sprint notes any escalations of the ticket. This allows anyone reading the ticket notes, including the Government, to view any escalation activity that takes place. The BSAC records information on any verbal notification Sprint provides to the customer. Verbal notification data includes the name of the Agency contact that was notified, time of the notification, and the name of the Sprint representative who performs the notification. Sprint also employs a group of individuals whose sole function is to analyze Key Performance Indicators (KPIs) as they relate to the trouble and complaint processes. KPIs include, but are not limited to the following: Average speed of answer Call handling time Time to repair individual trouble tickets Time to close individual trouble tickets Number of tickets opened and closed per month Percentage of timely statuses on individual trouble tickets. These KPIs, as well as many others, are monitored, hourly, daily, weekly, and monthly to ensure that operations support groups comply with service level expectations. The Sprint BSAC trends all activities as it relates to ticket volumes, mean time to repair, average speed of answer, etc. This trending allows Sprint to adequately plan the staffing resources necessary to ensure that trouble and complaint handling is maintained up to the required standards and practices. Figure shows how Sprint measures KPIs related to ticket resolution and closure include time to repair (TTR), time to fix (TTF), time to notify (TTN), and time to close (TTC). Page 213 March 5, 2007

8 9.5 POLICIES (L ) To implement our strategy of focused trouble management and complaint handling, the Sprint BSAC adheres to a customer friendly, responsive, and ethical help desk policy that the Government can count on during every interaction. We maintain corporate standards of conduct for all employees; these standards require acknowledgement by every employee. Additionally, Figure BSAC Key Performance Indicators Analyzing metrics associated with key performance indicators enables us to identify early any potential problems and to take appropriate action. the BSAC organization has operational procedures in place that address both process and conduct. BSAC management is responsible for manage these Page 214 March 5, 2007

9 processes and policies though monitoring, coaching, and corrective action when necessary. Maintaining the Sprint reputation in the industry and with the Government for high standards in customer service and honesty is of great importance to us. These principles guide our employees daily and support our commitment to deliver to GSA and the Agencies the highest level of support available in a prompt, professional, and courteous manner. 9.6 PROCESSES (L ) Government personnel reach the BSAC by: Calling Networx trouble reporting toll free number Calling the trouble reporting number and selecting the option to report a trouble or complaint Initiating a ticket online via the Sprint Networx Web Portal. Non-domestic callers can reach the BSAC via telephone by either calling the toll free number, (from countries where the number has been established), or by placing a collect call to a standard 10-digit number that will route through to the domestic Networx VRU. The VRU accepts collect calls. Once the callers are at the VRU, all non-domestic troubles and complaints employ and operate on the same system and procedures as the domestic callers. The BSAC performs the following functions: Receives trouble reports from the customer and collects all pertinent information necessary to support timely resolution Provides online chronic issue identification at the time of ticket entry Tracks resolution activity via the Sprint trouble reporting system Coordinates repair activities with the customer and the appropriate fix agencies Obtains trouble report status updates and closure information Page 215 March 5, 2007

10 Provides trouble report status feedback to the customer on a periodic basis, including reporting and verifying closure of service trouble report Provides information to the NSM in critical situations Communicates and coordinates trouble report information with the fix agencies to the appropriate technical support group. Once a ticket is open, the BSAC representative routes the ticket to the appropriate internal organization for analysis and resolution. The representative routes complaints to Customer Service and service issues to the appropriate fix agency. Sprint has well-defined processes and procedures for resolving troubles. By constantly monitoring performance and applying quality principles, including root cause analysis and continuous process improvement, Sprint has refined our fault management procedures over the years and will continue to do so. Terms used in the trouble management process are defined in Table The ticket flow process during isolation and repair is shown in Figure following the table further depicts this process. Table Terms Used During Fault Isolation and Repair Repair Stage Definition Stage 1 The trouble ticket has been entered into the fault management system, Auto Test or electronically forwarded to the repair agency, automatically screened, Pending populated with appropriate test information, and picked up by a repair agency Completion technician. Stage 2 The technician has reviewed the information and performed the appropriate Segmentation tests to segment the problem. Based on the results, the technician will route and Referral the ticket appropriately: 1) electronically routed to the responsible fix agency responsible, 2) electronically bonded or manually referred to the Telco to open a ticket in their respective system, or 3) referred to the User via a telephone call to provide the tests. Stage 3 Isolation If the Stage 2 testing indicated the fault was caused by a managed network or security element, the ticket will be routed to an IOC technician who performs additional tests to further isolate the specific failing segment and analyze the various service elements. Successful completion of this stage enables the technician to identify the suspected failing hardware, circuit element, or software. Page 216 March 5, 2007

11 Repair Stage Definition Stage 3 Telco: If the Stage 2 testing indicated the fault was caused by a trouble with Segmentation the local access loop, the ticket will be routed to Telco. The Telco technician performs additional tests to further segment the problem. International: If the Stage 2 testing indicated the fault was caused by an international network problem, the ticket is routed to a Sprint technician who Stage 4 Dispatch Stage 4 Isolation Stage 5 Repair Stage 5 Dispatch Stage 6 Repair performs additional tests to further segment the problem. If the tests isolate the problem to equipment located on the Agency premises, the IOC will electronically refer the ticket for dispatch to the premises. The dispatcher will provide an estimated time of arrival (ETA) to reach the location. Telco: The Telco technician performs additional tests to further isolate the specific failing segment and analyze the various circuit elements, enabling the technician to identify the failing hardware, circuit element, or software. International: The technician performs additional tests to further isolate the specific failing segment and analyze the various circuit elements, enabling the technician to identify the suspected failing hardware, circuit element, or software. Once the failing software element is identified and/or the technician arrives on the customer premises, the fault is repaired. The BSAC representative notifies the User and confirms that service has returned to normal operation. Telco: If Telco tests isolate the problem to Telco equipment located on the Agency premises, the Telco will dispatch a technician. The dispatcher will provide an estimated time of arrival (ETA) to reach the location. International: If the tests isolate the problem to off-shore equipment located on the Agency premises, we will dispatch a technician. Sprint International will obtain an estimated time of arrival (ETA) to reach the location. Once the failing software element is identified and/or the technician arrives on the customer premises, the fault is repaired. The BSAC representative notifies the User and confirms that service has returned to normal operation. Page 217 March 5, 2007

12 Figure Trouble Ticketing Process Training on the trouble repair process will assist the Government in executing fault management. Page 218 March 5, 2007

13 Sprint has established a system for prioritizing service problems based on the severity of the problem reported. Table depicts these classifications. This process along with our aggressive escalation procedures, provide a foundation to guide the standard operating procedures for BSAC support. (J.9 ID 10351) Table Ticket Classifications Severity Level Status Reporting Definition/Examples of Severities Severity 1 Initial status of reported to Multiple services failed or reported unusable Government caller within Total out of service condition at multiple sites thirty minutes of ticket entry, and every hour thereafter Host line down Total loss of a Network element Circuit must be available for testing. Severity 2 Initial status reported to Single service failed or reported unusable Government caller within Remote site circuit/service down one hour of ticket entry, and every two hours thereafter Inability to use VPN on-net services as necessary Circuit available for immediate intrusive testing. Severity 3 Severity 4 Status will be provided to Government caller every two hours as necessary Service impacting condition for which circuit is released for delayed or scheduled intrusive testing Chronic but transient degradation of service Continued slow or degraded service or response Service problem with work around Multiple attempts to successfully reach a single destination Duplicate RVA or poor voice quality (Echo, clipping, static, fading), or voice product feature failure (calls still completing). Status will be provided to Routine Government caller a Intermittent quality or connectivity problems with minimum of one a day or as switched voice services additions necessary Sprint will resolve each service outage within 4 hours for any service not requiring dispatching of personnel. Similarly, for each service outage requiring dispatching of personnel except for non-domestic POPs, we will Page 219 March 5, 2007

14 resolve the problem within 8 hours, with the exception of Fixed Satellite Service (FSS). For FSS, Sprint will resolve each service outage within 72 hours for restoration requiring dispatching of personnel. (J.9 ID 10147, 10351) On a monthly basis, Sprint will resolve all service outages, measured at the Agency level: 90% within 4 hours with out personnel dispatch 90% within 8 hours with personnel dispatch. Escalations Table shows the internal escalation policy. Through ongoing ticket monitoring, the BSAC representative identifies when a trouble has fallen outside of process based on its severity level and escalates the problem. The representative will escalate to all levels of the fix agency. As escalation within the BSAC organization progresses, corresponding escalation to increasing levels of technical expertise will be made by the BSAC current level of escalation to their counterparts within the appropriate fix agency. Escalation will continue until Sprint resolves the problem. (J.9 ID 10370) Sprint also has established procedures for customer driven escalations for when a trouble or complaint is not resolved within a required performance target or the customer is dissatisfied with the way Sprint is handling the issue. The Sprint Networx Program Office is always available for escalations. The Communications Plan will detail escalation points of contact and procedures. (J.9 ID 10129, 10371) Table Escalation Intervals If the ticket is severity: Severity 1 (Out Of Service) And the ticket has been opened: XXXXXXXXXXXX XXXXXXXXXX XXXXXXXXX BSAC Escalation path is: XXXXXXXXXXXXXXXX XXXXXXXXXXXXXX XXXXXXXXXXXXXXXXX Page 220 March 5, 2007

15 If the ticket is severity: And the ticket has been opened: BSAC Escalation path is: XXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXX Severity 2 XXXXXXXXXXX XXXXXXXXXXXXX (Impaired XXXXXXXXXXX XXXXXXXXXXXXXXX Service) XXXXXXXXXXXX XXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXX XXXXXXXX Severity 3 XXXXXXXXXXX XXXXXXXXXXXXXXXX (Service XXXXXXXXXXXXXX XXXXXXXXXXXX Related) XXXXXXXXXXX XXXXXXXXXXXXXX XXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXX XXXXXXX Severity 4 XXXXXXXXXXX XXXXXXXXXXXXXXX (Routine) XXXXXXXXXX XXXXXXXXXXX XXXXXXXXXXXX XXXXXXXXXXXXXXXX XXXXXXXXXX XXXXXXXXXXXXXXXXXXXXX XXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXX Telecommunications Service Priority (TSP) Sprint is well versed in TSP restoration and has established methods and procedures to perform restoration of services. Sprint fully complies with NS/EP and the TSP system for priority restoration of installed services. Sprint reports resolution efforts at least hourly for all services classified as NS/EP and for services covered by the TSP system. Should the Government designate other services as priority services not under a TSP, Sprint will make the appropriate arrangements to handle these in a priority manner. Services assigned TSP priority take precedence over other services except as noted on E-designated (emergency) orders. Page 221 March 5, 2007

16 Table outlines the guidelines that Sprint operations personnel adhere to when restoring priority services. Table Operational Guidelines for Restoring Priority Services Stage Initiation Restoration Priorities Patching Services Procedure Personnel observing service failure and/or upon receipt of a trouble ticket affecting services with TSP restoration codes assigned will take immediate action to restore the service. Problems will be worked in the order indicated by the assigned restoration priority code. If two or more services have the same restoration priority code, operations personnel will work the problem in the order that most efficiently restore the affected services. When restoring priority service, restoration of lower or no priority services that are on parallel channels may occur. This is acceptable if the parallel channel restoration does not delay restoration of the priority service. Temporarily patching services to spare channel is the preferred method of minimizing restoration time. Operations personnel will patch the service in the order indicated by the assigned priority until the priority traffic exceeds the altroute s capacity. Services will be returned to their original configuration as soon as the trouble is repaired. Emergency In an emergency, restorations activities take priority over all other tasks. Restoration activities are documented in TRS. Following Hurricane Katrina, our Circuit Restoration Team assembled to review, analyze, and prioritize circuits that were impaired or out of service. Our plans and experience supporting critical Government operations may be measured by more than just the speed and scope of our response. To maximize the effectiveness of our response, we worked closely with Federal Agencies and State Emergency Operations Centers to identify services that required prioritized restoration and handled TSP requests so that emergency personnel could effectively communicate. In just over a week, we had made considerable progress in network restoration. One wireline highlight was that we successfully rehomed numerous circuits including nearly all recoverable TSP circuits many critical to Government operations. With wireless services, within that same time we had restored service completely in Alabama, and had re-established coverage in 75 percent and 45 percent of our footprint in Mississippi and Louisiana, respectively. Page 222 March 5, 2007

17 9.7 PROCEDURES (L ) Sprint provides a BSAC customer package (Appendix D) to all Agencies. This trouble reporting reference package includes detailed information on information required to generate trouble tickets, explanation of trouble ticket scenarios, status and escalation guidelines, confirmation of ticket resolution, and network events. This document functions as a how to manual for Government users for the trouble and complaint handling process. When reporting a problem, the caller will need common information including: circuit Identification, problem description, and time the problem occurred. The caller is given a trouble ticket tracking number that is used to track all activity associated with the repair. Users also use the ticket number view status via the Sprint Networx Web Portal. 9.8 TOOLS (L ) The BSAC and Customer Service both rely primarily on applications developed internally at Sprint to manage troubles and complaints. These applications provide a method for employees to track issues to resolution as well as providing notification on tickets opened online by the Government and proactively opened by Sprint. Fix agencies use a variety of specialized tools including inventory-loaded trucks, two-way communication devices, wireless laptops, firebird test sets, scopes, sniffers, and various other hand and electronic tools to perform their job. 9.9 SYSTEMS (L ) Figure provides a high-level overview of existing architecture used to support FTS2001 customers today and is same systems architecture for the trouble management system that supports the Networx program. (J.9 ID 10351) Page 223 March 5, 2007

18 Figure Trouble Handling High Level System Overview Use of tried and true systems provides users continuity and reduced risk REPORTS (L ) To monitor its success Sprint will track the performance of the trouble tickets against the Networx service level agreement and provide both the Trouble Management Performance Summary Report and the Trouble Management Incident Performance Report. These reports can be delivered to the PMO and customer Agencies in any of the media, transport, and format types described in the Reports Table in RFP Section C For Agencies that request it, the assigned NSM will prepare a Service Quality Review (SQR). The SQR provides detailed information on an Agency-wide basis on previous month s trouble reports. A sample SQR can be found in Appendix J. While SQRs are available to all Agencies, our experience with FTS2001 shows that not all request to receive one. Page 224 March 5, 2007

19 In conclusion, Sprint understands the importance of a trouble and complaints process and support organization that understands the critical role that telecommunications plays in enabling the Government to succeed in its mission and that acts with the appropriate sense of urgency to resolve problems when they occur. Our BSAC and NSM organizations support the Government today and will be there from the start of the Networx program. Page 225 March 5, 2007