AWS Managed Service Program Partner Validation Checklist. Authors: Kelly Hartman and Allen Brown August 2015 Version 2.3

Transcription

1 AWS Managed Service Program Partner Validation Checklist Authors: Kelly Hartman and Allen Brown August 2015 Version 2.3

2 Table of Contents Purpose of this Document... 3 Expectations of Parties... 3 Audit Process and Timing... 4 Definitions... 5 Program Prerequisites... 6 AWS Managed Service Program Partner Validation Checklist... 7 Discover, Plan, Migrate, Integrate, Validate Customer Capabilities Demonstration Business Management Solution Design Capabilities Infrastructure Migration Design Capabilities Application Migration Design Capabilities Security Operate Service Desk Operations and Customer Support Service Level Agreements Customer Obsession Service Reporting Optimize Internal Process Optimization SLA Optimization Capability Identifiers (Optional) Business Capability Identifiers (Optional) Technical Capability Identifiers (Optional) Appendix A: Technical Competencies AWS Core Technical Capabilities (Required) Appendix B: Best Practice Guides and Reference Materials Appendix C: White Label terminology, Expectations of Parties, and Process for Audit Terminology Audit Process for White Label Vendors Expectation of Parties for Partners Providing White Label Solutions Summary of Changes AWS MSP Partner Validation Checklist, v2.3 pg. 2

3 Purpose of this Document The AWS Managed Service Program Partner Validation Checklist is intended for APN Partners ( Partners ) who are interested in applying for the Managed Service Program. This checklist provides the criteria necessary to achieve the Managed Service Program Partner designation. Partners should fill out this checklist based on their own assessment of capabilities, and the assessment will serve as the basis for discussion during the Partner capabilities audit. The goal is to recognize APN Partners that provide the best AWS Cloud managed service experience for their customers. Partners will undergo a validation of their capabilities upon applying for entry into the AWS Managed Service Program, and every 12 months thereafter. AWS will leverage an objective third-party auditing firm to facilitate the review in the Partner s preferred language and location, when feasible, which results in costs incurred to the Partner. No costs will occur without Partner first agreeing to the engagement with the third-party firm. AWS reserves the right to make changes to this document at any time, but will do so with effective version control, by including a summary of changes with each document released after Version 1.0. Expectations of Parties APN Partner Participation and Process. It is expected that APN Partners will review this document in detail before submitting an application for the AWS Managed Service Program, even if all of the prerequisites are met. If items in this document are unclear and require further explanation, please contact your Partner Development Representative (PDR) or Partner Development Manager (PDM) as the first step. Your PDR/PDM will contact the Program Office if further assistance is required. When ready to submit a program application, APN Partners should complete the Partner Self- Assessment column of the AWS Managed Service Program Partner Validation Checklist set forth below in this document (for scoring matrix, award only full point values; no partial values will be given. The scoring is meant to be binary in nature; however, some of the items hold more weight to the overall score). After submitting your program application, your completed self-assessment to [email protected]; with cc: to your PDR/PDM. AWS will review and aim to respond back with any questions within 10 business days to initiate scheduling of your capabilities audit. Partners should prepare for the Managed Service Program audit by reading the Partner Validation Checklist, performing a self-assessment using the checklist, and gathering and organizing objective evidence to share with the auditor on the day of the audit. AWS recommends that Partners have individuals who are able to speak in-depth to the requirements at the audit (remote or onsite). Best practice is for the Partner to have one or more highly technical AWS engineers/architects, an operations manager who is responsible for the service desk and support AWS MSP Partner Validation Checklist, v2.3 pg. 3

4 elements (and or managed service practice manager), and a business development executive to give the overview presentation. Program Participation and!benefits.!aws may revoke a Partner s status as an AWS Managed Service Partner if at any time AWS determines in its sole discretion that such Partner does not meet the AWS Managed Service Program requirements or otherwise fails to represent the high standards expected of AWS Managed Service Partners. If a Partner s status as an AWS Managed Service Partner is revoked, such Partner will (i) no longer receive, and will immediately cease taking advantage of, any AWS Managed Service Program benefits, (ii) immediately cease use of all materials provided to it in connection with the AWS Managed Service Program and (iii) immediately cease to identify itself or hold itself out as an AWS Managed Service Partner.! Audit Process and Timing After the audit occurs, the Partner will receive an audit summary (within 24 hours) from the auditor detailing Partner strengths, opportunities for improvement, and action items. A preliminary score will be provided with the audit summary, although the passing score threshold will not be disclosed. Partners have 5 business days from receipt of the audit summary to respond to and address any identified action items, which will be categorized as either Mandatory Action Items or as additional Score-Impacting Action Items. Mandatory Action Items must be closed out prior to approval of entry into the AWS Managed Service Program. If Partner is not able to fully close a Mandatory Action Item in 5 business days, an action plan detailing how and when the item will be closed must be provided to the AWS MSP Program Manager. Score-Impacting Action Items may be closed by providing evidence of closure to the auditor within 5 business days. Any such items satisfactorily closed within the five (5) business days, as determined by the auditor, will raise the Partner s score, and the new score will become the final score submitted to AWS with the final audit report. Any Score-Impacting Action Items not addressed, or not fully closed within the 5 business days will result in no change to the Partner s score, and will not be included on the final audit report. The auditor will submit the final audit report to AWS after the 5 business days have passed, and no longer than 10 business days after the audit. Partners will not receive the final audit report. The final determination of acceptance into the Managed Service Program will be made after AWS receipt of the final audit report and no longer than 20 business days from receipt of the report. AWS MSP Partner Validation Checklist, v2.3 pg. 4

5 Definitions Mandatory Action Items (MAIs): Mandatory Action Items are non-negotiable items that must be addressed by the Partner to be accepted into the AWS Managed Service Program. Score-Impacting Action Items (SIAIs): Score-Impacting Action Items are action items that arise from not having sufficient evidence at the time of the audit for the Partner to receive a full score in that area. SIAI s are opportunities for Partners to increase their scores from the time the audit occurs, to within 5 business days after the audit. SIAIs need to be closed out with the auditor directly after the audit in order for the score to be included in the Partner s final score. Any SIAIs not closed 5 business days are treated as not meeting requirements and the final score will reflect the corresponding point value. Preliminary Partner Score: The preliminary score is determined and disclosed in the audit summary. Final Partner Score: The final score is the score provided by the auditor to AWS, after addressing any action items. Passing Score Threshold: The passing score threshold is the minimum score required for Partner acceptance into the AWS Managed Service Program. The minimum score serves as one of two requirements for acceptance: in addition to meeting the passing score threshold, all Mandatory Action Items must be satisfactorily closed as determined by the auditor and/or AWS. AWS MSP Partner Validation Checklist, v2.3 pg. 5

6 Program Prerequisites AWS Managed Service Program Partner Prerequisites APN Membership Advanced or Premier APN Consulting Partner (view requirements) AWS Billings $25,000/month in AWS Revenue (direct/indirect) Customer Engagements AWS Certifications 3 AWS Customer References (nonpublic or public) 4 AWS Certified Staff 2 Trained Staff System Operations on AWS or DevOps Engineering on AWS AWS Trainings Note: AWS Training requirements can be superseded by the following Certifications: AWS Certified Solutions Architect Professional; AWS Certified SysOps Administrator Associate; AWS Certified DevOps Engineer Professional. If superseded, the minimum personnel requirement is 4 AWS Certifications. Technical Competencies Validation of core technical capabilities; see Appendix A. AWS MSP Partner Validation Checklist, v2.3 pg. 6

7 AWS Managed Service Program Partner Validation Checklist In preparation for the validation process, Partners should become familiar with the items outlined in this document, and prepare objective evidence, including but not limited to: prepared demonstration to show capabilities, process documentation, and/or actual customer examples. Discover, Plan, Migrate, Integrate, Validate 1.0 Customer Capabilities Demonstration 1.1 Company Overview Partner has a company overview presentation to set the stage for customer conversations, in addition to demonstration capabilities. Subtract if Does Not Meet Capability Add if Does Meet Capability Partner Self- Assessment Auditor Validation Overview presentations generally contain: Company history Office locations Number of employees Customer profile, including number and size of customers, including industry Service differentiators AWS partnership overview/details, including APN participation, monthly AWS billings, etc. Evidence must be a presentation delivered during validation meeting. Presentation should be limited to no more than 30 minutes. 2.0 Business Management 2.1 Resource/ Capacity Planning Partner conducts resource/capacity planning to ensure that adequate resources are available for the business activity. 2.2 Job Roles/ Staffing Evidence must be in the form of resource planning documentation detailing how Partner ensures that appropriate personnel, processes, and infrastructure are available to meet business demand. This may include, for example, ensuring that there are sufficient AWS Certified Solutions Architecture Professionals available based on the number of customers Partner has an overview of the job roles within their company, supporting the AWS business. Evidence must be in the form of a document or spreadsheet that describes the role, job title, % of time on AWS business, any AWS trainings or certifications achieved, and any other industry relevant trainings/certifications Partner has defined processes for on- and offboarding of personnel. Evidence must be in the form of completed on- and off-boarding records; examples may include completed checklists, training plans, or other records AWS MSP Partner Validation Checklist, v2.3 pg. 7

8 2.3 Customer Contracts 2.4 Supplier Management Partner has at least one person at a leadership position certified to ITIL Foundation or above Partner has a secession plan in place to address loss of key leadership personnel. Evidence must be in the form of a documented secession plan Partner has signed contracts with customers defining the specific billing model and any other contractual agreements. Evidence must be in the form of three (3) records of signed customer contracts Customer contracts define the specific legal ownership of data, including arrangements for handling of customer data upon termination of the contract by either party, including: Time commitment as to when data/account is handed to customer Format and method for transfer of data/account credentials If applicable, the process for removal of noncustomer IAM accounts, groups, roles, and federation. Evidence must be in the form of two (2) signed customer contracts addressing the above requirements. Partner may use the same contracts used in Partner has defined processes for selection and evaluation of suppliers (e.g., SaaS vendors or any other third parties to whom activities or services are subcontracted). Where Partner uses SaaS solutions for internal systems, they must show that due diligence has been carried out to assess the security compliance of these solutions with a focus on customer privacy and security AWS Support Plan Evidence must be in the form of records of supplier selection and evaluation. As evidence of assessment of security compliance, Partner must show SaaS providers security overview and documentation, authentication and authorization validation, MFA capabilities, overview of availability characteristics, data backup plan, and disaster recovery plan. Partner has an AWS Support plan in place. Evidence must be in the form of a current AWS Support agreement. 3.0 Solution Design Capabilities 3.1 Solution Capabilities Partner demonstrates that during customer engagements, a complete detailed design document is delivered such that customers and partners are both assured that due diligence, capacity planning, architectural review and long term operational process have been assessed for the customer engagement. Evidence must be in the form of two (2) actual customer system detailed design documents that contain the following components. AWS MSP Partner Validation Checklist, v2.3 pg. 8

9 3.1.1 Documentation of customer requirements Assessment of current infrastructure/application environment Architectural details of the proposed design Details of the system performance, capacity management and availability measurement systems to be put in place to measure success of proposed design Assessment of customer s security policies and procedures with gap identification Detailed design shows that customer infrastructure is architected as per AWS security best practices as outlined in _Best_Practices.pdf Detailed design shows that the proposed design allows for governance and risk management at scale as per _at_scale_governance_in_aws.pdf and _Risk_and_Compliance_Whitepaper.pdf For each customer engagement, Partner provides an initial and ongoing assessment of that customer s architectural status by maintaining the AWS Basic Operations Checklist and Enterprise Operations Checklist (where applicable) contained in nal_checklists.pdf Evidence must be in the form of at least one (1) completed Basic Operations Checklist and Enterprise Operations Checklist (if applicable) for a current or past customer, and evidence that ongoing assessments are scheduled with current customers where appropriate. 4.0 Infrastructure Migration Design Capabilities 4.1 Infrastructure Migration Capabilities Partner consistently provides customers with infrastructure that is aligned with AWS architecture best practices and reference architectures. The detailed design document from section 3 should include an architectural overview that provides the following details: Infrastructure architecture reliably utilizes services like Multi-AZ Auto Scaling, Amazon Virtual Private Cloud, Elastic Load Balancing, and Multi-AZ Amazon Relational Database Service to provide highly available and reliable infrastructure. Evidence must be in the form of two (2) actual customer design recommendations with explanation of the customer scenario for which it was developed. 5.0 Application Migration Design Capabilities 5.1 Application Migration Capabilities Partner has application migration capabilities and provides continual integration, automated deployment and takes advantage of elastic, highly available AWS MSP Partner Validation Checklist, v2.3 pg. 9

10 infrastructure. The detailed design document from section 3 should include an application migration overview that provides the following details: Partner provides tooling that abstracts application deployment from infrastructure deployment and allows customers to, independently or in conjunction with the managed service, deploy and configure their applications. Evidence must be in the form of two (2) actual customer recommendations with explanation of the customer scenario for which it was developed. Section 1-5 Total: 6.0 Security 6.1 Security Management Partner has established security policies and procedures to protect their own systems from attacks. Evidence of security policies and procedures may also be in the form of industry certification related to information security (e.g., ISO 27001) or proof of infrastructure security and information management processes Partner has systems that provide segregated responsibilities and Quorum-based access mechanisms before allowing direct access to customer resources or data. Evidence must be in the form of a live demonstration of internal capabilities that show that only specific engineers are allowed specific functions on customer accounts and that it requires some form of approval from another person before access is allowed to customer resources and data Partner has security policies and procedures to protect their customer's systems from attacks. Evidence may be in the form of industry certification related to information security management (e.g., ISO 27001) or proof of infrastructure security and information management processes Partner deploys all supported AWS infrastructure into Amazon Virtual Private Cloud. Evidence must be in the form of a technology demonstration of deployment framework, specifically that VPC deployment is implemented Partner does not administrate AWS accounts by use of root account credentials for day-to-day operations and any exceptions are closely approved, monitored, and approved. Evidence must be in the form of a technology demonstration that shows how root credentials are protected from general engineering usage and usage is approved and audited when exceptions require their use. AWS MSP Partner Validation Checklist, v2.3 pg. 10

11 6.2 Security Event Logging and Retention Partner provides encryption at rest services for AWS infrastructure as outlined in Data_at_Rest_with_Encryption.pdf Evidence must be in the form of design documentation specifying the use of encryption at rest services Partner ensures customers understand AWS security processes and technologies as outlined in y_whitepaper.pdf Evidence must be in the form of onboarding and educational documents provided to customers that specifically cover customer security considerations in the partner s environment Partner ensures that multi-factor authentication is activated on all partner and customer AWS root accounts. Partner must show processes, procedures, or technology as evidence that they activate MFA on new AWS root accounts. In addition Partner must show technology that demonstrates regular audits of MFA usage on all customer and partner accounts Security Event Logs are retained for the duration contractually agreed with the customers. Security events are stored in a log for regulatory and analysis purposes. Use of technologies as specified in t_scale_logging_in_aws.pdf is recommended. Evidence must be In the form of an example of a customer Security Event Log Partner can show that customer-agreed retention periods for logs are honored and systems exist to support and maintain these logs. Evidence must be in the form of an example of a Security Event Log that has been maintained for the agreed retention period. Partner must also show where retention times and methods of log support and maintenance are defined and agreed with the customer, e.g., in an MSA or other document. If log is maintained by customer, Partner must provide evidence that they are taking steps to ensure that it is maintained Partner can demonstrate use of AWS Config service for security components auditing. Evidence must be in the form of an example of security components auditing using AWS Config Partner has AWS CloudTrail enabled on all managed accounts Evidence must be in the form of a technology demonstration, in the absence of which, documented policies and processes ensure that CloudTrail is enabled on all existing and new accounts may be presented. 6.3 Access Partner has a documented Access Management AWS MSP Partner Validation Checklist, v2.3 pg. 11

12 Management/ AWS API Credential Strategy 6.4 Service Continuity/ Disaster Recovery Strategy, including but not limited to: AWS Identity and Access Management (IAM) users, symmetric access keys, asymmetric.509 certificates, console passwords, and hardware or virtual multifactor authentication (MFA) devices. Evidence must be in the form of a technology demonstration, process documentation that addresses the above, and one customer example. Partner has the ability to monitor their own internal systems to ensure that the customer services are not compromised by internal failures, and that there are reasonable and tested processes to respond to internal outages and failures. Partner has evidence that they test their own systems and processes on an annual basis and document lessons learned. Evidence must be in the form of process documentation that addresses the above, as well as records of last test, and documented lessons learned. Additional evidence may be in the form of industry certification related to business continuity management (e.g., ISO 22301). Section 6 Total: Operate 7.0 Service Desk Operations and Customer Support 7.1 Customer Service Availability Partner provides 24x7 customer service available over multiple communication means; may be a staffed 24x7 call center or 8x5 service with after-hours support (e.g., pager/alert support after-hours on a rotational basis). 7.2 Service Desk Operations Partner must explain or show how customer service is provided; if Partner does not maintain a staffed call center on a 24-hour basis, there must be documented procedures for after hours, weekend, and holiday support Support Priority and Severity levels are defined, documented, and conveyed to customers. Partner must provide documentation defining support priority and severity levels, and must explain or show how this information is communicated to customers Local Language Support and In-Country Phone Numbers: In support of multinational customers, Partner has a published customer service number in-country with calls answered in local language. This may be outsourced, and is required only for Partners supporting multinational customers. Partner must explain or show how local language support and in-country phone numbers are published and provided to customers All calls are logged immediately after the initial communication with the customer. Partner must explain or show how calls are logged; AWS MSP Partner Validation Checklist, v2.3 pg. 12

13 time-stamping or other method can be used to prove that calls are immediately logged Partner provides callback SLAs Ticketing System Partner must explain or show how customer callbacks are done with documentation proving that they are occurring within the specified SLA timeframe. Partner has an IT Service Management (ITSM) ticketing system capable of the following: Event/Incident ticket creation and escalation. 7.4 Proactive Monitoring and Alerting Partner must show how event/incident tickets are created and escalated Immediate logging and time stamping of tickets. Partner must provide evidence of immediate logging and time stamping of tickets Documented escalation process for escalating to AWS Support, including flowchart of process, timeframes for escalating to AWS, definition of the types of cases that get escalated with defined criteria, and closed loop process to ensure smooth handoff and ticket resolution. Partner must provide a documented escalation process addressing the above requirements Escalation process provides automated escalation alerts. Partner must demonstrate how automated escalations occur Ticketing system integration with AWS Support Center is preferred via AWS Support API, but other documented and tested methods are acceptable. Partner must demonstrate integration of their ticketing system with AWS Support Center or must provide evidence of documentation and testing of an equivalent method Read/Write remote access for Partner employees to make updates. Partner must demonstrate that employees have read/write remote access for making updates in the ticketing system Verification by customer that the case has been closed satisfactorily. Partner must provide evidence of customer verification of case closure, e.g., by providing examples of closed cases that have been customer approved. Partner has systems, tools, or applications capable of monitoring the performance of all AWS Services that are part of the customer s managed service agreement. Proactive Monitoring looks for patterns of events to predict possible future failures. (ITIL Service Operation) The monitoring and alerting functionality must also be accompanied by corresponding service desk AWS MSP Partner Validation Checklist, v2.3 pg. 13

14 7.5 Event and Incident Management 7.6 Problem Management functionality to take action on events/alerts according to SLAs/contractual obligations. Partners must show their capabilities within the following two (2) categories: 1) Infrastructure monitoring; examples may include: Amazon CloudWatch out-of-the-box metrics for AWS monitoring, alerting, and automated provisioning Amazon CloudWatch custom metrics for application monitoring, alerting, and automated provisioning Other 3 rd party AWS infrastructure monitoring tools 2) Service monitoring; examples may include: Operating system monitoring tools for OS-level monitoring Application monitoring tools for application-level monitoring Simulated transaction monitoring tools for end-toend system monitoring Evidence must be in the form of a technology demonstration of tools used to carry out proactive monitoring and alerting. Partner can demonstrate the ability to programmatically add value to customers operations by differentiating between problems that require customer engagement and those that don t. In ITIL language an event is any identified or reported problem or anomaly in a running system. An incident is the classification of event where the problem or anomaly is impacting availability or correct operations of the system. This standard stipulates that Partners add value to customer operations by only exposing customers to immediate engagement when the system availability or accuracy is directly impacted. Partner must provide an example of filtering used to identify impacting incidents and sending this incident information to customers Partner has a documented process for problem management encompassing incidents with no known or available resolution or those that are proactively identified based on performance trending or monitoring. A problem is defined as a cause of one or more incidents. The cause is not usually known at the time a problem record is created, and the problem management process is responsible for further investigation. Problem management is the process responsible for managing the lifecycle of all problems. Problem management proactively prevents incidents from happening and minimizes the impact of incidents that cannot be prevented. (ITIL Service Operation) Evidence must be in the form of examples where AWS MSP Partner Validation Checklist, v2.3 pg. 14

15 incidents were handed off or were proactively identified based on performance trending/monitoring/pattern analysis Partner has the ability to identify and document root causes, and store in a Known Error Database that is searchable by appropriate support personnel Asset Management 7.8 Configuration and Change Management A Known Error Database (KEDB) is a database containing all known error records. This database is created by problem management and used by incident and problem management. The known error database may be part of the configuration management system, or may be stored elsewhere in the service knowledge management system. (ITL Service Operation). Evidence must be in the form of problems that were identified, logged, analyzed, and subsequently entered into the Known Error Database. Partner must demonstrate that the database is searchable; may be a simple Excel file or a more sophisticated tool. Partner has a strategy for tracking and managing its AWS deployed assets. An asset is defined as any resource or capability that could contribute to the delivery of a service. A generic activity or process responsible for tracking and reporting the value and ownership of assets throughout their lifecycle. (ITIL Service Strategy/Service Transition.) Partner s asset management strategy answers the following questions: Is your organization leveraging AWS provided instance and service-specific metadata as part of its asset management strategy? Is your organization leveraging custom resource tags to track and identify AWS resources? Does your organization have a resource tagging strategy? How will AWS assets be integrated with internal asset management systems? More details specific to these questions can be found at: l_checklists.pdf Evidence must be in the form of a technology demonstration Partner has configuration and change management processes. Processes address the following questions specific to the AWS business: How will your organization manage server images (AMIs)? Will instances be automatically configured at launch or manually configured later? How will patches and upgrades be applied? Will applications be managed as homogeneous fleets? How will your organization manage changes to OS AWS MSP Partner Validation Checklist, v2.3 pg. 15

16 hardening baselines, configure security groups or OS firewalls, and monitor their instances for intrusions or unauthorized changes? More details specific to these questions can be found at: l_checklists.pdf Evidence must be in the form of a technology demonstration of a change against a test or pseudoproduction environment The change management process includes a change rollback process. Evidence must be in the form of a technology demonstration and documented change management process that addresses change rollback; an example must be provided Configuration Management Database (CMDB) is highly recommended Release and Deployment Management A Configuration Management Database is a database used to store configuration records throughout their lifecycle. (ITIL Service Transition) Evidence must be in the form of a demonstrable Configuration Management Database Partner has application release and deployment management processes. Release and deployment management is defined as the process responsible for planning, scheduling and controlling the build, test and deployment of releases, and for delivering new functionality required by the business while protecting the integrity of existing services. (ITIL Service Transition) In the cloud, the traditional lines between infrastructure changes and application deployments can be blurred, if not completely erased. Continual integration and continual deployment mechanisms are broadly supported and recommended for cloud application and infrastructure deployments. In addition to traditional code development, testing, and versioning concepts, organizations should also consider the following cloud integration points for application releases or deployments: How does your practice enable DevOps methodology for development for your customer? How does your practice enable self-service or managed CI/CD pipelines? Will weighted load distribution patterns be used to intentionally deploy, test, migrate, and roll-out or rollback new application releases? How can your organization leverage infrastructure boot-strapping and application deployment tools to more quickly and effectively introduce or roll-back changes? AWS MSP Partner Validation Checklist, v2.3 pg. 16

17 How can your organization make its applications more infrastructure-aware so applications can become active participants in making the infrastructure changes required to support a specific software release or deployment?! To meet this standard partners should show how they enable customer application deployment and release management as either a self-service continuous integration and continuous deployment pipeline endpoint or a managed function.! Evidence must be in the form of a technology demonstration and process documentation that addresses the above, and one customer example Partner has infrastructure release and deployment management processes. Infrastructure release and deployment should utilize a highly configurable, reusable and repeatable mechanism for defining, customizing, provisioning, and updating customer operating environment and infrastructure stacks To meet this standard Partner must show how they enable infrastructure deployment and release management with a repeatable and reusable mechanism that ensures accurate deployment of designed operating environments and infrastructure stacks. Partner must also demonstrate how updates to existing operating environments and infrastructure stacks are performed through the infrastructure and deployment management process. Evidence must be in the form of a technology demonstration and process documentation that addresses the above, and one customer example. Section 7 Total: 8.0 Service Level Agreements 8.1 Foundational SLAs Partner has foundational SLAs. Foundational SLAs are those that relate to response times, actions, and notifications by the Partner to their customers. 8.2 Infrastructure- Level SLAs SLAs may include response times when customer open ticket/initiates request, time from event or incident trigger to remediation, and turnaround time for customer initiated changes/requests. Evidence must be in the form of SLA documentation and supporting processes and metrics. Partner provides infrastructure-level SLAs beyond AWS Service SLAs to guaranteed uptime or availability of resources. This may include service specific SLA and should go beyond pure infrastructure SLAs. For example offering 100% availability SLAs for specific software products that the partner has added value in the deployment and management mechanism that extend the service level beyond the base EC2 SLA AWS MSP Partner Validation Checklist, v2.3 pg. 17

18 8.3 Workload or Solution- Specific SLAs Evidence must be in the form of SLA documentation and supporting processes and metrics. In addition to foundational response and restoration SLAs, Partner has SLAs based on the customer workloads operating in the AWS Cloud. Evidence must be in the form of SLA documentation and supporting processes and metrics. Section 8 Total: 9.0 Customer Obsession 9.1 Customer Satisfaction Partner has the ability to objectively capture customer satisfaction data. This is done via formal survey process, contact-based surveys (after customer case is closed) or as part of customer review meetings. Evidence must be in the form of documentation of feedback collected Partner has a process for following up on lowscores or customer dissatisfaction and document resolution. 9.2 Customer Review Evidence must be in the form of a low-score follow up process and a customer example showing where this process was used Partner has regular customer review meetings to discuss the performance of their services/slas and to share reports with the customer. The purpose is to ensure customers understand the value of a managed solution; particularly since proactive services that work well may appear unnecessary to an end customer. Evidence must be in the form of documentation from a customer review meeting (may be same example used above), complete with recommendations and reports provided to customer Partner regularly assesses customer infrastructure cost and highlights opportunities to optimize these costs to their customers through reporting. Evidence must be in the form of documentation from a customer review meeting (may be same example used above), including evidence that recommendations for infrastructure cost optimization were provided. Section 9 Total: 10.0 Service Reporting 10.1 Incident Management Reports Partner provides reports detailing the current work activities to correct incidents; metrics on the management of incidents, such as number of incidents, average time to resolve, common causes identified. Evidence must be in the form of sample reports Non- Partner maintains non-service affecting incident reports: AWS MSP Partner Validation Checklist, v2.3 pg. 18

19 Service Affecting Incident Reports 10.3 Performance Analysis Reports 10.4 Asset/ Resource Reports 10.5 Exception Reports 10.6 Web- Accessible Reports Includes incidents where action was taken to proactively re-route, new services provisioned according to automated triggers, etc. Evidence must be in the form of sample reports. Partner provides historical performance analysis. This would typically be available over a number of sample periods (daily, weekly, monthly) and would include data to allow the customer to understand how the overall service is performing, e.g., how much traffic is being generated by a particular application. Evidence must be in the form of sample reports. Partner provides reports of assets/resources under management for the customer. Evidence must be in the form of sample reports. Partner provides reports generated by customerspecified thresholds or ranges; provides ability for customers to self-select parameters on individual devices and determine thresholds for the raising of exception reports. Evidence must be in the form of sample reports. Partners make customer reports accessible via the web or web portal. Evidence must be in the form of a web portal or customer accessible repository Section 10 Total: Optimize 11.0 Internal Process Optimization 11.1 Internal Process Optimization Partner has established a regular cadence to review internal performance, and provide recommendations for improvement. Internal optimization involves looking for efficiencies within the Partner s operations that result in financial efficiencies, process efficiencies, and/or greater customer satisfaction. Evidence must be in the form of an internal review cadence report, and any efficiencies implemented as part of the process (e.g., billing alerts, etc.) SLA Optimization 12.1 SLA Optimization Partner takes actions to continually improve performance to objectives. Evidence of continual improvement includes records of actions taken to improve performance, particularly when established objectives are not being met For example a partner may set an internal objective to reduce customer impacting EC2-related incidents, this metric should be tracked over time and trends used to improve internal processes, mechanisms and technology, driving down EC2 related incidents time and improving customer availability. AWS MSP Partner Validation Checklist, v2.3 pg. 19

20 Evidence must be in the form of examples where long term operational inefficiencies and improvements were identified and implemented. Section Total: TOTAL PARTNER SCORE: AWS MSP Partner Validation Checklist, v2.3 pg. 20

21 Capability Identifiers (Optional) 13.0 Business Capability Identifiers (Optional) All capability identifiers, business or technical, are optional. We encourage Partners to apply for only those that represent their core competencies, as they are intended to help Partners differentiate their unique skills to customers Application Migration and Management (Optional) Partners should only apply for the Application Migration and Management specialization if they have repeated customer success in this area. Required Optional Partner Validated Self Customer Samples Public Customer References Partner has ten (10) unique customer samples related to application migration and ongoing management, with the following details: Partner must explain their process for moving customers business applications from an on premise environment to the AWS cloud; process must describe Partner s unique value proposition in handling application migration and ongoing management. Details must include, at minimum: Customer situation (how they are/were using the application(s) Architecture they were moving from (including network, operating system, management tools, etc.) Architecture they are moving to, based on partner s recommendations How the applications were managed in the previous environment and how they are being managed in the AWS cloud environment Efficiencies gained by migrating their applications to AWS Partner must also describe middleware products used to bridge gaps between technologies, if/when applicable. Partner must provide five (5) public references (unique customers, not unique projects) specific to application migration and management. Assessment 13.2 Hybrid IT Expertise (Optional) Partners should only apply for the Hybrid IT Expertise specialization if they have repeated customer success in this area Customer Samples Partner has ten (10) unique customer samples related to Hybrid IT Expertise, with the following details: Partner must explain their process for designing and managing workloads in a hybrid IT environment; process must AWS MSP Partner Validation Checklist, v2.3 pg. 21

22 Public Customer References describe Partner s unique value proposition in handling hybrid IT solutions. Details must include, at minimum: Customer situation (Why a hybrid environment? What are the business objectives the customer is trying to achieve? What led them to the decision to operate in a hybrid environment?) Architecture they were moving from Architecture they are moving to, based on partner s recommendations Managed Service tools used by the Partner to seamlessly combine the hybrid environment into their existing processes Partner can also describe middleware products used to bridge gaps between technologies, if/when applicable. Partner can provide five (5) public references (unique customers, not unique projects) specific to managed Hybrid IT environments Custom Application Development for Managed Environments (Optional) Partners should only apply for the Custom Application Development for Managed Environments specialization if they have repeated customer success in this area. Note: This identifier is NOT intended for APN Technology Partners, but rather APN Consulting Partners in the Managed Service Program who have developed unique solutions for their internal use or for partner differentiation. (In other words, custom applications for managed environments are not commercially available applications sold by the MSP.) Custom Application Samples Partners must provide examples/ demonstrations of custom applications they have developed that serve one or more of the following purposes: 1) Custom applications to increase managed service efficiencies; such as self-healing applications. This can be a combination of applications developed in-house that are triggered by specific events or event correlation to remediate customer issues and may or may not be combined with AWS Services/AWS CloudFormation templates. 2) Custom applications developed to create more seamless interoperability in hybrid managed service environments. 3) Custom applications that provide business value on top of the Partner s AWS Managed Service offerings. AWS MSP Partner Validation Checklist, v2.3 pg. 22

23 Application Developers/Programmers Details must include, at minimum: How the concept originated What the application is/does Why did you develop in house rather than seek an alternative solution? How the application(s) add value for the customers. Why this application is unique to the AWS Cloud environment Partner must have a minimum of four (4) application developers on full-time payroll to qualify for this identifier Digital Media Competency (Optional) Partners should only apply for the digital media competency specialization if they have repeated customer success in this area. AWS will not be evaluating a Partner s ability to meet regulatory or governing standards for this area, but will look for evidence of having achieved through certifications/audits. Also, AWS will not endorse the Partner for specific industries or standards, but will allow Partner to have the APN Digital Media competency Industry Expertise Partner must provide at least six (6) customer references for media and entertainment customers that are involved in creation, management, storage, or distribution of digital media content, at least two (2) must be public references. All references should show implementation on AWS Global reach Ability to demonstrate product or solution can support 3 or more AWS regions Public documentation and references Public documentation on company s solutions, tools, and guidance in digital media Healthcare Competency (Optional) Partners should only apply for the healthcare competency specialization if they have repeated customer success in this area. AWS will not be evaluating a Partner s ability to meet regulatory or governing standards for this area, but will look for evidence of having achieved through certifications/audits. Also, AWS will not endorse the Partner for specific industries or standards, but will allow Partner to have the APN Healthcare competency Industry Expertise Partner must provide at least four (4) customer references for healthcare customers that help healthcare providers and payers securely store, process, transmit, and analyze clinical information, at least two (2) must be public references. All references should show implementation on AWS Public documentation and references All references must demonstrate the design, architecture, deployment and operation of HIPAA compliant workloads. Public documentation on company s solutions, tools, and guidance in healthcare 13.6 Life Sciences Competency (Optional) Partners should only apply for the life sciences competency specialization if they have repeated customer success in this area. AWS will not be evaluating a Partner s ability to meet regulatory or governing standards for this area, but will look for evidence of having achieved through certifications/audits. Also, AWS will not endorse the Partner for specific industries or standards, but will allow Partner to have the APN Life Sciences competency Industry Expertise Partner must provide at least four (4) AWS MSP Partner Validation Checklist, v2.3 pg. 23

24 Public documentation and references customer references for life sciences customers that help life sciences customers conduct drug discovery, research and develop novel geneticbased treatments, manage clinical trials, and engage in bio pharma manufacturing and distribution activities, at least two (2) must be public references. All references should show implementation on AWS. All references must demonstrate the design, architecture, deployment and operation of HIPAA or CLIA compliant workloads. Public documentation on company s solutions, tools, and guidance in life sciences White Label (Optional) Partners should only apply for the White Label specialization if they are a white label provider operating in an AWS cloud environment and have repeated customer success in this area. Partners with the white label competency are allowed, under specific program rules to let their MSP customers (referred to as white label vendors) inherit standards that they have been audited for. Depending on which standards are inherited, this can exempt white label vendors from audit while under contract with the Partner Contractual requirements White Label vendor contracts include language that: Protects the White Label Brand from disclosure and contain an exception for reporting relationship information to Amazon AWS. Lists exactly which standards the white label vendor inherits. Process for handling of customer operations upon termination of the contract by either party, including:! Time commitment as to when operations is handed to customer! Format and method for transfer of data/account credentials! If applicable, the process for removal of non-customer IAM accounts, groups, roles, and federation Escalation Process Evidence must be in the form of two (2) signed contracts addressing the above requirements. Partner must provide an escalation process that identifies roles, responsibilities, timelines, and escalation contact information for inclusion in customer s operational processes AWS MSP Partner Validation Checklist, v2.3 pg. 24

25 Enablement/ Training Partner must provide instructions and training to sales, customer relationship managers, and engineers; training may include the following: Service Activities, deliverables, SLAs Management platform technical overview Management portal training Positioning of the service Service Process Partner must provide a document that defines roles and responsibilities for the service activation process and the day-to-day operational support activities, including Request for Changes (RFCs), relating to the codelivery of the service with customer Reports to Customer Partner must provide monthly and quarterly metrics reports to measure the service quality for customers under management; Reports to AWS Partner must provide details of relationship and reports to AWS on an as-needed basis Marketing Collateral Partner must provide the following marketing collateral that can be reused and branded by the customer: service description, service data sheet, customer presentations, service overview, and FAQ Ticket e-bonding Partner must provide tickets and performance, inventory, and availability information electronically for integration into customer s ticketing and reporting systems Branded portal Partner must provide a customer portal that branded for the customer; this must include a portal user guide Branded customer deliverables Partner must provide customer deliverables that are branded for the customer; deliverables may include: Customer e-notifications Service activation kit addresses 14.0 Technical Capability Identifiers (Optional) All capability identifiers, business or technical, are optional. We encourage partners to apply for only those that represent their core competencies, as they are intended to help Partners differentiate their unique skills to customers Application Development (Optional) Partners wishing to meet the Application Development identifier should meet all required capabilities below; meeting optional competencies can offset missing required competencies at AWS's discretion Application Development Capabilities Partner must provide application development capabilities that utilize AWS best practices to Required Optional Partner Self- Assessment Validated AWS MSP Partner Validation Checklist, v2.3 pg. 25

26 build stateless, elastic and highly available applications. Specifically applications should be built in an ecosystem that includes continual deployment methodologies, automated deployment and is aware of the elastic nature of AWS. In addition applications should be built to survive failure of any individual AWS resource and be designed with the principles of don t fix, replace with a new one. See Appendix A for links to architecture whitepapers and reference architectures Customer Partner must provide two (2) customer References references for application that were designed, built and managed as part of a single project or customer engagement Application Amazon Simple Workflow Service (SWF) Services Amazon CloudSearch Amazon Elastic Transcoder Amazon AppStream Amazon Kinesis 14.2 Data Management (Optional) Partners wishing to meet the Data Management specialist requirements should meet all required capabilities below; meeting optional competencies can offset missing required competencies at AWS's discretion Database Administration Customer References Partners must provide database administration services to customers to help manage the database services required below. Partners must provide two (2) customer references for database specific infrastructure and DB administration services in a single project or customer engagement Databases Amazon Redshift Amazon DynamoDB Amazon ElastiCache (Memcache and Redis) Analytics Amazon Kinesis Amazon Elastic MapReduce (EMR) AWS Data Pipeline 14.3 Mobile Applications (Optional) Partners wishing to meet the Mobile Application requirements should meet all required capabilities below; meeting optional competencies can offset missing required competencies at AWS's discretion Custom Application Development Capabilities Partner has demonstrated application development capabilities that utilize AWS best practices to build stateless, elastic and highly available applications Customer References Specifically applications should be built in an ecosystem that includes continual deployment methodologies, automated deployment and is aware of the elastic nature of AWS. In addition applications should be built to survive failure of any individual AWS resource and be designed with the principles of don t fix, replace with a new one. Partner must provide two (2) customer references for application that were designed, built and managed as part of a single project or customer engagement Mobile Services Amazon Cognito Amazon Mobile Analytics AWS MSP Partner Validation Checklist, v2.3 pg. 26

27 Amazon Simple Notification Service (SNS) Database Amazon DynamoDB Miscellaneous AWS Mobile SDK 14.4 End User Services Specialist (Optional) Partners wishing to meet the End User Services Specialist requirements should meet all required capabilities below; meeting optional competencies can offset missing required competencies at AWS's discretion End User Services Partner must provide end user service provisioning, administration and support for customers Customer Partner must provide two (2) customer References references for the design, implementation and management of end user services. Partner must provide two (2) customer references of successful integration into on premise end user infrastructure Enterprise Amazon WorkSpaces Applications Amazon WorkDocs Networking AWS Direct Connect (D) Administration and Security AWS Directory Services AWS MSP Partner Validation Checklist, v2.3 pg. 27

28 Appendix A: Technical Competencies AWS Core Technical Capabilities (Required) In addition to their operational capabilities, Partner will undergo validation of their technical competencies in order to receive the AWS Managed Service Partner designation. For each of the following items marked required, Partner should be prepared with the following: Description of the AWS service Examples of customer solutions leveraging each service marked as required If required solution is not being leveraged by an active customer, Partner must provide a hypothetical use case when it would be appropriate to use that service Description of how services are supported by Partner, alone or as part of a solution comprising multiple services Required Optional Partner Self- Assessment Validated Compute Amazon Elastic Compute Cloud (EC2) Amazon Elastic Block Store (EBS) Elastic Load Balancing (ELB) Networking Amazon Virtual Private Cloud (VPC) AWS Direct Connect (D) Amazon Route 53 (R53) Databases Amazon Relational Database Service (RDS) Storage and Content Distribution Amazon ElastiCache Amazon Simple Storage Service (S3) Amazon Glacier Amazon CloudFront AWS Storage Gateway Application Services Amazon Simple Queue Service (SQS) Deployment and Management Administration and Security Amazon Simple Service (SES) AWS Elastic Beanstalk AWS CloudFormation AWS OpsWorks AWS Directory Service AWS Identity and Access Management (IAM) AWS CloudTrail Amazon CloudWatch (CW) AWS Trusted Advisor Mobile Services Amazon Simple Notification Service (SNS) AWS MSP Partner Validation Checklist, v2.3 pg. 28

29 Appendix B: Best Practice Guides and Reference Materials Always check the whitepapers URL for the latest versions or subscribe to the AWS Whitepapers RSS feed Amazon Web Services Whitepapers: Amazon Web Services Whitepapers RSS Feed: Start Here > Basic Operational Checklist and Enterprise Operational Checklist: AWS Security Center: Overview of Security Processes Whitepaper: Security Best Practices: AWS Compliance: Auditing Security Checklist: Introduction to AWS Security Credentials: Amazon Identity and Access Management: AWS IAM Best Practices: Making Secure Requests to Amazon Web Services: Building Fault Tolerant Applications on AWS: AWS MSP Partner Validation Checklist, v2.3 pg. 29

30 Appendix C: White Label Terminology, Expectations of Parties, and Process for Audit Terminology Throughout this document when referring to white label solution providers the following terminology will be used: Partner: The white label solution provider being audited. White Label Vendor: The organization that will inherit the standards and offer the Partner's services to their customers. Customer: The end client to whom managed services will be provided. Audit Process for White Label Vendors Due to the nature of capability inheritance and complexity of relationships an exception in the audit process is in place. This section lays out the process that Partners and White Label Vendors must follow to complete an MSP program audit. 1. Upon certification with the White Label business capability identifier the Partner must disclose which standards a) will always be passed onto White Label Vendors and b) are optionally passed onto White Label Vendors. This can be updated by contacting the MSP program office. Updates can affect the audit status of the Partner AND all white label vendors associated with that Partner. 2. The MSP program will assess which of the inherited standards must be passed to a White Label Vendor, in order for that vendor to avoid a separate audit on attempting to join the program; this will be provided to the Partner. This can include technical or business capability identifiers that the Partner may claim. 3. When a White Label Vendor wishes to join the program, the Partner must provide an assessment checklist to the MSP program office showing the contractually agreed standards that the White Label Vendor will inherit, as well as the contract length. This must include all standards listed as always provided to White Label Vendors, and optional standards that the White Label Vendor has selected. 4. White Label Vendors that receive all the standards as indicated by the MSP program will be admitted into the program, but will not receive credits as those Partners who are audited. White Label Vendors who do not inherit all the standards stipulated by the MSP program must submit a self-assessment in addition to the Partner assessment and will be audited, but standards passed by Partner will be automatically marked as complete. Expectation of Parties for Partners Providing White Label Solutions Partners must report any changes that may affect whether the Partner still meets a standard they have been previous validated as meeting (during an audit). Partners may report new standards that they feel are now met by their service to the MSP program office for assessment. AWS MSP Partner Validation Checklist, v2.3 pg. 30

31 Partners must follow the process stipulated above to admit new White Label Vendors to the MSP program. Partners must report White Label Vendors that have left their service within five (5) business days of that White Label Vendor no longer consuming their services. Partner is responsible for communicating to the White Label Vendor that this will result in the loss of MSP program audit status. Partners should also ensure that White Label Vendors are aware that MSP program audit status is only valid for the length of the contractual agreement to provide services between the Partner and White Label Vendor. Failure to meet the reporting or information requirements laid out in this section may result in the loss of audit status for the Partner and all associate White Label Vendors. AWS MSP Partner Validation Checklist, v2.3 pg. 31

32 Summary of Changes The following changes resulted in version change from 2.2 to Training requirement changed in prerequisite section from Advanced Operations on AWS to DevOps Engineering on AWS 2. White label business capability identifier section expanded 3. Program rules for white label partners expanded 4. Digital Media, Healthcare and Life Sciences competencies added to business capability identifiers 5. Phrasing and grammar updates made to core competency standards 6. Changes and clarifications to requirements made in sections 6.1 (Security Management), 7.4 (Proactive Monitoring and Alerting), 7.5 (Event and Incident Management), 7.9 (Release and Deployment Management) and 12.0 (SLA Optimization) 7. Changed prerequisites to Advanced or Premier APN Partners with minimum $25k monthly revenue The following changes resulted in version change from 2.1 to 2.2: 1. Expanded Expectations of Parties section 2. Added requirements for Business Management 3. Revised and/or moved requirements in several sections as a result of feedback from initial auditing activity 4. Updated scoring as appropriate 5. Moved Technical Competencies to Prerequisites; moved table to Appendix A 6. Added White Labeling to the Business Capability Identifiers The following changes resulted in version change from 2.0 to 2.1: 7. Added section-by-section scoring 8. Added section numbers for easier reference 9. Added contract language in section 6.0 The following changes resulted in version change from 1.0 to 2.0: 1. Added table of contents 2. Added scoring matrix 3. Added expectations of parties section 4. Detailed requirements under main headers to explicitly call out evidence partners should prepare in advance of validation meeting 5. Split cloud operations and technical support into event, incident, problem management, and service desk functions 6. Added AWS core technical requirements 7. Created sections specific to capability identifiers 8. Added Appendix 1: Best Practice Guides and Reference Materials AWS MSP Partner Validation Checklist, v2.3 pg. 32