Virtual Platforms Addressing challenges in telecom product development

Transcription

1 white paper Virtual Platforms Addressing challenges in telecom product development

2 This page is intentionally left blank.

3 EXECUTIVE SUMMARY Telecom Equipment Manufacturers (TEMs) are currently facing numerous challenges during product development. There is the constant need to reduce Time to Market (TTM) and cut down on costs, as well as the customer demands for higher quality products. At the same time, telecom products are becoming increasingly complex due to market requirements such as the need to increase capacity and bandwidth, reduce power consumption and benefit from cloud transformation. There has also been pressure from the market to reduce the high level of investment and operational expense during product development, which has resulted in limited access to the necessary hardware during the development and testing phases. Hardware limitations usually lead to lengthy and expensive development projects, which have a negative impact on product quality. Even with unlimited access to hardware, there are technical challenges during the integration and debugging of software in the target environment, which makes the creation of products very time-consuming. Cloud transformation is pushing TEMs into creating new network functionalities as pure cloud-born solutions, with the need to define a migration path for existing network functionalities to new cloud architecture. The following paper addresses all of the above-mentioned challenges and illustrates the potential of virtual platform utilization. A virtual platform solution replaces costly hardware with a software solution that mimics the function and characteristics of the hardware. The software can run on standard servers. Furthermore, a virtual platform can be used across the entire product life cycle: to reduce TTM before the new hardware is available, to lower investment and operational expenses during the software development phase, and to enable unique debugging capabilities that increase product quality. The solution can also be used at a later stage during the support and maintenance phases. The benefits of using a full-system virtual platform are numerous and can result in considerable cost savings. This paper also describes a customer case study in which the customer achieved a reduction in TTM of 6 months and annual savings of EUR six million.

4 CHALLENGES OF TELECOM EQUIPMENT DEVELOPMENT TTM REDUCTION AND HIGHER QUALITY The main challenges of telecom product development are related to efficiency and resources. Market pressure requires short development cycles with a constant need to reduce costs and TTM. This has obliged TEMs to cut development and testing costs. At the same time, product quality needs to be improved, a need that is not always prioritized ahead of TTM reduction. During the software development process, developers depend on accessing real hardware to test and integrate the software into the physical hardware in order to finalize the product. Delayed hardware impacts lead times and also impede the development and integration of software, with a subsequent delay in product TTM. See image 1. only reflect parts of the target system and not the real hardware behaviour and limitations of the target system, all of which has a negative impact on end-product quality. AGILE APPROACH Traditional project management methods (PM) that include planning and static execution phases tend to have a high cost for changes, and the changes can easily have a major impact on design, testing quality and lead times. Late changes are also extremely costly for any project. See image 2. In order to minimize risk and manage the constant changes in telecom development, an increasing number of organizations are adopting an agile approach. Use of agile development methods, including Continuous Integration, requires testing environment availability across the entire project life cycle, and later for maintenance and support. Image 1: Conventional R&D design Flow Developing world-class real-time applications for embedded systems is extremely complex. Due to high OPEX and CAPEX costs, it is not feasible to enable sufficient access to the systems to achieve the necessary quality in either the development or the testing phases. Either way, this only provides limited access and visibility to the hardware states and software for tracing, profiling and debugging. Telecom systems are multi core CPU systems that run in parallel. Debugging and tracing such systems is extremely tricky, if not impossible. Problems are seldom repeatable due to race and timing conditions with the rest of the system. Software module development and testing is often limited by less advanced simulations that Image 2: Traditional vs Agile project costs However, the hardware design process and usage of typical hardware from TEMs for software development does not support an agile approach. Developing an ASIC (Application Specific Integrated Circuit), a lab board, final boards and ultimately a full system is a more sequential process from a software development and integration perspective An agile approach focuses on developing systems iteratively and incrementally, which is an effective way of developing software rapidly and flexibly,

5 one which allows quick design adaptations across the project life cycle. It greatly improves software development and testing, and has a direct positive impact on product quality. CLOUD IMPACTS Emerging cloud technologies have obliged telecom operators and TEMs to re-evaluate their future product strategies. Telecom equipment has traditionally been deployed on the basis of very proprietary hardware solutions that facilitate carrier grade characteristics such as real-time performance and resilience. The rapid evolution of IT server technology and related solutions has enabled a new telecom cloud approach. This cloud transformation is causing TEMs both to create new network functionalities as pure cloudborn solutions and to define a migration path that enables migrating existing network functionalities to new cloud architecture. As the rate of transformation is highly dependent upon the type of network node, this will most likely lead to a situation in which both the vendor-specific legacy network and the cloud-based network will co-exist in parallel within the operator landscape. Virtual platforms can be the enabler for faster cloud adaptations. VIRTUAL PLATFORMS AS THE SOLUTION The use of specialized and costly hardware in the telecom domain constrains the development and testing process. Using standard hardware across the entire software development process would eliminate many obstacles. However, telecom grade applications are not designed to be executed in standard operating systems, and they are dependent upon hardware features that do not exist in standard servers. Virtual platforms offer a solution to these problems. THE VIRTUAL PLATFORM CONCEPT A virtual platform mimics the function and characteristics of a target computer system. It loads and executes unmodified target-compiled software. See image 3. The challenges of telecom product development relate to: TTM Reduction and Costs: Markets are requiring shorter TTM and lower costs for new products Quality: Simplified test environments and limited trace and debug capabilities have a negative impact on target code quality Agility: Traditional hardware design flow is not according to modern agile approaches Cloud impacts: Cloud strategy is obliging telecom operators to re-evaluate their product strategy Image 3: A virtual platform uses a target binary executable The virtual platform is built on a virtualization framework, which provides an infrastructure for managing software models of the hardware blocks. A platform, or a specific part of the platform, typically includes models of several Central Processing Units (CPU), various memories, interfaces such as Ethernet or serial ports, proprietary models of HW accelerators and other IP blocks. All models are assembled into one virtual platform that mimics the real hardware. See image 4

6 Building a virtual platform from scratch is a challenging and complicated task. It requires a high level of technical knowledge in virtualization techniques as well as knowledge of the target system. Image 4: A virtual platform is built on a framework including models that mimics all the HW in the target system To minimize risk and to focus on prioritized features while developing the virtual platform, agile methods are applied for incremental validation and use. Choosing the correct framework and technical solution is crucial and differs based on requirements. Basing the solution on a framework of commercial tools is one feasible alternative that typically adds enhanced introspection and debugging features out of the box. It also ensures a good head start with a set of models of standard IP blocks. Commercial tools usually have the drawback of some lock-in effect regarding tool or model support. The solutions and standards of the Open Source community for virtual platforms have evolved over the past decade and represent a viable alternative to commercial products. Open source allows users to benefit from the work of other developers and provides design and control freedom. Selecting a framework and creating models that support the SystemC/TLM2 open standard enables the development of IP block models and the ability to share with other projects. It also facilitates requests to third-party IP-block manufacturers for virtual models of specific IP blocks, thereby gaining development and lead time for the product. A virtual platform could well be as simple as a few IP blocks or systems on a chip, SoC, a complete board with multiple IP blocks and SoCs, or a full virtual system comprised of several interacting boards. It can even be set up with several racks of boards. The speed of the virtual platform is one of the key issues in most cases. There are several virtualization techniques that support the various needs of system verification and usage, and to achieve high speed, model accuracy has to be compromised. Using models with no or low timing notion allows fast execution, thereby enabling full software stacks to be run at end-user level. Using accurate models that include fine or approximate details about timing will allow system and performance analysis at the expense of speed. Furthermore, an increasing number of target systems tend to be either Intel x86 or ARM based. By selecting the right host server architecture, none or little target code instruction needs to be translated, which improves simulation performance dramatically. In other words, running an x86 target code in a virtual HW on an x86 based server, or an ARM target code on an ARM based server. A virtual platform always runs target code binaries regardless of accuracy level. There are other technical solutions available that are also known as simulators, but these should not be confused with the term virtual platform. Simulators fulfil the external behaviour of an Application Programming Interface (API), typically at Operating System (OS) level. A simulator does not test for the correct environment and requires a host build of the software. Other simulators operate at the network protocol level and replace the surrounding network environment typically by responding to requests or generating network traffic (payload and/or signalling). In a typical test scenario, virtual platforms and simulators co-exist, but they do not serve the same purpose.

7 Virtual Platform as the solution A virtual platform mimics the function and characteristics of a target computer system It loads and executes unmodified target compiled software Selecting the appropriate technology to achieve speed in the virtual system is a key factor Product requirements determine whether the solution is based on COTS or Open source In a typical test scenario, virtual platforms and simulators co-exist BENEFITS OF VIRTUAL PLATFORMS AVAILABILITY OF HARDWARE REDUCES TTM Telecom nodes traditionally consist of proprietary and specialized hardware, which makes them expensive to purchase. With access to virtual hardware, the need for costly prototype hardware in test laboratories is significantly reduced. A virtual platform makes it possible to develop and begin the software verification process before the actual hardware has been produced. Design and implementation of the software can be initiated as early as during hardware design by means of iterative virtual platform development, which results in reduced TTM and enhanced SW quality. Since the virtual platform runs on standard hardware, this increases its availability for development and integration. Having access to non-existent hardware during software development greatly facilitates the interaction between software and hardware teams. Image 5: Conventional design flow is limited by HW availability. Using a virtual platform enables agile development methods as early as the HW design phases Closer co-operation between the teams enables faster development of products, and enhanced product quality at the same time. In addition, a virtual prototype board can be modified and improved in tight cycles then delivered to the software projects. In the early phases, the SW organizations can even feed back to the HW design organization that actually impacts the upcoming HW design. See image 5. ENHANCED PRODUCT QUALITY WITH ADVANCED TESTING A virtual platform provides a debug and tracing environment that is superior to real hardware. Faults and defects can be identified at an earlier stage in the product lifecycle, which enhances product quality and reduces overall product risk. In complex systems, the lead time for each fault correction can be shortened by days or even several weeks. Another improvement when using a virtual platform instead of real hardware is predictable repeatability when performing debugging or fault analysis. The new multicore systems and external dependencies to real hardware make it difficult to reproduce errors. In a virtualized environment all HW models and SW states can be inspected, which enables the reproduction of an identified error at any given time.

8 Virtual platform solutions also enable nonintrusive inspection and modification of internal hardware and software data and states as well as advanced tracing, logging and data monitoring. Full monitoring of the system not only allows introspection, it also opens up the possibility of injecting faults, which allows the creation of test suites for negative test cases and to increase code coverage. PROCESS IMPACTS The process of designing hardware and reaching the phase whereby the hardware is ready for software development and testing has traditionally resembled a waterfall process. The benefits of the virtualized platform can be achieved once the development organizations have adopted and changed their approach. The introduction of agile and iterative software development for target software organizations requires constant access to the testing environment. Since virtual platform solutions allow both software and hardware design teams better access, it naturally suits this new approach. By using early drops of the virtual platform, the software design teams can begin using it and quickly feed back to the hardware design team. This close-collaboration approach both reduces the overall implementation time and has a positive impact on product quality because the feedback can be delivered to virtual platform and hardware teams alike. In modern software practices such as Continuous Integration (CI), the code changes of each software designer are tested in isolation. These methodologies also require access to test equipment, in which case the scalability characteristics of the virtual platform are especially beneficial. Since the virtual platform is software, this makes it possible to participate in product development and troubleshooting from different geographical locations, thereby creating a shared environment for colleagues all around the world. A virtual platform can also be used beyond R&D activities, for example for education and training purposes and to reduce the number of real systems needed during the support and maintenance phases of the product life cycle. PRIVATE CLOUD OPENS UP NEW OPPORTUNITIES The introduction of private clouds as virtual labs in product development opens up new opportunities to deploy virtual platforms in virtualized servers. Using virtual servers enables power to be scaled dynamic shrink and growth based on current capacity needs, which means there is less or no need for costly proprietary HW. The migration path and co-existence of legacy networks and telecom cloud is pushing telecom R&D into rethinking their product development strategy, especially in regard to product and system verification. Virtual platforms are one of the enablers in the transformation to telecom cloud in the context of product testing and verification. Migrating from proprietary HW solutions into cloud poses challenges because the vendor-specific legacy networks have to co-exist with the cloudbased network, which requires double investments in the test environments. In network infrastructure, virtual domain platforms provide a natural migration path from proprietary hardware deployments to COTS server solutions, which offers cost savings for the equipment manufacturer. Benefits of using virtual platforms: Virtual platforms enable the development and software verification process before the hardware has been produced, which reduces TTM and enhances SW quality Virtual platforms are well suited for agile development methods Virtual platforms are preferably integrated in continuous integration loops Faults and defects can be identified at an earlier stage in the product life cycle, which enhances product quality and reduces overall product risk.

9 CASE STUDY: VIRTUAL PLATFORM WITH ANNUAL SAVINGS OF EUR 6 MILLION Customer: A platform organization at a leading telecom network equipment provider that supplies the platform for the latest generation of mobile networks. The customer had two overall issues that impacted directly on TTM and product quality: lack of availability of hardware systems for the development and verification phases due to high costs. There were also deficiencies in the debug environment. The product was based on standard as well as customer-specific hardware. The challenge was to virtualize the platform and to provide an enhanced debug and trace environment. The prerequisite for the solution was that it needed to be flexible and enable the design of the new virtual hardware based on new or updated hardware specifications. The overall cost of ownership had to be lower than using real hardware. The assignment began with several important architectural decisions, including the selection of the virtualization framework that was to be used, in this case a commercially available product. The implementation phase included the creation of software models for both COTS components and customer proprietary hardware elements. An extended debug API was developed for integration with the existing debug functionality to allow inspect and inject features. were successfully debugged and rectified using the virtual platform, which reduced the lead time for each fault correction by approximately two weeks. In the platform maintenance department, 55% of the test cases ran on the virtual platform. The lab infrastructure was reduced by approximately 6 hardware systems. The customer was able to perform several critical customer delivery systems, even if they lacked the proper hardware for release verification. By using the virtual platform instead, they avoided penalties for late delivery. In this customer case, the full-system virtual platform solution not only reduced TTM by almost 6 months, it also generated annual cost savings of approximately EUR 6 million. The customer is now using a virtual platform regularly for all types of delivery. Case Study: A leading telecom equipment manufacturer was experiencing problems due to inadequate availability of test and development equipment and needed to increase debugging capabilities. The use of a virtual platform resulted in the following improvements: Reduced time to market: 75% of the code was tested before the real HW existed, which reduced TTM by 6 months CAPEX and OPEX reduction: 6 hardware systems were replaced, which generated annual cost savings of EUR 6 million When the solution was ready for implementation, deployment and roll-out support were key success factors that ensured the benefits of the solution in the company. The results that the solution achieved were convincing. During software design, 75% of the code was tested before the real hardware existed. Furthermore, problems that would have been almost impossible to solve with real hardware

10 CONCLUSIONS Previous development methods no longer provide the means for TEMs to meet overall market requirements, i.e. more features, faster deliveries and enhanced quality at a lower cost. A virtual platform offers the means to overcome many of the mentioned challenges and provides multiple benefits: Minimizes investments and operational costs as well as the need to maintain physical hardware during development, test, integration and functional test. Using a virtual platform is also cost efficient for the aftermarket. Enables software design, verification and integration before the actual hardware has been produced, which reduces TTM. Provides a scalable and cost-efficient test infrastructure, which opens up agile approaches and the possibility of early, extensive software verification. Offers improved debugging and tracing capabilities, which enhances end-product quality significantly. The advanced debugging features accelerate root cause analysis, which impacts positively on the quality of the products. In order to maximize the use of a virtual platform solution, working processes within the organization need to be adapted to agile approaches. It is not enough merely to create the virtual platform; organizations need to pay due attention to applying the solution. Deployment, environment adaptations, on-site training, project-specific consultation, guidance and early recognition of problems are other important considerations. In this paper, we have described how a fullsystem virtual platform can solve several of the challenges that TEMs need to address. We have also presented actual benefit figures from a customer case that describe how the solution can reduce TTM by up to 6 months and generate overall cost savings of several millions euros. TIETO AS A VIRTUAL PLATFORM PARTNER Tieto provides virtual platform services divided into expert, project and product-level deliveries that unlock value across the complete life cycle of the product. See image 6. Image 6: Tieto Virtual Platform Services Expertise consultancy services include operational analysis and guidance in technical solutions. Creation services include the creation of software models for proprietary hardware, design and implementation of virtual platform solutions that comprise full system (node-level), board-level and hardware component (SoC) virtual platforms. Integration services prioritize the improvement of the end-user experiences, for example virtual platform bring-up and integration of virtual platforms with surrounding development, tools and test systems. Support services provide customer-specific training, deployment and user support. Tieto is committed to providing Telecom Equipment Manufacturers with virtual platforms and related services in order to promote all aspects of software development, which enhances product TTM, quality and cost efficiency. Tieto has been providing virtual platform services to major telecom equipment providers for more than 15 years. The outlined customer case is just one example of the capabilities and wide range of services that Tieto can provide. Tieto is widely experienced in a range of virtual platform technologies, but experts in creating solutions based on the QEMU framework [1] and Wind River Simics [2]. For more information visit [3]

11 ABBREVIATIONS API ASIC CAPEX CI COTS CPU HW OPEX OS R&D SoC SW TEM TTM Application Programming Interface Application Specific Integrated Circuit Capital Expenditures Continuous Integration Commercial off-the-shelf Central Processing Unit Hardware Operational Expenditures Operating System Research and Development System on a Chip Software Telecom Equipment Manufacturer Time-to-Market REFERENCES [1] Wind River Simics. Available at [2] QEMU. Available at [3] Tieto Virtual Platforms. Available at