Practical Benefits of Multi-threaded RISC/DSP Processing

Transcription

1 Practical Benefits of Multi-threaded RISC/DSP Processing David W. Knox Vice President, Software. Imagination Technologies The Imagination Technologies Ltd. (IMG) META processor is the first commercially available multi-threaded IP (Intellectual Property) core. A common architecture for both RISC and DSP instructions, in combination with hardware multi-threading, allows complex systems to be built around a single processor core, where previously two or more different processors would have been used together. This has already proved beneficial in volume real-time systems supporting broadcast and multimedia requirements, including digital radio and digital television receivers. These systems used early versions of META It is expected that the licensable META 120 IP core will form the basis of many future SoC products. Introduction The Imagination Technologies Ltd. (IMG) META processor core is the first commercially available multi-threaded IP (Intellectual Property) core. It is available either stand-alone, in the form of synthesisable VHDL, or as a component of System-on-Chip (SoC) designs. Architecture Overview The processor contains two 32-bit data units, two 32-bit address units and a control unit. Each data unit provides both conventional ALU operations and, optionally, extended DSP arithmetic operations. The address units are cut-down data units, with operations limited to those required for a range of indexed and auto-incremented memory accesses via pointer registers. The control unit looks after program flow, with features to support branch-prediction, lowoverhead looping, exceptions, interrupts, and synchronous triggers. Main memory is accessed via two caches, one for instructions and one for data. Optionally, the data units may include closely coupled DSP RAM which is accessed like an extended register file, bypassing the main memory system. The processor is intended for high volume, multi-function consumer SoC applications. Many SoC systems have performance critical functions which are best implemented as custom hardware accelerators. Accelerators may be memory mapped devices or they may be implemented as coprocessors connected to the CPU s coprocessor port. Co-processor devices benefit from special instructions to optimise the flow of data between CPU registers and the accelerator. DSP Features The standard processor configuration provides a 32-bit fixed point RISC architecture suitable for general purpose applications (typically compiled C code). All processor options benefit from a 64-bit memory architecture which allows two 32-bit operands to be fetched or stored in the same cycle. In addition, an optional package of DSP extensions includes: Larger register files in the data and address units for additional scratch variable storage 40-bit accumulators with sustainable single cycle Multiply-Accumulate (MAC) performance. Extended arithmetic modes including saturation, rounding and scaling. Zero overhead hardware loop support in the control unit. Parallel ALU operation and memory access. DSP RAM memory banks, built into the data units to provide scratch RAM space with register style access

2 Non-linear address modes (bit-reversed and modulo addressing) in the address units. Split arithmetic: a form of SIMD in which a data unit treats a 32-bit register as either two 16-bit values or four 8-bit values and applies the same operation to all of them. Dual bank operation: a form of SIMD in which both data units perform the same operation in parallel. By using both dual bank operation and split-16 arithmetic together, a sustained performance of 4 MACs per cycle is readily achievable. A system of DSP instruction Template registers which allows a rich DSP instruction set to be made available without resorting to an excessively long instruction word. The DSP Template registers implement a table driven VLIW system which keeps the number of DSP op-codes required low enough for both general purpose (RISC) and DSP instructions to be combined in a unified instruction stream. Hardware Multi-Threading The most significant feature of the processor is that it maintains a number of separate hardware execution threads. System designers can tailor the processor implementation by choosing the number of threads and how many of the threads have DSP capability. A typical implementation might have 4 threads of which two would be DSP-capable. A lightweight system might have one general purpose thread and one DSP-capable thread. A heavyweight system might have 4 DSP-capable threads. Although the processing resources (multiplier, adder etc) are shared, each thread has its own registers, pipeline and program counter. On every cycle, a hardware scheduler looks at the next candidate instruction from each thread and chooses which to execute. This choice is based on availability of processing resources and a thread prioritisation system. There is an opportunity for a context switch on every instruction cycle, without overhead. Over fifty internal resources are considered by the scheduler to determine whether a thread can run on the next cycle but most significant, from a programmers viewpoint, is whether a previous memory access has completed the time for a memory access can be lengthy (depending on memory type) and unpredictable (depending on cache state, refresh cycles etc). For high performance DSP code, programmers will attempt to control these memory stalls using techniques such as cache pre-fetches and pipelined address pre-issue. With hardware multi-threading, memory stalls are much less of a problem. When one thread stalls, other threads can execute. Simply dividing a system into a few major subsystems, with one executing on each thread, can be all that is required to eliminate wasted cycles from the system as a whole. Figure 1 and Figure 2 show the difference between the conventional and the multithreaded approaches. By way of example, we have a system of four tasks, two are hard real time tasks in the sense that they must be scheduled in response to external events and complete before the next event arrives. Two further tasks are lower priority continuous background activities. The histogram in Figure 1 shows, as a function of time, the proportion of CPU cycles on which an instruction is issued for a particular CPU and memory combination. This histogram does not reveal all the details of the task scheduling, but we can see that, while the utilisation varies a little as the software scheduler switches tasks, more than 25% of the available CPU cycles are unused. Figure 1 Single Threaded Operation In Figure 2 we see what happens when we use hardware multi-threading to run each of the four tasks on its own hardware thread. The tasks on threads 0 and 1 are the repetitive real-time tasks and these have been given higher priority to ensure they meet their deadlines. The tasks on threads 2 and 3 are the background load tasks

3 We can see that, even though it has a lower priority, thread 1 makes some progress when thread 0 is running by fitting into the unusable (for thread 0) cycles. More significantly, the total CPU utilisation has increased to nearly 100% as shown by the lowest graph. Figure 2 Prioritised Multi-Threading Super-Threading Although there is only one set of processing resources shared between all the threads, not every resource is needed by every instruction. In principle, instructions which do not conflict in their resource requirements can execute simultaneously. Often, more than one thread can issue an instruction on each cycle. We call this super-threading. The performance improvement obtained in practice depends on the particular instruction sequences in the program, but throughput improvements from 50% to 100% relative to simple single-instruction multithreading are not uncommon. Figure 3 shows the additional improvement obtained in our example system as a result of super-threading. This system is identical to that illustrated in Figure 2, but superthreading has increased instruction issues to about 150% of the clock rate. Figure 3 Super-Threading Automatic MIPS Allocation For simplicity, in the examples shown so far, we have used a fixed priority scheme to control the relative execution rates of the four threads. While this is adequate for a system of regular processing in response to periodic events, real systems are more complex and more sophisticated control is required. The scheduler includes a patented system called Automatic MIPS Allocation (AMA ), which dynamically adjusts thread priorities to give each thread, a controlled proportion of the CPU capacity. The control can be applied to both short and long term observation periods, for example to allow a thread to respond to a real-time event with a burst of activity before settling back to its background activity level. Figure 4 illustrates a simple AMA rate control configuration using the same system as in the previous examples. Here we have limited threads 0 and 1 to an instruction issue rate which is just sufficient for them to meet their real-time deadlines (about 40% each). Thread 2 has been set at 20% and thread 3 has been allowed to free-run and absorb the remaining capacity. In particular, this has enabled us to control the relative resource use of threads 2 and 3 with greater precision than simple fixed priorities allowed

4 Figure 4 AMA Rate Control Figure 5 shows a more realistic configuration in which we have used a combination of thread prioritisation and AMA to achieve the desired behaviour. Threads 0 and 1 have elevated priorities for timely real-time response. Thread 2 is controlled by AMA for an execution rate of about 40% instructions/clock and thread 3 is a free-running background task. Because of the interference from the higher priority threads, at times thread 2 cannot achieve the desired run rate and builds up a processing deficit. Whenever possible, for example immediately after the thread 1 activity burst is complete, thread 2 s execution rate is increased to make up the deficit. After catching up, the execution rate returns to the set value. Over short periods, the desired execution rate can not be provided, due to total resource demand exceeding what is available. However, over longer periods (the major time divisions) the average rate of 40% instructions/clock is maintained. Figure 5 AMA & Priority Combination Benefits In hardware multi-threading, execution contexts are replicated but processing logic, external interfaces etc. are shared. This means that META systems introduce new points in the set of cost/performance options available to a system designer. The combination of multi-threading and a unified instruction set for both DSP and RISC operations means that we can often replace a multi-processor RISC+DSP system with a single multi-threaded core. It turns out that the performance benefits of the multi-threaded architecture are easy to realise at the system design level. Often a fairly simplistic partitioning across two or three hardware threads is all that is required to convert a significant proportion of stall cycles into useful processing. If fine-tuning, through load balancing, is required, then the fact that each thread executes the same instruction set makes it simple to move functions from one thread to another. A second benefit of hardware multithreading occurs during system construction and integration. Software authors can develop each thread as a separate subsystem, largely independent of the others. A software tool combines the separate thread images into a single executable system unit at the end of the build process. Each thread can have a completely different software environment. It - 4 -

5 is even possible to run different software operating systems simultaneously on different threads. Hardware multi-threading allows us to separate real-time tasks having widely differing scheduling requirements into different software scheduling domains. For example a subsystem based on a complex multi-functional OS could run on one thread at the same time as a real-time data-driven DSP task runs on another. The DSP task might be based on a simple synchronous IO driven scheduling strategy which would be completely independent of the interrupts and device drivers in the other subsystem. This approach avoids the problems which arise when trying to schedule tasks with wildly differing event rates and activity patterns under a common OS. Such difficulties are another reason why many systems use two or more processors. causes no interference or break-up on the DAB audio an excellent example of how software tasks on different threads can operate independently on the META core. Devices based on META and currently reaching production are applying the same design approaches to bring the cost of digital TV decoders down to a level where they can be integrated into TV sets rather than requiring a separate set-top box. Now, with the new META 120 core, IMG have a set of standard IP configurations which can be licensed alone or as a component of SoC designs. With the core design and architecture validated by real consumer products in volume manufacture, and the availability of comprehensive software algorithm support, IMG expect that META will form the basis of many future high volume products. Application Examples The META core has been used in System on Chip designs for a range of consumer entertainment products. Perhaps the best known, so far, is the Frontier Silicon CHORUS device which implements a Digital Audio Broadcast (DAB) receiver for the internationally adopted Eureka-147 system. In this design three threads are deployed as follows: (i) COFDM symbol demodulation, (ii) DAB protocol and MPEG Audio decoding, (iii) support and control of a dedicated Viterbi decoder hardware device. The fourth thread looks after the user interface keyboard, display, volume and tuning controls. Many different variants of the fourth thread software have been developed to give a variety of product and brand identities ranging from Hi-Fi rack tuners to portable battery powered devices. The CHORUS chip has rapidly established itself as the world s leading DAB receiver. More recent versions of the DAB receiver software have been designed to use only three threads, leaving the fourth available for completely separate applications. For example, both Linux and an MPEG-4 video decoder have been run on the fourth thread at the same time as the DAB radio receiver. Booting Linux on the fourth thread when DAB is being decoded on the other threads - 5 -