TMS320C6000 Programmer s Guide

Transcription

1 TMS320C6000 Programmer s Guide Literature Number: SPRU198K Revised: July 2011 Printed on Recycled Paper

2

3 Preface Read This First About This Manual This manual is a reference for programming TMS320C6000 digital signal processor (DSP) devices. Before you use this book, you should install your code generation and debugging tools. This book is organized in five major parts: Part I: Introduction includes a brief description of the C6000 architecture and code development flow. It also includes a tutorial that introduces you to the tools you will use in each phase of development and an optimization checklist to help you achieve optimal performance from your code. Part II: C Code includes C code examples and discusses optimization methods for the code. This information can help you choose the most appropriate optimization techniques for your code. Part III: Assembly Code describes the structure of assembly code. It provides examples and discusses optimizations for assembly code. It also includes a chapter on interrupt subroutines. Part IV: C64x Programming Techniques describes programming considerations for the C64x. iii

4 Related Documentation From Texas Instruments Related Documentation From Texas Instruments The following books describe the TMS320C6000 devices and related support tools. To obtain a copy of any of these TI documents, call the Texas Instruments Literature Response Center at (800) When ordering, please identify the book by its title and literature number. TMS320C6000 Assembly Language Tools User s Guide (literature number SPRU186) describes the assembly language tools (assembler, linker, and other tools used to develop assembly language code), assembler directives, macros, common object file format, and symbolic debugging directives for the C6000 generation of devices. TMS320C6000 Optimizing C Compiler User s Guide (literature number SPRU187) describes the C6000 C compiler and the assembly optimizer. This C compiler accepts ANSI standard C source code and produces assembly language source code for the C6000 generation of devices. The assembly optimizer helps you optimize your assembly code. TMS320C6000 CPU and Instruction Set Reference Guide (literature number SPRU189) describes the C6000 CPU architecture, instruction set, pipeline, and interrupts for these digital signal processors. TMS320C6000 Peripherals Reference Guide (literature number SPRU190) describes common peripherals available on the TMS320C6201/6701 digital signal processors. This book includes information on the internal data and program memories, the external memory interface (EMIF), the host port interface (HPI), multichannel buffered serial ports (McBSPs), direct memory access (DMA), enhanced DMA (EDMA), expansion bus, clocking and phase-locked loop (PLL), and the power-down modes. TMS320C64x Technical Overview (SPRU395) The TMS320C64x technical overview gives an introduction to the C64x digital signal processor, and discusses the application areas that are enhanced by the C64x VelociTI. iv

5 Trademarks Trademarks Solaris and SunOS are trademarks of Sun Microsystems, Inc. VelociTI is a trademark of Texas Instruments Incorporated. Windows and Windows NT are trademarks of Microsoft Corporation. The Texas Instruments logo and Texas Instruments are registered trademarks of Texas Instruments Incorporated. Trademarks of Texas Instruments include: TI, XDS, Code Composer, Code Composer Studio, TMS320, TMS320C6000 and 320 Hotline On-line. All other brand or product names are trademarks or registered trademarks of their respective companies or organizations. v

6 vi

7 Contents Contents 1 Introduction Introduces some features of the C6000 microprocessor and discusses the basic process for creating code and understanding feedback 1.1 TMS320C6000 Architecture TMS320C6000 Pipeline Code Development Flow to Increase Performance Optimizing C/C++ Code Explains how to maximize C performance by using compiler options, intrinsics, and code transformations. 2.1 Writing C/C++ Code Tips on Data Types Analyzing C Code Performance Compiling C/C++ Code Compiler Options Memory Dependencies Performing Program-Level Optimization ( pst Option) Profiling Your Code Using the Standalone Simulator (load6x) to Profile Refining C/C++ Code Using Intrinsics Wider Memory Access for Smaller Data Widths Software Pipelining Compiler Optimization Tutorial Uses example code to walk you through the code development flow for the TMS320C Introduction: Simple C Tuning Project Familiarization Getting Ready for Lesson Lesson 1: Loop Carry Path From Memory Pointers Lesson 2: Balancing Resources With Dual-Data Paths Lesson 3: Packed Data Optimization of Memory Bandwidth Lesson 4: Program Level Optimization Lesson 5: Writing Linear Assembly vii

8 Contents 4 Feedback Solutions Provides a quick reference to techniques to optimize loops. 4.1 Understanding Feedback Stage 1: Qualify the Loop for Software Pipelining Stage 2: Collect Loop Resource and Dependency Graph Information Stage 3: Software Pipeline the Loop Loop Disqualification Messages Bad Loop Structure Loop Contains a Call Too Many Instructions Software Pipelining Disabled Uninitialized Trip Counter Suppressed to Prevent Code Expansion Loop Carried Dependency Bound Too Large Cannot Identify Trip Counter Pipeline Failure Messages Address Increment Too Large Cannot Allocate Machine Registers Cycle Count Too High. Not Profitable Did Not Find Schedule Iterations in Parallel > Max. Trip Count Speculative Threshold Exceeded Iterations in Parallel > Min. Trip Count Register is Live Too Long Too Many Predicates Live on One Side Too Many Reads of One Register Trip var. Used in Loop Can t Adjust Trip Count Investigative Feedback Loop Carried Dependency Bound is Much Larger Than Unpartitioned Resource Bound Two Loops are Generated, One Not Software Pipelined Uneven Resources Larger Outer Loop Overhead in Nested Loop There are Memory Bank Conflicts T Address Paths Are Resource Bound Optimizing Assembly Code via Linear Assembly Describes methods that help you develop more efficient assembly language programs. 5.1 Linear Assembly Code Assembly Optimizer Options and Directives The on Option The mt Option and the.no_mdep Directive The.mdep Directive The.mptr Directive viii

9 Contents The.trip Directive Writing Parallel Code Dot Product C Code Translating C Code to Linear Assembly Linear Assembly Resource Allocation Drawing a Dependency Graph Nonparallel Versus Parallel Assembly Code Comparing Performance Using Word Access for Short Data and Doubleword Access for Floating-Point Data Unrolled Dot Product C Code Translating C Code to Linear Assembly Drawing a Dependency Graph Linear Assembly Resource Allocation Final Assembly Comparing Performance Software Pipelining Modulo Iteration Interval Scheduling Using the Assembly Optimizer to Create Optimized Loops Final Assembly Comparing Performance Modulo Scheduling of Multicycle Loops Weighted Vector Sum C Code Translating C Code to Linear Assembly Determining the Minimum Iteration Interval Drawing a Dependency Graph Linear Assembly Resource Allocation Modulo Iteration Interval Scheduling Using the Assembly Optimizer for the Weighted Vector Sum Final Assembly Loop Carry Paths IIR Filter C Code Translating C Code to Linear Assembly (Inner Loop) Drawing a Dependency Graph Determining the Minimum Iteration Interval Linear Assembly Resource Allocation Modulo Iteration Interval Scheduling Using the Assembly Optimizer for the IIR Filter Final Assembly If-Then-Else Statements in a Loop If-Then-Else C Code Translating C Code to Linear Assembly Drawing a Dependency Graph Determining the Minimum Iteration Interval Contents ix

10 Contents Linear Assembly Resource Allocation Final Assembly Comparing Performance Loop Unrolling Unrolled If-Then-Else C Code Translating C Code to Linear Assembly Drawing a Dependency Graph Determining the Minimum Iteration Interval Linear Assembly Resource Allocation Final Assembly Comparing Performance Live-Too-Long Issues C Code With Live-Too-Long Problem Translating C Code to Linear Assembly Drawing a Dependency Graph Determining the Minimum Iteration Interval Linear Assembly Resource Allocation Final Assembly With Move Instructions Redundant Load Elimination FIR Filter C Code Translating C Code to Linear Assembly Drawing a Dependency Graph Determining the Minimum Iteration Interval Linear Assembly Resource Allocation Final Assembly Memory Banks FIR Filter Inner Loop Unrolled FIR Filter C Code Translating C Code to Linear Assembly Drawing a Dependency Graph Linear Assembly for Unrolled FIR Inner Loop With.mptr Directive Linear Assembly Resource Allocation Determining the Minimum Iteration Interval Final Assembly Comparing Performance Software Pipelining the Outer Loop Unrolled FIR Filter C Code Making the Outer Loop Parallel With the Inner Loop Epilog and Prolog Final Assembly Comparing Performance Outer Loop Conditionally Executed With Inner Loop Unrolled FIR Filter C Code Translating C Code to Linear Assembly (Inner Loop) Translating C Code to Linear Assembly (Outer Loop) x

11 Contents Unrolled FIR Filter C Code Translating C Code to Linear Assembly (Inner Loop) Translating C Code to Linear Assembly (Inner Loop and Outer Loop) Determining the Minimum Iteration Interval Final Assembly Comparing Performance C64x Programming Considerations Describes programming considerations for the C64x. 6.1 Overview of C64x Architectural Enhancements Improved Scheduling Flexibility Greater Memory Bandwidth Support for Packed Data Types Non-Aligned Memory Accesses Additional Specialized Instructions Accessing Packed-Data Processing on the C64x Packed Data Types Storing Multiple Elements in a Single Register Packing and Unpacking Data Optimizing for Packed Data Processing Vectorizing With Packed Data Processing Combining Multiple Operations in a Single Instruction Non-Aligned Memory Accesses Performing Conditional Operations With Packed Data Linear Assembly Considerations Using BDEC and BPOS in Linear Assembly Avoiding Cross Path Stalls C64x+ Programming Considerations Describes programming considerations for the C64x Overview of C64x+ Architectural Enhancements Improved Scheduling Flexiblity Additional Specialized Instructions Software Pipelined Loop (SPLOOP) Buffer Exceptions Compact Instruction Set Utilizing Additional Instructions Improvng Multiply Throughput Combining Addition and Subtraction Instructions Improved Packed Data Instructions Software Pipelined Loop (SPLOOP) Buffer Introduction to SPLOOP Buffer Terminology Hardware Support for SPLOOP Contents xi

12 Contents The Compiler Supports SPLOOP Single Loop With SPLOOP(D) Nested Loop With SPLOOP(D) Do While Loops Using SPLOOPW Considerations for Interruptible SPLOOP Code Compact Instructions Structure of Assembly Code Describes the structure of the assembly code, including labels, conditions, instructions, functional units, operands, and comments. 8.1 Labels Parallel Bars Conditions Instructions Functional Units Operands Comments Interrupts Describes interrupts from a software programming point of view. 9.1 Overview of Interrupts Single Assignment vs. Multiple Assignment Interruptible Loops Interruptible Code Generation Level 0 - Specified Code is Guaranteed to Not Be Interrupted Level 1 Specified Code Interruptible at All Times Level 2 Specified Code Interruptible Within Threshold Cycles Getting the Most Performance Out of Interruptible Code Interrupt Subroutines ISR with the C/C++ Compiler ISR with Hand-Coded Assembly Nested Interrupts Linking Issues Explains linker messages and how to use run-time-support functions How to Use Linker Error Messages How to Find The Problem Executable Flag How to Save On-Chip Memory by Placing Run-Time Support Off-Chip How to Compile Must #include Header Files Run-Time-Support Data How to Link Example Compiler Invocation xii

13 Contents Header File Details Changing Run-Time-Support Data to near Contents xiii

14 Figures Figures 2 1 Dependency Graph for Vector Sum # Software-Pipelined Loop Dependency Graph for Lesson_c.c Dependency Graph of Fixed-Point Dot Product Dependency Graph of Floating-Point Dot Product Dependency Graph of Fixed-Point Dot Product with Parallel Assembly Dependency Graph of Floating-Point Dot Product With Parallel Assembly Dependency Graph of Fixed-Point Dot Product With LDW Dependency Graph of Floating-Point Dot Product With LDDW Dependency Graph of Fixed-Point Dot Product With LDW (Showing Functional Units) Dependency Graph of Floating-Point Dot Product With LDDW (Showing Functional Units) Dependency Graph of Fixed-Point Dot Product With LDW (Showing Functional Units) Dependency Graph of Floating-Point Dot Product With LDDW (Showing Functional Units) Dependency Graph of Weighted Vector Sum Dependency Graph of Weighted Vector Sum (Showing Resource Conflict) Dependency Graph of Weighted Vector Sum (With Resource Conflict Resolved) Dependency Graph of Weighted Vector Sum (Scheduling ci +1) Dependency Graph of IIR Filter Dependency Graph of IIR Filter (With Smaller Loop Carry) Dependency Graph of If-Then-Else Code Dependency Graph of If-Then-Else Code (Unrolled) Dependency Graph of Live-Too-Long Code Dependency Graph of Live-Too-Long Code (Split-Join Path Resolved) Dependency Graph of FIR Filter (With Redundant Load Elimination) Bank Interleaved Memory Bank Interleaved Memory With Two Memory Blocks Dependency Graph of FIR Filter (With Even and Odd Elements of Each Array on Same Loop Cycle) Dependency Graph of FIR Filter (With No Memory Hits) Four Bytes Packed Into a Single General Purpose Register Two halfwords Packed Into a Single General Purpose Register Graphical Representation of _packxx2 Intrinsics Graphical Representation of _spack xiv

15 Figures 6 5 Graphical Representation of 8-Bit Packs (_packx4 and _spacku4) Graphical Representation of 8-Bit Unpacks (_unpkxu4) Graphical Representation of (_shlmb, _shrmb, and _swap4) Graphical Representation of a Simple Vector Operation Graphical Representation of Dot Product Graphical Representation of a Single Iteration of Vector Complex Multiply Array Access in Vector Sum by LDDW Array Access in Vector Sum by STDW Vector Addition Graphical Representation of a Single Iteration of Vector Multiply Packed Multiplies Using _mpy Fine Tuning Vector Multiply (shift > 16) Fine Tuning Vector Multiply (shift < 16) Graphical Representation of the _dotp2 Intrinsic c = _dotp2(b, a) The _dotpn2 Intrinsic Performing Real Portion of Complex Multiply _packlh2 and _dotp2 Working Together Graphical Illustration of _cmpxx2 Intrinsics Graphical Illustration of _cmpxx4 Intrinsics Graphical Illustration of _xpnd2 Intrinsic Graphical Illustration of _xpnd4 Intrinsic C64x Data Cross Paths Nested Loops Flow Labels in Assembly Code Parallel Bars in Assembly Code Conditions in Assembly Code Instructions in Assembly Code TMS320C6x Functional Units Units in the Assembly Code Operands in the Assembly Code Operands in Instructions Comments in Assembly Code Contents xv

16 Tables Tables 1 1 Three Phases of Software Development Code Development Steps Compiler Options to Avoid on Performance Critical Code Compiler Options for Performance Compiler Options That Slightly Degrade Performance and Improve Code Size Compiler Options for Control Code Compiler Options for Information TMS320C6000 C/C++ Compiler Intrinsics TMS320C64x/C64x+ C/C++ Compiler Intrinsics TMS320C64x+ C/C++ Compiler Intrinsics TMS320C67x C/C++ Compiler Intrinsics Memory Access Intrinsics Status Update: Tutorial example lesson_c lesson1_c Status Update: Tutorial example lesson_c lesson1_c lesson2_c Status Update: Tutorial example lesson_c lesson1_c lesson2_c lesson3_c Status Update: Tutorial example lesson_c lesson1_c lesson2_c lesson3_c Comparison of Nonparallel and Parallel Assembly Code for Fixed-Point Dot Product Comparison of Nonparallel and Parallel Assembly Code for Floating-Point Dot Product Comparison of Fixed-Point Dot Product Code With Use of LDW Comparison of Floating-Point Dot Product Code With Use of LDDW Modulo Iteration Interval Scheduling Table for Fixed-Point Dot Product (Before Software Pipelining) Modulo Iteration Interval Scheduling Table for Floating-Point Dot Product (Before Software Pipelining) Modulo Iteration Interval Table for Fixed-Point Dot Product (After Software Pipelining) Modulo Iteration Interval Table for Floating-Point Dot Product (After Software Pipelining) Software Pipeline Accumulation Staggered Results Due to Three-Cycle Delay Comparison of Fixed-Point Dot Product Code Examples Comparison of Floating-Point Dot Product Code Examples Modulo Iteration Interval for Weighted Vector Sum (2-Cycle Loop) Modulo Iteration Interval for Weighted Vector Sum With SHR Instructions Modulo Iteration Interval for Weighted Vector Sum (2-Cycle Loop) Modulo Iteration Interval for Weighted Vector Sum (2-Cycle Loop) Resource Table for IIR Filter xvi

17 Tables 5 17 Modulo Iteration Interval Table for IIR (4-Cycle Loop) Resource Table for If-Then-Else Code Comparison of If-Then-Else Code Examples Resource Table for Unrolled If-Then-Else Code Comparison of If-Then-Else Code Examples Resource Table for Live-Too-Long Code Resource Table for FIR Filter Code Resource Table for FIR Filter Code Comparison of FIR Filter Code Comparison of FIR Filter Code Resource Table for FIR Filter Code Comparison of FIR Filter Code Packed Data Types Supported Operations on Packed Data Types Instructions for Manipulating Packed Data Types Unpacking Packed 16-Bit Quantities to 32-Bit Values Intrinsics Which Combine Multiple Operations in One Instruction Comparison Between Aligned and Non-Aligned Memory Accesses Intrinsics With Increased Multiply Throughput Intrinsics Combining Addition and Subtraction Instructions on Common Inputs Intrinsics Combining Multiple Data Manipulation Instructions SPLOOPD ILC Values Execution Unit Use for SPLOOP of Autocorrelation Execution Unit Use for Outer Loop of SPLOOP of Autocorrelation Selected TMS320C6x Directives Functional Units and Operations Performed Definitions Command Line Options for Run-Time-Support Calls How _FAR_RTS is Defined in Linkage.h With mr Contents xvii

18 Examples Examples 1 1 Compiler and/or Assembly Optimizer Feedback Basic Vector Sum Use of the Restrict Type Qualifier With Pointers Use of the Restrict Type Qualifier With Arrays Incorrect Use of the restrict Keyword Including the clock( ) Function Saturated Add Without Intrinsics Saturated Add With Intrinsics Vector Sum With restrict Keywords,MUST_ITERATE, Word Reads Vector Sum with Type-Casting Casting Breaking Default Assumptions Additional Casting Breaking Default Assumptions Rewritten Using Memory Access Intrinsics Vector Sum With Non-Aligned Word Accesses to Memory Vector Sum With restrict Keyword, MUST_ITERATE Pragma, and Word Reads (Generic Version) Dot Product Using Intrinsics FIR Filter Original Form FIR Filter Optimized Form Basic Float Dot Product Float Dot Product Using Intrinsics Float Dot Product With Peak Performance Int Dot Product with Nonaligned Doubleword Reads Using the Compiler to Generate a Dot Product With Word Accesses Using the _nassert() Intrinsic to Generate Word Accesses for Vector Sum Using _nassert() Intrinsic to Generate Word Accesses for FIR Filter Compiler Output From Compiler Output From Compiler Output From Automatic Use of Word Accesses Without the _nassert Intrinsic Assembly File Resulting From Trip Counters Vector Sum With Three Memory Operations Word-Aligned Vector Sum Vector Sum Using const Keywords, MUST_ITERATE pragma, Word Reads, and Loop Unrolling FIR_Type2 Original Form xviii

19 Examples 2 35 FIR_Type2 Inner Loop Completely Unrolled Vector Sum Use of If Statements in Float Collision Detection (Original Code) Use of If Statements in Float Collision Detection (Modified Code) Vector Summation of Two Weighted Vectors lesson_c.c Feedback From lesson_c.asm lesson_c.asm lesson1_c.c lesson1_c.asm lesson1_c.asm lesson2_c.c lesson2_c.asm lesson2_c.asm lesson3_c.c lesson3_c.asm Profile Statistics Using the iircas4 Function in C Software Pipelining Feedback From the iircas4 C Code Rewriting the iircas4 ( ) Function in Linear Assembly Software Pipeline Feedback from Linear Assembly Stage 1 Feedback Stage Two Feedback Stage 3 Feedback Linear Assembly Block Copy Block Copy With.mdep Linear Assembly Dot Product Linear Assembly Dot Product With.mptr Fixed-Point Dot Product C Code Floating-Point Dot Product C Code List of Assembly Instructions for Fixed-Point Dot Product List of Assembly Instructions for Floating-Point Dot Product Nonparallel Assembly Code for Fixed-Point Dot Product Parallel Assembly Code for Fixed-Point Dot Product Nonparallel Assembly Code for Floating-Point Dot Product Parallel Assembly Code for Floating-Point Dot Product Fixed-Point Dot Product C Code (Unrolled) Floating-Point Dot Product C Code (Unrolled) Linear Assembly for Fixed-Point Dot Product Inner Loop With LDW Linear Assembly for Floating-Point Dot Product Inner Loop With LDDW Linear Assembly for Fixed-Point Dot Product Inner Loop With LDW (With Allocated Resources) Linear Assembly for Floating-Point Dot Product Inner Loop With LDDW (With Allocated Resources) Contents xix

20 Examples 5 19 Assembly Code for Fixed-Point Dot Product With LDW (Before Software Pipelining) Assembly Code for Floating-Point Dot Product With LDDW (Before Software Pipelining) Linear Assembly for Fixed-Point Dot Product Inner Loop (With Conditional SUB Instruction) Linear Assembly for Floating-Point Dot Product Inner Loop (With Conditional SUB Instruction) Pseudo-Code for Single-Cycle Accumulator With ADDSP Linear Assembly for Full Fixed-Point Dot Product Linear Assembly for Full Floating-Point Dot Product Assembly Code for Fixed-Point Dot Product (Software Pipelined) Assembly Code for Floating-Point Dot Product (Software Pipelined) Assembly Code for Fixed-Point Dot Product (Software Pipelined With No Extraneous Loads) Assembly Code for Floating-Point Dot Product (Software Pipelined With No Extraneous Loads) Assembly Code for Fixed-Point Dot Product (Software Pipelined With Removal of Prolog and Epilog) Assembly Code for Floating-Point Dot Product (Software Pipelined With Removal of Prolog and Epilog) Assembly Code for Fixed-Point Dot Product (Software Pipelined With Smallest Code Size) Assembly Code for Floating-Point Dot Product (Software Pipelined With Smallest Code Size) Weighted Vector Sum C Code Linear Assembly for Weighted Vector Sum Inner Loop Weighted Vector Sum C Code (Unrolled) Linear Assembly for Weighted Vector Sum Using LDW Linear Assembly for Weighted Vector Sum With Resources Allocated Linear Assembly for Weighted Vector Sum Assembly Code for Weighted Vector Sum IIR Filter C Code Linear Assembly for IIR Inner Loop Linear Assembly for IIR Inner Loop With Reduced Loop Carry Path Linear Assembly for IIR Inner Loop (With Allocated Resources) Linear Assembly for IIR Filter Assembly Code for IIR Filter If-Then-Else C Code Linear Assembly for If-Then-Else Inner Loop Linear Assembly for Full If-Then-Else Code Assembly Code for If-Then-Else Assembly Code for If-Then-Else With Loop Count Greater Than If-Then-Else C Code (Unrolled) Linear Assembly for Unrolled If-Then-Else Inner Loop Linear Assembly for Full Unrolled If-Then-Else Code xx

21 Examples 5 55 Assembly Code for Unrolled If-Then-Else Live-Too-Long C Code Linear Assembly for Live-Too-Long Inner Loop Linear Assembly for Full Live-Too-Long Code Assembly Code for Live-Too-Long With Move Instructions FIR Filter C Code FIR Filter C Code With Redundant Load Elimination Linear Assembly for FIR Inner Loop Linear Assembly for Full FIR Code Final Assembly Code for FIR Filter With Redundant Load Elimination Final Assembly Code for Inner Loop of FIR Filter FIR Filter C Code (Unrolled) Linear Assembly for Unrolled FIR Inner Loop Linear Assembly for Full Unrolled FIR Filter Final Assembly Code for FIR Filter With Redundant Load Elimination and No Memory Hits Unrolled FIR Filter C Code Final Assembly Code for FIR Filter With Redundant Load Elimination and No Memory Hits With Outer Loop Software-Pipelined Unrolled FIR Filter C Code Linear Assembly for Unrolled FIR Inner Loop Linear Assembly for FIR Outer Loop Unrolled FIR Filter C Code Linear Assembly for FIR With Outer Loop Conditionally Executed With Inner Loop Linear Assembly for FIR With Outer Loop Conditionally Executed With Inner Loop (With Functional Units) Final Assembly Code for FIR Filter Vector Sum Vector Multiply Dot Product Vector Complex Multiply Vectorization: Using LDDW and STDW in Vector Sum Vector Addition (Complete) Using LDDW and STDW in Vector Multiply Using _mpy2() and _pack2() to Perform the Vector Multiply Vectorized Form of the Dot Product Kernel Vectorized Form of the Dot Product Kernel Final Assembly Code for Dot-Product Kernel s Inner Loop Vectorized Form of the Vector Complex Multiply Kernel Vectorized Form of the Vector Complex Multiply Non-Aligned Memory Access With _mem4 and _memd Vector Sum Modified to Use Non-Aligned Memory Accesses Clear Below Threshold Kernel Clear Below Threshold Kernel, Using _cmpgtu4 and _xpnd4 Intrinsics Contents xxi