Hardware Acceleration for Just-In-Time Compilation on Heterogeneous Embedded Systems

Transcription

1 Hardware Acceleration for Just-In-Time Compilation on Heterogeneous Embedded Systems A. Carbon, Y. Lhuillier, H.-P. Charles CEA LIST DACLE division Embedded Computing Embedded Software Laboratories France Contact: 24th IEEE International Conference on Applicationspecific Systems, Architectures and Processors June , Washington D.C., USA.

2 Cliquez pour modifier le style du Outline titre Context : Virtualization JIT emergence JIT optimization opportunities (based on the LLVM framework) Hardware accelerator proposal Experiments results Conclusion 2

3 Cliquez pour modifier le style Context du titre Parallelism emergence CPU CPU CPU CPU CPU 3

4 Cliquez pour modifier le style Context du titre Parallelism emergence Heterogeneity development in computing systems ILT/TLP-based multi-cores and wide SIMD GPUs CPU CPU CPU CPU GPU 4

5 Cliquez pour modifier le style Context du titre Parallelism emergence Heterogeneity development in embedded systems Heterogeneous asymmetric many-core processors (AMP) emergence CPU DSP CPU HW acc. GPU PPE: a major concern of embedded systems designers 5

6 Cliquez Context: pour Virtualization modifier le style emergence du titre Code deployment on such architectures High development cost to efficiently target one AMP Code portability has become a major issue Software? Hardware 6

7 Cliquez Context: pour Virtualization modifier le style emergence du titre Emergence of virtualization abstraction layers First mention in 60 s (IBM 360/67) Java Virtual Machines, CLI, LLVM => Virtual Machines (VMs) development Software Virtualization layer Hardware 7

8 Cliquez Context: pour Virtualization modifier le style emergence du titre Emergence of virtualization abstraction layers Based initially on interpretation Suffer from considerable performance overheads Software Virtualization Interpretation layer Hardware 8

9 Cliquez Context: pour modifier Just-In-Time le style emergence du titre Emergence of virtualization abstraction layers Coupling today interpretation and Just-In-Time compilation Software Interpretation Virtualization layer Just-In-Time compilation Hardware 9

10 Cliquez Context: pour modifier Just-In-Time le style emergence du titre JIT compilation: widely used in GPP Performance consumption overheads in embedded 2 kinds of existing optimizations to reduce JIT impact Software optimizations System design specialization Specialized dedicated resources Additional standard dedicated resources 1 JIT compilation complexity limits performance gains 1 Pointer-based algorithms Proposing tuned hardware associated to these dedicated resources to manage JIT compilation algorithms 1 Ting Cao et al. «The yin and yang of power and performance for asymmetric hardware and managed software», ISCA

11 Cliquez pour modifier le style du Outline titre Context : Virtualization JIT emergence JIT optimization opportunities (based on the LLVM framework) Hardware accelerator proposal Experiments results Conclusion 11

12 Cliquez pour modifier LLVM le style framework du titre LLVM bytecode compiler (LLC): used in many projects Profiling: identifying LLC s most critical parts Experiments on a ARM Cortex-A5 model Associative array management dynamic memory allocation: on average 24% of LLC execution time 12

13 LLVM: Cliquez existing pour modifier software le optimizations style du titre New abstract-data-types (optimized versions of STL C++ ADT) Provide [multi]map multi[set] abstract data types Hash table implementations rather than sorted-tree STL C++: still used when performance is not a key issue Specialized allocators (eg. RecycleAllocator) Keeps track of recently dealloccated objects to reuse them Avoiding frequent allocations and deallocations Despite these software optimizations, associative arrays memory allocation still prevail 13

14 Cliquez pour modifier LLVM le normalization style du titre Our goal: proposing an alternative acceleration of associative arrays dynamic memory allocation Mean: standardization of LLC Proposing a solution based on standard libraries for reuse Using only STL C++ library for [multi]map multi[set] ADT C s memory allocation standard library (dlmalloc-based) Transferring optimizations to a hardware accelerator Solution portability: accelerator reuse Benefit to all pointer-based algorithms using massively associative arrays and dynamic memory allocation 14

15 Cliquez pour modifier le style du Outline titre Context : Virtualization JIT emergence JIT optimization opportunities (based on the LLVM framework) Hardware accelerator proposal Experiments results Conclusion 15

16 Cliquez RB-Tree pour modifier Hardware le style accelerator du titre Current implementation of associative arrays memory allocation Standard C++ Map Set libraries Using Red-Black Tree representation Binary tree with coloring property C s memory allocator: dlmalloc Using associative arrays to associate data sizes with free memory chunks Using hash table double linked-lists Systematic usage of RB-Trees Proposing an implementation of dlmalloc using RB-Trees Modifying the allocator without modifying user interface 16

17 Cliquez Hardware pour modifier acceleration style description du titre New RB-Tree node structure: held in 128-bits Digest key in 31-bits, color in the last bit, preserving the sorting order rb_tree_node_t* X rb_tree_node_t* X 31 0 COLOR PARENT LEFT RIGHT KEY D_KEY C 128 bits PARENT LEFT 128 bits RIGHT Key_size (a) initial structure (b) proposed structure Proposing hardware accelerated instructions Specialized instructions for RB-Tree management functions Accelerating traversals, key look-ups, balanced insertion and removal 17

18 Cliquez pour modifier Proposed le ISA style extension du titre 15 new instructions over 400 in ARM ISA Instruction Function Used by RBTINC Rd, Rm increment map::iterator, set::iterator RBTDEC Rd, Rm decrement map::iterator, set::iterator RBTLOW Rd, Rn, Rm lower bound map::lower_bound, set::lower_bound, malloc, realloc RBTUP Rd, Rn, Rm upper_bound map::upper_bound, set::upper_bound RBTDEL Rn, Rm rebalance for erase map::erase, set::erase, malloc, realloc, free RBTINS <L R> Rn, Rm, Rs insert rebalance map::insert, set::insert, malloc, realloc, free Read at most 3 registers, write at most 1 register Multi-cycles instructions: hiding iterative computations 18

19 Cliquez HW pour accelerator modifier implementation style du titre New Cortex-A5 pipeline HW accelerator proposal Full FSM size estimation: 161 states, 223 transitions 19

20 Cliquez pour modifier le style du Outline titre Context : Virtualization JIT emergence JIT optimization opportunities (based on the LLVM framework) Hardware accelerator proposal Experiments results Conclusion 20

21 Cliquez pour modifier Experiments le style du Results titre Instrumented ARM Cortex-A5 ISS with a cache simulator Speedups obtained for SW and HW accelerated versions of LLC SW: 29% of gain, HW: 50% / LLC standardized version 15% of gain for HW acc / SW acc 21

22 Cliquez pour modifier Experiments le style du Results titre Evolution of time spent in memory allocation associative array management (relative to total execution time) From 41% to 24% (SW) 12% (HW) Raw speedup on memory allocation associative arrays: 5x 22

23 Cliquez pour modifier le style du Outline titre Context : Virtualization JIT emergence JIT optimization opportunities (based on the LLVM framework) Hardware accelerator proposal Experiments results Conclusion 23

24 Cliquez pour modifier le style Conclusion du titre Interest of dedicated resources for virtualization services Limited gains for JIT compilation with SW optimizations Impact of dynamic memory allocation associative array management on execution time Proposing tuned HW for JIT compilation, coupled to dedicated resources HW accelerator hidden behind standard libraries Valuable for all pointer-based algorithms ISA extension for RB-Tree management functions 15% of gain comparing to SW opt in LLVM code gen 5x raw speedup for memory allocation associative array management Next acceleration opportunities: instruction graph handling 24

25 Thank you Questions? Centre de Grenoble 17 rue des Martyrs Grenoble Cedex Centre de Saclay Nano-Innov PC Gif sur Yvette Cedex