The continuum of data management techniques for explicitly managed systems

Transcription

1 The continuum of data management techniques for explicitly managed systems Svetozar Miucin, Craig Mustard Simon Fraser University MCES Montreal

2 Introduction Explicitly Managed Memory systems lack conventional hardware cache mechanisms. Instead, they have small but fast scratchpad memories close to the cores, and slower global memories far away across an interconnect. And a DMA controller between them! Difference in access times between local/global is a couple of orders of magnitude.

3 Introduction EMM systems are important: Lower power consumption Higher computational power if used correctly But, they re hard to use correctly Lots of DMA transfers = lots of data management decisions.

4 Our setup STMicroelecronics STHORM Cluster = 16 PE cores + 1 Cluster controller core 256kB scratchpad per cluster

5 The discrete view Convolution kernel 56x faster!

6 What is the difference? We can run most code with global accesses without much modification. To get the manual version speedups, we need to try hard. Is there a compromise? Yes, there is.

7 Common bits, abstractions Many kernels have similar data access patterns. This can be exploited to abstract a lot of stuff away. Our example: NxM Convolution A textbook case for image processing.

8 Convolution accesses A single iteration of 3x3 convolution Read pattern Write pattern input: Access all data inside a 2D tile output: Access the central pixel relative to the input tile

9 Iterator interface Convolution access abstraction case is simple: init_area(...) Initializes the area that a core (or a cluster) will process get_next_tile(...) Returns a structure containing a batch of pixels to process and metadata

10 Double buffering Hiding the transfers behind processing. DMA transfer #1 Processing #1 DMA transfer #2 Example: DMA transfer #1 DMA transfer #2 Processing #1 initiate transfer of data chunk #2 process data chunk #1 wait for the transfer of chunk #2 to complete initiate transfer of data chunk #3 process data chunk #2 A tedious task, but it can be automated when accesses are predictable.

11 Border processing and Out-of-image pixels To process a pixel, we need to fetch its neighbours. The iterator interface returns metadata about which area of the fetched tile contains this extra data. We can also generate border pixels that fall outside of the image by providing custom generator functions.

12 The real benefit All of this can be done manually. Abstracting it through a library gives clean, small kernels that are easy to reason about: init_area(...); while (get_next_tile(&tile)!= NULL) { process(tile); }

13 Where does this place us?

14 Software cache Hide all data management from the programmer. data = softcache_fetch(address); 1. How well can software cache do versus manual management? Why? 2. How to achieve this performance. High-level.

15 The driver - Hough transform Algorithm for geometrical shape detection in images. Pseudo-random access patterns. Takes time to carefully organize memory transfers.

16 Software cache on Hough First try (Pseudo random access to data) 80x faster than naive Second try Software cache 2x as fast as the first manual 1.5x slower than the best manual version Manual data management code can be SLOWER than using software cache!

17 Software cache on Hough 2x as fast as one manual version. Why? 1. Software cache has less overhead than the manual management on the access pattern of Hough. 2. Software cache took better advantage of available space than the first version. Bonus #1: Software cache inspired the best performing manual version Bonus #2: Software cache was much easier to implement than either manual version (15 minutes: naive->softcache)

18 Software cache on Convolution (Very regular access to data) Classic interface New interface Classic interface: 10x faster than naive 2.5x slower than manual versions New interface: 1.2x slower than best manual version Manual management performs very well for regular data access patterns

19 Software cache on Convolution 2.5x slower than manual versions. Why? Overhead of function calls in software cache New interface reduces # of function calls by returning more data with a single function call. softcache_fetch(addr) - one item softcache_fetch_more(addr, &num) - N items

20 Software cache: the gory details We tested all features of a classic cache. Private/shared, line-size vs # of lines, prefetching, write policies To get the performance we showed: Per core (private) caches of 8K Read-only Tuned line-size vs number of lines (12 tests) Prefetching disabled (rarely ever good)

21 The continuous view Both the library and the software cache approach are just tools. They can be used together in any combination, to hit the desired ease-ofprogramming/performance trade-off.

22 The end