Aspects Consommation & Thermique dans les Circuits 3D

Aspects Consommation Thermique dans les Circuits 3D ECOFAC 2014, 19-23 Mai 2014, Lorient, France Pascal Vivet Minatec Campus, CEA-LETI, Grenoble, Fr. pascal.vivet@cea.fr www.cea.fr

3D Technology Cliquez pour modifier le style Outline du titre Introduction Technology Challenges 3D Design CAD Challenges 3D : an exemple for Power optimisation WIDEIO 3D memory, an exemple for addressing memory and power wall Thermal Analysis in 3D systems WIOMING Circuit Comparison of 3D versus 2D Exploration of Packaging materials Thermal impact of TSV Impact of Thermal effects on DRAM performances Power and Thermal Optimization in 3D systems Massive multiprocessing, Active interposer and power managment Thermal Monitoring Thermal Mitigation exemple within a multi-core Conclusions Perspectives CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 2

3D : some definitions 3-D chip stacking Cliquez pour modifier le style du titre With the possibility of backside routing Passivation BEOL FEOL Through Si via (TSV) Si Redistribution layer (RDL) Chip 2 Chip-to-chip interconnection (with underfill) Chip-to-package interconnection Si Chip 1 pitch Source : wikipedia CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 4

Via first, via middle and via last TSV Cliquez pour modifier le style du titre FEOL + Middle End CMOS process BEOL Backside process Dicing Stacking Via First TSV process Transistors + W contacts Cu interconnects + passivation Temp carrier (W, Poly Si) Via Middle (Cu) TSV process Temp carrier TSV process Via Last (Cu) Temp carrier CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 5

Stacking strategy Cliquez pour modifier le style du titre Wafer-to-wafer (WtW) Easier to process but require the same die size with very good yields Die-to-Wafer (DtW) Possibility to select the known good dice Die-to-Die (DtD) More flexible CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 6

3D Technology in LETI : Open 3D processes Cliquez pour modifier le style du titre Pitch 50µm Thickness 120µm Pitch 120µm CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 7

3D Technology : Fine pitch connections Cliquez pour modifier le style du titre 3D connections Top chip Bottom chip Pitch CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 8

3D Technology : Fine pitch TSV Cliquez pour modifier le style du titre 1µm diameter TSV, 10µm thin Si 400 C TSV process flow 3µm Feasibility demonstrated on daisy chains, In advanced RD phase 15µm Si ~300nm TSV open (a) (b) (c) Cu pad TiN Cu (d) (e) (f) P. Leduc et al, 2009 3D IC CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 9

Design perspectives Cliquez pour modifier le style du titre What can we expect today from the technology? Less than 10 µm diameter TSV is challenging Must add guard interval thus reduce effective interconnect pitch ~500 interconnects/mm² Coarse to medium grain partitioning for 3D SoC and Silicon board 3D SoC 2.5D or Interposers 3D-stacked dies Memory-on-processor: 3D memory hierarchy Processor-on-processor: Many-core cluster High bandwidth Fine grain architecture partitioning High density for vertical interconnects Face-to-back Dies stacked on a silicon interposer Heterogeneous integration: Digital, analog, memory, input/output, power management Medium bandwidth System partitioning High density for horizontal interconnects Face-to-face Large size silicon interposer 10 CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 10

3D Design Cliquez pour will modifier dream become le style reality du titre? Stackable Processor DRAM Standard DRAM Parallel low speed IF Mem Standard Non Volatile Memory Proc Heterogenous with RF Proc RF/Analog NVM Active interposer (mature technology): Services communication infrastructure ; IO, IF and ESD ; Test ; Power management CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 11

Cliquez 3D pour Design modifier Flow le : Challenges style du titre? 3D Design? Mostlyreuseexisting«2D» design tools but due to 3D, many other design challenges!!! 3D Architecture Physical Implementation Architecture partitionning TSV, Bumps, Power Grids, Additionnal Effects Analysis Thermal Power effects Test DFT Final Verification 3D faults, 3D DFT pattern reuse Signal Integrity, Power Integrity CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 12 12

3D Design Challenges Cliquez pour modifier le style du titre 3D Design Complexity Maturity 13 Design IPs Currently Available Short Term Long Term - LowSpeed interfaces for digital memory - High Speed interfaces for digital memory - Photonic interfaces Design Kit Models - Individual 2D Design Kits with 3D objects - PreliminaryTSV models(rc, noise, keepout zone, ) - Multi-technology 3D Design Kits - AccurateTSV 3Dobjectmodels - 3D AMS design environement - Full modelingof 3D physical effects(thin wafer, Xtalk effect betweendies, ) CAD Tools - 2Dindependantdesigns with3d constraints(placeroute, DRC/LVS/Extract) - 3D prototyping co-design, early partionning tools, including substrate/package - Full 3D DRC LVS - 3D IP reusemethology - Full 3D physicaldesign (Place, Route, Clocking, Extraction, etc.) Power Thermal - EarlyPower Thermal model simulation tools(2d with some 3D constraints) - Full 3D Power Thermal model simulation (early protyping signoff) - Optimized 3D Power Delivery networks (IRdrop, noise, etc.) - 3D Power Thermaleffect optimizationtechniques (Hardware and/or Software) - Thermal cooling solutions Design for Test Test - 2D compatible DFT - Early3D test flow - Full 3D DFT, includingspecifictest methods(delay, leakage,...) - 3D modulartest flow - Power Thermal Test time tradeoffs(dft and ATPG levels) CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 13

3D Design Flow : Collaboration with EDA partners Cliquez pour modifier le style du titre 3D Architecture - Die partitioning - TSV location exploration - 3D Thermal Power Profile 3D Design : CAD tools challenges - Handle complexity - From System level down to implementation - In strong coordination with existing tools and standards 3D Implementation - 3D Floorplan - 3D Power planning - 2D Place CTS Route - 3D DFT ATPG Multiple partnership required Collaboration through common labs and collaborative programs 3D Sign-Off - 3D thermal validation - 3D IR-Drop analysis - 3D parasitic extraction -3D DRC LVS verif Standardization Efforts CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 14

3D Technology Cliquez pour modifier le style Outline du titre Introduction Technology Challenges 3D Design CAD Challenges 3D : an exemple for Power optimisation WIDEIO 3D memory, an exemple for addressing memory and power wall Thermal Analysis in 3D systems WIOMING Circuit Comparison of 3D versus 2D Exploration of Packaging materials Thermal impact of TSV Impact of Thermal effects on DRAM performances Power and Thermal Optimization in 3D systems Massive multiprocessing, Active interposer and power managment Thermal Monitoring Thermal Mitigation exemple within a multi-core Conclusions Perspectives CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 15

Motivation Objectives Cliquez pour modifier le style du titre In 2010 3D-IC PoP Competition was between WideIO and quad-channel LPDDR2 Source: JEDEC Mobile Memory Forum, 2011 ST-Ericsson CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 17

WideIO Memory Interface Cliquez pour modifier le style du titre Uses chip-level three dimensional (3D) stacking for high-bandwidth capability (>12.8 GB/s) Quad-channel interface: 128 bit data bus per channel SDRAM protocol Single Data Rate sampling mode 3D Interface: More than 1k I/Os I/O Pitch: 50µm x 40µm Placed in the center area of the memory die Boundary scan test mode Package Substrate Package Balls SoC: face down Metal stack Channel 0 Channel 1 Bank 0 Bank 2 Bank 0 Bank 2 Bank 1 Bank n Bank 1 Bank n Bank 0 Bank 2 Bank 0 Bank 2 Bank 1 Bank n Bank 1 Bank n Channel 2 Channel 3 WideIO DRAM floorplan Wide IO DRAM: face down Metal stack 3D Interface Package molding 3D-IC with WideIO DRAM CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 18

Asynchronous NoC based MP-SoC Cliquez pour modifier le style du titre Network-on-Chip: Asynchronous, 16-routers, 550 Mflit/s, GALS interfaces, Local Clock. 24 programmable units: 4x WideIO controllers, 5x memory units, 14x processing units, 1x ARM core. DFT logic for WideIO. Embedded Heaters and Thermal Sensors. CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 19

WideIO Traffic Controller Cliquez pour modifier le style du titre 8 GB/s overall data BW between NoC, SRAM and WideIO One traffic controller per WideIOmemory channel. Handles multiple data paths. Interfaces with: ANoC router, WideIO. Dedicated WideIO memory controller. 128KB SRAM. Dedicated DFT. Asynchronous NoC: 32-bit data WideIO Channel: 128-bit, 200 MHz, SDR CK Config. NoC Port TSV DCM-WIDEIO JTAG Clock CTRL TSV A[11:0] TSV Test Mux. 128-bit WideIO Physical Layer Interface (PHY) 128-bit DMQS[15:0] TSV 200 MHz WideIO Memory Controller 128-bit 400MHz AXI3 Streaming Communication Controller Mover DQ[127:0] TSV 200 MHz DFI2.1 MBIST SRAM 4x32Kbyte Slide 20 CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 20

WIOMING Circuit CPU 4 * Heaters Cliquez pour modifier le style du titre Circuit Technology 3D NOC TSVs 3D NOC TSVs WideIO TSVs + Controllers 4 memory channels 3D NOC TSVs 3D NOC TSVs -High speed CMOS TSV middle process -Face2Back, Die2Die, Flip-Chip 3Dassembly Main features - WideIO memory controllers -3D ANOC -3GPP LTE multi core CPU backbone -Host CPU Circuit numbers -125 Million Transistors -400 Macros -270 pads -1980 TSV for 3D NoC -1016 TSV for WideIOmemory -933 Bumpsfor flip chip Circuit performances -WideIO200MHz / 512 bits -Unitsin the [350-400] MHz range -Asynchronous NoC~ 550 MHz CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 21 21

Cliquez pour modifier Power le Consumption style du titre Less than 1 pj/bit for full 3D-I/O link: micro-buffers and die-to-die interconnect with TSV Power during at speed (200 MHz) 13N MBIST (12.8 GB/s) 0.8 pj/bit VDD- Mem 0.9 pj/bit VDD-I/O 1.2 pj/bit VDD-MPSoC Temperature CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 23

WIOMING Cliquez : LPDDR3 pour modifier / WideIO le style comparison du titre 4x gain in power efficiency with 3D-TSV interconnect Memory Type LPDDR3 - [1] WideIO - WIOMING - [2] Package PoP / Discrete 3D-IC BW (Gbyte/s) 6.4 GB/s 12.8 GB/s Total power 293 mw * VDD-MPSoC VDD-Mem VDD-I/O MPSoC power 121 mw * Memory Power 4x gain 81 mw* I/O power 91 mw * I/O power efficiency 3.7 pj/bit** 0.9 pj/bit * [1] Yong-Cheol Bae, et al. "A 1.2V 30nm 1.6Gb/s/pin 4Gb LPDDR3 SDRAM with input skew calibration and enhanced control scheme," ISSCC-2012. [2] D. Dutoit, et al. "A 0.9 pj/bit, 12.8 GByte/s WideIO Memory Interface in a 3D-IC NoC-based MPSoC, VLSI-Symposium, 2013. CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 24

Cliquez pour modifier WideIO le 1 style : conclusion du titre WideIO provided very good promises : WideIO1 performances 512 bits data path, more than1000 3D connections 200 MHz Simple Data Rate protocol, Memory bandwidth performances : 12.8 Gbit/s. Low Power consumption Due to short 3D connections (smaller cap) compared to flip chip and POP connections 4x better power consumption of memory interface, for energy per bit, compareed to LPDDR2 Design implementation verification Full Std Cell approach, compared to complex analog PHY from LPDDR2 Easy setup for test and qualification, no complex PHY But Compareda few weeksof test comparedto monthsof qualiffor LPDDR setup Supplychainwasnot readyyetfor volume production LPDDR3 new standard interface was raising interests, wtih main stream technology Perspectives WIDEIO2 : 400 MHz, Double data rate, bandwdith48 Gbit/s Some new memory interfaces and consortium : HMC, etc. CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 25

Cliquez Thermal pour modifier issues in le modern style du SoCs titre Increasing thermal issues Technology scaling = higher power density 3D stacking with TSVs = even higher power density and reduced heat dissipation properties Higher lateral thermal resistance due to thinned wafers Poor conductive materials used to bond stacked dies Temperature impacts Power consumption Peak performance Ageing Package costs How to perform thermal modeling to enable early exploration of thermal issues including 3D? CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 27

Cliquez pour modifier Thermal le style design du flow titre 12-24 months 2-3 months 6 months New architecture concept High level model (e.g. systemc) RTL model netlist Area power consumption Fixed floorplan constraints Tape out chip sales Architecture exploration VHDL design Synthesis Refined Power Studies Floorplan exploration Place Route w. Thermal sign off Fab Circuit test Development Validation Online thermal control Early power evaluation Need for thermal evaluation tools that take into account a complete SoC environment and its dynamic behaviour Need early HW/SW thermal effects evaluation => ESL Linking thermal/power/functional tools in the same design flow is mandatory CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 28

Low-level tools: Cliquez Thermal pour Analysis: modifier le state style of du the titre art Multiphysics simulation : FloTherm (Mentor Graphics), Icepack (ANSYS), Marc (MSC), COMSOL General-purpose FEM solutions with no specific support for IC design flows Post-layout level : HeatWave (Gradient DA), Apache (ANSYS) Adapted for short term analysis at die level but long simulation times for multiple power scenarios Those solutions are time consuming methods thus not suitable for system-level simulations High-level tools: Architectural and system-level thermal simulators must support: Fast transient thermal analysis to be used in power-thermal coupled simulations Different granularity scales: from large structures (package, interposer and board) to very fine-grain elements (TSVs, C4-bumps and copper pillars) with both high impact on model accuracy Existing solutions: Mostly academic tools like HotSpot(Virginia) and 3D-ICE (EPFL) No support for fine-grain structures with heterogeneous distribution No link with virtual platforms No solution for fast transient thermal simulation of complex systems with unrestricted support for fine grain structures required for 3D integration, flip-chip designs or BGA-like packaged circuits CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 29

Thermal Exploration at High Level Cliquez pour modifier le style du titre DOCEA Tools offer a unique CAD solution to simulate jointly in close loop : - Thermal Effect (ATM tool) - Power Effect (Aceplorer tool) - co-simulated with a HW/SW platform using SystemC/TLM A single plaftorm to develop early in phase Thermal Mitigation Algorithms Can also be used for 3D Architectures T. Sassolas All, «Early Validation of MPSoCs Thermal Mitigation through Integration of Thermal Simulation in SystemC Virtual Prototyping», DATE 14 CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 30

Cliquez Thermal pour modifier Modeling le Methodology style du titre Thermal modeling based on DOCEA ATM tool: Based on a numerical finite difference method Heat transfer modeled via full 3D heat diffusion in solid materials with no restriction to heat flow paths Highly compacted thermal model (CTM) with good tradeoff between accuracy and efficiency API + GUI for easy model creation/updates CTM generation fully automated by Python scripts CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 31

Cliquez pour Thermal modifier Model le style : input du data titre LEF/DEF parser implemented within ATM Die size and placement of power sources, TSVs, u-bumps and C4-bumps can be read into ATM from initial explorative floorplans or final layout databases Input data organized as follows: Material thermal property library: CSV file/python Circuit description: parsed from floorplan(lef/def format) and saved as CSV files Technology settings: thickness, diameter and material used for every structure of a specific technology CSV format is widely used for system-level exploration Allow fast iteration and manual modifications CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 32

Thermal Cliquez Model: pour Material modifier Homogenization le style du titre Material Homogenization: This technique consists in calculating the equivalent anisotropic material propertiesfor a heterogeneous layer containing multiple structures A set of geometries are identified to go through material homogenization Once the equivalent material is calculated, those original geometries are then replaced by simplified structures Package solder balls Array of TSVs and u-bumps Structures usually considered for material homogenization Objective is to reduce the number of geometries going for extraction while keeping the spatial accuracy CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 33

WIOMING, a Memory-on-Logic 3D Circuit Cliquez pour modifier le style du titre D. Dutoit, et al. "A 0.9 pj/bit, 12.8 GByte/s WideIO Memory Interface in a 3D-IC NoC-based MPSoC, VLSI-Symposium, 2013. WIOMING circuit floorplan WIOMING circuit Many-core architecture, STMicroelectronics 65nm TSV middle (diam 10µm), u-bumps (diam 20um) 3D Assembly : Die2Wafer, Face2Back, FlipChip Stacking a WideIO compatible DRAM WIOMING circuit instrumented with 8 Heaters [Can generate each 1Watt] 7 Thermal Sensors [Accuracy ~1 C after calibration] Full Thermal Software Control on chip board to perform accurate 3D Thermal Characterization CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 35

WIOMING ATM model: from board to chip level Cliquez pour modifier le style du titre +6K circuit structures Board-level including PCB, socket and package Chip-level with stacked dies, TSVs and u-bumps 27 power sources 26 areas for heat exchange Multi-corner CTM generation +2k lines of Python script CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 36

Model Compaction Results Tool Performance Results Cliquez pour modifier le style du titre Material Homogenization Model Reduction Results 130x less geometries after material homogenization 60x less nodes after material homogenization 570x less nodes after model reduction System physical representation Before homogenization After homogenization After reduction # geometries +6k 71 -- # defined materials 23 44 -- # extracted nodes 18 million 231k 405 ATM Tool Performances a Per simulation time step Highly compacted thermal model for static and dynamic thermal analysis Very fast transient simulations compatible with system-level exploration CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 37

WIOMING: Cliquez Steady-state pour modifier Analysis le style : du results titre Model is able to capture the hotspot effect and spatial temperature distribution Simulation has a linear relation between power dissipation and temperature increase Very good accuracy of hot spot evaluation : avg=3.96% and worst=13.41% ON CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 38

WIOMING: Cliquez Transient pour modifier analysis le step style response du titre Good fitting between simulation and measurement data Thermaltimeconstanterrorinthe5% 40%range Same static offset can be noticed ON CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 39

Cliquez pour WIOMING modifier : PWM le style Response du titre Transient response(pwm shape) PWM response for various PWM periods and duty cycles Response Amplitude decrease with PWM increase frequency, achieving same steady state average tempearture(~30 increase) Good fitting between simulation and measurement data CTM able to model the multiple thermal time constants of the system PWM response for various PWM periods (same duty cycle 50% max power) [C. Santos, 3DIC 2013] CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 40

Accuracy Impact of the Material Homogenization Cliquez pour modifier le style du titre Steady-State analysis experiments with: 1) No homogenization procedures(all fine-grain structures are ignored) 2) Homogenization without selecting localized areas (flat) Full homogenization approach(reference) Case 1 ->averageerror=51% worstcaseerror=81% Case 2 ->averageerror= 7% worstcaseerror = 14% Properly selecting regions to apply material homogenization brings considerable accuracy to the spatial temperature distribution No Homogenization Flat Homogenization CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 41

Impact of board components on silicon Cliquez pour modifier le style du titre Not accounting for the components mounted on the board introduces an additional average error of 24% in the steady-state temperature. Worst case error is 43.66%. Those results show that: Poorpowermodelingoftheboardelementswillhavestrongaccuracyimpact! Thermal modeling approach being focused either only on board-package or silicon level is not enough for accurate thermal analysis. Error when removing PCB components CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 42

Cliquez pour Comparison modifier of le 3D style versus du titre 2D 2 versions of the same circuit WIOMING (3D with WideIO memory) 80um bottom die + 260um top die StandAlone(w/o WideIO memory) die =200um/300um * (* still unknown) ATM model fully implemented With 2D 3D options Objective: Study impact of die thickness CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 44

Cliquez pour modifier le style du titre Scenario : Hot Spot=4 W on BottomLeftHeaters Xaxes= distance to hot spot Yaxes= 3D versus 2D temperatureincreasein % 3D vs 2D : Simu versus Measurements Simulation (300um) and measurements with same non-monotonic behavior : -For points close to the hotspotsstand alone version benefits of the thicker bottom die -For points far from the hotspot Heat spreads better through the top die in 3D version Measurements : with relative values, same behavior as simulations CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 45

Impact of Die thickness Cliquez Impact pour of modifier die thickness le style of du 2D titre die Change in the DeltaT(3D vs2d) when playing with multiple thickness of the 2D die (200um 300um 500um) Thin 2D die has worse behavior than 3D die Thick 2D die provides better dissipation Spreading through the top die can be seen in a ideal case where PGS is considered instead of silicon CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 46

PGS Graphite ordinaire Cliquez pour Heatspreader modifier le style Materials du titre 20µm Anisotropes K(z) 15 W/mK K(xy) > 600 W/mK Fabricant Panasonic Panasonic Pyrolytic Graphite Sheet(PGS) Cuivre K(isotrope) = 390 W/mK Deux épaisseurs testées: 17 µm 105 µm CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 48 48

Cliquez pour Heat modifier Spreader le style evaluations du titre 3 versions of the same circuit WIOMING (3D with WideIO memory) 80um bottom + 260um top die standalone(w/o WideIO memory) die =200um/300um standalone + heat spreader Die 180um * (measured) PGS 70um Glue 30um ATM model fully implemented PGS 17/25/70/100 Silicone and acrylic glue options CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 49

Cliquez Heatspreader pour modifier : first le style evaluations du titre Scenario : Hot Spot=4 W on BottomLeftHeaters Xaxes= distance to hot spot Yaxes = Relative temperature Heat spreader with smaller deltatand smaller maxtthan standaloneversion (200um) and standalone version (300um) HeatSpreader can reduce hot spot, and spread heat through the overall die area CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 50

3D vs 2D Cliquez vs packaging pour modifier materials le : style conclusions du titre 3D vs 2D Fab Packaging requires very thin dies, for product reduced form factors VerystrongImpact of die thicknessof 2D die!doesnot help thermal dissipation! In case of WIOMING, 3D DRAM - with reduced power -helpsto dissipateheatof bottomdie For full 3D system, withmore power on all layers, itisexpectedto beworse. Heat Spreader PGS = Strong anisotropic materials Withverythinmaterial, helpsto dissipateheat, as a lessthinneddie Better thermal response, in a smaller form factor Next Steps Compare simulation with measurements for PGS die version Continue exploration by simulation of various parameters HS thickness, HS size, dynamicresponse, HS withcu versus PGS CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 51

Cliquez pour modifier le Thermal style du TSVs titre Use of TSVs to mitigate the thermal issue in 3D ICs Well known/accepted method for temperature reduction in 3D ICs Several techniques for smart/optimized TSV insertion can be found in the literature However... Are TSVs really efficient to mitigate thermal issues? Proposed methods do not properly consider/expose the poor thermal conductive SiO2 oxide when modeling the thermal resistance of TSVs SiO2 is either ignored or not well modeled since only one vertical resistance is used for each TSV Even when SiO2 is considered, used power scenarios do not reveal the reduced lateral thermal resistance caused by TSVs CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 53

Cliquez pour modifier le Thermal style du TSVs titre Homogenization procedures using ATM: SiO2 = 0.25um / TSM diam = 10um / TSV pitch = 40um Homogenized material: Kxy = 133 Kz = 159 Kxyz for silicon is 150 It slightly reduces the vertical resistance (good for average power) It results in a increase of the horizontal resistance (bad for hotspots): Negative impact on hotspot temperatures (heat spreads first horizontally Eric Beyne in 3DIC Conf 2011) No extensive exploration of the TSV+SiO2 impact on thermal results of 3D ICs CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 54

Cliquez pour Uniform modifier power le style dissipation du titre It is not effective even with unreachable TSV density Heat still has to overcome considerable thermal resistance from underfill layer Using 1um underfill (CuCu bonding) is much more efficient than TSVs TSV density Number of TSVs Underfill thickness Temperature Reduction 30% 169 33um 0,15% 100% 576 33um 0,43% 30% 169 1um 66,2% CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 55

Cliquez pour Hotspot modifier power le style dissipation du titre TSVs cause considerable lateral thermal blockage Combined thinned dies + TSVs may cause exacerbated hotspots CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 56

WIOMING Cliquez : Impact pour modifier of TSVs on le Temperature style du titre TSV Thermal Properties : TSV diam = 10um / TSV pitch = 40um / SiO2 layer around TSV = 0.25µm Thermal Conductivity of a TSV array (Homogenized material) Kxy = 133 TSV slightly increase the horizontal resistance (negative impact for hotspots): Kz = 159 TSV slightly reduces the vertical resistance (good for average power) Kxyz = 150 for silicon only (without TSV) Exploration of WIOMING testcase : Comparing with TSVs vs. without TSVs Power dissipation Temperature difference Hotspot a) + 2.22% Hotspot b) + 5.89% Uniform - 0.02% TSV does not reduce temperature for uniform power Due to very low TSV density (3%) But may lead to temperature increase for hotspot Must take care of TSV Hotspot placement!!! CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 57

General Cliquez Thermal pour impact modifier of TSV le : style Conclusions du titre Techniques for thermal TSV insertion should be clearly reviewed Analyzed in the context of a single die (not in 3D context) Vertical conductivity is slightly increased at the cost of reduction of lateral conductivity Paper to be submitted to ICCAD 14 Analytical (ANSYS) and numerical (ATM) models in agreement with measurements Implementation Once properly described, ATM is able to capture the negative effects of the very fine grain SiO2 oxide (0.3um thick) around TSVs ATM results matching ANSYS ATM faster than ANSYS (x40) Mesh grid strongly impacting the temperature results All experiments should consider the same grid Next Steps Extend analysis to include coppers pillars Evaluate the effectiveness of thermal TSVs traversing multiple layers in full 3D context CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 58

Cliquez pour Conclusion modifier le Perspectives style du titre Thermal Model for 3D using ATM tool Fully automated using Python scripting Material homogenization : reduce complexity, maintain accuracy System-level thermal model: accuracy versus simulation time Must focus on all levels: from board package to die internal structures Achieved Compact Thermal Model includes macroscopic components fine grain structures Efficient compaction engine allowing fast thermal simulation and system-level exploration Thermal model correlation with a Memory-on-Logic 65nm 3D circuit Steady State analysis : less than 4% error with silicon measurements, accurate evaluation of hot spots Transient step response : 5% 40% error in thermal response time (thermal time constant) Exploration Test Cases Study Advanced Packaging Materials (HeatSpreader, etc.) Study thermal impact of TSVs (vertical conductivity versus lateral blockage) Study impact of thermal on DRAM refresh rate CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 59

WIOMING Cliquez pour : Thermal modifier impact le style on du DRAM titre Understand processor hot spot effects on memory data integrity: 1. System-level localization of errors versus hot spot topology configuration. 2. Memory data error rate versus processor die temperature. 3. Correlation between processor hot spots and memory thermal sensor information. Memory data plane: errors 1: channel #, bank #, row @, column @ 2: nb errors vs processor Temp Memory die: hot spot Processor die: hot spot Memory thermal sensor value 3: correlation? Processor thermal sensor value CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 61

Test Case setup : Cliquez Thermal pour impact modifier on le DRAM: style du How titre? Simulation and board environment set-up done. Simple cpu memory application has been developed and running on the board Developped traces to dump memory analysis results to laptop (memory error location, etc.). Measurement Software, using ARM core WideIO memory, and Heaters Scenario Inject Hotspot using Bottom Left Heaters, 4 Watt Modify the refresh rate of the DRAM Still issues to read back WideIO thermal sensor Collaboration with UKL (Univ. Kaiserlautern, Germany) 4 independant memory channels with WideIO DRAM memory + associated WideIO TSV matrixes Heat Generation CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 62

Cliquez pour modifier le style du titre DRAM Thermal Behavior Qualified up to 95 C But still works a bit above Thermal Impact on DRAM : Results (1) Heaters can generate up to 105 C Heat does not spread due to TSV blockage Can generate errors on Channel 3 4 CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 63

Thermal Cliquez pour Impact modifier on DRAM le style : Results du titre (2) Study impact of DRAM refresh rate on DRAM errors : Less refresh, more DRAM errors, with increased temperature From max 25 to max 70 errors Thermal has clear impact on System reliability performances Next? Validate measure on more devices Explore power performance impact of refresh rate on real applications Explore SW control loops at system level CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 64

Where is 3D in massively parallel computing? Cliquez pour modifier le style du titre Silicon board: 3D Integrated Circuit to fill the integration gap for massively parallel computing M M M SoC P P P On-chip ICN M M M M M M M M M M M M M M M M M M P P P P P P P P P P P P P P P P P Silicon Chip-to-chip ICN board M ICN M M M M Chip-to-chip M Node-to-node M M ICN M ICN M M M M M M Interconnect Board M Rack Computer Board-to-board ICN M M P Rack-to-rack ICN M P M ICN Processor (could be Multi-core CPU with HW accelerators) Memory device (could be RAM, NVM, SSD, HDD) Interconnect (could be electrical, optical) 3D integration from WideIO to computing D. Dutoit - D43D Lausanne June 2012 66 CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 66

Active interposer for computation systems Cliquez pour modifier le style du titre Digital/mixed-signal on a mature CMOS node Allows for buffering and much more 3D Program Top dies:, 6 dies, FDSOI 28 nm, 16 mm², Many-core compute fabric, Cache COherent Active Interposer: 65 nm, ~160 mm² NoCInterconnect hub Include test DFT architecture Embedded power management Demonstration in 2015 In Nanoelec 3D program Buffering: increased chip-to-chip distance Lowthroughtputper line ( 1 Gbps) but high density(7-8 ML) => high bandwidth Allowing: power delivery network, test architecture 3D Technology needed for active interposer: Reduce µbump pitch for better performance : 40µm => 20µm pitch Stress management : reduce warpage of large interposers CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 67

Active Interposer : system integration Cliquez pour modifier le style du titre Focus is miniaturization and energy efficient intra-chip communications. Active interposer technology benefits for system integration are: High horizontal interconnect density with metal layers Aggressive vertical interconnect thanks to TSV technology Backbone for heterogeneous integration of small dies and passives Backbone for integration of IOs, Peripherals, test, Power Management Better thermal conductivity with silicon Keep small filling factor to preserve large size interposer yield and costs 3D integration from WideIO to computing D. Dutoit - D43D 2012 CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 68 68

Embedding Power management Cliquez pour modifier le style du titre Integrate the power management of the stacked digital ICs on the interposer Use mature technology 65nm to supply advanced FDSOI28nm node Generate in 65nm Ultra Wide Voltage Range (UWVR) to supply 28nm DC-DC conversion using switched capacitors (from 0.6V to 1.3 V) Integrated power supplies for generate the back biasing for FBB Using High Voltage Charge Pumps (-3/+3) Technical objectives Reduce the overall power consumption (by fine grain DVFS at chip level) Provide a clean internal power rails (avoid IO impedance) Reduce the extra cost due to an external power management unit Limit the number of the external passive components Reduce the number of the IO power pins No extra cost because of the free area available on the interposer CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 69

On-Chip Power Converter Proposal Cliquez pour modifier le style du titre Distributed power supply units dedicated to each stacked chiplets Achieve power efficiency up to 80% from 0.6 to 1.3V Generate various voltages for fine DVFS Regulate the power rails with limited drop Use no external components Robust to CMOS 65nm process variation Use the unused area on the interposer Need (only) one external power management unit to provide 1.8V digital IC (FDSOI) digital IC (FDSOI) digital IC (FDSOI) Power Unit Interposer (CMOS) Power Unit 1.8V (only) CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 70

Energy Cliquez Efficiency pour modifier Control le for style Multi-core du titre Multi-Cores challenge? Power Management and Process monitoring, per Core / PE Proposal of a CVP module (Clock, Variability and Power ) to maximize PE energy efficiency. The CVP integrates : Clock generation Power control Thermal monitors Variability Slack monitors CVP implementation Highly configurable Modular IP Each sub-module is optional CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 72

Cliquez pour CVP modifier Multiprobe le style Principle du titre Test Start / Stop MultiProbe based on Ring Oscillators 8-to-1 multiplexer Stage 1 Stage 2 Stage n Ring-oscillator #1 Ring-oscillator #2... Ring-oscillator #7 Adress decoder Outputs a set of frequencies : Exploit behavioral differences to estimate a P,V,T operating point Kolmogorov-Smirnov method to estimate Temperature( Ringo Freq) 28 bits Counter Overflow bit 3 bits Config Scan in Scan out Low cost HW ; Some complex SW Can afford one probe per core or cluster CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 73

Cliquez Thermal pour sensors modifier (FDSOI le style 28nm) du titre ST Reference Sensor (TH-SENS) Proposed Multiprobe Surface 29 919 µm² 450 µm² Integration constraints HardMacro, Analog supply voltage Full StdCell approach, Small hard macro Accuracy +/- 6 C Mean: 0,6 C Standard dev: 7,46 C * Temperature access Direct access through HW register Indirect Access (Oscillators frequencies) SW Access Easy Requires esw Calibration Simple shift More complex Available * : worst accuracy (from spice simulation), when considering reduced PV ranges on board, we can achieve 3 C standard deviation Both Thermal sensors : absolute multiprobes can be integrated within CVP unit CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 74

ESL Modeling Simulation Framework Thermal Power Aceplorer TM (Power modeling and coupled Power/Thermal simulation) Functional AceTLMConnect TM (Library for functional, power, thermal parameter monitoring providing co-simulation link) AceThermalModeler TM (Dynamic Compact Thermal Model) / TLM Pascal Vivet DAC 2013 June 2013 76 CEA. All rights reserved

Power Thermal Aware Virtual Platform Existing Virtual Platform Power Thermal Application Application Runtime Runtime sync Power actuator Thermal management Hardware dependent SW Timed HW platform or PVT platform Sync HAL ISS HW blocks DVFS HAL Power unit Thermal sensor HAL Thermal sensor AceTLMConnect Power thermal analysis tools activity Clockdomain voltage supply Static Dynamic power Temperature Pascal Vivet DAC 2013 June 2013 77 CEA. All rights reserved

MPSoC use case: LOCOMOTIV LOCOMOTIV architecture Multi-Core with shared memory Thermal sensors Power management Local: adapts to process/ageing/temperature Global: DVFS control per core Hardware Assisted Runtime Software (HARS) Pedestrian Detection Application Variable execution time Parallel execution Pascal Vivet DAC 2013 June 2013 78 CEA. All rights reserved

Cliquez pour Online modifier thermal le management style du titre Software stack enhanced with thermal control Cooperative runtime (Threads queue handed by master core) Schedule power hungry tasks to low temperature cores Reactive thermal control: Adapt DVFS to meet thermal budget Profit from data dependant computation to reduce PE performances Slack Reclamation + Idle mode when reaching temperature threshold Slack Reclamation with Temperature Thresholds: If t > WCET Temp < Thigh, then DVFS + If t < WCET, then DVFS - If t > Tframe, then loose next frame If Temp > Tcrit., go to IDLE mode If Temp < Tlow, start next frame CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 79

No thermal mitigation Cliquez Thermal pour management modifier le style results du (1/3) titre Fast Simulation of Temperature Warm-up phase (Simulators are decoupled) Thermal budget iscrossed: T > 95 C Peak temperature = 114 C Thermal control is needed Tools are mandatory for early exploration and validation CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 80

Cliquez Thermal pour management modifier le style results du (2/3) titre No thermal mitigation Idle Time Management + Thresholds Thermal budget isrespected: T < 95 C Usingthresholds: 70 C < T < 90 C Peak temperature = 90,71 C Thermal control works but with poor application results: Skipped frames : 10/52 but successive frames! CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 81

Cliquez Thermal pour management modifier le style results du (3/3) titre No thermal mitigation Idle Time Management + Thresholds Slack Reclamation Slack reclamation + Thresholds Fine grain power and thermal analysis simulation time: 16 minutes (SystemC VP simulation time only : 5 minutes) Compliant with early design phases! Thermal budget respected: T < 95 C, samethresholds: 70 C < T < 90 C Peak temperature = 91,2 C Slack reclamation: use estimated remaining time to reduce DVFS Skipped frames : 3/52. Application rendering is preserved. CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 82

3D Technology Cliquez pour modifier le style Conclusions du titre 3D is today a mature and available technology, proven on various applications XILINX FPGA, Imagers, Sony play station, WideIO1, HMC memories. More Technology developments for further 3D scaling: smaller TSV µbumps pitch, CuCu copper bonding, large interposer size, 3D assembly 3D : CurrentPower Thermal Issues? For embedded markets, medium range power densities, not yet real Thermal issues Some new packaging solutions, with HeatSpreaders to help dissipation For more agressive designs, temperature control is mandatory 3D CAD Tools for Power Thermal DOCEA Power tools available for accurate Thermal model For system levelexploration in close loop: Power + Thermal + Virtual Platform Mentor Graphics: somenew toolsto beavailable For signoff analysis, closingthe gap betweendie design and package design CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 84

Cliquez pour Challenges modifier le style Perspectives du titre Some More Design Challenges : 3D Power Delivery Network : Complex power grid Design Analysis Power design, IR drop + EM analysis, power swithes within interposer, etc. Requires new tools: APACHE, Mentor Graphics 3D DFT test : more dies to test, increasedpower thermal issues DFT and test schedulinghas been widelystudiedfor 2D Needto beextendedfor 3D includingpower thermal budgetting 3D ManyCore Computation Fabric Active Interposer as a backbone for ManyCore integration within chiplet fabric Including advanced Network-on-Chip with 3D plugs including embedded DC/DC converters for efficient power managment On-going 2 Circuit Design developments, circuits tape out end of 2014. INTACT : 3D Cache CoherentManyCore, basedon MIPS core, withagressive 3D tech GAIA2 : 3D ManyCore, baseedon ARM CoretexA53 cores, withavailable3d tech Larger power densities: from 0.03W/mm2 to 0.1W/mm2 Advanced Power Management and Thermal Mitigation to be developped CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 85

Cliquez pour modifier Acknowledgments style du titre LETI design technology teams : C. Bernard, F. Clermidy, D. Dutoit, E. Guthmuller, I. Miro Panadès, G. Pillonnet, Y. Thonnart, C. Santos, J. Pontès T. Sassolas, J. Mottin, A. Molnos, S. Cheramy, P. Leduc, Our Partners DOCEA Power Mentor Graphics STMicrolectronics UFRGS LIRMM LIP6 UKL CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 86

Cliquez pour modifier le Save style the du date titre! D43D 2014, 23-24 June 2014 @ EPFL, Lausanne, Suisse http://www.d43d.com 2 days workshop dedicated on 3D Design Technology CEA. All rights reserved ECOFAC 14 Workshop, Lorient, Thermal Power in 3D Design 20-May-14 87