Hybrid Memory Cube Controller Design Example User Guide

Hybrid Memory Cube Controller Design Example User Guide Subscribe Last updated for Quartus Prime Design Suite: 16.0 UG-20027 101 Innovation Drive San Jose, CA 95134 www.altera.com

TOC-2 Contents Hybrid Memory Cube Controller Design Example Quick Start Guide...1-1 Design Example Directory Structure... 1-2 Design Example Components...1-2 Generating the Design Example... 1-4 Understanding the Testbench...1-6 Simulating the Design Example Testbench...1-6 Setting Up the Board... 1-7 Compiling and Testing the Design Example in Hardware... 1-9 Hybrid Memory Cube Controller Design Example Description... 2-1 Design Example Description...2-1 Features... 2-1 Hardware and Software Requirements... 2-1 Functional Description... 2-1 Interface Signals... 2-2 Design Example Register Map... 2-3 Additional Information...A-1 HMC Controller Design Example User Guide Revision History... A-1 How to Contact Altera... A-1 Typographic Conventions... A-2

Hybrid Memory Cube Controller Design Example Quick Start Guide 1 UG-20027 Subscribe To help you understand how to use the Hybrid Memory Cube Controller IP core, the core features a simulatable testbench and a hardware design example that supports compilation and hardware testing. When you generate the design example, the parameter editor automatically creates the files necessary to simulate, compile, and test the design in hardware. You can download the compiled design to the Arria 10 GX FPGA Development Kit. Figure 1-1: Development Steps for the Design Example Compilation (Simulator) Functional Simulation Example Design Generation Compilation (Quartus Prime) Hardware Testing Related Information Hybrid Memory Cube Controller IP Core User Guide 2016. All rights reserved. ALTERA, ARRIA, CYCLONE, ENPIRION, MAX, MEGACORE, NIOS, QUARTUS and STRATIX words and logos are trademarks of and registered in the U.S. Patent and Trademark Office and in other countries. All other words and logos identified as trademarks or service marks are the property of their respective holders as described at www.altera.com/common/legal.html. Altera warrants performance of its semiconductor products to current specifications in accordance with Altera's standard warranty, but reserves the right to make changes to any products and services at any time without notice. Altera assumes no responsibility or liability arising out of the application or use of any information, product, or service described herein except as expressly agreed to in writing by Altera. Altera customers are advised to obtain the latest version of device specifications before relying on any published information and before placing orders for products or services. ISO 9001:2008 Registered www.altera.com 101 Innovation Drive, San Jose, CA 95134

1-2 Design Example Directory Structure UG-20027 Design Example Directory Structure Figure 1-2: Directory Structure for the Generated Design Example hmcc_mtaps_0_example_design (default name of <example_design_install_dir>) ip Supporting IP.qsys and other files src Supporting IP and testbench files example_design sim. hmcc_tb.sv hmc_cfg.sv Makefile <Simulation Script> par Hardware design files hmcc_example.qpf hmcc_example.qsf hmcc_example.sdc sysconsole_testbench.tcl <Simulation Script> The hardware configuration and test files (the hardware design example) are located in <example_ design_install_dir>/example_design/par. The simulation files (testbench for simulation only) are located in <example_design_install_dir>/example_design/sim. Design Example Components The HMC Controller hardware design example includes the following components: HMC Controller IP core with CDR reference clock set to 125 MHz and with default RX mapping and TX mapping settings. Note: The design example requires these settings to operate properly on the Arria 10 GX FPGA Development Kit with the HMC daughter card. Client logic that coordinates the programming of the IP core, and packet generation and checking. JTAG controller that communicates with the Altera System Console. You communicate with the client logic through the System Console. Hybrid Memory Cube Controller Design Example Quick Start Guide

UG-20027 Design Example Components 1-3 Figure 1-3: HMC Controller Design Example Block Diagram Board Clocks & Reset Arria 10 Device TX PLLs HMC Controller IP Core Data Path Request Generator and Response Monitor TX TX FIFO MAC TX Lane Swapper Transceiver x16 HMC Device RX MAC RX Lane Swapper LEDs Test Controller Avalon-MM Avalon-MM I 2C Master Control and Status Interface Initialization State Machine Arria 10 Transceiver Reconfiguration Interface Table 1-1: HMC Controller IP Core Design Example Testbench File Descriptions Lists the key files that implement the example testbench. File Names Description Testbench and Simulation Files /src/hmcc_example.sv /sim/hmcc_tb.sv Top-level hardware design example file. Top-level file for simulation. Testbench Scripts Note: Use the provided Makefile to generate these scripts. /sim/run_vsim.do /sim/run_vcs.sh /sim/run_ncsim.sh The ModelSim script to run the testbench. The Synopsys VCS script to run the testbench. The Cadence NCSim script to run the testbench. Hybrid Memory Cube Controller Design Example Quick Start Guide

1-4 Generating the Design Example Table 1-2: HMCC Controller IP Core Hardware Design Example File Descriptions UG-20027 File Names hmcc_example.qpf hmcc_example.qsf hmcc_example.sdc../src/hmcc_example.sv Quartus Prime project file Description Quartus Prime project settings file Quartus Prime project Synopsys Design Constraints file Top-level Verilog HDL design example file Scripts sysconsole_testbench.tcl Main file for accessing System Console Generating the Design Example Figure 1-4: Procedure Start Parameter Editor Specify IP Variation and Select Device Select Design Parameters Specify Example Design Initiate Design Generation Hybrid Memory Cube Controller Design Example Quick Start Guide

UG-20027 Generating the Design Example 1-5 Figure 1-5: Example Design Tab in Hybrid Memory Cube Controller Parameter Editor Follow these steps to generate the Arria 10 hardware design example and testbench: 1. In the IP Catalog (Tools > IP Catalog), select the Arria 10 target device family. 2. In the IP Catalog, locate and select Hybrid Memory Cube Controller. The New IP Variation window appears. 3. Specify a top-level name for your custom IP variation. The parameter editor saves the IP variation settings in a file named <your_ip>.qsys. 4. You must select a specific Arria 10 device in the Device field, or keep the default device the Quartus Prime software selects. 5. Click OK. The IP parameter editor appears. 6. On the IP tab, specify the parameters for your IP core variation. 7. On the Example Design tab, choose the following settings for the design example: a. For Select Design, select the HMCC Daughter Board option. b. For Example Design Files, select the Simulation option to generate the testbench, and select the Synthesis option to generate the hardware design example. c. For Generated HDL Format, only Verilog is available. d. For Target Development Kit select the Arria 10 GX FPGA Development Kit (Production Silicon). Hybrid Memory Cube Controller Design Example Quick Start Guide

1-6 Understanding the Testbench Note: When you choose this kit, the hardware design example overwrites your previous device selection with the device on the target board. When you generate the design example, the Quartus Prime software creates Quartus Prime project, setting, and pin assignments for the board you selected. If you do not want the software to target a specific board, select None. 8. Click the Generate Example Design button. UG-20027 Understanding the Testbench Altera provides an design example with the HMC Controller IP core. The design example is available both for simulation of your IP core and for compilation. The design example in simulation functions as the HMC Controller IP core testbench. If you click Generate Example Design in the HMC Controller parameter editor, the Quartus Prime software generates a demonstration testbench. The parameter editor prompts you for the desired location of the testbench. To simulate the testbench, you must provide your own HMC bus functional model (BFM). Altera tests the design example testbench with the Micron Hybrid Memory Cube BFM. The testbench does not include an I 2 C master module, because the Micron HMC BFM does not support and does not require configuration by an I 2 C module. In simulation, the testbench controls a TX PLL and the data path interfaces to perform the following sequence of actions: 1. Configures the HMC BFM with the HMC Controller IP core data rate and channel width, in Response Open Loop Mode. 2. Establishes the link between the BFM and the IP core. 3. Directs each of the IP core's four ports to write four packets of data to the BFM. 4. Directs the IP core to read back the data from the BFM. 5. Checks that the read data matches the write data. 6. If the data matches, displays TEST_PASSED. Simulating the Design Example Testbench Figure 1-6: Procedure Change to Testbench Directory. Run <Simulation Script>. Analyze Results. Hybrid Memory Cube Controller Design Example Quick Start Guide

UG-20027 Setting Up the Board 1-7 Follow these steps to simulate the testbench: 1. At the command line, change to the <design example>/sim directory. 2. Type make scripts. 3. Type one of the following commands, depending on your simulator: Simulator License Mentor Graphics QuestaSim Synopsys VCS Cadence NCSIM Command Line make vsim HMC_MODEL=<HMC BFM directory> make vcs HMC_MODEL=<HMC BFM directory> make ncsim HMC_MODEL=<HMC BFM directory> 4. To view simulation results: When you run the testbench in any of the three supported simulators, the script executes the testbench sequence and logs the simulator activity in <design example directory>/example_design/ sim/<simulator>.log. <simulator> is "vsim", "ncsim", or "vcs". When you run the testbench in any of the three supported simulators, the script generates a waveform file. You can run the command make <simulator>_gui to load the waveform in the simulator-specific waveform viewer. To view the waveform file in your simulator, type one of the following commands: Simulator License Command Line Waveform File Mentor Graphics ModelSim Synopsys Discovery Visual Environment Cadence SimVision Waveform make vsim_gui make vcs_gui make ncsim_gui <design example directory>/example_design/sim/ mentor/hmcc_wf.wlf <design example directory>/example_design/sim/ hmcc_wf.vpd <design example directory>/example_design/sim/ cadence/hmcc_wf.shm 5. Analyze the results. The successful testbench sends and receives ten packets per port, and displays Test_PASSED" Setting Up the Board Set up the board to run the hardware design example. Note: Ensure that power is turned off before you change any settings. 1. Set the DIP switches on the daughter card as follows: Set DIP switch SW1 to indicate cube ID 0: Switch Function Setting 1 CUB[0] Open 2 CUB[1] Open Hybrid Memory Cube Controller Design Example Quick Start Guide

1-8 UG-20027 Setting Up the Board Switch Function Setting 3 CUB[2] Open 4 Don't Care Set DIP switch SW2 to specify clock settings: Switch Function Setting 1 CLK1_FSEL0 Open (125 MHz) 2 CLK1_FSEL1 Open (125 MHz) 3 CLK1_SEL Open (Crystal) 4 Don't Care 2. Connect the HMC daughter card to the Arria 10 FPGA Development Kit using the daughter card's J8 and J10 connectors. 3. Set the jumpers on the Arria 10 GX FPGA Development Kit: Add shunts to the J8 jumper to select 1.5 V as the VCCIO setting for FMC connector B. Add shunts to the J11 jumper to select 1.8 V as the VCCIO setting for FMC connector A. Figure 1-7: Arria 10 GX FPGA Development Kit and HMC Daughter Card HMC Daughter Card DIP Switches SW1 and SW2 Arria 10 GX FPGA Development Kit Jumpers J8 and J11 Hybrid Memory Cube Controller Design Example Quick Start Guide

UG-20027 Compiling and Testing the Design Example in Hardware 1-9 Compiling and Testing the Design Example in Hardware To compile and run a demonstration test on the hardware design example, follow these steps: 1. Ensure hardware design example generation is complete. 2. In the Quartus Prime software, open the Quartus Prime project <example_design_install_dir>/ example_design/par/hmcc_example.qpf. 3. In the Compilation Dashboard, click Compile Design (Quartus Prime Pro Edition) or choose Processing > Start Compilation (Quartus Prime Standard Edition). 4. After you generate a.sof, follow these steps to program the hardware design example on the Arria 10 device: a. Choose Tools > Programmer. b. In the Programmer, click Hardware Setup. c. Select a programming device. d. Select and add the Arria 10 GX FPGA Development Kit to which your Quartus Prime session can connect. e. Ensure that Mode is set to JTAG. f. Click Auto Detect and choose any device. g. Double-click the Arria 10 device. h. Open the.sof in <example_design_install_dir>/example_design/par/output_files, Note: The Quartus Prime software changes the device to the one in the.sof. i. In the row with your.sof, check the box in the Program/Configure column. j. Click Start. k. After the software configures the device with the hardware design example, observe the board LEDs: A blinking red LED signifies the design is running. Two green LEDs near the red blinking LED signifies that the HMC link is initialized and the test passed. One red LED near the red blinking LED signifies that the test failed. l. Optional. Use the System Console testbench to observe additional test output. Note: Use the System Console to monitor status signals in the design example when the board is connected to your computer via the JTAG interface. The System Console shows the board's LED status for remote monitoring, the initialization status for each step, and the status of each port s request generator and response checker. The System Console also provides an interface to start or re-start the test. a. Choose Tools > System Debugging Tools > System Console. b. In the System Console, choose File > Execute Script. c. Open the file <example_design_install_dir>/example_design/par/sysconsole_testbench.tcl. d. The software loads graphical test output. Choose Re-start to run the test again. Hybrid Memory Cube Controller Design Example Quick Start Guide

1-10 Compiling and Testing the Design Example in Hardware Figure 1-8: System Console Testbench Output UG-20027 Related Information Programming Altera Devices Analyzing and Debugging Designs with System Console Quartus Prime Standard Edition Handbook, Volume 3: Verification For information about programming an Altera device, refer to the "Programming Altera Devices" chapter. Hybrid Memory Cube Controller Design Example Quick Start Guide

Hybrid Memory Cube Controller Design Example Description 2 UG-20027 Subscribe Design Example Description Features The design example demonstrates the functionality of the Hybrid Memory Cube Controller IP core. You can generate the design from the Example Design tab of the Hybrid Memory Cube Controller graphical user interface (GUI) in the IP parameter editor. I 2 C master and I 2 C initialization state machine for HMC daughter card and HMC configuration ATX PLL and transceiver recalibration state machine Request generator Request monitor System Console interface Hardware and Software Requirements Altera uses the following hardware and software to test the design example: Quartus Prime software System Console ModelSim-AE, Modelsim-SE, NCsim (Verilog HDL only), or VCS simulator Arria 10 GX FPGA Development Kit HMC daughter card Functional Description Altera provides a compilation-ready design example with the HMC Controller IP core. This design example targets the Arria 10 GX FPGA Development Kit with an HMC daughter card connected through the FMC connectors. You can use the design as an example for correct connection of your IP core to your design, or as a starter design you can customize for your own design requirements. The design example includes an I 2 C master module, a PLL/CDR recalibration module, one external transceiver PLL IP core, and logic to generate and 2016. All rights reserved. ALTERA, ARRIA, CYCLONE, ENPIRION, MAX, MEGACORE, NIOS, QUARTUS and STRATIX words and logos are trademarks of and registered in the U.S. Patent and Trademark Office and in other countries. All other words and logos identified as trademarks or service marks are the property of their respective holders as described at www.altera.com/common/legal.html. Altera warrants performance of its semiconductor products to current specifications in accordance with Altera's standard warranty, but reserves the right to make changes to any products and services at any time without notice. Altera assumes no responsibility or liability arising out of the application or use of any information, product, or service described herein except as expressly agreed to in writing by Altera. Altera customers are advised to obtain the latest version of device specifications before relying on any published information and before placing orders for products or services. ISO 9001:2008 Registered www.altera.com 101 Innovation Drive, San Jose, CA 95134

2-2 Interface Signals check transactions. The design example assumes a Micron HMC 15G-SR HMC device, which is a fourlink device, on the daughter card. The design example includes one instance of the IP core and connects to a single link on the HMC device. Figure 2-1: HMC Controller Design Example Block Diagram UG-20027 Board Clocks & Reset Arria 10 Device TX PLLs HMC Controller IP Core Data Path Request Generator and Response Monitor TX TX FIFO MAC TX Lane Swapper Transceiver x16 HMC Device RX MAC RX Lane Swapper LEDs Test Controller Avalon-MM Avalon-MM I 2C Master Control and Status Interface Initialization State Machine Arria 10 Transceiver Reconfiguration Interface After you configure the Arria 10 FPGA with the design example, the I 2 C controller configures the onboard clock generators and the HMC device. When calibration completes, the design example calibrates the ATX PLL. During operation, the request generator generates read and write commands that the HMC Controller IP core then processes. The request monitor captures the responses from the IP core and checks them for correctness. Interface Signals Table 2-1: HMC Controller IP Core Design Example Signals Signal Name Direction Width (Bits) Description clk_50 Input 1 50 MHz input clock. hssi_refclk Input 1 CDR reference clock for HMC and HMCC IP core. Hybrid Memory Cube Controller Design Example Description

UG-20027 Design Example Register Map 2-3 Signal Name Direction Width (Bits) Description hmc_lxrx Input Channel Count (16 or 8) hmc_lxtx Output Channel Count (16 or 8) FPGA transceiver receive pins. FPGA transceiver transmit pins. hmc_ctrl_lxrxps Input 1 FPGA transceiver power save control. hmc_ctrl_lxtxps Output 1 HMC transceiver power save control. hmc_ctrl_ferr_n Input 1 HMC FERR_N output. hmc_ctrl_p_rst_n Output 1 HMC P_RST_N input. hmc_ctrl_scl Bi-Directional 1 HMC I 2 C configuration clock. hmc_ctrl_sda Bi-Directional 1 HMC I 2 C configuration data. fmc0_scl Output 1 Unused. Driven low to protect the FPGA I/O pins from the 3.3 V pullup on the daughter card. fmc0_sda Output 1 Unused. Driven low to protect the FPGA I/O pins from the 3.3 V pullup on the daughter card. push_button Input 1 Push button input used for reset. heart_beat_n Output 1 Heartbeat LED output. link_init_complete_n Output 1 Link initialization complete LED output. test_passed_n Output 1 Test passed LED output. test_failed_n Output 1 Test failed LED output. Design Example Register Map Table 2-2: HMC Controller IP Core Design Example Register Map Offset 0x00 0x01 0x02 0x03 Register DESIGN_INFO and RESET BOARD_LEDS TEST_INITIALIZATION_STATUS PORT_TEST_STATUS Hybrid Memory Cube Controller Design Example Description

2-4 Design Example Register Map Table 2-3: DESIGN_INFO AND RESET Registers UG-20027 Writing to these registers resets the design. Bits Field Name Type Value on Reset Description 1:0 Port Count RO Varies Number of ports for the IP core instance. 7:2 Reserved RO 0x00 Table 2-4: BOARD_LEDs Register This register reflects the status of the board's LEDs. Bits Field Name Type Value on Reset Description 0 Test Failed RO 0x00 Test failed. 1 Test Passed RO 0x00 Test passed. 2 HMCC Link Initialization Complete RO 0x00 HMC link initialization complete and ready for traffic. 3 Heartbeat RO 0x00 Toggles when the design is running. 7:4 Reserved RO 0x00 Table 2-5: TEST_INITIALIZATION_STATUS Register Bits Field Name Type Value on Reset Description 0 I 2 C Clock Generator Set 1 ATX PLL and Transceiver Recalibration Complete 2 I 2 C HMC Configuration Complete 3 HMC Link Initialization Complete 7:4 Reserved RO 0x00 Table 2-6: PORT_STATUS Register RO 0x00 On- board clock generators configured. RO 0x00 ATX PLL and transceivers recalibrated to the input clock. RO 0x00 HMC device configuration over I 2 C complete. RO 0x00 HMC link initialization complete and ready for traffic. Bits Field Name Type Value on Reset Description 0 Port 0 Requests OK RO 0x00 Port 0 request generation complete. 1 Port 0 Responses OK RO 0x00 Port 0 response checking passed. 2 Port 1 Requests OK RO 0x00 Port 1 request generation complete. 3 Port 1 Responses OK RO 0x00 Port 1 response checking passed. Hybrid Memory Cube Controller Design Example Description

UG-20027 Design Example Register Map 2-5 Bits Field Name Type Value on Reset Description 4 Port 2 Requests OK RO 0x00 Port 2 request generation complete. 5 Port 2 Responses OK RO 0x00 Port 2 response checking passed. 6 Port 3 Requests OK RO 0x00 Port 3 request generation complete. 7 Port 4 Responses OK RO 0x00 Port 3 response checking passed. Hybrid Memory Cube Controller Design Example Description

Additional Information A UG-20027 Subscribe HMC Controller Design Example User Guide Revision History Table A-1: Document Revision History Summarizes the new features and changes in the design example user guide for the HMC Controller IP core. Date ACDS Version Changes 16.0 Initial release. How to Contact Altera Table A-2: How to Contact Altera To locate the most up-to-date information about Altera products, refer to this table. You can also contact your local Altera sales office or sales representative. Contact Contact Method Address Technical support Website www.altera.com/support Technical training Website Email www.altera.com/training custrain@altera.com Product literature Website www.altera.com/literature Nontechnical support: general Email nacomp@altera.com Nontechnical support: software licensing Email authorization@altera.com 2016. All rights reserved. ALTERA, ARRIA, CYCLONE, ENPIRION, MAX, MEGACORE, NIOS, QUARTUS and STRATIX words and logos are trademarks of and registered in the U.S. Patent and Trademark Office and in other countries. All other words and logos identified as trademarks or service marks are the property of their respective holders as described at www.altera.com/common/legal.html. Altera warrants performance of its semiconductor products to current specifications in accordance with Altera's standard warranty, but reserves the right to make changes to any products and services at any time without notice. Altera assumes no responsibility or liability arising out of the application or use of any information, product, or service described herein except as expressly agreed to in writing by Altera. Altera customers are advised to obtain the latest version of device specifications before relying on any published information and before placing orders for products or services. ISO 9001:2008 Registered www.altera.com 101 Innovation Drive, San Jose, CA 95134

A-2 Typographic Conventions Related Information www.altera.com/support www.altera.com/training custrain@altera.com www.altera.com/literature nacomp@altera.com authorization@altera.com UG-20027 Typographic Conventions Table A-3: Typographic Conventions Lists the typographic conventions this document uses. Visual Cue Bold Type with Initial Capital Letters bold type Italic Type with Initial Capital Letters Meaning Indicate command names, dialog box titles, dialog box options, and other GUI labels. For example, Save As dialog box. For GUI elements, capitalization matches the GUI. Indicates directory names, project names, disk drive names, file names, file name extensions, software utility names, and GUI labels. For example, \ qdesigns directory, D: drive, and chiptrip.gdf file. Indicate document titles. For example, Stratix V Design Guidelines. italic type Indicates variables. For example, n + 1. Variable names are enclosed in angle brackets (< >). For example, <file name> and <project name>. pof file. Initial Capital Letters Subheading Title Indicate keyboard keys and menu names. For example, the Delete key and the Options menu. Quotation marks indicate references to sections in a document and titles of Quartus Prime Help topics. For example, Typographic Conventions. Additional Information

UG-20027 Typographic Conventions A-3 Visual Cue Meaning Courier type Indicates signal, port, register, bit, block, and primitive names. For example, data1, tdi, and input. The suffix n denotes an active-low signal. For example, resetn. Indicates command line commands and anything that must be typed exactly as it appears. For example, c:\qdesigns\tutorial\chiptrip.gdf. Also indicates sections of an actual file, such as a Report File, references to parts of files (for example, the AHDL keyword SUBDESIGN), and logic function names (for example, TRI). 1., 2., 3., and a., b., c., and so on Numbered steps indicate a list of items when the sequence of the items is important, such as the steps listed in a procedure. Bullets indicate a list of items when the sequence of the items is not important. The Subscribe button links to the Email Subscription Management Center page of the Altera website, where you can sign up to receive update notifications for Altera documents. The Feedback icon allows you to submit feedback to Altera about the document. Methods for collecting feedback vary as appropriate for each document. Related Information Email Subscription Management Center Additional Information