Create a Custom Embedded Linux Distribu3on for Any Embedded Device Using the Yocto Project

Transcription

1 Yocto Project Developer Day 2014 Intro to Yocto Project It's not an embedded Linux distribu2on It creates a custom one for you Create a Custom Embedded Linux Distribu3on for Any Embedded Device Using the Yocto Project Chris Hallinan Mentor Graphics October 16,

2 Agenda!! Introduc3on to the Yocto Project! Key Concepts " Build System Overview & Workflow " Exercise 1: Exploring the Host System! Recipes In- Depth " Standard Recipe Build Steps " Exercise 2: Examining Recipes! Building and Boo3ng an Image " Exercise 3: Building Your First Linux Image " Exercise 4: BooNng Your Linux Image Using QEMU! Layers and BSPs " Exercise 5: CreaNng a Custom Layer " Exercise 6-7: Adding a graphical boot logo and SSH server " Exercise 8-9: BooNng an embedded hardware board 2

3 Yocto Project Overview!! Collec3on of tools and methods enabling " Rapid evaluanon of embedded Linux on many popular off- the- shelf boards " Easy customizanon of distribunon characterisncs! Supports x86, ARM, MIPS, Power! Based on technology from OpenEmbedded other layers Project meta- yocto! Layer architecture allows for easy re- use of code meta- yocto- bsp oe- core (meta) 3

4 Yocto Project Overview!! Supports use of popular package formats including: " rpm, deb, ipk! Releases on a 6- month cadence! Latest (stable) kernel, toolchain and packages, documenta3on! App Development Tools including Eclipse plugin, ADT, hob 4

5 Yocto Project Overview!! Governance " Organized under the Linux FoundaNon " Split governance model " Technical Leadership Team " Advisory Board made up of parncipanng organizanons 5

6 Yocto Project Release Versions!! Major Version Releases Name Revision Poky Release Date Bernard Apr 5, 2011 Edison Oct 17, 2011 Denzil Apr 30, 2012 Danny Oct 24, 2012 Dylan Apr 26, 2013 Dora Oct 19, 2013 Daisy Apr 24, 2014 (unnamed) Oct 31, 2014 (planned) 6

7 Intro to OpenEmbedded!! Bitbake " Powerful and flexible build engine " Determines dependencies and schedules tasks Metadata a structured collecnon of "recipes" which tell BitBake what to build 7

8 Yocto is based on openembedded-core! Metadata describing approximately 900 popular "core" recipes used for building boot images. Includes support for graphics, Qt, networking, kernel recipes, tools, much more. 8

9 OK, so what is Poky?!! Poky is both a reference distribu3on and a build system! Poky has its own git repo " git clone git://git.yoctoproject.org/poky! Primary poky layers " oe- core (poky/meta) " meta- yocto- bsp " meta- yocto! Poky is the basis of YP Other layers meta- yocto meta- yocto- bsp oe- core (meta) 9

10 What is Metadata? Metadata exists in four general categories: Recipes (*.bb) Usually describe build instrucnons for a single package PackageGroups (special *.bb) Oeen used to group packages together for a FS image Classes (*.bbclass) Inheritance mechanism for common funcnonality Configura3on(*.conf) Drives the overall behavior of the build process 10

11 OE-CORE Breakdown" *.bb: 892 *.conf 73 *.bbclass 161 packagegroup* 40 11

12 Recipe Basics Most common form of metadata The Recipe Provides the list of ingredients and cooking instruc3ons to build a package(s) Has a common set of tasks busybox Metadata glibc sysvinit BitBake inetutils mtdutils 12

13 Recipe Basics!! A recipe is a set of instruc3ons for building packages, including: " Where to obtain the upstream sources and which patches to apply o SRC_URI! " Dependencies (on libraries or other recipes) o DEPENDS, RDEPENDS! " ConfiguraNon/compilaNon opnons o EXTRA_OECONF, EXTRA_OEMAKE! " Define which files go into what output packages o FILES_*! 13

14 Example Recipe ethtool_3.15.bb" 14

15 Example Recipe ethtool_3.15.bb" 15

16 Example Recipe ethtool_3.15.bb" 16

17 Example Recipe ethtool_3.15.bb" 17

18 Recipe Basics Default Tasks* do_setscene! do_fetch! do_unpack! do_patch! do_configure! do_compile! do_install! do_package! If a cache object exists and is valid, use it Locate and download source code Unpack source into working directory Apply any patches Perform any necessary pre- build configura3on Compile the source code Installa3on of resul3ng build ar3facts Create binary package(s) Note: to see the list of all possible tasks for a recipe, do this: $ bitbake -c listtasks <recipe_name>! 18 *Simplified for illustration

19 Quick Start Guide in a Slide!! Download Yocto Project sources: " $ wget yocto-1.6.1/poky-daisy tar.bz2! " $ tar xf poky-daisy tar.bz2! " $ cd poky-daisy ! " Can also use git and checkout a known branch i.e. Daisy o $ git clone git://git.yoctoproject.org/poky.git! o $ git checkout -b daisy!! Build one of the reference Linux distribu3ons: " $ source oe- init- build- env " Check/Edit local.conf for sanity o Modify MACHINE=qemux86! " $ bitbake core-image-base!! Run the image under emula3on: " $ runqemu qemux86 19

20 Lab 1: Exploring the Host System!! Objec3ves " Familiarize yourself with how the YP metadata sources are organized " Learn where you can find conf files, BitBake class files, and recipe files Log into your lab cloud-based host:! Instructor will supply URL/credentials! 20

21 Host System Layout! /scratch! ---sandbox! Storage for your software and mods! ---working!! Project build directory! ---yocto! Yocto sources and caches. Do Not Modify! Note: This is your instructor's preferred setup others may have their own favorite working layout 21

22 Host System Layout! /scratch/yocto! ---binary-images! Pre-built binary boot images for convenience! ---downloads (DL_DIR)!! Source cache (downloaded for each recipe)! ---sources! Yocto source code. Do Not Modifiy! ---sstate-cache (SSTATE_DIR)!! Common binary cache! Note: This is your instructor's preferred setup others may have their own favorite working layout 22

23 Host System Layout!!/scratch/yocto/sources! " ---meta-fsl-arm! " ---meta-fsl-arm-extra! " ---meta-fsl-demos! " ---meta-intel! " ---poky! Note: This is your instructor's preferred setup others may have their own favorite working layout 23

24 Poky Layout!! /scratch/yocto/sources/poky! " ---bitbake!!!!(the BitBake applica3on)! " ---documentation! " ---meta!!!!! (OE- CORE) " ---meta-yocto!!! (Yocto distro policy) " ---meta-yocto-bsp! (Yocto reference BSPs)! " ---oe-init-build-env (project setup u3lity)! " ---README! " ---README.hardware! " ---scripts!!!! (Various helper scripts)! Note: some items omiked for simplicity 24

25 Exercise 2: Examining Recipes!! meta/recipes- extended/bc/bc_1.06.bb " Uses LIC_FILES_CHKSUM and SRC_URI checksums " Note the DEPENDS build dependency declaranon indicanng that this package depends on flex to build 25

26 Exercise 2: Examining Recipes!! meta/recipes- mul3media/flac/flac_1.3.0.bb " Includes custom source patches to apply to the sources " Customizes autoconf configure opnons (EXTRA_OECONF) based on TUNE features " Breaks up output into mulnple binary packages 26

27 Exercise 2: Examining Recipes!! meta/recipes- connec3vity/ofono/ " Splits recipe into common.inc file to share metadata between mulnple recipes " Sets a condinonal build DEPENDS based on a distro feature (in the.inc file) " Sets up an init service via do_install_append()! " Has a _git version of the recipe 27

28 EXERCISE 3 Building a Linux Image 28

29 What board are you building?!! You must follow the correct procedure based on the board you are using! This Lab supports " Beaglebone Black " Minnowboard Max " WandBoard " QEMU x86! Make sure you follow the correct set of steps for your board 29

30 Beaglebone Black Build - Overview!! Checkout sources " Daisy branch! Create project directory " Edit local.conf o MACHINE = "beagleboard" o Change DL_DIR and SSTATE_DIR! Bitbake image " Bitbake core- image- base! Format SD card (details in Ex. 4) " 2 parnnons, boot (FAT 32 LBA 'c') and root (ext3) " Copy MLO, u- boot to 'boot'boot " Copy roops to 'root' 30

31 Beaglebone Black Build - Detailed!! Sources are already downloaded for you! Create project directory, edit local.conf and bblayers.conf $ cd /scratch/working! $. /scratch/yocto/sources/poky/oe-initbuild-env build-beaglebone! $ vi conf/local.conf! o Modify MACHINE = "beaglebone"! o Modify DL_DIR = "/scratch/yocto/downloads"! o Modify SSTATE_DIR = "/scratch/yocto/sstate-cache"! " Nothing to change in bblayers.conf, beaglebone is supported in meta- yocto- bsp! Bitbake your system image recipe $ bitbake core-image-base! 31

32 WandBoard Build - Overview!! Checkout sources " Daisy branch! Create project directory " Edit local.conf, bblayers.conf o MACHINE = "wandboard- quad" in local.conf o Modify DL_DIR and SSTATE_DIR o ACCEPT_FSL_EULA = "1"! Bitbake image " core- image- base! Copy sdcard image and boot (details in Ex. 4) " Use SD slot on cpu module on bokom of board 32

33 WandBoard Build - Detailed!! Sources are already downloaded for you! Create project directory, edit local.conf and bblayers.conf $ cd /scratch/working! $. /scratch/yocto/sources/poky/oe-init-build-env build-wbquad! $ vi conf/local.conf! o Modify MACHINE = "wandboard-quad"! o Modify DL_DIR = "/scratch/yocto/downloads"! o Modify SSTATE_DIR = "/scratch/yocto/sstate-cache"! o ACCEPT_FSL_EULA = "1" (bokom of file is good locanon for this) $ vi conf/bblayers.conf! o Add layers to BBLAYERS BBLAYERS?= " \! /scratch/yocto/sources/meta-fsl-arm \! /scratch/yocto/sources/meta-fsl-arm-extra \! /scratch/yocto/sources/meta-fsl-demos \!!! Bitbake your system image recipe $ bitbake core-image-base! 33

34 Minnowboard Max Build - Overview!! Checkout sources " Daisy branch! Create project directory " Edit local.conf o Change DL_DIR and SSTATE_DIR! Bitbake image " Bitbake core- image- base! Format SD card (details in Ex. 4) " 2 parnnons, boot (FAT 32 LBA 'c') and root (ext3) " Copy MLO, u- boot.img to boot parnnon " Copy roops to root parnnon 34

35 Minnowboard Max Build - Detailed!! Sources are already downloaded for you! Create project directory, edit local.conf and bblayers.conf $ cd /scratch/working! $. /scratch/yocto/sources/poky/oe-init-build-env build-minnowmx! $ vi conf/local.conf! o Modify MACHINE = "intel-corei7-64"! o Modify DL_DIR = "/scratch/yocto/downloads"! o Modify SSTATE_DIR = "/scratch/yocto/sstate-cache"! $ vi conf/bblayers.conf! o Add layers to BBLAYERS BBLAYERS?= " \! /scratch/yocto/sources/meta-intel \!!! Bitbake your system image recipe $ bitbake core-image-base! 35

36 QEMUx86 Build - Overview!! Checkout sources " Daisy branch! Create project directory " Edit local.conf, bblayers.conf o MACHINE = "qemux86" in local.conf (Note: this is the default) o Modify DL_DIR and SSTATE_DIR! Bitbake image " core- image- minimal! Boot using Yocto's helper script "runqemu" " Use -nographic if using SSH to your host 36

37 QEMUx86 Build - Detailed!! Sources are already downloaded for you! Create project directory, edit local.conf and bblayers.conf! $ cd /scratch/working! $. /scratch/yocto/sources/poky/oe-init-build-env build-qemux86! $ vi conf/local.conf! o Modify DL_DIR = "/scratch/yocto/downloads"! o Modify SSTATE_DIR = "/scratch/yocto/sstate-cache"! " Bblayers.conf: OK with the defaults, no addinonal layers necessary! Build your image using BitBake!$ bitbake core-image-minimal! 37

38 EXERCISE 4 Boo3ng your board 38

39 What board are you booting?!! You must follow the correct procedure based on the board you are using! This Lab supports " Beaglebone Black " Minnowboard Max " WandBoard " QEMU x86! Make sure you follow the correct set of steps for your board 39

40 Beaglebone Black Boot - Detailed!! Format blank SD Card for Beaglebone Black $ export DISK=/dev/sdc $ umount ${DISK}1 <<<Note the addition of the '1' $ sudo dd if=/dev/zero of=${disk} bs=512 count=20 $ sudo sfdisk --in-order --Linux --unit M ${DISK} <<- EOF 1,12,0xE,*,,,- EOF $ sudo mkfs.vfat -F 16 ${DISK}1 -n boot $ sudo mkfs.ext4 ${DISK}2 -L rootfs! Now unplug and replug your SD Card for automount $ cd <path-to-your-binary-images> $ sudo cp -v MLO-beaglebone /media/boot/mlo $ sudo cp -v u-boot.img /media/boot/ $ sudo tar xf core-image-base-beaglebone.tar.bz2 -C /media/rootfs $ sync (flush to device, not neccesary, but illustrative) $ umount /media/rootfs /media/boot 40

41 Beaglebone Black Boot - Detailed!! Copy sdcard image and boot " Use SD slot on cpu module on bokom of board! For convenience, we will only replace the root file system! On your laptop Linux, insert your microsd: " $ scp - P <port> ilab01@devdayn:/scratch/working/build- beaglebone/tmp/deploy/images/beaglebone/core- image- base- beagleboard.tar.bz2. " $ sudo rm - rf /media/roops/* " $ sudo tar xf core- image- base- beaglebone.tar.bz2 - C /media/roops " $ umount /media/roops 41

42 Minnowboard Max Boot Detailed (1)!! Note: Due to the size of the HDD Image, we offer two op3ons for upda3ng the image on your board. Chose only one of the following: " Copy the ennre image (.hddimg) to the microsd or USB flash drive, creanng a brand new image " Update only the roops image (much smaller download from cloud) 42

43 Minnowboard Max Boot Detailed (2)!! To build a brand new microsd or USB Flash drive image: " Download the.hddimg from your cloud- based host: o $ scp -P <port> ilab01@devdayn.yoctoproject.org:/ scratch/working/build-minnowmx/tmp/deploy/images/ intel-corei7-64/core-image-base-intelcorei7-64.hddimg. " On your laptop Linux, insert your microsd or USB flash drive: $ umount /media/boot!!$ dd if=core-image-base-intel-corei7-64.hddimg of=/ dev/sd(n) <<<replace with proper dev 43

44 Minnowboard Max Boot Detailed (3)!! To replace only the root file system image, do the following:! On your laptop Linux, insert your usb flash drive or microsd: $ scp -P <port> ilab01@devdayn.yoctoproject.org:/ scratch/working/build-minnowmx/tmp/deploy/images/intelcorei7-64/core-image-base-intel-corei7-64.ext3.! $ cp core-image-base-intelcorei7-64.ext3 /media/boot/rootfs.img! $ umount /media/boot! 44

45 Minnowboard Max Boot Detailed (4)!! Boo3ng the board " Insert the new MicroSD card Shell> connect -r! Shell> map -r! Shell> fs0:! Shell> bootx64!! That should get you to a Linux login prompt 45

46 WandBoard Boot - Detailed!! Prepare new SD Card for Wandboard " Insert SD card in Card Reader $ sudo dd if=core-image-basewandboard-quad.sdcard of=/dev/sdc Obviously, you'll need to verify your SD card device node 46

47 WandBoard Boot - Detailed!! For convenience, we will only replace the root file system on a pre- populated SD Card! On your laptop Linux, insert your microsd! It should auto- mount. If it doesn't this won't work $ scp -P <port> ilab01@devdayn.yoctoproject.org:/ scratch/working/build-wbquad/tmp/deploy/images/ wandboard-quad/core-image-base-wandboardquad.tar.bz2. $ sudo rm -rf /media/rootfs/* $ sudo tar xf core-image-base.tar.bz2 -C /media/ rootfs $ umount /media/rootfs 47

48 A Detailed Look at Layers!! Introduc3on to Layers! Stacking Customiza3ons! Adding Layers! Board Support Packages Patch Layers Developer Layers Commercial Layers UI layers meta- yocto meta- yocto- bsp oe- core (poky/meta) 48

49 Layers!! The Yocto Project build system is composed of layers " A layer is a logical collecnon of recipes represennng the core, a BSP, an applicanon stack, patches, or other funcnonality.! All layers have a priority and can override policy and config semngs of the layers of lower priority 49

50 Layer Hierarchy" Patch Layers Developer Layers Commercial Layers UI layers meta- yocto meta- yocto- bsp oe- core (poky/meta) 50

51 Using Layers!! Layers are added to your build by inser3ng them into the global BBLAYERS variable within your /conf/bblayers.conf file: BBLAYERS =!!!!! \! /<install-path>/poky/meta!!! \! /<install-path>/poky/meta-yocto! \! /<my-meta-dir>/meta-my-custom-layer \!! 51

52 Board Support Packages!! BSPs are layers to enable support for specific hardware planorms! Defines machine configura3on for the board! Adds machine- specific recipes and customiza3ons " Kernel config " Graphics drivers (e.g, Xorg) " AddiNonal recipes to support hardware features 52

53 Exercise 5: Create a Custom Layer!! When doing development with Yocto, never edit files within the Yocto/Poky source tree " Use a custom layer for modularity and maintainability! Create custom layers to hold a custom image recipe! Let's call this layer meta- ypdd! This layer must include: " Meta-ypdd/conf/layer.conf " Recipes directory (meta-ypdd/recipes-core/) " A meta- ypdd/readme file (basic documentanon for the layer, including maintainer info) 53

54 Exercise 5: Create a Custom Layer!! Yocto includes a helper script to create a new layer! It can also supply an example recipe or bbappends file " $ yocto-layer <new-layer>!! For this exercise, a layer is already provided " /scratch/sandbox/meta-ypdd! 54

55 meta-ypdd/conf/layer.conf! Must match layer name in three places 55

56 Exercise 5: Custom Layer!! Add this new layer to your project's bblayer.conf file " Note that this layer is already provided on your VM $ vi /conf/bblayers.conf! $ BBLAYERS?= " \!!!! /scratch/sandbox/meta-ypdd \!! Rebuild core- image- minimal: $ bitbake core-image-minimal! " NoNce the new login banner line!! 56

57 Exercise 6: Creating a Custom Image Recipe!! Derive from core- image- minimal " Add support for a graphical boot logo (via psplash) " Add support for SSH server (dropbear)! Use the layer we created in Exercise 5: meta- ypdd! We'll name our custom image ypdd-image! " The recipe will be named: meta-ypdd/recipes-core/images/ypdd-image.bb!! Simplest way to add packages to a predefined image " Append them to IMAGE_INSTALL within the image recipe 57

58 Exercise 6: Creating a Custom Image Recipe" 58

59 Exercise 7: Build and Boot Your Custom Image!! We added the new layer in Exercise 5 " /scratch/sandbox/meta-ypdd!! Now build your custom image: " $ bitbake ypdd-image! Boot the image using instruc3ons specific to your target board " Replace the roops contents on the SD/USB! If using QEMU target, simply do this: " $ runqemu qemux86 -nographic tmp/ deploy/images/ypdd-imageqemux86.ext3! 59

60 Common Gotchas When Getting Started!! Do not try to re- use the same shell environment when moving between copies of the build system! oe-init-build-env script appends to your $PATH, so repeated use in same shell is not recommended and may lead to unpredictable results! Do not try to share sstate- cache between hosts running different Linux distros even if they say it works ;) 60

61 Project Resources!! The Yocto Project is an open source project, and aims to deliver an open standard for the embedded Linux community and industry! Development is done in the open through public mailing lists: " openembedded- " " And public code repositories: " hkp://git.yoctoproject.org and " hkp://git.openembedded.net! Bug reports and feature requests " hkp://bugzilla.yoctoproject.org 61

62 Tip: ack-grep!! Much faster than grep for the relevant use cases! Designed for code search! Searches only relevant files " Knows about many types: C, asm, perl " By default, skips.git,.svn, etc. " Can be taught arbitrary types! Perfect for searching metadata " $ bback 62

63 Tip: ack-grep" alias bback='ack-grep --type bitbake'! 63

64 TIP: VIM Syntax Highlighting!! hrps://github.com/openembedded/bitbake/tree/master/contrib/vim " Install files from the above repo in ~/.vim/ " Add "syntax on" in ~/.vimrc $ tree ~/.vim/! /home/ilab01/.vim/! ftdetect! bitbake.vim! ftplugin! bitbake.vim! plugin! newbb.vim! syntax! bitbake.vim!! You do use VI, right? ;- ) 64

65 TIP: VIM Syntax Highlighting" 65

66 It s not an embedded Linux distribu2on It creates a custom one for you 66