A Smart Way to Manage Packages in Yocto Project

Transcription

1 A Smart Way to Manage Packages in Yocto Project Mar 11th, 2016 Fan Xin, Fujitsu Computer Technologies Limited 1378ka02

2 whoami Fan Xin, Fujitsu Computer Technologies Ltd. Embedded Linux Developer In-House Embedded Linux Distributor of Fujitsu Our Distribution includes LTSI Kernel and is built with Yocto Project Our Distribution is used for IVI, Server System Controller, Storage System, Network Equipment, Printer, etc. IVI:In-Vehicle Infotainment 1

3 Fujitsu s contribution for Yocto Project meta-openembedded.git top contributors by employer ( to ) None 625 (46.1%) Wind River 309 (22.8%) Fujitsu 160 (11.8%) Intel 82 (6.0%) Mentor Graphics 34 (2.5%) O.S. Systems 24 (1.8%) National Instruments 19 (1.4%) MontaVista 16 (1.2%) OpenDreambox 12 (0.9%) Freescale 9 (0.7%) Linaro 8 (0.6%) ENEA AB 5 (0.4%) Leica 5 (0.4%) Aker Security Solutions 4 (0.3%) BMW 3 (0.2%) 2

4 Agenda Package Manager Comparison Package Management Problem Analysis in Yocto Project Introduction of Smart Package Manager Fujitsu s Contribution and Next Step 3

5 Package Manager Comparison 4

6 Package Manager Package Manager A collection of software tools that automates the process of installing, upgrading, configuring, and removing computer programs Deals with packages, distributions of software and data in archive files Maintain a database of software dependencies and package information Common Package Management System rpm, deb, ipkg, opkg base of package manager Common Package Managers Advanced Packaging Tool (APT) Yellowdog Updater Modified (YUM) Dandified Yum (DNF) 5

7 Advanced Packaging Tool (APT) APT ( Advanced Packaging Tool ) The Advanced Package Tool, or APT, is a free software user interface that works with core libraries to handle the installation and removal of software Used on Debian Linux distribution and its variants. Depends on deb package format Works with repositories, which are collections of packages, 6

8 Yellowdog Updater Modified (YUM) A package manager for automatic install, updates, uninstall package and dependency management. Originally developed to manage Red Hat Linux systems at Duke University's Physics department Used on RPM-based distributions, such as RedHat Yum depends on RPM Works with repositories, which are collections of packages, 7

9 Dandified Yum (DNF) Next generation version of the Yellowdog Updater Modified (yum) Using RPM, libsolv and hawkey libraries Introduced in Fedora 18 and become the default package manager for Fedora since version

10 Comparison of Package Managers Items Apt Yum DNF Latest Version Support Package Format.deb.rpm (apt-rpm) Distribution Debian Ubuntu RHEL CentOS Fedora OpenSUSE Fedora (Since version 22) License GNU GPL GPL v2 GPL v2 Website yum.baseurl.org dnf.baseurl.org

11 Comparison of Package Management Systems Items rpm deb opkg ipkg Check Package License Information File format Used by OpenEmbedded.rpm.deb.udeb Develop Status Active Active.ipk Active ( fork from ipkg ).ipk Discontinued License GPL GPL GPL v2 GPL v2 Website g git.yoctoprojec t.org/cgit/cgit.c gi/opkg/ s.org/moin/moi n.cgi/ipkg

12 Present Situation in Yocto Project Recipe files Build Build Build Build Packages Deploy Deploy Deploy Deploy Embedded Devices Flexibility Flex to select package before building Efficiency Create root file system, binary packages (rpm, ipkg, opkg) Optimization Easy to optimize for each distribution 11

13 Package Management Problem Analysis in Yocto Project 12

14 Problems Yocto creates a custom one for you The Advantage of custom embedded Linux The value of custom The balance between custom and minimize fragmentation 13

15 Problems Can not know what package is built due to the packages dependence. Have to rebuild the root file system every time when have some updates or modifications for some packages. Produce fragmentation Flexibility : Prefer graphical interface and detailed info Efficiency : Inefficiency, in some degree Optimization : Produce fragmentation 14

16 Image Build Deploy Build Deploy Build Deploy Build Deploy Build and Deploy for Each Targets These cars use same architecture and own individual build directory. 15

17 Image Build Deploy Build Deploy Build Deploy Build Deploy Build and Deploy for Each Targets 16

18 Image Build Deploy Build Deploy Build Deploy Build Deploy Build and Deploy for Each Targets Using Package Manager Build Deploy Build Once, Deploy Anywhere Simulation Full build time: 5~6 hours Deploy time: 1 hour Target Type: 50 Before: Cost = (5+1)*50=300 hours After: Cost = 5+1*50= 55 hours 17

19 Cost Comparison Build glibc package and deploy for car A Build glibc package and deploy for car B Build glibc package and deploy for car C Build glibc package and deploy for car A, B, C Feb 17 th, glibc publics critical security flaw. This vulnerability should be fixed in all targets immediately. For each kind of target, the glibc package has to been builded and deployed. Simulation Cost = ( Build Time + Deploy Time ) * Device Number = ( ) * 50 = 60 hours Cost = Build Time + Deploy Time * Device Number = * 50 = 11 hours Glibc Build Time: 1 hours Deploy Time: 0.2 hours Device Number: 50 kinds 18

20 What we should do Rebuild the root file system and redeploy them into production every time? Use the package manager to manage individual package efficiently? 19

21 Introduction of Smart Package Manager 20

22 Smart Package Manager Overview Aim to create smart and portable algorithms for solving adequately the problem of managing software upgrading and installation. Homepage: smartpm.github.io/smart License: GPL v2 Works with APT, APT-RPM, YUM, etc. Started on May 4 th, 2004, and version 1.0 was released on Aug 14 th, 2008 The latest version is 1.5 Merged into Yocto Project 1.4 (dylan), but unavailable in Yocto Project so far

23 Features Modular Support PRM, DPKG, and Slackware package management systems Smart Transactions Thousands of packages and relations are being considered Not only find a solution, but find the best solution (install, remove, upgrade, etc). Channels Many different channel types are supported Priority Handing Hand integration of multiple channels and setup preferred package versions Downloading Mechanism Fast parallel downloading mechanism, multiple connections 22

24 Smart Usage Smart package manager uses the following command to manage packages. Usage: smart command Action commands update, install, reinstall, upgrade, remove, check, fix, download, clean Query commands search, query, newer, info, stats Setup commands config, channel, priority, mirror flag Run smart command help for more information 23

25 Project Statistics Statistics about Commit change by year month Commit change by year, author Commit change by author File change by year Tag Name Date Commits Copyright FUJITSU COMPUTER TECHNOLOGIES LIMITED

26 Problems Smart still has some problems needed to solve before using it. Problem #1 There is bug in the rpm and smart command in default toolchain. Send the patch to community to fix it Problem #2 Need to configure the environment before using smart, or it can not use normally Prepare the script to configure environment automatically 25

27 Problems Problem #3 Smart Package Manager uses GTK/Qt4 as the graphical framework, but GTK/Qt4 is disabled in toolchain Develop Text GUI with python to make smart user-friendly and easy to use 26

28 Smart Package Manager Usage Install toolchain./libc-x86-meta-toolchain-i586-toolchain.sh Initial environment. /opt/environment-setup-core2-64-linux Install package into root file system Command line interface Graphical interface We use this script to finish the environment initialization, such as environment variable setting smart --data-dir =${rootfs-dir}/var/lib/smart install smart --data-dir=${rootfs-dir}/var/lib/smart --interface=tgui 27

29 Select install type 28

30 Select packages 29

31 Search package 30

32 Select dbg packages 31

33 Installing 32

34 Created Root Filesystem 33

35 Demo Smart Package Manager 34

36 Conclusion and Next Step 35

37 License Management Packages Package Manager Recipe files Build SRPM packages Embedded Devices SPDX files Accompanied with the package files, SRPM packages and SPDX files are created to manage license information SRPM Packages are classified by license type 36

38 Community Activity Send our patches to community, but not accepted In order to solve the problem, need to active the community again 37

39 Conclusion Summary the current package managers and problems Compare the package managers Introduce package management problem in Yocto Project The package manager is necessary Contribution to Ycoto Project Introduce the Smart Package Manager Fix the bugs and make Smart Package Manager easy to use License management Not stick to Smart Package Manager, but want to solve package management problem 38

40 Next Step Continue to make contribution for Yocto Project The interface of Smart is not so smart currently. Fujitsu will improve the interface to make it easier to use. Remote access over a network connection like APT, YUM and DNF Add the option to create image automatically 39

41 Copyright 2015 FUJITSU COMPUTER TECHNOLOGIES LIMITED