path: root/src/devel/1_preparing.xml

<?xml version="1.0" encoding="utf-8"?>
<chapter label="1" id="preparing">
    <title>Preparing Your System</title>
    <para>Welcome to the Adélie Linux 1.0 Developer's Handbook.  This handbook will teach you how to develop the software and packages that powers the Adélie Linux distribution.  You may follow along at your own pace.</para>
    <para>We'll start with installing the software and writing the configuration files necessary to perform development.</para>
    <section>
        <title>Installing Development Software</title>
        <para>In this section, you will learn:</para>
        <itemizedlist>
            <listitem><para>How to install software on Adélie Linux;</para></listitem>
            <listitem><para>What software you will use to develop Adélie Linux; and</para></listitem>
            <listitem><para>What task each utility in the Adélie Linux development system performs.</para></listitem>
        </itemizedlist>
        <para>Please make sure that you are at an Adélie Linux terminal before continuing.</para>
        <section>
            <title>Using <command>apk</command> to Install Software</title>
            <para>The <productname>Alpine Package Keeper</productname> or <command>apk</command> package manager is the central software management tool on the Adélie Linux distribution.
                For a more thorough introduction to the <command>apk</command> package manager, please consult the Adélie Linux 1.0 Administrator's Guide, chapter 3: "Packages".</para>
            <para>The <command>apk</command> package manager automatically resolves dependencies for requested software packages, and will automatically remove or "clean up" unneeded dependencies when a software package is removed.
                If a software package's files become corrupted, it can be reinstalled with the <parameter class="command">fix</parameter> subcommand.</para>
            <para>The <parameter class="command">add</parameter> subcommand will add the specified package(s) to the system's list of packages, and then resolve all necessary dependent libraries and other software packages to ensure the system has the requested packages installed.
                The <parameter class="command">del</parameter> subcommand will remove the specified package(s) from the system's list of packages, and then remove any other libraries and software packages that are no longer required for the system's operation.  For example, the command:</para>
            <para><prompt>#</prompt> <command>apk <parameter>add</parameter> <userinput>build-tools</userinput></command></para>
            <para>will install the <userinput>build-tools</userinput> package, and any packages and libraries it requires.  Similarly,</para>
            <para><prompt>#</prompt> <command>apk <parameter>del</parameter> <userinput>libressl-dev</userinput></command></para>
            <para>will remove the <userinput>libressl-dev</userinput> package, and any packages and libraries that it required and no other package on the system requires.</para>
        </section>
        <section>
            <title>Installing the Adélie Linux Development System</title>
            <para>The development environment is made of two essential parts: the <package>build-tools</package> package, which provides the basic compiler infrastructure, including a C compiler, build system, linker, assembler, and other utilities; and the <package>abuild</package> package, which takes package "recipe" files with build instructions, and creates the <filename class="extension">APK</filename> package files using them.  You may install these two packages by executing the following command:</para>
            <para><prompt>#</prompt> <command>apk <parameter>add</parameter> <package>abuild</package> <package>build-tools</package></command></para>
        </section>
        <section>
            <title>Components of the Adélie Linux Development System</title>
            <para>The development system uses many open-source technologies that work together to provide a responsive, cohesive, libre environment.</para>
            <variablelist>
                <title>Development System Components</title>
                <varlistentry>
                    <term><package>abuild</package> (<command>abuild</command>)</term>
                    <listitem><para>Builds <filename class="extension">APK</filename> package files from <filename>APKBUILD</filename> package "recipe" files.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>autoconf</package> (<command>autoconf</command>)</term>
                    <listitem><para>The <orgname>GNU</orgname> package for generating <filename>./configure</filename> scripts in packages that require them to be generated.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>automake</package> (<command>automake</command>)</term>
                    <listitem><para>The <orgname>GNU</orgname> package for generating <filename>Makefile</filename>s in packages that use the <package>autoconf</package> package.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>attr</package></term>
                    <listitem><para>A framework for <firstterm>ACL</firstterm>s or <firstterm>access control list</firstterm>s.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>bash</package> (<command>bash</command>, <command>sh</command>)</term>
                    <listitem><para>A Unix shell mostly compliant with the <productname>POSIX</productname> standard, used by many packages for build scripts and testing.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>binutils</package> (<command>as</command>, <command>ld</command>, <command>nm</command>, …)</term>
                    <listitem><para>Low-level development utilities including assembler (<command>as</command>), linker (<command>ld</command>), and others.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>bison</package> (<command>yacc</command>)</term>
                    <listitem><para>The <orgname>GNU</orgname> version of the <command>yacc</command> compiler generation command.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>bzip2</package> (<command>bzip2</command>, <command>bunzip2</command>, <command>bzless</command>, …)</term>
                    <listitem><para>A popular compression format, included for the ability to unpack <filename class="extension">.bz2</filename> archives within which packages may be distributed.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>cmake</package> (<command>cmake</command>, <command>ctest</command>)</term>
                    <listitem><para>The CMake build system, used to build many packages, and the CTest test system, used to prove packages are functioning correctly on the local system.</para></listitem>
                </varlistentry>
                 <varlistentry>
                    <term><package>diffutils</package> (<command>diff</command>, <command>cmp</command>)</term>
                    <listitem><para>Compares two or more files together, showing the differences.  Used by many test suites, and also useful for troubleshooting and generating <filename class="extension">.patch</filename> files.</para></listitem>
                </varlistentry>
               <varlistentry>
                    <term><package>fakeroot</package></term>
                    <listitem><para>An environment where a package believes it is running as the root user, but is actually running as the build user.  <!-- Why is this necessary? --></para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>flex</package> (<command>lex</command>)</term>
                    <listitem><para>The <orgname>GNU</orgname> version of the <command>lex</command> lexer, used for creating parsers in many projects.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>gcc</package> (<command>c99</command>, <command>c11</command>, <command>gcc</command>, <command>g++</command>)</term>
                    <listitem><para>The <orgname>GNU</orgname> Compiler Collection, a mostly standards-compliant C and C++ compiler.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>git</package> (<command>git</command>)</term>
                    <listitem><para>A distributed <firstterm>version control system</firstterm> used by many projects to keep a record of file history; the current distribution method for Adélie's package repository.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>gzip</package> (<command>gzip</command>, <command>gunzip</command>)</term>
                    <listitem><para>A popular, long-standing compression format, included for the ability to unpack <filename class="extension">.gz</filename> archives within which packages may be distributed.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>libarchive</package> (<command>cpio</command>, <command>tar</command>)</term>
                    <listitem><para><command>tar</command> allows creation of APK files and extraction of a popular archive format; <command>cpio</command> is used to create the initial boot disk for CD images.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>libtool</package> (<command>libtool</command>)</term>
                    <listitem><para>The <orgname>GNU</orgname> library manipulation tool, used by many packages to create shared (<filename class="extension">.so</filename>) and static (<filename class="extension">.a</filename>) library files.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>lzip</package> (<command>lzip</command>)</term>
                    <listitem><para>A compression format, included for the ability to unpack <filename class="extension">.lz</filename> archives within which packages may be distributed.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>make</package> (<command>make</command>)</term>
                    <listitem><para>The <orgname>GNU</orgname> version of the standard Unix <command>make</command> command.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>mawk</package> (<command>awk</command>)</term>
                    <listitem><para>A <productname>POSIX</productname> compliant implementation of the AWK programming language.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>openssl</package> (<command>openssl</command>)</term>
                    <listitem><para>A command to sign packages cryptographically, ensuring they are not tampered with when a user downloads them.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>patch</package> (<command>patch</command>)</term>
                    <listitem><para>The <orgname>GNU</orgname> version of <personname><firstname>Larry</firstname><surname>Wall</surname></personname>'s standard <command>patch</command>utility.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>pax-utils</package> (<command>dumpelf</command>, <command>symtree</command>)</term>
                    <listitem><para>A collection of utilities to enable the building of packages for the PaX secure operating environment.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>pkgconf</package> (<command>pkgconf</command>, <command>pkg-config</command>)</term>
                    <listitem><para>A freely licensed, mathematically proven <command>pkg-config</command> implementation, used by package build systems to determine the location of libraries on the system.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>sed</package> (<command>sed</command>)</term>
                    <listitem><para>The <orgname>GNU</orgname> version of the Unix <command>sed</command> command.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>sudo</package> (<command>sudo</command>)</term>
                    <listitem><para>Allows a user to run a command as the root user without using the root user's password.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><package>xz</package> (<command>xz</command>, <command>unxz</command>)</term>
                    <listitem><para>An efficient compression format, included for the ability to unpack <filename class="extension">.xz</filename> archives within which packages may be distributed.</para></listitem>
                </varlistentry>
            </variablelist>
        </section>
    </section>
    <section>
        <title>Packaging Primer</title>
        <para>In this section, you will learn:</para>
        <itemizedlist>
            <listitem><para>How repositories and packages work in the Adélie Linux distribution;</para></listitem>
            <listitem><para>How to use the <command>abuild</command> command; and</para></listitem>
            <listitem><para>The options and commands available in <command>abuild</command>.</para></listitem>
        </itemizedlist>
        <section>
            <title>Packages and Repositories</title>
            <para>The <command>abuild</command> command enables you to build a package file for later installation on to one or more computers running the Adélie Linux dsitribution.  A <firstterm>package file</firstterm> is a kind of archive file that contains two key parts: package metadata, and package content files.</para>
            <para><firstterm>Package metadata</firstterm> is a collection of information used by a package manager, such as <command>apk</command>, to determine information about the package and the package file, including its name, version, maintainer contact information, installed size on disk, and more.  The <firstterm>package content files</firstterm> are a collection of one or more files that comprise the package itself, including executables, default configuration, data files, and any other files needed for the package to work properly.  In some cases it may also include templates, examples, or documentation.</para>
            <para>All packages belong to a <firstterm>package repository</firstterm> or <firstterm>repo</firstterm>.  A repository is a collection of related packages.  The Adélie Linux distribution has two main repositories: the <database>system</database> repository, used for system software and important libraries; and the <database>user</database> repository, used for the majority of packages that users may be interested in.
                While you are developing on your own computer, you may have as many, or as few, repositories as you like.  It can use any name except the above two ('system' and 'user').  You may use a name such as <database>mypackages</database>, or just your username.  The repository is a directory on your computer's hard disk.  You may create it anywhere, but it is highly recommended that you store your repository as a subdirectory of your home directory (<filename class="directory">/home/<userinput>yourname</userinput>/<userinput>yourrepository</userinput></filename>).</para>
            <para>Each package you create will have its own directory inside your repository.  If you use the <command>newapkbuild</command> command (discussed later), this will be created for you automatically.  Inside this directory will be at least an <filename>APKBUILD</filename> file, which tells <command>abuild</command> the commands to run to build the package.  It may also contain patches or other files necessary for building the package, if necessary.</para>
            <para>Typically, the <command>abuild</command> command is executed inside the package's directory (for example, <filename class="directory">/home/<userinput>yourname</userinput>/packages/<userinput>pkgname</userinput></filename>).  However, by setting the <envar>APKBUILD</envar> environment variable to the absolute path on disk to an <filename>APKBUILD</filename> file, you may run <command>abuild</command> from any directory.</para>
        </section>
        <section>
            <title>Using the <command>abuild</command> Command</title>
            <para>Running the <command>abuild</command> command with no arguments in a directory containing an <filename>APKBUILD</filename> file will cause it to fetch the package's sources (if necessary), build the package, and create the package file.  Passing the <parameter>-r</parameter> parameter to <command>abuild</command> will cause it to install the dependencies of the package as listed in the <filename>APKBUILD</filename>, and then build the package.  Afterwards, the dependencies of the package that were not already installed on your computer are removed.
                If the build concluded successfully, the directory tree created for the source code and build system are removed by default.  You may configure what <command>abuild</command> removes when a package concludes building.
                If the build fails to conclude successfully, no directories are removed by default.  You may configure what <command>abuild</command> removes when a package fails to build.</para>
            <para><command>abuild</command> has many subcommands that you may use, for finer-grained control over the build process.  You may use the <command>abuild <parameter class="command">deps</parameter></command> subcommand to only install the dependencies of the package, and use the <command>abuild <parameter class="command">unpack</parameter></command> subcommand to unpack the source archive into the directory tree where it can later be built.  These and many other subcommands are described in more detail in the following section.</para>
            <para>There are additionally a number of options that you may pass to <command>abuild</command>, which are also described in more detail in the next section.</para>
        </section>
        <section>
            <title><command>abuild</command> Subcommands</title>
            <para>The <command>abuild</command> command provides many subcommands that you may use while packaging software.</para>
            <variablelist>
                <varlistentry>
                    <term><command>checksum</command></term>
                    <listitem><para>This subcommand instructs <command>abuild</command> to recreate the checksum lines at the end of the <filename>APKBUILD</filename>.  This is typically used when "bumping" or upgrading the software package (which causes it to have new source code).  It is also used when creating, modifying, or removing patches from the package.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><command>fetch</command></term>
                    <listitem><para>This subcommand instructs <command>abuild</command> to fetch external source(s) specified in <varname>source</varname> to the <varname>SRCDEST</varname> specified in your <filename>abuild.conf</filename>, and ensure they match the checksum(s) specified in the <filename>APKBUILD</filename>.  The default <varname>SRCDEST</varname> is <filename class="directory">/var/cache/distfiles</filename>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><command>unpack</command></term>
                    <listitem><para>This subcommand instructs <command>abuild</command> to unpack the archive files specified in <varname>source</varname> to the source directory.  For instance, if <varname>source</varname> contained "https://my-package.org/foobar-1.2.zip", the contents of "foobar-1.2.zip" would be extracted to the source directory.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><command>prepare</command></term>
                    <listitem><para>This subcommand instructs <command>abuild</command> to run the <function>prepare()</function> function from the <filename>APKBUILD</filename>.  If no <function>prepare()</function> is provided in the <filename>APKBUILD</filename>, <command>abuild</command> will run a default <function>prepare()</function> which will apply all patches specified in <varname>source</varname> to the source tree.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><command>build</command></term>
                    <listitem><para>This subcommand instructs <command>abuild</command> to run the <function>build()</function> function from the <filename>APKBUILD</filename>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><command>package</command></term>
                    <listitem><para>This subcommand instructs <command>abuild</command> to run the <function>package()</function> function from the <filename>APKBUILD</filename>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><command>rootpkg</command></term>
                    <listitem><para>This subcommand instructs <command>abuild</command> to run the <function>package()</function> function, any split functions, and then create the APK files.</para></listitem>
                </varlistentry>
            </variablelist>
        </section>
    </section>
    <section>
        <title>Configuring Your System</title>
        <para>In this section, you will learn:</para>
        <itemizedlist>
            <listitem><para>How to configure <command>abuild</command> for your environment;</para></listitem>
            <listitem><para>How to create and use package signing keys; and</para></listitem>
            <listitem><para>The directory layout used by <command>abuild</command> and how to create it.</para></listitem>
        </itemizedlist>
        <para>Please make sure that you are at an Adélie Linux terminal before continuing.</para>
        <section>
            <title>Configuring <command>abuild</command> for Your Environment</title>
            <para>The <command>abuild</command> system reads its configuration from two files on your system, if they exist.
                The <filename>/etc/abuild.conf</filename> file contains system-wide configuration, used by all users on a system.
                The <filename><envar>$HOME</envar>/.abuild/abuild.conf</filename> contains user-specific configuration, including the packager name and key pair path.  This allows multiple users to create their own repositories on a single system.
                Configuration set in the <filename>/etc/abuild.conf</filename> will apply to all packages on the system unless overridden by the user-specific configuration file.</para>
            <para>The most common configuration variables are explained below.  For a list of all available configuration variables, consult the <filename>abuild.conf(5)</filename> manual page by running <command>man 5 abuild.conf</command> on a computer where the <package>abuild-doc</package> package is installed.</para>
            <variablelist>
                <title><filename>abuild.conf</filename> Configuration Variables</title>
                <varlistentry>
                    <term><envar>CFLAGS</envar></term>
                    <listitem><para><envar>CFLAGS</envar> contains the flags passed to the C compiler.  For more information about available flags, consult the documentation for your compiler.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>CXXFLAGS</envar></term>
                    <listitem><para><envar>CXXFLAGS</envar> contains the flags passed to the C++ compiler.  If <envar>CXXFLAGS</envar> is not set, and <envar>CFLAGS</envar> is set, then <envar>CXXFLAGS</envar> is set to the value of <envar>CFLAGS</envar>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>CPPFLAGS</envar></term>
                    <listitem><para><envar>CPPFLAGS</envar> contains the flags passed to the C and C++ preprocessors.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>LDFLAGS</envar></term>
                    <listitem><para><envar>LDFLAGS</envar> contains the flags passed to the linker.  For more information about available flags, consult the documentation for your toolchain (likely <package>binutils</package>).</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>JOBS</envar></term>
                    <listitem><para><envar>JOBS</envar> should be set to the number of concurrent build processes you would like to run on your system.  Most dedicated build servers use a number equal to the number of CPU cores.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>MAKEFLAGS</envar></term>
                    <listitem><para><envar>MAKEFLAGS</envar> contains the flags passed to <command>make</command>.  The default value is <literal>-j$JOBS</literal>.  For more information about available flags, consult the documentation for <command>make</command>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>USE_COLORS</envar></term>
                    <listitem><para><envar>USE_COLORS</envar> controls whether or not the <command>abuild</command> command will output colours to the terminal.  If you prefer a colourless output, or your terminal is not capable of ANSI colour codes, set this to <literal>0</literal>.  The default value is <literal>1</literal>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>SRCDEST</envar></term>
                    <listitem><para><envar>SRCDEST</envar> should be an absolute path where source files are downloaded.  The default, and typical, value is <filename class="directory">/var/cache/distfiles</filename>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>REPODEST</envar></term>
                    <listitem><para><envar>REPODEST</envar> should be an absolute path where finished packages are placed when built.  The default is <filename class="directory"><envar>$HOME</envar>/packages/</filename>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>PACKAGER</envar></term>
                    <listitem><para>The value of <envar>PACKAGER</envar> is added to every produced package as the person responsible for packaging it.  Typically, this follows an RFC822-style "Name &lt;Email Address&gt;" format.  There is no default value; it must be set before running <command>abuild</command>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>PACKAGER_PRIVKEY</envar></term>
                    <listitem><para><envar>PACKAGER_PRIVKEY</envar> should be an absolute path to the packager's private key, which will be used to sign every produced package.  Typically, this is set for you by the <command>abuild-keygen</command> command.  There is no default value; it must be set before running <command>abuild</command>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>MAINTAINER</envar></term>
                    <listitem><para>The value of <envar>MAINTAINER</envar> is used as a point of contact if a user has an issue with a package, or if a security patch or update is available for the patch.  Typically, this follows an RFC822-style "Name &lt;Email Address&gt;" format.  The default value is the value of <envar>PACKAGER</envar>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>CLEANUP</envar></term>
                    <listitem><para><envar>CLEANUP</envar> controls the cleanup procedure after a build that concludes with success.  The default value is <literal>srcdir pkgdir deps</literal>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>ERROR_CLEANUP</envar></term>
                    <listitem><para><envar>ERROR_CLEANUP</envar> controls the cleanup procedure after a build that concludes with failure.  The default value is <literal>deps</literal>.</para></listitem>
                </varlistentry>
                <varlistentry>
                    <term><envar>DEFAULT_DBG</envar></term>
                    <listitem><para><envar>DEFAULT_DBG</envar> controls whether a debug package will be created for compiled packages.  It is unset by default; being set to any value enables the option.  The typical value to enable the option is <literal>YesPlease</literal>.</para></listitem>
                </varlistentry>
            </variablelist>
        </section>
    </section>
</chapter>