DRU Disc Remastering Utility

From DellLinuxWiki

Contents 1 Summary 2 Documentation 3 Dependencies 3.1 Current Dependencies 3.2 Suggested Packages 4 Installation 5 Quick Start 6 Usage 7 Automated Customization Basics 8 Automated Customization Development 9 Command List 10 Option List 11 Framework 12 Difference Directory 13 The dru-framework Directory Layout 14 Development Notes 15 To Do

Summary

The Disc Remastering Utility (DRU) was designed to apply modifications to Ubuntu distributions for Dell systems. The resulting images are to be posted to the public, so that customers have the best possible experience when installing.

DRU is a utility for modifying and creating custom releases of Ubuntu. We have two different types of users in mind. Individual users who want to simply make a custom ISO or bootable flash device and users what want to actually remaster an ISO for some type of distributing. Individual users will most likely want to use the quick and simple GUI.

Please note that this page currently has information for both DRU and DRUdell, together.

Documentation

This Wiki goes over some of the basics to get started, some light development notes, and DRUdell usage notes. For a much more robust documentation, see the manual pages for dru, dru-framework, dru-diff, and drudell.

The manual pages can be found in PDF form for DRU [1] and DRUdell [2].

Information on the result images for the public can be found at Ubuntu_Dell_Install_Media.

Dependencies

Current Dependencies

• root access
• Ubuntu box
• squashfs-tools
• rsync
• dpkg-dev
• debhelper

Suggested Packages

• mercurial (>= 0.9.4)
• syslinux
• mtools
• build-essential
• docbook2x
• imagemagick
• dialog

Installation

Installation can be done automatically by pasting the following command into a terminal:

wget -q -O- http://linux.dell.com/dru/install.sh | bash

NOTE: This also installs DRUdell as the dell subdirectory.

If you prefer a more interactive approach, try the following:

sudo apt-get install dialog
wget -q -O- http://linux.dell.com/dru/install-gui.sh | bash

The above commands require sudo privileges, along with DRU itself to be used.

The latest copy of DRU can be installed using:

Quick Start

1. man dru Is a good starting place to read.
2. Automatically create a new ISO by applying changes from drudell.

sudo ./dru --diff=dell --iso=ubuntu-*-desktop-i386.iso auto

1. Start an interactive GUI walk-through for basic operations.

./dru

Usage

1. Set up working directory
   1. Create a working directory on an Ubuntu system.
   2. In this directory, install the dru scripts and framework, and the source ISO file to modify (the source ISO can be from ubuntu.com).
   3. You can put the ISO anywhere. I tend to keep my ISOs in a separate subdirectory.
2. Initialize the working directory for modifying an image.

./dru -i=original-image.iso init

1. You should now have an ISO mount directory, a new directory, and a casper directory.
   1. ISO mount directory: A R/O, as it is a live mount of the source ISO.
   2. new: A R/W copy of the ISO, with select files missing.
   3. casper: A R/W copy of the Casper file system, which is rooted in the live CD environment.
2. Make modifications
   1. Apply a set of changes.

./dru -d=diff-directory apply

1. 1. Manually modify the content.

./dru shell

1. Create new.iso, which will use the R/W content as the source.

./dru build-iso

Automated Customization Basics

If you wish to use a previous customization set, download the latest image modification tar, and install it into the same directory as the dru script.

If your directory is not clean already, you may wish to start with a fresh copy by:

1. Make sure any changes to new, and casper are saved if you want to keep them, as they will be discarded for fresh copies.
2. Run ./dru mrclean

To simply apply the modifications from the scripts directory (supplied by the modification tar), run ./dru to use the GUI. After running this, you should end up with the build folders new and casper, plus a new.iso file.

Automated Customization Development

The DRU system consists of three basic elements:

1. DRU itself The dru script manages the build environment, calls the framework, generates the results, and handles user interaction.
2. Framework This contains any scripts and hooks which are injected into the target ISO, and part of the scripts to target specific image types. This is designed to be interchangeable for different distributions, but is very behind in being operational as such.
3. Difference Directories This holds the OEM specific changes which are applied by the framework.

As in the basics above, you may want to run mrclean first. Also, it may be wise to start with a pre-existing difference directory, as they have a lot of work in them. The DRU script provides the skel command to start out with a basic skeleton difference directory.

Command List

The following commands are available for more advanced usage, when the GUI isn't enough. Please read any manual pages for more information.

 auto
    Automatically converts the source ISO into the resulting ISO, with zero user interaction. Requires a
    valid diff directory. This is the same as calling the targets: init apply build-iso, for convenience.
 init
    Initialize the build environment by mounting the original ISO, copying the contents to a writable
    folder, and decompressing the Casper filesystem. This command requires the --iso option to
    specify the source.
    Important: It is important to use the desktop Ubuntu ISO, which is based on Casper.
 shell
     Starts a bash shell within the Casper filesystem, emulating the Live CD environment on the
     developer machine. Changes made persist onto the result filesystem. This is useful for making
     manual adjustments to the end-user experience.
           Note: Changes made will (almost always) also persist onto the installed image. This is due to
           Ubiquity using the Casper filesystem as a base image.
 apply
     Applies the diff directory, specified by the --diff option, to the current build environment. apply
     installs the DRU framework inside the Casper and ISO environments if it has not been done yet.
     Installation of this framework is a prerequisite of any diff directory.
 build-iso
     Generates a bootable ISO file from the current build environment. The result is a Joliet filesystem
     with an isolinux bootloader. Note the last line of this command, to tell if it can fit on a CD-ROM.
 build-vfat
     Generates a directory which is intended to be copied to the first partition of a device, for use with
     syslinux. This enables most media other than optial to be booted, given the proper amount of space.
     See format-vfat.
 format-vfat
     Automatically formats a device, and installs a bootable VFAT partition with syslinux, and an
     optional syslinux MBR. This is useful for creating USB keys. It is not necessary to call build-vfat
     first, as this command directly targets the device without buffering.
 clean
     Clean the build environment from stale mounts and DRU files. This will not remove any progress.
     This is useful after using mount to mount the original ISO, before continuing.
 mrclean
     Cleans the build environment with loss of progress.
 mount
     Mounts the original ISO. This implies the --leave-mounts option, to leave the path mounted
     after exiting. This must be unmounted before running most other commands, for safety purposes.

Option List

Each command may require specific parameters, or provide optional parameters. These are provided by prepending any of the following options.

 -m, --leave-mounts
    Keep mounts active after dru exits.
 -s, --skip-casper
    Skip building Casper filesystem.
 -I, --skip-initrd
    Skip building a new initrd with initramfs.
 -n, --no-rebuild
    Skip entire build process, and just copy files.
 --no-mbr
    When formatting a device, this forces no MBR mode. If a device refuses to boot ("Boot error"), use
    this flag. You cannot use the --repartition or --linux-part options with this flag.
 -l, --linux-part
    When formatting device, create a second primary partition as EXT2. This requires the
    --repartition option.
 -p, --repartition
    When formatting a device, create a new partition table.
 -D, --dev=devfile
    Specify a target device. This is required for any formatting targets.
 -d, --diff=diffpath
    Specify the DIFF directory to use.
 -f, --framework=frameworkpath
    Specify an alternate Framework to use for the apply command.
 -i, --iso=isofile
    Specify a source ISO image to use. This is required by the init command.
 -t, --title=title
    Set the title for the target volume.
 --log=logfile
    Set the output log file to use. Default is dru.log.
 --debug
    Sets more verbose output into log file.
 --debug-diff
    Modifies the source code of scripts which are executed on the host machine, so that they are
    extremely verbose. The target log files include /var/log/casper.log, /var/log/dru.log, /var/log/dru-update.log, and /var/log/syslog
    on the client.
 --force
    Assume yes for any prompts. This is potentially unsafe when used in combination with a format
    command.
 -h, --help
    Displays a short help message.

Framework

The DRU framework directory is (supposed to be) the modular heart of the apply command in dru. It produces hooks in the resulting image, and applies the modifications listed in the difference directory. The hooks then get executed at the proper time to incur dynamic changes, based on criteria configured in the difference directory.

Difference Directory

The difference directory is the location of all the package, initramfs, and script differences to be applied to the base Linux image. Each difference directory has a certain expected layout required by DRU for proper execution. Extra files may be included around the expected layout for customization. The expected files are described below.

Difference directories require a specific layout as described in this manual page. You can create a directory layout by using:

./dru --diff=new_diff skel

The dru-framework Directory Layout

• dru-framework/rules
  • Executes ISO and live modifiers when building ISO. Used as event handler for dru script.
• dru-framework/casper.sh
  • Copies Casper files directly into Casper FS
  • Installs main debs directly into Casper FS
  • Installs live-only debs directly into Casper FS
  • Generates new filesystem.manifest and filesystem.manifest-desktop, based on Casper-only and main debs list, plus original file system
• dru-framework/dru/
  • Contains source for the DRU package which is installed on the Live environment.
  • This package manages everything past the changes in dru-framework/casper.sh.
  • Note that the files installed by Makefile in this directory are automatically removed from the target filesystem by Ubiquity. These should only contain hooks, and never persistent files.
• dru-framework/dru/debian/
  • Contains the packaging spec files for the DRU package.
  • Creates init.d file which executes scripts/bootstrap.sh on Live boot.
• dru-framework/dru/init-hook
  • Runs when mkinitramfs is executed with the developer machine or in the live environment.
• dru-framework/dru/casper-script
  • Runs at bottom of Casper bootup in developer and live environments.
  • Executes dru-framework/dru/scripts/bootstrap.sh rooted in /root, which is the target live file system for Casper.
• dru-framework/scripts/bootstrap.sh
• Executes the system seeds and scripts.
  • Determines current system required setup scripts, and executes them
• dru-framework/ubiquity-hook
  • Executes scripts/bootstrap.sh rooted in /target, which is the target file system for Ubiquity.
• dru-framework/scripts/dru-functions:
  • Includes utilities which are used in scripts throughout DRU, and difference directory scripts.
  • Should be included with . $DRU/dru-functions

Development Notes

Debugging difference directory changes can be a time-taking challenge. Here are a few tips to help you out. For quick reference, I will refer to the following phases as such:

• Phase 1 - The modifications completed when calling dru on the developer machine. This includes hooking the Casper FS, installing main packages, and running main scripts.
• Phase 2 - The modifications completed when booting the Live environment, in bootstrap.sh, while $1 = boot. This is the set of changes which are not reflected directly on the installation target FS, but are present in the Live environment.
• Phase 3 - The modifications completed when running Ubiquity, in bootstrap.sh, while $1 = ubiquity. This set of changes does not affect the Live environment, but instead the target installation FS.

During phase 1, debug messages are piped by default to the dru.log file. This can be changed by the --log= flag, and can be made more verbose (bash set -x) by using --debug.

Phase 2 is the most complex to debug. If you are using the casper-script file, the break=dru-bootstrap option in the kernel line when booting may be helpful. You will be dropped to an operable shell immediately before the bootstrap.sh file is executed. You can use this opportunity to make changes in the scripts that are about to run. The results from the actual phase 2 boot run will be stored in /var/log/dru.log.

Phase 3 output is stored in two separate files. Standard out from the scripts is redirected into /var/log/dru-update.log, and standard error (more useful normally) is piped to /var/log/syslog by Ubiquity.

To Do

1. Create a verify target, which should run tools such as 'apt-get check' after doing a mock Ubiquity install for each system. This would most likely prove to be a very expensive operation. If each system is not verified independently, packages may not get properly purged through Ubiquity on the target system.
2. Fix PXE target build.
3. Categorize seeds such that PXE and VFAT may have a different set of installations, depending on the target's size.
4. make KVER update action be a kernel hook instead of updating after each batch. This potentially fixes an issue if someone places a kernel upgrade in the same seed file as packages depending on that new kernel.
5. Support more architectures.