Metztli Information Technology Blog Bits

I happened to came across the need to enable advanced multi layered unification filesystem (Aufs) -- now at version 3.x (thus, Aufs3) -- module support into a GNU/Linux Debian deployment. It had been a while that I had done something similar by customizing the kernel to enable that feature. Well, surprise! Nothing has changed. Well, let me rephrase that, Aufs development continues; however, we still have to somehow hack the feature into the linux kernel tree version desired. Below I describe a procedure that worked on Debian and linux kernel 3.12.2, that is, the latest stable kernel source on this date.

Given the fact that cahuitl means the space of time in Nahuatl (Mexico's language par excellence), and a certain place, area, or surface in n-dimensional space is referred to as tlacauhtli, I was inspired to create a working directory appropriately named for the task at hand :P

mkdir -p --verbose /usr/src/tlacauhtli/build

Of course, prefixing sudo for the privilege to execute the above command (and similar others) is necessary if you are not already wielding root privilege; either way, the root (or super user) privilege is necessary to create directory(ies) at /usr/src directory since in Debian the default owner and group is root as shown with command:

ls -ld /usr/src
(sample output)...
drwxr-xr-x 8 root root 376 Nov 28 00:10 /usr/src/

We will not touch those default permissions but will manipulate those of the directories we initially created for our operations.

Now find out if your_username already belongs to Debian's src group:

groups your_username

Membership in the group src must be included in the output since, for security, we will want to operate as a non-root user. If the output does not show that your_user is a member of src group, then, wielding root privilege we should add the normal user who will be doing the Aufs3 and kernel build operations as a member of Debian's src group:

adduser your_username src

For the privileged permissions to take effect, your_username must logout and then login back again. Subsequently entering command again:

groups your_username

the output should show that your_username is a privileged member of Debian's src group (amongst others).

your_username should now be ready to begin her Aufs3 hack into the kernel tree.

But first root should grant and/or modify access permissions on our work area tlacauhtli and its subdirectory build, as below.

Still wielding root privilege, we subsequently grant src group ownership and writing privilege on the tlacauhtli directory where the operations will take place.

chmod -R g+w /usr/src/tlacauhtli

grants writing privilege to the group on directories tlacauhtli and its subdirectory build

chgrp -R src /usr/src/tlacauhtli

grants group ownership to the members of src group (which includes your_user) on directories tlacauhtli and its subdirectory build

Below is a rather verbose snapshot summary from bash shell on Debian:

Now, we assume the identity of a normal non-root user, of course, you will assume your_username that we elaborately granted src group membership and privileges to operate on tlacauhtli and its subdirectory build directories. Now empowered, we change to tlacauhtli directory:

$ cd /usr/src/tlacauhtli

We start by downloading the latest stable kernel by visiting The Linux Kernel Archives. For security you are encouraged to check integrity of the files downloaded and scan them for malware.

For instance, if you have clamav, you may scan the uncompressed files as:

$clamscan downloaded_file

If you already uncompressed the files, you can recursively check the directory trees and only print out if infected files were found as:

$clamscan -ri uncompressed-file_directory

(Please do man clamscan and read for further information).

NOTE: If your Debian does not have the clamav anti-virus, wielding root privilege download and install as: apt-get install clamav

Verify Integrity Of Your Downloaded Kernel Releases By Verifying The Corresponding Signatures...

Linux kernel releases PGP signatures

Assuming we downloaded the compressed linux kernel tree linux-3.12.2.tar.xz and it is at our current location, we download its corresponding signature with wget utility (all in a single line directive)

NOTE: If your Debian does not have wget and/or bzip2, wielding root privilege download and install as: apt-get install wget bzip2 bc (indeed, you will need bc - An arbitrary precision calculator language - in your kernel building efforts.

$wget https://www.kernel.org/pub/linux/kernel/v3.0/linux-3.12.2.tar.sign

In order to be verified, linux-3.12.2.tar.xz should be decompressed and left in tar format (see Using GnuPG to verify kernel signatures ). As illustrated, the procedure can be accomplished step by step or in a one liner concatenation directive. I have decide to go with the latter:

$xz -dc linux-3.12.2.tar.xz | gpg --verify linux-3.12.2.tar.sign -

(sample output)...
gpg: Signature made Fri 29 Nov 2013 11:29:20 AM PST using RSA key ID 6092693E
gpg: Can't check signature: public key not found

I take as input the RSA key ID to download the public key from the PGP keyserver in order to verify the signature, thus:

$gpg --recv-keys 6092693E --keyserver subkeys.pgp.net

alternatively, if the above directive fails, try:

$gpg --keyserver pgp.mit.edu --recv-keys 6092693E
(sample output)...
gpg: requesting key 6092693E from hkp server subkeys.pgp.net
gpg: key 6092693E: public key "Greg Kroah-Hartman (Linux kernel stable release signing key) <greg@kroah.com>" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)

I rerun gpg --verify...:

$xz -dc linux-3.12.2.tar.xz | gpg --verify linux-3.12.2.tar.sign -
(sample output)...
gpg: Signature made Fri 29 Nov 2013 11:29:20 AM PST using RSA key ID 6092693E
gpg: Good signature from "Greg Kroah-Hartman (Linux kernel stable release signing key) <greg@kroah.com>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 647F 2865 4894 E3BD 4571 99BE 38DB BDC8 6092 693E

For this post, we will stop our verification process here. Nevertheless, you may decide to continue the procedure by following:

'You will now need to verify that the key used to sign the archive really does belong to the owner...'

I refresh my Debian repositories:

Shell

and download the necessary tools/utilities to build my kernel:

$apt-get install build-essential kernel-package patch fakeroot libncurses5 libncurses5-dev git


Now I change from my current tlacauhtli to subdirectory build

$cd build

and decompress my kernel, simultaneously untar'ing it:

$tar -xvJPf ../linux-3.12.2.tar.xz

"it becomes clear that 'Aufs was rejected. Let's give it up.' According to Christoph Hellwig, linux rejects all union-type filesystems but UnionMount."

Aufs at SourceForge

Since I am interested in building Aufs3 as a module, at this particular time the aufs3-standalone GIT tree enables that CONFIG_AUFS_FS=m option; hence, I download it:

git clone git://git.code.sf.net/p/aufs/aufs3-standalone aufs3-standalone.git

and change into the referenced directory:

$cd aufs3-standalone.git

and remember to specify the minor number of the kernel we are building. Thus, in our case, I have emphasized the minor number of the kernel 3.12 for inclusion in the next command:

$git checkout origin/aufs3.12

(sample output)...

Note: checking out 'origin/aufs3.12'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b new_branch_name

HEAD is now at 4a7364f... aufs3.12 20131111

and proceed to create a couple of directories to prepare a patch for the linux kernel tree:

$mkdir --verbose ../x ../y

How you name those remains at your discretion but I named them x and y. The following sequence of commands should operate on the directory ../y/.

Code

In the above commands, the verbosty (-v) is optional.

Make sure to remove Kbuild:

$rm -v ../y/include/uapi/linux/Kbuild

else you should specify 'no' when prompted by the patch utility whether it should replace Kbuild in your linux kernel tree.

We go back to our previous directory, i.e., build:

$cd ..

And perform the following operations:

Code

We are ready to apply the patch; we change directory to our linux kernel tree source:

$cd linux-3.12.2

And apply the patch:

$patch -p1 < aufs.patch

You will observe a long output to your bash shell but no errors.

Yolahuialtia [Cheers!] You have just patched your latest stable linux kernel tree :)

As I am upgrading from my currently running linux kernel 3.12.0, I will import the relevant .config into my spanking new and Aufs-patched linux kernel tree:

$cat /boot/config-`uname -r` >.config

and do:

$make oldconfig
(sample output)...
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
SHIPPED scripts/kconfig/zconf.tab.c
SHIPPED scripts/kconfig/zconf.lex.c
SHIPPED scripts/kconfig/zconf.hash.c
HOSTCC scripts/kconfig/zconf.tab.o
HOSTLD scripts/kconfig/conf
scripts/kconfig/conf --oldconfig Kconfig
*
* Restart config...
*
*
* Miscellaneous filesystems
*
[...]
SquashFS 4.0 - Squashed file system support (SQUASHFS) [M/n/y/?] m
Squashfs XATTR support (SQUASHFS_XATTR) [Y/n/?] y
Include support for ZLIB compressed file systems (SQUASHFS_ZLIB) [Y/n/?] y
Include support for LZO compressed file systems (SQUASHFS_LZO) [Y/n/?] y
Include support for XZ compressed file systems (SQUASHFS_XZ) [Y/n/?] y
Use 4K device block size? (SQUASHFS_4K_DEVBLK_SIZE) [Y/n/?] y
Additional option for memory-constrained systems (SQUASHFS_EMBEDDED) [N/y/?] n
[...]
OS/2 HPFS file system support (HPFS_FS) [M/n/y/?] m
[...]
Aufs (Advanced multi layered unification filesystem) support (AUFS_FS) [N/m/y/?] (NEW) m

Indeed, the directive above prompts as to whether install Aufs as a mdule, which I agree to by specifying to enable Aufs support as a module by typing: m

And subsequently follow additional options:
(sample output)...
Maximum number of branches
> 1. 127 (AUFS_BRANCH_MAX_127) (NEW)
2. 511 (AUFS_BRANCH_MAX_511) (NEW)
3. 1023 (AUFS_BRANCH_MAX_1023) (NEW)
4. 32767 (AUFS_BRANCH_MAX_32767) (NEW)
choice[1-4?]:
Detect direct branch access (bypassing aufs) (AUFS_HNOTIFY) [N/y/?] (NEW)
NFS-exportable aufs (AUFS_EXPORT) [N/y/?] (NEW) y
Readdir in userspace (AUFS_RDU) [N/y/?] (NEW) y
Respect the attributes (mtime/ctime mainly) of special files (AUFS_SP_IATTR) [N/y/?] (NEW)
Show whiteouts (AUFS_SHWH) [N/y/?] (NEW)
Ramfs (initramfs/rootfs) as an aufs branch (AUFS_BR_RAMFS) [N/y/?] (NEW) y
Fuse fs as an aufs branch (AUFS_BR_FUSE) [N/y/?] (NEW) y
Hfsplus as an aufs branch (AUFS_BR_HFSPLUS) [Y/n/?] (NEW)
Debug aufs (AUFS_DEBUG) [N/y/?] (NEW)

If I do a:

$make xconfig

I can observe and configure graphically Aufs support in the kernel tree:

The moment to build our Debian custom linux kernel has finally arrived! Do:

$fakeroot make-kpkg clean

And I proceed to build my kernel: Xonecuiltzin which :yes: prefixed with a leading dot, I provide as an argument to --append-to-version in the directive that follows:

$time fakeroot make-kpkg --append-to-version=.xonecuiltzin --stem aufs -j8 --initrd kernel_image kernel_headers

I also provide aufs as an argument to --stem because I want to know the main reason for my kernel customization. Additionally, I provide 8 as an argument to -j. Eight(8) represents the number of threads I wish to launch, hence I make it equal to the number of cores in my machine where I am building the kernel -- for optimum performance during compilation.

After a few minutes, the build procedure ends and I take a look a my newly created Debian kernel and headers DEB packages, thus:

$ls ..
(sample output)...
aufs3-standalone.git/ linux-3.12.2/
aufs-headers-3.12.2.xonecuiltzin_3.12.2.xonecuiltzin-10.00.Custom_amd64.deb x/
aufs-image-3.12.2.xonecuiltzin_3.12.2.xonecuiltzin-10.00.Custom_amd64.deb y/

To install our newly built Xonecuiltzin kernel on Debian, we need to acquire root privilege or prefix the command below with sudo:

$dpkg -i aufs-image-3.12.2.xonecuiltzin_3.12.2.xonecuiltzin-10.00.Custom_amd64.deb

... yeah, include the path if your kernel is not at your current directory location ;-)

---

Nahuatl quote found in an IAEA Publication (PDF) titled: Analytic Number Theory & The Nuclear Level Density.

The quote is attributed to Cuauhtemoc, last ruler of the Mexicah. On August 13, 1521, after the death of some 250,000 Mexicah and besieged for 79 days, Cuauhtemoc's metropolis Mexico-Tenochtitlan fell to the large invading armies of indigenous enemies of the Mexicah led by a small number of Spanish mercenary plunderers. After being tortured unsuccessfully to reveal hidden gold by the avaricious Spaniards, Cuauhtemoc was revered by the surviving Mexicah as Xonecuiltzin, "The Limping one", the same name/title by which one of their Deities, Tezcatlipoca, was also known.

REFERENCES:
How To Roll A Kernel the Ubuntu/Debian Way
Kernel 3.9 on Debian Wheezy/Testing
Compile Debian Kernel (Squeeze) 3.0 and Above with Aufs and squashfs
COMPILING Linux kernel version 3..2.6 | r4mi5

DISCLAIMER:P although due diligence has been applied, this resource is made available for testing/evaluation purposes on an AS IS basis. The procedure only reflect my own modifications, my limited testing, and the potential user who executes the procedures assumes all risks.

Please do not hold me or Metztli Information Technology responsible if the information provided here does not achieve the desired result. The information is provided AS IS and with the hope that it may be useful to the Internet community --especially those who need Aufs support on Debian.

Notwithstanding, There is no implicit or explicit guarantee that the information presented here is accurate --even though due diligence was exercised during the procedure. Accordingly, if an user(s) decide to implement the procedure or shell commands described here she, he, or them, do so at her, his, or their own risk. You have been forewarned.