Download: zstd-for-kernel-v5.5.16.patch

Downloading: zstd-for-kernel-v5.5.16.patch Exposing Hacks for ZSTD -compressed Metztli Reiser4 / Debian Buster bps Linux 5.5.caxtolli∙omome and initramfs

"Monamacaz amahuan nicpia oncan ca hueyhuey caxtolli niman tepitoton matlactli".
The books I own should be sold, there are some fifteen very large and some ten small ones.

...

"Auh mochintin ynin que teteuhctin amo omazicamat ynin tlachihualiz. Ipampa inicuac ocalla que ynixiocaxtiltecah mochi opoliuhque ynin tlacuilol amahuan"
But of the feats of all these personalities there is nothing because the Spanish scum destroyed all the books of chronicles.

From the above phrase, it is evident the Nahuatl word caxtolli represents the number 15. So yes, this post was about utilizing the last Debian packaging for Linux kernel 5.5 series, i.e., 5.5.13, and applying upstream delta patches to bring it up to 5.5.15. However, as I was writing this post kernel 5.5.16 became available --which is what I will aim to patch and, subsequently, I will 'Add support for ZSTD-compressed kernel and initramfs' via a customized patch. And lastly, I will apply reiser4 for 5.5.5 patch --which still applies smoothly over kernel 5.5.15 and hopefully 5.5.16 and build our hacked kernel.

As illustrated above, I began by creating a directory for the task, relative to current directory, and changed my location into it:

Shell

I cloned locally the Debian Sid (unstable) debian/5.5.13-2 tagged branch which provides the Debian Packaging for Linux 5.5.13:

Shell

and followed by creating a new branch:

Shell

Now we need to fetch the Linux kernel 5.5.13 -- to satisfy our previously cloned Debian packaging for kernel 5.5.13-2 -- as well as the necessary upstream delta patches to bring our kernel up to 5.5.caxtolli_omce, or equivalently, 5.5.16.

Shell

And, as illustrated above, proceed to verify our downloaded resources:

Shell

We then select the kernel source XZ compressed file and create a sym link in the form expected by Debian Packaging and follow by changing back into our linux directory:

Shell

As this reiser4 hack is going to be essentially a Debian Buster backports, we need to replace the GCC-9 compiler string in the Sid (unstable) Debian Packaging configuration files, to a lower compiler version GCC-8. I create file ../chingaGCC8.amatl where I will place a list of the Debian packaging target files to modify:

Shell

And proceed with the GCC8 text string modifications for Buster backports:

Shell

Additionally, since we know we will be building our kernel for AMD64 architecture, we may want to disable debug-info in relevant defines just below the heading [build]:

Shell

And below the [relations] heading we add the Debian developers' Buster backports specific directive:

Shell

As illustrated in screenshot above --where the cursor is located in the xvi text editor-- we perform one last modification to debian/config/amd64/defines by changing signed-code to false.

Side note: In the illustrations, I am calling xvi tiny text editor from the file iztaccihuatl, i.e., 'white woman' in Nahuatl -- an allusion to the white snow covering a dormant volcanic mountain which resembles a sleeping female (Russian, in this instance) -- which contents save me some typing:

Shell

Although not strictly necessary -- unless, of course, we are also building this kernel for i386 architecture -- we proceed similarly to modify debian/config/i386/defines. We begin by setting signed-code: false,

and proceed by inserting the Debian developers' Buster backports directive immediately below relations header:

Shell

Subsequently, we use sed to set [686-pae_build] header and debug-info: false either below existing text string pattern: ^hardware-long.*not supporting:

Shell

exclusive or above the [686_description] header text string pattern:

Shell

Finally, we can use wc -l find out how many lines exist in debian/templates/control.extra.in:

Shell

and visually we can locate the Package: linux-compiler-gcc-8-x86 relevant line in debian/templates/control.extra.in .
Realizing it is in the third block of text in the file -- which contains 26 lines -- I can insert Debian developers' text string (>= 8-20180123-1~) for Buster backports for i386 either by using the xvi tiny text editor and/or by using sed as:

Shell

∙

Enabling Reiser4 in Debian Packaging for Linux kernel 5.5.13-2

Subsequently, we need to create and/or modify appropriate configuration files to create a reiser4 module. Indeed, I have expounded on the procedure in prior blogs; however, upstream Debian Packaging maintainer(s) alter their code and we must figure out what changed.

Using sed we add reiser4 as module in debian/config/config, by inserting a blank line after existing capitalized text string CONFIG_COREDUMP=y:

Shell

then we locate the capitalized text string CONFIG_NUMA_BALANCING_DEFAULT_ENABLED and revert its default, y, to n, i.e.,
CONFIG_NUMA_BALANCING_DEFAULT_ENABLED=n

Note: this settting was modified to y by Debian kernel maintainers beginning with Debian packaging for linux 4.15.x. Subsequently, my Linux host running with that modification rendered my local build of VirtualBox type 2 hypervisor unable to format in reiser4 filesystem any of the virtual machines it created. Furthermore, any pre-existing reiser4 -formated virtual machine(s) would hang during operation and potentially introduces corruption.

Although your mileage may vary (YMMV), I can not overemphasize the importance of the configuration setting referenced above when the aim is to create and run reiser4 virtual machines.

We continue by creating an empty file to satisfy kernel-wedge and it does not complaint about 'duplicates'.

Shell

and tell Debian packaging to create reiser4 modules for amd64 and i386 architectures:

Shell

...including for the Debian Installer (d-i)

Shell

One blank line below the JFS code block, we use sed to insert our reiser4 block:

Shell

By now we have created and/or modified these files which will enhance with reiser4 the Debian packaging for linux 5.5.13-2:

• debian/config/config
• debian/installer/ignore-dups
• debian/installer/modules/amd64/reiser4-modules
• debian/installer/modules/i386/reiser4-modules
• debian/installer/modules/reiser4-modules
• debian/installer/package-list

The above modifications are enough to enable Debian packaging to create reiser4 module support for Debian Buster backports for AMD64 architecture, of course, after we apply the corresponding reiser4 patch to our linux source tree --later. Additionally, although I have not built for i386 architecture, the above modifications will likely work as well.

We finish this section by creating a patch: reiser4-gcc8-for-debian-packaging-5.5.13-2.patch (attached at end of post) after specifying my Ynic cecni (First) commit:

Shell

∙

Only if we will ‘Add support for ZSTD-compressed kernel and initramfs'

Fact is the debian build halts if we do not strip compress-modules from our Debian packaging because the ensuing patch modifies Zstd decompression from being a Debian packaging module to being built into the kernel.

Thus, we begin by deleting the character string ', compress-modules' reference from all the module blocks in the last file we modified:

Shell

Then proceed to match the pattern Package: compress-modules, delete line, as well as the next 5 lines below which compose the module block:

Shell

Now we remove any compress-modules files in our Debian packaging source tree:

Shell

∙

Preparing our initial Linux kernel 5.5.13 with Debian packaging for 5.5.13-2.

We will strip WireGuard and related crypto backport support from the debian/patches/series until we have applied all the kernel incremental patches. Thus we make a backup of debian/patches/series first:

Shell

Then invoke Debian packaging magic to acquire, patch, and 'debianize' the linux kernel 5.5.13 XZ compressed archive we had downloaded previously:

Shell

Debian packaging decompresses and untars the upstream linux 5.5.13 source onto ../orig/linux-5.5.13 prior to patching... debianizing source tree into our current directory.

∙

We should expect a couple of errors during the next Debian packaging incantation:

Shell

And finish this session with Debian packaging sourceing incantation:

Shell

∙

Applying upstream delta patches onto a partially debianized  linux source tree at our current directory.

We can test the application of ../patch-5.5.13-14.xz by specifying --dry-run option to patch utility:

Shell

Analyzing the output we can tell that kernel/bpf/verifier.c kernel/bpf/verifier.c has already been debianized, i.e., patched in this instance. Thus, in order for the incremental patch to be applied smoothly, we can overwrite the debianized file with an original from the ../orig/linux-5.5.13 --the backup will allow us to compare the differences after we apply the incremental patch. Subsequently, we may decide to discard the backup file --especially if diff utility shows no differences:

Shell

An additional note: an .orig extension will be concatenated to files which the patch does not match the target exactly. As this is a hack over a partially debianized kernel source, we will have a few; from man patch output we can specify

--no-backup-if-mismatch
Do not back up a file if the patch does not match the file exactly and if backups are not otherwise requested. This is the default if patch is conforming to POSIX.

However, it is always instructive to compare the backups with the patched files as we may detect some glaring irregularities which will affect the build, stability, and/or functioning, of our kernel. In this particular post, notwithstanding, I already engaged in some educated overview and I will specify the POSIX conforming option when patching so as to save some time doing side-to-side comparisons.

Shell

We proceed to select and apply the next delta 5.5.14-15 increment --it applies smoothly

Shell

Next, as I was to apply final delta 5.5.15-16 increment available from upstream, I realize that there is a latest addition, patch-5.5.16-17.xz!
I download it and check its integrity:

Shell

And pick up where I had left off: apply the next delta 5.5.15-16 increment --it applies smoothly too!

Shell

and finish off by applying the latest upstream delta patch 5.5.16-17 increment that we just downloaded:

Shell

WireGuard backport for Debian packaging for 5.5.13-2: long option one(1)

We can now apply WireGuard backports that we had previously stripped from debian/patches/series. Editing our previously saved ../series file by deleting all text except for WireGuard patch numbered entries and subsequently writing our modifications to the file wg.amatl, we can do something like...

Notwithstanding, be aware that there is a file that stops the build procedure -- as it seems was not updated by the Debian kernel maintainers: drivers/net/wireguard/queueing.h

For the WireGuard backport patch(es) -- which was initially hacked for Debian 5.5.13-2 -- to apply smoothly we need to fetch two(2) files from the original linux 5.5.13 source tree:

lib/crypto/chacha20poly1305-selftest.c
arch/arm64/crypto/chacha-neon-glue.c

and we will overwrite the ones modified by our ensuing application range of patches-5.5.13-17. If we write those two(2) files as content in in file ../original-wg.amatl then we can copy over the files from ../orig/linux-5.5.13/ onto our current linux directory and at the same time appending a suffix PatchedRecent to our corresponding backups. We will compare them at the end of the procedure.

Shell

Shell

Hence, the WireGuard backport for Debian packaging for 5.5.13-2 applied smoothly; we verify that the two replacement files discussed previously show no differences --accordingly, we delete our two(2) backups.

Finally, replace drivers/net/wireguard/queueing.h with updated queueing.h (attached below)
Thus, assuming we place queueing.h at our parent directory:

Shell

Exclusive OR: WireGuard backport for 5.5.16(-17): Option two(2)

Forget about the lengthy Debian procedure described in option one(1) above and just download linux-5.5.16-WireGuard.patch
-- attached below -- as it will apply smoothly onto our current linux 5.5.17.
Again, assuming we place linux-5.5.16-WireGuard.patch at our parent directory, we could instead have patched as:

Shell

If Zstd-compressed kernel and initramfs

Either fetch from [PATCH v4 0/8] Add support for ZSTD-compressed kernel... and create your own hack OR get it from here: zstd-for-kernel-v5.5.16.patch attached below) but we assume patch is located at our parent directory, then:

Shell

CAUTION: the next sed one liner will replace the default kernel gzip compression method for that of Zstd.

Shell

04/27/2020 We make sure our new default overrules any others by explicitly stating it in debian/config/config

Shell

Make sure that your initramfs-tools package supports Zstd; in other words, you must install a priori Zstd and Zstd -enabled initramfs-tools. For instance here is a quick hack where we backport Sid Package: initramfs-tools 0.136 for Debian Buster backports by applying initramfs-tools-core: Enable zstandard support -based metztli-initramfs-tools-0.136-zstd-default-compressor.patch (attached below, as well):
zstd_1.4.4+dfsg-3.1_amd64.deb                  zstd_1.4.4+dfsg-3.1_amd64.deb.SHA256SUM
initramfs-tools_0.136+nmu1_all.deb             initramfs-tools_0.136+nmu1_all.deb.SHA256SUM
initramfs-tools-core_0.136+nmu1_all.deb     initramfs-tools-core_0.136+nmu1_all.deb.SHA256SUM
Zstd optional:
libzstd1_1.4.4+dfsg-3.1_amd64.deb               libzstd1_1.4.4+dfsg-3.1_amd64.deb.SHA256SUM
libzstd-dev_1.4.4+dfsg-3.1_amd64.deb          libzstd-dev_1.4.4+dfsg-3.1_amd64.deb.SHA256SUM

∙

Applying reiser4 for 5.5.5 patch onto our partially debianized linux 5.5.17 source tree.

Download reiser4-for-linux-5.x/reiser4-for-5.5.5.patch.gz into our parent directory.

And apply patch:

Shell

It should end with no errors nor generate .rej extension files.

We should now modify a couple of configuration files for our Debian packaging. First, I summon iztaccihuatl -- which in turn calls xvi to edit; and we will concatenate the string '+reiser4.0.2' to the ABI number:

Shell

I peek into debian/changelog:

Shell

and proceed to correct the kernel version --remember this is a hack where we are using Debian packaging for linux 5.5.13-2 as base -- and we have patched the kernel in delta increments in the range of 5.5.14-17. Indeed, we are also concatenating +reiser4.0.2 string (but -D metztli is totally optional )

Shell

the above Debian incantation will summon whatever text editor is your default. Once you are done with your annotations, save and quit your editor. Above illustration was taken after I had quit my default text editor and saved my annotations. We continue...

As in prior occasion earlier in this post, we should expect a couple of 'intentional' output errors by invoking the next debian incantation.

Shell

...but we should only expect one(1) output error after the following debian magic string directive

Shell

For the next -- previous to last debian incantation -- it may be desired to redirect the output and potential errors, if any, into a file to analyze afterwards. It is a fact that if any errors are logged the build will fail:

Shell

04-28-2020 Updated during more recent build of 5.5.caxtolli∙nahui (5.5.19) to illustrate more detailed log.
NOTE 04/27/2020 due to applying prior command 75, above command 82 will yield a couple of warnings; please disregard:

.config:4934:warning: override: KERNEL_ZSTD changes choice state

Debian packaging has created the kernel .config. Hmmm...

As seen in illustration above, from time to time Debian packaging will override default KERNEL_ZSTD but we can modify debian/build/build_amd64_none_amd64/.config as: 04/27/2020  By explicitly stating our directive(s) in debian/config/config (see prior command 75), the illustration immediately above and next command 83 are likely redundant:

Shell

∙

VERY IMPORTANT! Starting with linux kernel 5.5.16 we must modify Debian packaging by adding the following compiler directive at the end of the target file. Failing to do so will stop the build procedure.

Shell

∙

We are ready to build our kernel the 'Debian way'... Tlayecuel : Поехали : Let's go!

Shell

where X represents...

Number of jobs allowed to be run simultaneously, number of jobs matching the number of online processors if auto is specified (since dpkg 1.17.10), or unlimited number if jobs is not specified, equivalent to the make(1) option of the same name (since dpkg 1.14.7, long option since dpkg 1.18.8)

(read man dpkg-buildpackage for additional info)

∙∙∙

Metztli Reiser4 Linux kernel macuilli.macuilli.caxtolli∙omome (5.5.17) with Zstd-compressed kernel and initramfs build completes

---

hyperv-daemons_5.5.17-2+reiser4.0.2_amd64.deb
hyperv-daemons_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
hyperv-daemons-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb
hyperv-daemons-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
libbpf0_5.5.17-2+reiser4.0.2_amd64.deb
libbpf0_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
libbpf0-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb
libbpf0-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
libbpf-dev_5.5.17-2+reiser4.0.2_amd64.deb
libbpf-dev_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
libcpupower1_5.5.17-2+reiser4.0.2_amd64.deb
libcpupower1_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
libcpupower1-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb
libcpupower1-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
libcpupower-dev_5.5.17-2+reiser4.0.2_amd64.deb
libcpupower-dev_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
libtraceevent1_5.5.17-2+reiser4.0.2_amd64.deb
libtraceevent1_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
libtraceevent1-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb
libtraceevent1-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
libtraceevent1-plugin_5.5.17-2+reiser4.0.2_amd64.deb
libtraceevent1-plugin_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
libtraceevent1-plugin-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb
libtraceevent1-plugin-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
libtraceevent-dev_5.5.17-2+reiser4.0.2_amd64.deb
libtraceevent-dev_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-compiler-gcc-8-x86_5.5.17-2+reiser4.0.2_amd64.deb
linux-compiler-gcc-8-x86_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-config-5.5_5.5.17-2+reiser4.0.2_amd64.deb
linux-config-5.5_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-cpupower_5.5.17-2+reiser4.0.2_amd64.deb
linux-cpupower_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-cpupower-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb
linux-cpupower-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-doc_5.5.17-2+reiser4.0.2_all.deb
linux-doc_5.5.17-2+reiser4.0.2_all.deb.SHA256SUM
linux-doc-5.5_5.5.17-2+reiser4.0.2_all.deb
linux-doc-5.5_5.5.17-2+reiser4.0.2_all.deb.SHA256SUM
linux-headers-5.5.0-1+reiser4.0.2-all_5.5.17-2+reiser4.0.2_amd64.deb
linux-headers-5.5.0-1+reiser4.0.2-all_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-headers-5.5.0-1+reiser4.0.2-all-amd64_5.5.17-2+reiser4.0.2_amd64.deb
linux-headers-5.5.0-1+reiser4.0.2-all-amd64_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-headers-5.5.0-1+reiser4.0.2-amd64_5.5.17-2+reiser4.0.2_amd64.deb
linux-headers-5.5.0-1+reiser4.0.2-amd64_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-headers-5.5.0-1+reiser4.0.2-cloud-amd64_5.5.17-2+reiser4.0.2_amd64.deb
linux-headers-5.5.0-1+reiser4.0.2-cloud-amd64_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-headers-5.5.0-1+reiser4.0.2-common_5.5.17-2+reiser4.0.2_all.deb
linux-headers-5.5.0-1+reiser4.0.2-common_5.5.17-2+reiser4.0.2_all.deb.SHA256SUM
linux-headers-amd64_5.5.17-2+reiser4.0.2_amd64.deb
linux-headers-amd64_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-headers-cloud-amd64_5.5.17-2+reiser4.0.2_amd64.deb
linux-headers-cloud-amd64_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-image-5.5.0-1+reiser4.0.2-amd64_5.5.17-2+reiser4.0.2_amd64.deb
linux-image-5.5.0-1+reiser4.0.2-amd64_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-image-5.5.0-1+reiser4.0.2-cloud-amd64_5.5.17-2+reiser4.0.2_amd64.deb
linux-image-5.5.0-1+reiser4.0.2-cloud-amd64_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-image-amd64_5.5.17-2+reiser4.0.2_amd64.deb
linux-image-amd64_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-image-cloud-amd64_5.5.17-2+reiser4.0.2_amd64.deb
linux-image-cloud-amd64_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-kbuild-5.5_5.5.17-2+reiser4.0.2_amd64.deb
linux-kbuild-5.5_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-kbuild-5.5-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb
linux-kbuild-5.5-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-libc-dev_5.5.17-2+reiser4.0.2_amd64.deb
linux-libc-dev_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-perf_5.5.17-2+reiser4.0.2_amd64.deb
linux-perf_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-perf-5.5_5.5.17-2+reiser4.0.2_amd64.deb
linux-perf-5.5_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-perf-5.5-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb
linux-perf-5.5-dbgsym_5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
linux-source_5.5.17-2+reiser4.0.2_all.deb
linux-source_5.5.17-2+reiser4.0.2_all.deb.SHA256SUM
linux-source-5.5_5.5.17-2+reiser4.0.2_all.deb
linux-source-5.5_5.5.17-2+reiser4.0.2_all.deb.SHA256SUM
linux-support-5.5.0-1+reiser4.0.2_5.5.17-2+reiser4.0.2_all.deb
linux-support-5.5.0-1+reiser4.0.2_5.5.17-2+reiser4.0.2_all.deb.SHA256SUM
usbip_2.0+5.5.17-2+reiser4.0.2_amd64.deb
usbip_2.0+5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM
usbip-dbgsym_2.0+5.5.17-2+reiser4.0.2_amd64.deb
usbip-dbgsym_2.0+5.5.17-2+reiser4.0.2_amd64.deb.SHA256SUM

---

References: Виктория Одинцова : Viktoriya Odintcovo ~ Viki- Xochiquetzal's Instagram

Linux: Using sed to insert lines before or after a match

sed - perform only first (nth) matched replacement?

sed insert line with spaces to a specific line

Remove line containing certain string and the following line

SED: insert text after the last line?

---

DISCLAIMER although due diligence has been applied, this resource is made available for testing/evaluation purposes on an AS IS basis. The procedure only reflects my own modifications, my limited testing, and the potential user(s) who execute(s) the procedures assumes all risks.

Please do not hold me or Metztli Information Technology (and/or its associates) responsible if the information provided here does not achieve the desired result. The information is provided AS IS and with the hope that it may be useful to the Internet community --especially those open-minded individuals interested in Reiser4 continued development.

Notwithstanding, There is no implicit or explicit guarantee that the information presented here is accurate --even though due diligence was exercised during the procedure. Accordingly, if an user(s) decide to implement the procedure or shell commands described here she, he, or them, do so at her, his, or their own risk. You have been forewarned.

Metztli IT, but not other entities, reserves the right to modify the content -- to correct and/or elucidated procedure(s), for instance -- and/or even delete all or partial, including blog post, without previous notice.