Tag: "ModSecurity Nginx Connector"

Build PHP 7.3.5̶ 6 for Nginx 1.17.0 on Metztli Reiser4 'the Debian Way' for Stretch Backports on AMD64.

By Jose in

Linux, Current Trends, Cloud Computing, Reiser4, nginx, Debian

Виктория Одинцова¹: Viktoriya Odintcova, graces this post cover of
Tecpan Quetzalpapalotl : 'Palace of the Gorgeous Butterfly', located at Teuhctihuacan ['Teotihuacan'].
Any likeness to XochiQuetzal, i.e., 'Gorgeous Flower'
--the real Mexicah's collectively held archetype of female beauty--
is not a mere artistic coincidence.

After a couple of ensuing build iterations following previous successful build of Engine X (Nginx), I updated my original patch to include a sample b2evolution Blog/CMS configuration. At the same time Nginx 1.17.0 was released and I decided to upgrade a Metztli Reiser4 instance running Nginx 1.16.0 in the cloud. Once that instance was running Nginx 1.17.0, notwithstanding, Drupal CMS 8.x.y Status Report displayed a notice advising that PHP 7.2 would be a significant improvement over PHP 7.0; this latter, however, is the only currently available version even in Debian Stretch backports repository used by Metztli Reiser4. The following post thus illustrates 'the Debian way' guiding procedure that I hacked to build PHP 7.3.5̶ 6, a slightly newer version than that currently found in the upcoming Debian Buster repositories.

Preliminary Tasks Prior to PHP 7.3.5̶ 6 Build

First I created a tekitl- [work] tlacauhtli [space] area where I changed into --subsequently fetching the Debian Buster packaging for PHP 7.3.4-2 as well as the source for php-7.3.5̶ 6.
Note: As I was writing this blog entry, php-7.3.6 became available at PHP Downloads and I verified it builds, too.

Shell

mkdir --verbose tekitl-tlacauhtli && cd tekitl-tlacauhtli
wget http://deb.debian.org/debian/pool/main/p/php7.3/php7.3_7.3.4-2.debian.tar.xz
wget https://www.php.net/distributions/php-7.3.6.tar.xz.asc https://www.php.net/distributions/php-7.3.6.tar.xz

After expanding the Debian Buster packaging for PHP 7.3.4-2 archive, I then proceeded to fulfill required dependencies using its debian/control file as my guide for AMD64:

Shell

tar -xvPf php7.3_7.3.4-2.debian.tar.xz
ls debian
ls debian/control

We will be peeking into debian/control so as to evaluate necessary dependencies; but we must be careful not to install dependencies for architectures for which we do not intend to build our software as that will only introduce unnecessary errors in our build. We should not forget that this file is from upcoming Debian 10 Buster and thus we will need to fetch some newer resources than those found in Stretch repositories.

Note: to fulfill MySQL requirement, I selected MariaDB 10.3.15 instead; hence libmariadb-dev-compat is an alternative to libmysqlclient-dev, and I had to 'pin' a preferred source repository specified in a file with .pref extension written into /etc/apt/preferences.d/ directory. For instance, below is sample content if your preferred repository is, say, in New York:
Package: * Pin: origin 'nyc2.mirrors.digitalocean.com' Pin-Priority: 1000

Your mileage may vary (YMMV) as it will depend on your nearest and/or preferred location as you make your selection and follow instructions provided in the above referenced MariaDB resource link. Or you might as well use available, lower-version, MariaDB from Debian repositories and forget about 'pinning', etc..

Hence we peek into debian/control to figure out dependencies prior to our php-7.3.5̶ 6 build:

Shell

vim -R debian/control

Shell

apt-get -t stretch-backports update
apt-get -t stretch-backports install autoconf automake bison chrpath debhelper libmariadb-dev-compat dh-apache2 dpkg-dev flex freetds-dev libapparmor-dev libapr1-dev libargon2-0-dev libbz2-dev libc-client2007e-dev libcurl4-openssl-dev libdb-dev libedit-dev libenchant-dev libevent-dev libexpat1-dev libfreetype6-dev libgcrypt20-dev libgd-dev libglib2.0-dev libgmp3-dev libicu-dev libjpeg-dev libkrb5-dev libldap2-dev liblmdb-dev libmagic-dev libmhash-dev libnss-myhostname libonig-dev libpam0g-dev libpcre2-dev libpng-dev libpq-dev libpspell-dev libqdbm-dev librecode-dev libsasl2-dev libsnmp-dev libsodium-dev libsqlite3-dev libssl-dev libtidy-dev libtool libwebp-dev libwrap0-dev libxml2-dev libxmlrpc-epi-dev libxmltok1-dev libxslt1-dev libzip-dev locales-all netbase netcat-traditional re2c systemtap-sdt-dev tzdata unixodbc-dev zlib1g-dev dh-systemd libsystemd-dev firebird-dev mariadb-server

We will need the upcoming New Perl Compatible Regular Expression Library- 8, 16, 32 bit runtime files, the development files, and posix-compatible runtime files --which I downloaded manually from ensuing Buster packages repository

libpcre2-8-0_10.32-5_amd64.deb
libpcre2-16-0_10.32-5_amd64.deb
libpcre2-32-0_10.32-5_amd64.deb
libpcre2-dev_10.32-5_amd64.deb
libpcre2-posix0_10.32-5_amd64.deb

Additionally, we will need the memory-hard hashing function - runtime library and development files. Accordingly, downloaded these other two(2) packages from this other Buster repository
libargon2-1_0~20171227-0.2_amd64.deb
libargon2-dev_0~20171227-0.2_amd64.deb

I thus resolved an error which halted my initial build attempts and the log files were ambiguous as to its cause. Naturally, prior to my PHP 7.3.5̶ 6 successful build, I installed those seven(7) files from Buster manually as,

Shell

dpkg -i libpcre2-16-0_10.32-5_amd64.deb libpcre2-8-0_10.32-5_amd64.deb libpcre2-posix0_10.32-5_amd64.deb libpcre2-32-0_10.32-5_amd64.deb libpcre2-dev_10.32-5_amd64.deb libargon2-dev_0~20171227-0.2_amd64.deb libargon2-1_0~20171227-0.2_amd64.deb

Patching Debian Packaging Patch To Build PHP 7.3.5̶ 6

We must change a couple of strings for this debian/patches/0038-php-5.6.0-oldpcre.patch to apply cleanly

Shell

sed -i 's/group/\%s/g' debian/patches/0038-php-5.6.0-oldpcre.patch

Also in debian/patches/series file, we locate and neutralize
debian/patches/0039-hack-phpdbg-to-explicitly-link-with-libedit.patch as it seems to be redundant against newer PHP:

add ## to neutralize listed patch — debian/patches/series

Shell

sed -i 's/^0039/## 0039/g' debian/patches/series

Then we can expand the PHP 7.3.5̶ 6 that we downloaded a priori:

Shell

tar -xvPf php-7.3.6.tar.xz

and move our modified debian packaging for php into newly created directory as we change directory to the latter, as well:

Shell

mv -v debian php-7.3.6/.
cd php-7.3.6/.

Create a symbolic link to debian/patches so that we may subsequently apply those with quilt.

Shell

ln -s debian/patches
quilt push -a --fuzz=0

And note our new upstream php 7.3.6 in debian/changelog

Shell

dch -v 7.3.6-1

i.e.,

Shell

dch -v 7.3.6-1 -D metztli

We save our modifications and dch will close our text editor.

And finally, we begin our PHP 7.3.6 build for our Metztli Reiser4 and/or Debian 9.x Stretch Backports AMD64:

Shell

dpkg-buildpackage -F -us -uc -jX -T binary-arch,binary-indep

where X represents the number of CPU/cores we want to assign to our build task. Please read the manual page for dpkg-buildpackage, especially for option -d

Installing Our Newly Built PHP 7.3.5̶ 6 and/or Purging Stretch/Backports PHP 7.0 in Our Target System.

It is good practice to make a backup of a working Linux, Engine X, MariaDB, PHP (LEMP) stack prior to engaging in the installation of software which is not available in your default Debian repositories. For instance, my target Metztli Reiser4 instance was on Google Compute Engine (GCE) and I wanted to make sure I could recover relatively straightforward in case something went wrong. Thus, I created a snapshot of the target instance underlying disk, i.e., Best Practices for Persistent Disk Snapshots, prior to engaging in the task.

At the very least make a backup of /etc/ directory as that contains important configuration files that will be overwritten by our newly built PHP 7.3.6.

Additionally, depending upon having --or not-- an existing LEMP installation will determine whether some of the following steps are/or not applicable:

Assuming that you are using PHP 7.0 from Debian repositories, prior to installation of PHP 7.3.5̶ 6 packages, find out which packages you will need to replace those older ones. For instance,

Shell

dpkg-query -l "php7.0*" | grep ^ii | cut -d' ' -f3

will show which packages are installed that should be replaced with equivalent PHP 7.3.5̶ 6 ones. Consequently, after acquiring a list of older packages, I purged them as:

Shell

apt-get purge php7.0 php7.0-cli php7.0-common php7.0-fpm php7.0-gd php7.0-intl php7.0-json php7.0-mbstring php7.0-mysql php7.0-opcache php7.0-readline php7.0-sqlite3 php7.0-xml php7.0-xmlrpc php7.0-zip

Note: prior to installing PHP 7.3.5̶ 6 in another target VirtualBox instance running Metztli Reiser4, but not LEMP, I had to fulfill these dependencies:

Shell

apt-get -t stretch-backports install php-common libzip4 libsodium23 libgd3 libxmlrpc-epi0

Notwithstanding, unless you are currently running Debian Buster, these other dependencies are a must -- whether or not a current LEMP existed a priori:
Manually download memory-hard hashing function - runtime library libargon2-1_0~20171227-0.2_amd64.deb (or newer version from the date of this post) from Buster packages repository -- as well as libpcre2-8-0_10.32-5_amd64.deb (or newer version from the date of this post) runtime files from Buster packages repository -- and manually install as:

Shell

dpkg -i libargon2-1_0~20171227-0.2_amd64.deb libpcre2-8-0_10.32-5_amd64.deb

Then we could try to install (wielding root privilege) our newly built PHP 7.3.6 starting with four(4) packages:

Shell

dpkg -i php7.3_* php7.3-common_* php7.3-readline_* php7.3-cli_*

and/or all your desired DEB packages at once but putting ahead of the queue the four(4) packages above --so as to decrease potential dependency issues by the installation of ensuing packages:

Shell

dpkg -i php7.3_7.3.6-1_all.deb php7.3-common_7.3.6-1_amd64.deb php7.3-readline_7.3.6-1_amd64.deb php7.3-cli_7.3.6-1_amd64.deb php7.3-fpm_7.3.6-1_amd64.deb php7.3-sqlite3_7.3.6-1_amd64.deb php7.3-curl_7.3.6-1_amd64.deb php7.3-intl_7.3.6-1_amd64.deb php7.3-mbstring_7.3.6-1_amd64.deb php7.3-xmlrpc_7.3.6-1_amd64.deb php7.3-mysql_7.3.6-1_amd64.deb php7.3-gd_7.3.6-1_amd64.deb php7.3-xml_7.3.6-1_amd64.deb php7.3-json_7.3.6-1_amd64.deb php7.3-opcache_7.3.6-1_amd64.deb php7.3-zip_7.3.6-1_amd64.deb

Your mileage may vary (YMMV) as you may need more or less packages installed.

Hack Debian Packaging to Build Engine X (Nginx) 1.17.0 with ModSecurity v3 Nginx Connector module.

We covered Engine X build in prior blog post. Suffice it to say that ModSecurity v3 is a prerequisite prior to building Nginx 1.17.0. After all, if ModSecurity v3 Nginx Connector was being offered in Debian repositories we would not be engaged in this hack. Accordingly, we review our links to libmodsecurity3 and download:

libmodsecurity3_3.0.3-2_amd64.deb
libmodsecurity3_3.0.3-2_amd64.deb.SHA256SUM
libmodsecurity-dev_3.0.3-2_amd64.deb
libmodsecurity-dev_3.0.3-2_amd64.deb.SHA256SUM

And install these required libmodsecurity3 dependencies:

Shell

dpkg -i libmodsecurity3_3.0.3-2_amd64.deb libmodsecurity-dev_3.0.3-2_amd64.deb

Additionally, I assume that, from prior blog post, the Nginx build requirements have been fulfilled, i.e.,

Shell

apt-get -t stretch-backports install debhelper dpkg-dev libexpat-dev libgd-dev libgeoip-dev libhiredis-dev libluajit-5.1-dev libmhash-dev libpam0g-dev libpcre3-dev libperl-dev libssl-dev libxslt1-dev po-debconf quilt zlib1g-dev

I assume we are downloading ensuing resources to our tekitl-tlacauhtli [work-space] directory:
debian-buster-packaging-for-nginx-1.17.0_and_modsecurity-connector-module.patch
debian-buster-packaging-for-nginx-1.17.0_and_modsecurity-connector-module.patch.SHA256SUM
Alternatively, you may hack together your own components and/or patch --as elaborated in previous blog post.

Fetch Debian Buster packaging for (older) Nginx source -- as well as Engine X 1.17.0 source.

Shell

wget http://deb.debian.org/debian/pool/main/n/nginx/nginx_1.14.2-2.debian.tar.xz
wget https://nginx.org/download/nginx-1.17.0.tar.gz https://nginx.org/download/nginx-1.17.0.tar.gz.asc

Verify Engine X source integrity and apply patch to Debian Buster packaging for Nginx

Shell

gpg --verify nginx-1.17.0.tar.gz.asc nginx-1.17.0.tar.gz
tar -xPf nginx-1.17.0.tar.gz
tar -C nginx-1.17.0 -xPf nginx_1.14.2-2.debian.tar.xz
ls nginx-1.17.0
cd nginx-1.17.0/debian
cat ../../debian-buster-packaging-for-nginx-1.17.0_and_modsecurity-connector-module.patch | patch --fuzz=0 -p1

Shell

cd ..
ls debian
chmod a+x debian/libnginx-mod-http-modsecurity.nginx

MOST IMPORTANT NOTE: Your build will fail if libnginx-mod-http-modsecurity.nginx is not executable; thus, make sure not to overlook the last command immediately above. Analyze following screenshot:

chmod a+x debian/libnginx-mod-http-modsecurity.nginx — MOST IMPORTANT!

Then we build Engine X 1.17.0 'the Debian way':

Shell

fakeroot debian/rules binary

Install Engine X (Nginx) 1.17.0 Into Metztli Reiser4 and/or Debian Stretch Backports for AMD64.

Fulfill dependencies by installing these packages:

Shell

apt-get -t stretch-backports install libmaxminddb0 liblua5.3-0 libfuzzy2 libyajl2

Then nginx-full with the integrated ModSecurity v3 Nginx Connector module may be installed thus:

Shell

dpkg -i nginx-common_1.17.0-1_all.deb nginx-full_1.17.0-1_amd64.deb libnginx-mod-http-auth-pam_1.17.0-1_amd64.deb libnginx-mod-http-dav-ext_1.17.0-1_amd64.deb libnginx-mod-http-echo_1.17.0-1_amd64.deb libnginx-mod-http-geoip_1.17.0-1_amd64.deb libnginx-mod-http-headers-more-filter_1.17.0-1_amd64.deb libnginx-mod-http-image-filter_1.17.0-1_amd64.deb libnginx-mod-http-modsecurity_1.17.0-1_amd64.deb libnginx-mod-http-subs-filter_1.17.0-1_amd64.deb libnginx-mod-http-upstream-fair_1.17.0-1_amd64.deb libnginx-mod-http-xslt-filter_1.17.0-1_amd64.deb libnginx-mod-mail_1.17.0-1_amd64.deb libnginx-mod-stream_1.17.0-1_amd64.deb

Again, your mileage may vary (YMMV) as you may need less or more packages from the pool generated and listed previously.

Elsewhere at this site, in Amatzintli collection, there exists a paged illustration of working LEMP PHP 7.3.6 component phpinfo function, link:
PHP 7.3.6 and Nginx 1.17.0 builds, as well as PHP app b2evolution on Metztli Reiser4 Virtual Machine Instance in VirtualBox 6.0.8: phpinfo()

Create, Download, and Setup, Modsecurity v3 Configuration and Rules.

The following instructions, gleaned from the final phase of Linux Journal² topic, do work appropriately for our task. As a matter of fact, that article provided me with flash of insight to hack a 'Debian way' procedure to build LEMP for Metztli Reiser4 / Debian Stretch Backports for AMD64.

Wielding root/sudo privilege, create modsec directory anchored at /etc/nginx to nest ModSecurity v3 rules/configuration

Shell

sudo mkdir --verbose /etc/nginx/modsec
cd /etc/nginx/modsec
sudo git clone https://github.com/SpiderLabs/owasp-modsecurity-crs.git

and rename ModSecurity v3 rules configuration file:

Shell

sudo mv -iv owasp-modsecurity-crs/crs-setup.conf.example owasp-modsecurity-crs/crs-setup.conf

Fetch and rename the libmodsecurity3 configuration corresponding to our previously installed libmodsecurity3 DEB packages:

Shell

sudo wget https://raw.githubusercontent.com/SpiderLabs/ModSecurity/v3/master/modsecurity.conf-recommended
sudo mv -iv modsecurity.conf-recommended modsecurity.conf

Then we generate a main.conf by writing the three(3) directives below -- which start with 'Include' -- to bind everything ModSecurity v3 together:

Shell

sudo tee main.conf > /dev/null << EOF
Include /etc/nginx/modsec/modsecurity.conf
Include /etc/nginx/modsec/owasp-modsecurity-crs/crs-setup.conf
Include /etc/nginx/modsec/owasp-modsecurity-crs/rules/*.conf
EOF

verifying content for ModSecurity v3 and Nginx Configuration — Eterm -t nIx -e xvi main.conf

We can verify proper content by using, say, tiny text editor xvi to open main.conf, as I did in the snapshot above in an Eterm small console.

References:
¹ Виктория Одинцова: Viktoriya Odintcova: 'Ce:1, Ome:2, or Eyi:3 ?'
² Linux Journal: Modsecurity v3 and Nginx
Zsh 'Permission denied' issue when trying to append to EOF

DISCLAIMER although due diligence has been applied, this resource is made available for testing/evaluation purposes on an AS IS basis. The procedure only reflects my own modifications, my limited testing, and the potential user(s) who execute(s) the procedures assumes all risks.

Please do not hold me or Metztli Information Technology (and/or its associates) responsible if the information provided here does not achieve the desired result. The information is provided AS IS and with the hope that it may be useful to the Internet community --especially those interested in PHP 7.3.5̶ 6 and/or Engine X (Nginx) 1.17.0 on Metztli Reiser4 / Debian for stretch-backports AMD64.

Notwithstanding, There is no implicit or explicit guarantee that the information presented here is accurate --even though due diligence was exercised during the procedure. Accordingly, if an user(s) decide to use the resources available here and/or implement the procedure and/or shell commands described here she, he, or them, do so at her, his, or their own risk. You have been forewarned.

Metztli IT, but not other entities, reserves the right to modify the content and/or even delete it, including blog post, without previous notice.

	`gpg --verify nginx-1.17.0.tar.gz.asc nginx-1.17.0.tar.gz`
	`tar -xPf nginx-1.17.0.tar.gz`
	`tar -C nginx-1.17.0 -xPf nginx_1.14.2-2.debian.tar.xz`
	`ls nginx-1.17.0`
	`cd nginx-1.17.0/debian`
	`cat ../../debian-buster-packaging-for-nginx-1.17.0_and_modsecurity-connector-module.patch \| patch --fuzz=0 -p1`

	`sudo tee main.conf > /dev/null << EOF`
	`Include /etc/nginx/modsec/modsecurity.conf`
	`Include /etc/nginx/modsec/owasp-modsecurity-crs/crs-setup.conf`
	`Include /etc/nginx/modsec/owasp-modsecurity-crs/rules/*.conf`
	`EOF`

Mon	Tue	Wed	Thu	Fri	Sat	Sun
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30
<< <				> >>

	`mkdir --verbose tekitl-tlacauhtli && cd tekitl-tlacauhtli`
	`wget http://deb.debian.org/debian/pool/main/p/php7.3/php7.3_7.3.4-2.debian.tar.xz`
	`wget https://www.php.net/distributions/php-7.3.6.tar.xz.asc https://www.php.net/distributions/php-7.3.6.tar.xz`

	`tar -xvPf php7.3_7.3.4-2.debian.tar.xz`
	`ls debian`
	`ls debian/control`

	`wget http://deb.debian.org/debian/pool/main/n/nginx/nginx_1.14.2-2.debian.tar.xz`
	`wget https://nginx.org/download/nginx-1.17.0.tar.gz https://nginx.org/download/nginx-1.17.0.tar.gz.asc`

	`cd ..`
	`ls debian`
	`chmod a+x debian/libnginx-mod-http-modsecurity.nginx`

	`sudo mkdir --verbose /etc/nginx/modsec`
	`cd /etc/nginx/modsec`
	`sudo git clone https://github.com/SpiderLabs/owasp-modsecurity-crs.git`

	`sudo wget https://raw.githubusercontent.com/SpiderLabs/ModSecurity/v3/master/modsecurity.conf-recommended`
	`sudo mv -iv modsecurity.conf-recommended modsecurity.conf`

	`mv -v debian php-7.3.6/.`
	`cd php-7.3.6/.`

	`ln -s debian/patches`
	`quilt push -a --fuzz=0`