Metztli Information Technology Blog Bits

"The World Didn't End Yesterday" [El Mundo No Terminó Ayer] video released by NASA prior to the Maya December 21, 2012, phenomenon is evidently intended to assuage potential concerns over the Judeo-Christian apocalyptic tendency inherent in the western psyche. On the other hand, for the Maya in Mexico and Guatemala this date is more of a celebration of their survival against all odds. The Maya have withstood genocide and brutal oppression by the Spanish since the early 1500s and until modern times under the rule of the Spaniards' descendants (criollos) in colonies such as 'Mexico', and Guatemala.

When I saw the original NASA video released a few days prior to the end of the 13 baktuns, in the Maya Long Count Calendar, I wondered why it was not released in the Spanish language as well. And after a couple of days passed, I decided that it might also be useful if at least Spanish subtitles were included.

I use GNU/Linux Debian and GNOME 3 as my main desktop. Thus, if I were to add subtitles I'd use free and open source software throughout this effort.

For a project like this, I had to download the original video at Youtube. Indeed, there multiple ways to go about the procedure but I used youtube-dl -- a python script to download Youtube videos. Under Debian, youtube-dl can be downloaded and installed thus:

apt-get install youtube-dl

To download a Youtube video all that is needed is to provide its link reference at the site. For easier use, at Google's Youtube site selecting the Share button will yield a shorter URL than what is seen at the browsers' top field. Feed that shorter URL to youtube-dl and use the -o option to give a name to the video you are about to download. For instance:

youtube-dl -o WorldDid-ntEnd.flv YoutubeShortLink

Will download the original video released by ScienceAtNASA and -o will rename the video to a custom file name of your choice with an .flv extension.

Once our target file is downloaded, then we can begin the process of adding the subtitles.

I came across GNOME-subtitles which, after becoming root or superuser, I was able to easily download and install as:

apt-get install gnome-subtitles

The application, once executed, enables us to load (select Video from the topmost bar) our previously downloaded video file and open up a temporary (until we assign it a name) subtitles file. We will be going backward or forward in the video as we synchronize our modifications and timing.

After we are satisfied with the created subtitles file, we save it (by default with a subtitle format SubRip *.srt) as well as character encoding (UTF-8), as shown below:

And then proceed into the actual encoding of its contents into a copy of the video we are manipulating.

Please note that GNOME-subtitles detected my Debian system character encoding appropriately. Sometime ago, when I installed the current instance of Debian into this particular machine, I selected English as default language with Spanish as secondary --both UTF-8.

Encoding Video with mencoder

The first tool I tried is mencoder, a command line utility that is part of the MPlayer: "a movie player which runs on many systems." Under Debian I installed utility as:

apt-get install mencoder

For the uninitiated, the long string of commands and options seems overbearing. If we ask for help by typing at our shell:

man mencoder

we get pages and pages of seemingly unintelligeble verbosity. Accordingly, we may search online for some simple example that matches our intention. In this particular case, I had articulated my intention as: add subtitles to an .flv output format so that it could be played with flowplayer and/or upload it to Youtube.

Suffice it to say that my initial encoding would not be rendered appropriately for the Spanish writing that I intended them. Subtitles would show question marks where accentuated characters should be rendered. Hence, I had to combine several hints and mencoder output help with sections in mencoder's manual pages to put together an articulate command string to achieve my aim.

mencoder inputFile.flv -oac mp3lame -ovc lavc -sub subTitlesFile.srt -subcp enca:es:utf-8 -subfont-text-scale 4 -subfont-outline 6 -of mpeg -o outputFile.flv

Note: make sure to have both, the video file and the subtitles file, at your current directory location in your Linux shell prior to executing mencoder command above to operate on those.

Although the above command may seem too complicated at first sight, mencoder helps us:

First we ask mencoder output for help on -oac:

mencoder -oac help
MEncoder 1.1-4.7 (C) 2000-2012 MPlayer Team

Available codecs:
copy - frame copy, without re-encoding (useful for AC3)
pcm - uncompressed PCM audio
mp3lame - cbr/abr/vbr MP3 using libmp3lame
lavc - FFmpeg audio encoder (MP2, AC3, ...)
twolame - Twolame MP2 audio encoder
faac - FAAC AAC audio encoder

Subsequently we request mencoder outpu help on -ovc:

mencoder -ovc help
MEncoder 1.1-4.7 (C) 2000-2012 MPlayer Team

Available codecs:
copy - frame copy, without re-encoding. Doesn't work with filters.
frameno - special audio-only file for 3-pass encoding, see DOCS.
raw - uncompressed video. Use fourcc option to set format explicitly.
nuv - nuppel video
lavc - libavcodec codecs - best quality!
libdv - DV encoding with libdv v0.9.5
xvid - XviD encoding
x264 - H.264 encoding

And finally ask mencoder -of help:

mencoder -of help
MEncoder 1.1-4.7 (C) 2000-2012 MPlayer Team

Available output formats:
avi - Microsoft Audio/Video Interleaved
mpeg - MPEG-1/2 system stream format
lavf - FFmpeg libavformat muxers
rawvideo - (video only, one stream only) raw stream, no muxing
rawaudio - (audio only, one stream only) raw stream, no muxing

Most importantly for this project is to adequately render Spanish accents on vowels, as well as other characters, among those are: ¿ and ¡. I was able to achieve that above with option fragment: -subcp enca:es:utf-8

Please see man mencoder and find or use utility grep to locate text string option -subfonts in output for explanations of the other options in the command above.

Well, I ended up with half the size of the original video (and it can be seen in the quality of the video) but I believe I achieved what I set out to do in the first place: convey the NASA scientists' message in the Spanish dialect. Hopefully those Spanish colonies 'Mexico' and Guatemala, as well as Central America, will take conscience of the importance of the Maya civilization and stop trying to annihilate their wonderful inhabitants.

From Google's Corporate Blog for 'Latin' America: Fin del 13er Baktún de los Mayas [End of the Maya's 13 Baktun(s)].

Friday, December 21, 2012 | 00:01

Mathematics, science and astronomy are issues for which we always have a passion. Therefore, with our doodle today, we celebrate the end of 13 Baktun of the Maya Long Count calendar. But what is its significance?

"Los Maya fueron matemáticos y astrónomos avanzados que calculaban los ciclos de la luna y del sol. Tenían formas muy precisas de medir el tiempo, una de estas formas es el Sistema de Cuenta Larga, en el que cada año tiene 18 meses de 20 [días cada uno]¹. El sistema contempla también otras unidades como el Katún, equivalente a 20 años de nuestro calendario, y el Baktún, que equivale a 394 años."

The Maya were advanced mathematicians and astronomers who calculated the moon and the sun's cycles. They had very precise forms of measuring time, and one of these forms is the Long Count System, in which each year has 18 months of 20 days [each]². Their system also took into account other units such as the Katun, equivalent to 20 years in our [West's] calendar, and the Baktun, which equals 394 years.

The importance of arriving at 13 Baktun, is that, it's very distinct from what happens with our [West's] calendar, there is no 14° Baktun. The count goes back to zero...continues below

continuing... But unlike of all the disaster stories that surely you may have come across, at the end of 13 Baktun, [analogous to the sequence] that ends on Sunday in our [western] calendars, it will be Monday once again and a new week begins. Those who have studied the theme, explain that this is how their calendrical system functions, and, though it represents the end of a cycle, it has no catastrophic significance.

DuckDuckGo search engine parodied the media disinformation hype.

Our doodle today represents the actual date, December 21, 2012, as well as 13 Baktun, to form the word Google. We hope that you enjoy as much as us this doodle, and may the start of the next Baktun be prosperous for all.

¹mas un periodo conocido como Wayeb o Uayeb que consistían de 5 días adicionales sin nombre³.

²plus a period known as Wayeb or Uayeb consisting of 5 additional nameless days³.

³Wikipedia article: Maya Calendar

DISCLAIMER although due diligence has been applied, the above post is intended as a proof of concept for encoding Spanish language subtitles in videos.

Please do not hold me or Metztli Information Technology, or its associates, responsible if the information provided here does not achieve the desired result. The information is provided AS IS and with the hope that it may be useful to the Internet community.

Notwithstanding, There is no implicit or explicit guarantee that the information presented here is accurate. Accordingly, if an user(s) decide to implement the procedure or shell commands described here she, he, or them, do so at her, his, or their own risk. You have been forewarned.

I reserve the right to modify the blog and even to delete it without further notice.

Coming from OS/2 and learning my first GNU/Linux commands at the OS/2 command shell -in an ported public domain Korn shell (ksh) emulated environment- I can not help but peek, once in a while, into the pond from which I emerged. A couple of days ago, I visited the OS/2 World forum and scrolled the initial page for something new. Perhaps, I thought, Serenity Systems International -the last vendor of an fixpacked OS/2 known as eComStation- might have developed an booting procedure or hack that emulated the Linux booting procedure.

If true, I continued submerged in my revery, then I could build an OS/2 instance into an virtual machine to be hosted in an Infrastructure as a Service (IaaS) provider like Amazon. To my dismay, I only found what I regarded as an mundane issue that could be resolved in a jiffy (if an OS/2er were to be a little more open minded) -that of accessing OS/2 created JFS partition from an GNU/Linux distribution.

In preparation for a migration from an OS/2 installed base to an modern Linux alternative, the user was interested in knowing how to access OS/2 created JFS partitions from an Ubuntu Live distribution. Her/his concerned lied in the fact that whenever he tried to mount an certain OS/2 JFS partition from the Live Ubuntu distribution, the message in his Linux shell output would be along the form:

mount: wrong fs type, bad option, bad superblock on /dev/xyz,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so

Well I just happened to have reinstalled an Linux Debian unstable distribution. No, the install is not from a “live” CD but it will help to illustrate an potential solution to the OS/2 administrator evaluating methods to migrate existing OS/2 deployments into GNU/Linux.

First of all, to exercise administrative duties in enterprise Linux, the user becomes root (or super user). Please note that from the perspective of Debian cfdisk utility, OS/2 created JFS partitions will be seen as of file system (FS) Type: Unknown (35). The reason may possibly be due to the fact that the LVM.EXE or LVMGUI.CMD disk management utilities of an appropriately upgraded OS/2 require a created physical partition to be of type LVM; otherwise formatting in the OS/2 implementation of Journaling File System (JFS) will not proceed. Hence, the GNU/Linux cfdisk utility does not appear to recognize the type of the OS/2 created partition and regard those as Unknown (35).

Observe the screenshot below:

cfdisk /dev/hda

Additionally, note that if you hard drive is IDE, of course, you will see the Primary partitions as hda1, hda2, and hda3. Partition hda4 represents the whole extended partition; when this latter is sub-partitioned, the first extended (sub)partition is hda5, hda6, etc.. Also note from the snapshot that if Primary partitions hda2 and/or hda3 are non existent, their places are reserved; as can be observed, the very first logical (sub)partition skips over the reserved slots for Primary partitions and starts at hda5. Obviously, if your hard disk is SCSI the device seen with the cfdisk utility will be of the form sda1, sda2, etc..

Please note that Red Hat and Fedora, at the very least, may also refer to IDE drives as sda1, sda2, etc.. I just want to call your attention to the fact that under all conditions, you should verify that the Linux operating system and you are on the same page. Fedora 11, in my limited tests, was able to mount JFS partitions. Notwithstanding, for this blog I used Debian.

BACK UP YOUR JFS PARTITIONS DATA.

Avoid unnecessary woes by diligently backing up all your OS/2 JFS partitions data during the migration planning stage. Technology, being a human creation and subject to unanticipated variables, is not perfect.

Additionally, please verify that you have the Debian jfsutils package. As root user, from the command line in your Linux shell you can simply type:

apt-get update

apt-get install jfsutils

Or if you prefer to install those visually with the synaptic package manager, from the Debian desktop menu, select System, Administration, and subsequently press the Synaptic Package Manager item from the cascading options.

Alternatively, as the root user, you may just type:

synaptic

and the package manager will come into existence, as well. Either way you access Synaptic, press the Search button from the wide menu of the utility and when the search utility appears, proceed to enter the text string "jfs" into the field. After some seconds, the jfsutils will be listed among other packages that met the criteria. Selecting the item jfsutils from the list will provide additional information: Utilities for managing IBM's Journaled File System (JFS) under Linux. Hence, after installtion of the package, you will have in your Linux Debian distribution: fsck.jfs, logdump, logredo, mkfs.jfs, xchkdmp, xchklog, xpeek.

MAKE SURE YOUR TARGET DEVICE IS NOT MOUNTED.

Reading the man page for jfs_fsck (or fsck.jfs) we note the warning: serious damage may occur if we check a mounted file system in other than READ ONLY access.

man jfs_fsck

After verifying that our target JFS partition(s) is(are) not mounted, and scrolling further down in the man jfs_fsck information, we come across the command to form to check our OS/2 created JFS partiton(s): jfs_fsck -v -f /dev/hdxy.

As an specific example, I selected the 26GB /dev/hda10 seen in the initial cfdisk snapshot at the beginning of this blog. Accordingly, if the hypothetical Linux user xiuhtecuhtli became root (or super user) of his GNU/Linux Debian sytem, he would enter the following command to check and fix (-f) his previous OS/2 created and formatted JFS partition:

jfs_fsck -v -f /dev/hda10

After performing the adequate JFS modifications on the OS/2 partition, the jfs_fsck front end utility for fsck.jfs finishes; and xiuhtecuhtli would be able to mount his 26GB OS/2 created JFS partition in an relevant directory created under /mnt, be typing:

mount /dev/hda10 /mnt/hda10

Please note that the above procedure will not work on OS/2 Logical Volume Manager (LVM) volumes spanning multiple devices/partitions. If an user tries to access one of the spanned entities composing an OS/2 LVM volume, the mount process will fail; also, if an file system check is performed on a device/partition component of the spanned OS/2 LVM volume, it may corrupt the target. Accordingly, make sure that you know -from the OS/2 side- which units are part of an spanned OS/2 LVM volume. For instance, see this snapshot of an past OS/2 Java-based utility that can be invoked as lvmgui.cmd in your OS/2 command shell environment.

Java 2 SE 1.4.1 native port for OS/2 is by Golden Code Development (GCD).

One final note. If an GNU/Linux root (or super user) mounts an OS/2 created and formatted partition -even if JFS checks and fixes are performed from the Linux side- an regular non-root user will likely not be able to view the contents of that file system. The reason is because all or most of that data will not have appropriate permissions, as seen in this sample ls -l command operating on the directory /mnt/hda10/xxe-std-3_5_1:

ls -l /mnt/hda10/xxe-std-3_5_1

total 28
d--------- 7 root root 256 2007-02-02 18:00 addon/
d--------- 7 root root 4096 2007-02-02 12:17 bin/
d--------- 2 root root 4096 2007-02-24 14:52 binHTML/
d--------- 9 root root 4096 2007-02-01 17:06 demo/
d--------- 17 root root 4096 2007-02-01 17:06 doc/
d--------- 2 root root 256 2007-02-24 12:44 Xiuhtecuhtli/
d--------- 2 root root 4096 2007-02-02 20:13 webstart/
---------- 1 root root 6778 2007-02-24 12:33 XXE-STD-3_5_2.zip

What the OS/2 turned Linux admin needs to do is to apply pertinent permissions to the files so that regular users are able to use and/or access the data. For an specific instance, if the root user needs to modify all the files in a directory to be world readable and or writable (r or w) use the chmod command with -R option to specify recursively all of the files in that directory.

chmod -R a+r /mnt/hda10/xxe-std-3_5_1

ls -l /mnt/hda10/xxe-std-3_5_1

total 28
dr--r--r-- 7 root root 256 2007-02-02 18:00 addon/
dr--r--r-- 7 root root 4096 2007-02-02 12:17 bin/
dr--r--r-- 2 root root 4096 2007-02-24 14:52 binHTML/
dr--r--r-- 9 root root 4096 2007-02-01 17:06 demo/
dr--r--r-- 17 root root 4096 2007-02-01 17:06 doc/
dr--r--r-- 2 root root 256 2007-02-24 12:44 Xiuhtecuhtli/
dr--r--r-- 2 root root 4096 2007-02-02 20:13 webstart/
-r--r--r-- 1 root root 6778 2007-02-24 12:33 XXE-STD-3_5_2.zip

Additionally please note that for an user in Linux to be able to use data s/he may need additional permissions such as being the owner and/or group member allowed to manipulate the data. Accordingly, the OS/2 admin turned to Linux admin (root user) should make use of the chown command to achieve her/his purpose.

As the OS/2 remains anchored in the past and with no supporting organization providing continued development -much less a clear road map for its future- OS/2 admins will be forced to migrate. Metztli Information Technology will be glad to provide migration and/or support services provided the target platform is resilient, secure, open source - open standards: GNU/Linux.

DISCLAIMER Please do not hold me or Metztli Information Technology responsible if the information provided here does not achieve the desired result. The information is provided AS IS and with the hope that it may be useful to the Internet community --especially OS/2 users and/or administrators considering GNU/Linux as their organizations' migration platform.

Notwithstanding, There is no implicit or explicit guarantee that the information presented here is accurate --even though due diligence was exercised during the procedure. Accordingly, if an user(s) decide to implement the procedure or shell commands described here she, he, or them, do so at her, his, or their own risk. You have been forewarned.

Metztli IT reserves the right to modify the content of the post or even to delete it altogether.

Note: This list collection of ia32-lib packages was successfully tested on x64 Fedora 10. I did not test them on x64 Fedora 11 nor x64 Fedora 12. Recently, however, I realized that the packages listed here do not even exist in x64 Fedora 13. Accordingly, I updated the list of packages in this list to reflect modifications done in the x64 Fedora 13 distribution. In order to minimize confusion, I deal with the issue and make available the relevant updated list in another blog entry: Hosting Providers Embrace of The Cloud Enabled by Open Source Virtualization Technologies. 08/10/2010

In an past how-to blog entry I described an procedure to install the 32-bit IBM Lotus Symphony office productivity suite under an 64-bit GNU/Linux Fedora 9 distribution. I mentioned that I had used the GNU/Linux 64-bit Debian ia32-libs suite package as a guide to figure out which packages would provide equivalent functionality in 64-bit Fedora 9 distribution. Notwithstanding, I was forced to realize that metztli-F9-ia32-libs.text, the list of packages that I made available for others as an aid to solve an similar issue as my own, was version specific. This short post is an update on metztli-F9-ia32-libs.text so as to make the list of packages more generically applicable to Fedora 9, Fedora 10, and Fedora 10+.

An kind comment by an user that used the pseudonym "Nobody" actually drew my attention to the need for the packages listed in metztli-F9-ia32-libs.text not to be version specific. At the time of Nobody's comment, well, I thought that making the packages listed in metztli-F9-ia32-libs.text version agnostic would be achieved simply by editing the file with vim, Elvis, or any other text editor. As an way of illustration, I downloaded metztli-F9-ia32-libs.text and opened it with my vim text editor:

In effect, once metztli-F9-ia32-libs.text had been opened for editing in the vim text editor, pressing the Esc key at our keyboard and following that with the colon character ":", the percent character "%", and the letter "s" for substitution, we achieve 77 substitutions at once. Below is an snapshot of the end of metztli-F9-ia32-libs.text file showing the substitutions in that relevant section:

Nevertheless, there are a few files that require special attention --if they are to support 32-bit applications in an distribution release agnostic GNU/Linux 64-bit Fedora. Those are listed below, first as the file was listed in the older metztli-F9-ia32-libs.text and subsequently as listed in metztli-Fedora_agnostic-ia32-libs.text:

• compat-libstdc++-33-3.2.3-63.i386 to compat-libstdc++-33.i386
• libaio-0.3.106-4.2.i386 to libaio.i386
• jack-audio-connection-kit-0.109.2-1.fc9.1.i386 to jack-audio-connection-kit.i386
• java-1.6.0-openjdk-1.6.0.0-0.15.b09.fc9.i386 to java-1.6.0-openjdk.i386
• compat-expat1-1.95.8-4.i386 to compat-expat1.i386
• libgcrypt-1.4.0-3.i386 to libgcrypt.i386

And I had a duplicate of jack-audio-connection-kit-0.109.2-1.fc9.1.i386 ; hence metztli-F9-ia32-libs.text should have contained a list of 80 74 packages (a big thanks to "an observer" 06-09-2009), instead of the 81 that I asserted in the post IBM Lotus Symphony 1 on 64-bit GNU/Linux Fedora 9 Sulphur. referenced initially.

Well, after detailing the procedure that I followed to generalize the prior 64-bit Fedora 9 specific package support for 32-bit applications listed in metztli-F9-ia32-libs.text, I now make available metztli-Fedora_agnostic-ia32-libs.text. This file will list version agnostic 32-bit packages that should work under 64-bit Fedora 9, Fedora 10 and possibly higher, since the packages providing the equivalent of Debian ia32-libs package likely are Fedora distribution release independent.

As referenced in the alluded past how-to blog entry, after the user downloads metztli-Fedora_agnostic-ia32-libs.text to his 64-bit Fedora distribution, s/he should install the listed packages by entering suggested commands at her 64-bit GNU/Linux Fedora shell terminal.

As an specific illustration, we assume that the user has become the root or super user in her 64-bit GNU/Linux Fedora 9, Fedora 10, or later; we further assume that the file metztli-Fedora_agnostic-ia32-libs.text was downloaded to her current file system location in her shell. Then the super user or root should install the support for 32-bit applications under her 64-bit Fedora by typing below commands:

for i in $(< metztli-Fedora_agnostic-ia32-libs.text); do yum -y install $i; done

DISCLAIMER although due diligence has been applied, the metztli-Fedora_agnostic-ia32-libs.text resource is made available for testing/evaluation purposes on an AS IS basis. Those files listed in the resource only reflect my own modifications on my limited testing --as expounded above and elswhere-- and the potential user that manipulates any or all of those (files) assumes all risks.

Please do not hold me or Metztli Information Technology responsible if the information provided here does not achieve the desired result. The information is provided AS IS and with the hope that it may be useful to the Internet community --especially AMD 64-bit GNU/Linux Fedora 9, Fedora 10 and higher distribution release, users.

Notwithstanding, There is no implicit or explicit guarantee that the information presented here is accurate --even though due diligence was exercised during the procedure. Accordingly, if an user(s) decide to implement the procedure or shell commands described here she, he, or them, do so at her, his, or their own risk. You have been forewarned.

Metztli Information Technology

...An [almost] infinite capacity for taking pains [engineering]¹:

IT solutions based on free and open source software (FOSS) and select open standards proprietary alternatives.

¹Jane Ellice Hopkins, Work Amongst Working Men, 1870
Camp's Unfamiliar Quotations From 2000 B.C. To The Present.