Have you ever seen a bunch of ^M characters in a text file? This odd character at the end of a line can also be represented as a Ctrl+M or <CTRL>M. You don’t know what it is, and you want it to go away.

Today, I’ll help you understand what that odd ^M character is, why it is in some of your documents, and how to get rid of them.

All About the Newline

In text documents, lines are separated by what is called a newline (also known as a line break or end of line). Different operating systems have different character codes that represent this newline.

For example:

• DOS-based systems, including Windows, as well as a number of other, older non-Unix OSes use a carriage return (CR) character followed by a line feed (LF) character.
• Commodore and the old Apple OSes before OS X used a CR character. Since OS X+ is based partly on BSD, which is in turn based on Unix, the new Apple OSes use the Unix newline method described below.
• Unix and its derivatives (Linux, BSD, and others) all use the single LF character to represent a newline.

So what does all of this mean to you? It means that text documents that come from a Windows system won’t always play nice in Linux. The converse is true however. If you create a text file in Linux, many programs will fail to recognize the single LF as a newline and will render the document without any line breaks.

dos2unix and unix2dos to the Rescue

Fortunately, there are a couple of very easy to use programs that make dealing with this file format mess much easier. They are dos2unix and unix2dos.

These programs basically do exactly what their name implies: dos2unix takes a file and converts all DOS-style newlines to Unix-style newlines. unix2dos takes a file and converts all Unix-style newlines to DOS-stlye newlines.

I put Ubuntu and CentOS in the title because I’m going to give instructions for installing these programs on each of these distros. Why just these two? They are the two that I work with most often and are representative of the lion’s share of what people are using these days. If you need help with a different distro, please let me know in a comment.

Installing dos2unix and unix2dos in Ubuntu

There aren’t any dos2unix or unix2dos packages that can be found in Synaptic; howver, there is a packages that will install them for you. Simply open up Synaptic and install the tofrodos package. If you are like me and prefer to do this from the command line, you can run the following command:

That’s all there is to it. Not only will dos2unix and unix2dos install, but alias programs fromdos and todos will be installed as well. These additional programs work in the same manner, so it’s purely a matter of preference which ones you use.

Installing dos2unix and unix2dos in CentOS

I really thought that I had to install these in CentOS, but amazingly, the programs are already installed by default. I tested this in a Virtual Machine fresh install, and the programs were there on the first boot.

So, CentOS users, you’re already good to go.

Using dos2unix

Fortunately, using these programs couldn’t be easier.

Let’s say that you are in a Terminal (Applications > Accessories > Terminal) viewing text files. Maybe you just downloaded a new WordPress plugin and you are reading the readme.txt file. It doesn’t really matter. However, there is a problem. The readme.txt file has a bunch of ^M characters at the end of each line, and it’s really distracting.

Simply exit out of the editor you are currently in, since the file will be modified, and run the following command:

If there are multiple files, you can specify each one with a space separating each. For example:

The program doesn’t produce any output. Simply reopen your text file and look at all the beautiful non-existant ^M’s.

Using unix2dos

The unix2dos program has the exact same syntax as dos2unix. However, I thought it might be helpful to describe a situation in which you might need to use it.

You’re working on a project. You’ve just sent out your batch of files, and another member on the project complains that you are being a jerk and remove all the newlines in your text files. This other member is most likely using a Windows application that doesn’t understand the Unix newline format. Rather than getting into a format war, it’s typically better and quicker to simply convert your text files to a DOS/Windows format.

If you have a folder full of files that all need to be converted, simply run:

Now you can send these new files to your project group and hopefully avoid any more unproductive drama.

File Formats in Vi

If you happen to use Vi, you can change Vi back and forth between DOS and Unix modes for newlines with a simple command. “:set ff=dos” sets the editor to use DOS newline encoding and will save the file in a DOS-encoded format. “:set ff=unix” sets the editor use Unix newlines and will save the file in a Unix format.

Note that changing the format to dos from unix will always work as expected. This is because any file that contains just LF characters for new lines will be converted to CRLF while lines that already end in CRLF will be left as is.

If your Vi config defaults to a unix format and you open a DOS file, you will see the ^M characters. You can either use the dos2unix conversion utility first or change Vi first to the dos format and then to unix.

If you’d like Vi to default to DOS or Unix formating each time you start a new Vi session, you can add the setting to your ~/.vimrc file. In that file, either add “set ff=dos” or “set ff=unix” depending on your needs. Note the lack of the colon, :, in the .vimrc entries.

For more information on the ff or fileformat setting in Vi, check out the official documentation.

Closing Thoughts

Maybe one day we won’t have to worry about these types of things. For now however, it’s good to know the tools that make these problems easily manageable.

BTW, Happy Daylight Savings Day everyone. I hope you enjoyed the loss of an hour of sleep.

You just got your new CentOS dedicated server, and you notice that times in your logs aren’t quite right. You check the time from the command line (run “date”), and find that the timezone is set to US Eastern or some other timezone. How do you get this changed?

Unfortunately, this is not an easy thing to figure out. Fortunately though, it’s not hard to do with the right directions.

Please note that you have to have root access to be able to make the changes detailed below.

There are a series of time zone files located at /usr/share/zoneinfo. Select the appropriate named timezone for your location. For my location, Oklahoma, USA, I actually have two that I can select from: America/Chicago or US/Central. Make note of the appropriate folder and file for your timezone.

The active timezone used on your system is in the /etc/localtime file. The default will vary depending on your server host but often seems to be EST or EDT (depending on the time of year you are checking). We simply need to replace this file with the file we selected in the previous step.

Now, I say replace, but I actually recommend creating a link to the pertinent file rather than actually making a copy. Let me explain the reasoning for this quickly.

A crucial part of the timezone shift calculations is the daylight savings calculations. Many people don’t realize this, but the daylight savings days were changed in 2007. When this change happened, all servers needed to be updated with new zoneinfo files.

If your server has an automated process to update these files when daylight savings calculations change, your /usr/share/zoneinfo files will be updated but the /etc/localtime file will not be. So, if you simply made a copy of the file, you’ll have to know when these updates come out and manually copy the file over. If you create a link, everything will take care of itself.

Enough of the banter. On with the show.

First, make a backup of the existing localtime file. It’s always good practice to make backups of original config files.

Next, create the link:

Make sure to replace “America/Chicago” with the directory (if your zone has one) and filename of the timezone you wish to use.

Now you just need to test your change. Run “date” from the command line, and ensure that the appropriate time, date, and timezone are reported.

Sorry about the late post everyone. I had a long week and went to bed before making sure a post was queued. Hopefully you’ll forgive me.

I’m working on a project where I needed to generate a MIME type given a file name. Not only did I need to create a solution that worked, I also needed the solution to be compatible with PHP 4/5 and not require any additional software to be installed on the host. I thought this would be a simple matter of finding a PHP function that does this. Unfortunately, things were not as simple as this.

Problems with finfo_open

I found a very helpful PHP, Mime Types and Fileinfo post on Jelly and Custard. The “Mime Types in PHP 4.x” seemed to be exactly what I wanted. I quickly tried it out on my server, and it failed instantly with the following error:

Since I have PHP 5.2.6 running on my dev server, I was very confused since the post said that this was a “PHP 4.x” solution. I pulled up the PHP doc on the finfo_open function and was very surprised to see that the function is a PHP 5.3.0+ function. Fortunately, my version of PHP was just before this so that I could actually catch the error.

Turns out that the Jelly and Custard post points to another post, Installing PECL Modules, where instructions are given for installing the Fileinfo PECL Package. Installing this package allows versions prior to 5.3.0 to use the finfo_* functions via PECL.

This is not what I want. I’m working toward a solution that doesn’t require the installation of any additional software, and so far I’ve only seen solutions that require this. There has to be a better way.

Problems with mime_content_type

I then found the mime_content_type function. This looks better. However, PHP has marked this function as deprecated due to the PECL Fileinfo package. Relying on a deprecated function has numerous problems: it may throw warnings if used on versions of PHP that know the function is deprecated, most likely won’t receive any updates in the future, and could possibly be removed from future versions of PHP.

In addition to the deprecated issue, the mime_content_type function is laced with problems. In order to use the function the PHP on your system must have been built with the “--with-mime-magic” option. The function also relies upon the “mime_magic.magicfile” ini configuration to tell it where to find the magic file used to detect the MIME type of the file. This magic file may or may not exist/may or may not be readable.

Initial Testing

These problems led to interesting results when I tested it on my CentOS dedicated and Hostgator servers.

On my CentOS 5.2 dedicated server, I have PHP 5.2.6. The PHP build on this server was not built with the “--with-mime-magic” option, so it doesn’t have access to the mime_content_type function at all. In addition to this, since the version is just before 5.3 and I haven’t installed the Fileinfo PECL package, my dedicated server does not have access to either of the official PHP solutions without installing additional software.

On my Hostgator shared server, I have access to PHP 5.2.8. Unlike my dedicated server, this server’s PHP build was built with the “--with-mime-magic” option. “This is great,” I thought. I ran a test, and the mime_content_type function did indeed exist. Like my CentOS server, my Hostgator server does not have the Fileinfo PECL package and does not have PHP 5.3, so the finfo_* functions are no go.

I did some more testing on my Hostgator server and was disappointed to find that the mime_content_type function exists yet is completely worthless. I tested file after file ranging from simple text files, to HTML documents, to a variety of image types. Every single test failed to produce a MIME type. When I say “failed”, I don’t mean that the program crashed with an error. The failure was worse than this, it simply returned an empty string to every single request.

I found that my Hostgator server’s PHP is set up to use the “/usr/local/apache/conf/magic” file to do it’s MIME magic. However, this file is not able to be read by my user. This means that the version of PHP might as well not have been built with the “--with-mime-magic” option at all.

It seems clear to me by now that there will not be an easy solution to this problem.

Final Solution

What I need is a solution that will first try to use the Fileinfo functions since they are the current standard. It will then fall back to using the mime_content_type function if and only if the function exists. Since the function is deprecated and PHP versions don’t actually package the replacement functions natively until 5.3.0, I need to also protect the code against conditions where neither Fileinfo nor mime_content_type are available. The final fallback will be manually generating the MIME type based upon an array match.

Using the fallback is not desirable since I probably won’t be updating the MIME types in the array very often, if at all. However, having it is better than having the code completely fail for common, present day MIME types. Hopefully, the conditions necessary to rely upon this fallback will become more and more rare as time goes on.

In addition to checking for the existence of the functions I’ll try to use, I need to make sure that the methods tried actually produce results. If a method fails to produce a non-empty value, I’ll move on to the next method.

The Code

I got the original idea for this code from a comment by svogal on the PHP doc site. I modified it to match my desired final solution.

The code is far to large to post here. You can download it here.

The array that I built for the mime types is quite large. Since I have a /etc/mime.types file (which is standard in most distros and provided by the mailcap package), I simply used it to generate my array. I quickly built a Perl script that parses through the mime.types file and outputs a file containing the PHP code to create the array. This script makes it easy for me to update the array any time the mime.types file is updated.

You can download my Perl script here. Simply run “perl generateMimeTypes” from the command line to build the array. The array code is put in a file called mime_type_var.code.

Examples

Simple Use

The code is fairly simple to use. Simply include or require the code in your own script and then call the get_file_mime_type function by passing the file’s path as the parameter.

This produces the following output:

Note that PHP’s more advanced functions have the ability to dig into the content of the file to identify the MIME type, so if your system can make use of those functions and the file is not a JPEG image, the results could vary.

Using the debug Parameter

I’ve also provided a debug parameter that allows you to also get information on what method was used to get the MIME type. This may be helpful if you need to determine how sure you are that the detected MIME type is accurate. When the debug parameter is used, an associative array will be returned with a mime_type key and a method key.

This produces the following output:

The possible methods are: fileinfo, mime_content_type, from_array, and last_resort. The first two should be self-explanitory. from_array means that the resulting MIME type was pulled from the array built into the function. last_resort means that the type could not be identified which results in a generic MIME type of “application/octet-stream” being used.

Conclusion

Frankly, I’m disappointed in the solutions provided by PHP to detect MIME types. I have access to a very stable, highly-regarded shared host and an extremely powerful, up-to-date dedicated host, neither of which have the ability to use either of the PHP solutions to detect MIME types without adding additional software.

I don’t remember which version of PHP my dedicated box started with, but I did upgrade to 5.1.6 using the standard repository back in October. I was only able to update to 5.2.6 by using the Utter Ramblings repository by Jason Litka. True, I could compile and install 5.3 or the PECL package myself, but I produce software for people who don’t have a clue what a compiler is, let alone what repositories or PECL extensions are.

It’s things like this that make development a pain. I could always just be a jerk of a developer, have the line “requires PHP 5.3+”, and tell all people that can’t run the software to check the requirements. However, I think that my job is to make the end-user’s life easier, not more complicated.

Hopefully others that have had woes dealing with this situation can make use of my code, and we can all just wait a few years until the majority of hosts have a PHP version that supports these calls natively.

Yesterday, I talked about how to get the most out of running regular expressions in PHP. The reason that I needed to dig in deep on regular expression syntax with PHP is because I needed to write some regular expressions that deal with Unicode characters.

After much reading, I believed that I knew everything that I needed. I started writing some regex strings and testing the code. Unfortunately, every time I ran a test with a string that contained Unicode characters, the match failed. When I removed the Unicode characters from the string and tested again, it would work. I was baffled.

Finding the Problem

I had the regex testing characters (‘\X’, ‘\pL’, etc) inside of a character class, such as ‘[\X-]‘, since I was creating a regex to test for domains. I wrote a really simple rule by simply looking for ‘/^\X$/’ and testing the regex with a single Unicode character. Amazingly, having the ‘\X’ outside of the square brackets changed everything as I now received the following very concerning warning:

Since PHP uses the PCRE engine to run regular expressions, I started to dig into it. I found out that I could query PCRE directly. I ended up with something very similar:

It looked like the error was coming from PCRE itself. I searched around for a while thinking that I could simply install a new package using yum. I hoped to find something like pcre-utf8, pcre-unicode, php-pcre-unicode, or something to make it simple and quick to add this support since I much prefer using package management tools rather than compiling and installing from source.

Unfortunately, no such package exists. This support is something that must be an option that PCRE is compiled with, and my CentOS repository only has packages that don’t include that support. After much digging around, I found that this isn’t necessarily CentOS’s fault as this package has carried over from the RHEL (Red Hat Enterprice Linux) side of things.

A great way of checking to see if this is an issue on your system is by running the following:

$ pcretest -C
PCRE version 6.6 06-Feb-2006
Compiled with
  UTF-8 support
  No Unicode properties support
  Newline character is LF
  Internal link size = 2
  POSIX malloc threshold = 10
  Default match limit = 10000000
  Default recursion depth limit = 10000000
  Match recursion uses stack

This is the output that I received. Notice the “UTF-8 support” and the “No Unicode properties support” lines. This means that PRCE was compiled with the “–enable-utf8″ configure option which allows PCRE to recognize and work with UTF-8 encoded strings. However, it wasn’t compiled with the “–enable-unicode-properties” configure option which works in conjunction with the enable-utf8 option to add support for the ‘\p’, ‘\P’, and ‘\X’ character classes.

This seems to have been an oversight when the rpm file was first put together. Fortunately, there is a way to fix it.

Fixing the Problem

Since I’m sure that many of you are like me and would rather not manually compile and install software outside of the package management system, the solution is to update the rpm to have the option that it needs and install it.

I had never done this before. Fortunately, I found a very helpful guide that details this process out very nicely: How to patch and rebuild an RPM package.

I have provided the new rpm file that I have built at the bottom of this post. If you don’t care about all this jibber-jabber, you can skip down there and grab the file. However, if you would like to learn how to address this issue yourself or have a system that my file will not support, please read on to see how I rebuilt the rpm with the new option.

Rebuilding the rpm

1. The first thing I did is set up my ~/.rpmmacros file and src/rpm folder structure as detailed in the Setup section of the guide that I’m following. I’ll simply refer you over there as it doesn’t need repeating here.
2. I needed to grab the source rpm for the current version of PCRE on my platform. I’m on CentOS 5.2 with version 6.6 of PCRE. I found the matching source rpm file (pcre-6.6-2.el5_1.7.src.rpm) here.
3. I then installed the source rpm in order to gain access to its files:

   $ rpm -ivh pcre-6.6-2.el5_1.7.src.rpm

   This put the necessary files into my ~/src/rpm/SOURCES and ~/src/rpm/SPECS folders.

4. I opened up the ~/src/rpm/SPECS/pcre.spec file and found the following line:

   %configure --enable-utf8

   I changed it to include the Unicode properties option:

   %configure --enable-utf8 --enable-unicode-properties

   I then saved and closed the file.

5. This is the only change that I needed to make. So, now it is time to build the new rpm file. I simply ran the following to build it:

   $ rpmbuild -ba ~/src/rpm/SPECS/pcre.spec

   Toward the end of the large amount of output, I received the following:

   Wrote: ~/src/rpm/SRPMS/pcre-6.6-2.7.src.rpm
   Wrote: ~/src/rpm/RPMS/x86_64/pcre-6.6-2.7.x86_64.rpm
   Wrote: ~/src/rpm/RPMS/x86_64/pcre-devel-6.6-2.7.x86_64.rpm
   Wrote: ~/src/rpm/RPMS/x86_64/pcre-debuginfo-6.6-2.7.x86_64.rpm

   This tells me exactly where I can find my new source rpm and rpm files.

Updated rpm File for CentOS 5.2 64-bit

If you are running a 64-bit version of CentOS 5.2, the following file should work for you. If you have a different architecture, Linux distro, or encounter any errors when trying to install this file, then you should follow the instructions above to build an rpm that is suitable for your distribution.

pcre-6.6-2.7.x86_64.rpm – PCRE 6.6 for CentOS 5.2 64-bit

Installing New rpm

Now that I have my new rpm file, I just need to install it. Since I already have a pcre package installed, I need to tell the rpm command to update rather than install. The following command does this for me:

# rpm -Uvh ~/src/rpm/RPMS/x86_64/pcre-6.6-2.7.x86_64.rpm

Notice that I need to be root to run this command.

Finally, to verify that everything worked, I ran the pcrecheck program again:

$ pcretest -C
PCRE version 6.6 06-Feb-2006
Compiled with
  UTF-8 support
  Unicode properties support
  Newline character is LF
  Internal link size = 2
  POSIX malloc threshold = 10
  Default match limit = 10000000
  Default recursion depth limit = 10000000
  Match recursion uses stack

Looks good.

Finally, time to move on with life.

This is going to turn into another one of my “I love package management” posts. When I first starting managing Linux servers with Apache, adding a new module to Apache required going back to the source code, remembering the configure parameters you compiled Apache with last time, configuring Apache again with the same parameters plus whatever changes need to be made, compiling, testing, and then installing. It took hours. Now, it takes a few minutes to add something like mod_ssl support to Apache without having to compile anything.

Installing

Once again, my server is running CentOS x64 5.2, and I’m using yum to do package management. Thus, the following instruction may need to be modified for your specific system.

I first need to install the mod_ssl library.

yum install mod_ssl

If you are using a different package manager, such as apt, you can run a similar command. The important part is that you can easily install mod_ssl with your package manager rather than having to recompile Apache.

Configuring Apache

There are a variety of different ways to configure Apache to load the necessary SSL options. Also, these options may need to be changed or tweaked for your setup. That said, here is how I configured my server.

My httpd.conf includes all the files from conf.d/*.conf. I took advantage of this and created a conf.d/ssl.conf file. My ssl.conf file contains the following:

LoadModule ssl_module modules/mod_ssl.so
Listen 443

AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl    .crl

SSLPassPhraseDialog  builtin

SSLSessionCache         shmcb:/var/cache/mod_ssl/scache(512000)
SSLSessionCacheTimeout  300

SSLMutex default

SSLRandomSeed startup file:/dev/urandom  512
SSLRandomSeed connect builtin

SSLCryptoDevice builtin
ServerName ssl.domain.com

DocumentRoot /home/site/html

ScriptAlias /cgi-bin /home/site/cgi-bin

SSLEngine on

SSLProtocol all -SSLv2

SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW

SSLCertificateFile /etc/httpd/conf/ssl/ssl.crt/ssl.domain.com.crt
SSLCertificateKeyFile /etc/httpd/conf/ssl/ssl.key/ssl.domain.com.key
SSLCertificateChainFile /etc/httpd/conf/ssl/ssl.crt/godaddy_intermediate_bundle.crt

SetEnvIf User-Agent ".*MSIE.*" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0

CustomLog /home/site/logs/access.log combined

The two lines at the top are extremely important. The first line actually loads the mod_ssl module. The second line tells the server to listen on port 443 (the default port for SSL and allows access using https).

Notice the SSLCertificateChainFile directive. This loads the chain certificate that was given to me by GoDaddy, the SSL provider I used. If you don’t load your provider’s chain file properly (if necessary), your SSL will not function.

I also have a sample file that you can use to create your own ssl.conf file. It contains comments that can help you understand all the options.

Once you’ve made your config changes, you need to ask Apache if everything is set up correctly. You can do so by running the following command:

service httpd configtest

Unless that command commands back with “Syntax OK”, you have a problem. Unless you know what the error means and how to fix it, the best thing to do is to copy the error and search for it in Google. That will be the quickest way of finding a solution.

For example, if I remove that first line in my ssl.conf, mod_ssl won’t load and I get the following error if I test my config:

Searching around for “Invalid command ‘SSLPassPhraseDialog’” produces some results that indicate that the mod_ssl module isn’t loaded correctly.

Another common error message is:

This is a common mistake where you try to load the mod_ssl module in the old Apache 1 config method rather than the new Apache 2 method. The old method looks like:

LoadModule mod_ssl modules/mod_ssl.so

The correct config line would be:

LoadModule ssl_module modules/mod_ssl.so

Configure Firewall

Now that you have Apache configured, it’s time to open up port 443 on your firewall. I’m using iptables, so my instructions will be specific to it.

The easiest way to open up the port immediately is to run the following:

This will immediately open up that port on your system. However, this change is only temporary, and a restart of iptables or of your server will remove this rule.

Making the rule permanent on your system is completely dependent upon your own personal setup. I have seen hundreds of different ways that people have set up their persistent firewall rules. I like to create a service that makes it easy to load, clear, and reload my firewall rules. Firewall management is beyond the scope of this topic, so I will do a firewall post and then link to it so that you can see how I manage my own firewall rules.

Now all that is left is to enjoy your new secure site.

FFmpeg is an amazing collection of open-source tools that can record and stream video and audio. However, it can also transcode video and audio (convert the files to different formats), and that is what has me so excited. There’s also a great PHP package called ffmpeg-php that allows for easy use of FFmpeg from inside PHP scripts. Today, I’m going to see if I can’t help you get both of these set up on your system.

Admittedly, it’s been a while since I’ve tried to install FFmpeg, about two years. I recently thought up some ideas on how I’d like to use FFmpeg, so I thought it was time to give it a try yet again. Today, I’m proud to say that installing FFmpeg is so much easier to install compared to the past, that I dare say it’s simple.

Here is my experience with installing FFmpeg on my server and how to fix the pitfalls that I encountered.

Preface

Note that I did all the following steps on a CentOS server. The specific version is CentOS x64 5.2.

These instructions can work for other distros with little or no modifications, but some distros will be completely different. For example, the DAG RPM Repository that I’m using (more info below) has support for the following distros: Red Hat Enterprise Linux, Fedora Core, Yellow Dog Linux, Aurora Linux, CentOS, Scientific Linux (they really need to get a verified SSL cert), TaoLinux, WhiteBox Linux, Lineox, and BLAG. Other distros will need to use a different repository.

If you successfully get FFmpeg running on another distro, please comment your changes here and I’ll update the post (and make sure you get credit of course).

Also note that I experienced some bumps in the road while installing everything. While many of you may not experience these issues, I found it important to document these problems and how I fixed them in case you encounter them.

Preparing

The first thing that you need to do is set up the DAG repository. This repository is an actively-maintained repository that provides a staggering number of packages with current or near current builds. Adding this repository is a great way to run the latest offerings of many packages.

Adding the DAG repository is simple. I’m using yum, so I did the following to add the repository:

1. Create and open a new file called /etc/yum.repos.d/dag.repo. I ran “vi /etc/yum.repos.d/dag.repo“.
2. Add the following text to the file:
   [dag]
name=DAG RPM Repository
baseurl=http://apt.sw.be/redhat/el$releasever/en/$basearch/dag
gpgcheck=1
enabled=1
3. Finally, save and close the file.

In order to successfully use the DAG repository with tools such as yum, you need to add DAG’s GPG key. Failure to do so will result in an error like the following:

In order to add the GPG key for DAG, run the following:

The DAG: Frequently Asked Questions page has additional instructions on how to get the repository loaded and working on your distro.

Now that DAG is setup, it’s a good idea to update all your packages.

Depending on the packages you currently have installed, this could potentially upgrade, install, or replace numerous packages that may or may not be very important to you. Make sure you carefully look through that list and do any necessary preparations before telling yum that it can go ahead.

For example, yum told me that it was going to replace my current MySQL interface library for Perl with a new one. I added to my check list a note to verify that my Perl code functioned correctly after the install.

I ran into another hitch when I told yum to go ahead with the update. It informed me that my current version of Subversion conflicted with the new version it wanted to install. When this happens, you need to remove the old package before proceeding. This time, I made backups of all of my repositories and my /etc/sysconfig/svnserve file before proceeding just in case. I then removed Subversion “yum remove subversion“, ran the update process “yum update“, and installed Subversion again “yum install subversion“.

Installing – FFmpeg

Now you are ready to install FFmpeg with yum. I wanted to install all the available FFmpeg packages, so I first asked yum what was available.

Searching through the results, I found that three packages need to be installed: ffmpeg, ffmpeg-devel, and ffmpeg-libpostproc.

Note: If you install ffmpeg-libpostproc, the entire FFmpeg software library changes from the LGPL license to the GPL license.

After a couple of minutes, the packages and the packages that they depend on were installed.

I simply ran “ffmpeg” from the command line, and I took the lack of threatening error or warning messages as a good sign that things were working.

Preparing for ffmpeg-php

I often work with programs through command line calls in code, but I wanted something more robust this time, so I looked around and found ffmpeg-php. Based on the API, it looks to be a great tool to interface PHP and FFmpeg.

There are four things that are required to successfully install and run ffmpeg-php; they are:

• ffmpeg-0.4.9_pre1 or higher
• php-4.3.0 or higher
• gd-2.0 or higher
• php-devel

PHP and FFmpeg should be good to go since at the time of this writing, DAG has PHP version 5.1.6 and FFmpeg version 0.4.9. GD and php-devel can be easily installed by running the following yum command:

In case you are wondering what php-devel is for, it installs the phpize program which is used to install ffmpeg-php.

Installing ffmpeg-php

Now we are ready to install ffmpeg-php. This can be done in six easy steps:

1. Download the latest ffmpeg-php release
2. Extract the archive:
   tar -xjf ffmpeg-php-X.x.x.tbz2
3. cd ffmpeg-php-X.x.x/
4. phpize
5. ./configure && make
6. sudo make install

Finishing Thoughts

This may seem like a lot of work when I earlier described this process as “simple,” but trust me that this is a thousand times easier than when I first tried installing FFmpeg. I think I spent three hours working on installing FFmpeg just to find out that it didn’t work the first time I tried.

Time and time again, package management has proven to be an extremely powerful tool. While I know the value of manually configuring and compiling code, the ease of simply using a package manager can reduce the time needed to install and manage software from hours or days to minutes.

I’m glad to see that FFmpeg has benefited from the use of these package managers and great repositories like the DAG RPM Repository.

Today I’m installing Ubersmith, a billing system solution, on a server to test it out. I have to say that Ubersmith has a more complex installation process than I’m used to these days. I guess that I’ve been spoiled.

Here’s a quick word about what I’m running. My test server is a dedicated system running CentOS 5.2 64-bit. The software setup is nothing special as most of the packages are straight from the repository.

I loaded the release files on the server, untarred them, and proceeded to read the instructions. Everything looked to be straight-forward. I quickly got down to the part about installing Ioncube. That’s when the trouble started.

I followed the instructions for getting Ioncube loaded, quickly modified my php.ini file, and restarted Apache. I then tried to access the site and got nothing. I looked at the logs and found that Ioncube wasn’t loading while reporting the following error:

Frankly, I had no idea what that meant. A bit of searching later, I find that I apparently have the wrong Ioncube loader. That didn’t make any sense since I loaded the exact loader that the ioncube-loader-helper.php program told me to use. I had a hunch that I knew what the problem was and decided to check it out. Yup, Ioncube Loader has 32-bit and 64-bit versions, the problem has to be that the Ubersmith code came with the 32-bit code. I downloaded a Linux (x86-64) archive from the Ioncube Loader packages page, loaded it up, restarted Apache again, and got Ioncube to load properly.

Ioncube was now loaded, but the page still didn’t render. Looking at the logs, I found this wonderful gem:

My first thought was, “great, the software is broken.” A quick google later, I found that mb_internal_encoding is part of PHP, but it is not a defaultly included option. I looked around at how to activate or add it, and everyone was going on about needing to recompile PHP. I’ve compiled everything from simple command-line tools to Apache2, but I always prefer using repository releases as it makes maintenance that much easier, so I was very upset to see that I needed to compile PHP with a new set of options just to run this software.

I was about to give up on Ubersmith altogether when I remembered how many amazing things package management can do. I ran “yum search mbstring” and found that there was a package called php-mbstring ready and waiting for me to install it. I ran “yum install mbstring“, restarted Apache, and finally the Ubersmith setup wizard was ready for me.

So, there are a few key pieces of wisdom that I have learned from my experience with installing Ubersmith:

1. Don’t trust that the installation documents/tools give you all the information you need. Ubersmith made a bad assumption that all systems will only require the 32-bit Ioncube while not making any notes that people on 64-bit platforms need to download new software. Ioncube’s installation helper neglected to check to see if the Ioncube files I had were built to work on my platform, and just like Ubersmith, the helper made no attempt to notify me that I may require different files depending on whether my platform is 32-bit or 64-bit.
2. Never trust that the third-party software packaged with another piece of software is the correct software for your system. If something breaks, first replace that packaged software with software freshly downloaded from the developer’s site.
3. When references say to compile your own software to get something to work, always remember to try using the package management tools first.

As I continued on the setup path, this wisdom had immediate use. The setup wizard indicated that two PHP extensions were missing: Mcrypt and XML-RPC. I immediately ran “yum search mcrypt” and “yum search xmlrpc” which returned positive results. I then ran “yum install php-mcrypt” and “yum install php-xmlrpc” and was golden. Just think if I had decided on compiling my own version of PHP earlier, I would have finally finished, gotten it working, and then had to do it all over again when I ran the setup and hit this snag. Thank you yum.

One thing that I have a gripe about is Ioncube’s navigation structure. I went to their site looking for download links for Ioncube Loader, and could not find them anywhere. I ended asking Google where I could find Ioncube Loader, and like a good little Google, it was right on the first try. I then tried again to find that page in the nav and found it under Products > Free Loaders for Encoded Files. I’m probably just daft, but I skipped right over that stuff because I was skimming the left side of the nav looking for something that started with “Ioncube …” not “Free …” At the very least, put links on the encoder pages that point to the loader page and say, “Looking for Ioncube Loader? Click here.”</rant>

I’ve poked around with some distros over the last few days. They’ve all been Live Distros running in VMWare. Some of the Distros ran very well like this (Fedora, Mandriva, openSUSE, and Ubuntu), but others failed miserably in one way or another (Linux Mint, Sabayon). Unfortunately, CentOS cannot be installed from a Live CD, so I was unable to test out its installer in VMWare.

Starting Over

I realized that running all these Distros in a Live environment and in VMWare may be unfair. So I’m going to start from scratch, and do all the processes by directly running each Live Distro on the machine directly. This time, I’ve created a list of tasks to do in each Distro. I’ll make notes as I try to do each step for each Distro.

Test Basics

I will run each Distro through as much of the list as I can (some Distros may not be able to do certain tests). For each item, I will record how easy it is for me to complete (five-point scale), how well errors or potential problems are reported (five-point scale), and make general notes about the process.

Live Distro Test

The idea for this test is to get a good idea of how much functionality each Distro can provide without having to be installed. I’ll be looking for the best out of box experience. Unless the Distro provides a clear way to allow the task to work, I won’t make any changes to the installation. Since this is a live run of the Distro, making a bunch of changes each time just to do a task would be very unproductive.

• Boot to GUI desktop. This will test how easily the Distro deals with a dynamic mix of hardware and how well it auto-configures itself.
• Test keyboard buttons. I’ll press the non-standard, Microsoft-specific and media keys) on my keyboard and see how many of them are recognized by the Distro.
• Run Firefox and load Google.  This will test for automatic discovery and connection to the network. The internet connection will be handled by the router, so connection to the network should be sufficient.
• While Firefox is still open, I’ll test to see if my additional mouse buttons are recognized and mapped to the back and forward functions of the browser.
• Mount the existing NTFS drives.
  • Copy files back and forth.
  • Test loading media from an NTFS drive.
• Play an audio CD. This will test the ability to auto-load an audio disk. It will also test whether or not the sound card was correctly identified and appropriate drivers were loaded for it.
• Play a DVD movie. Similar to the audio CD test, this test will test whether the Distro can automatically load a movie when popped in, how well the audio works, and how smoothly it renders the video. I expect at least a couple Distros to fail this test due to the DVD decoders being proprietary.
• I’d like to find out how well the automatic mounting of USB drives works, so I’ll connect a USB flash drive and a USB hard drive. I’ll then move and copy files around to get a good feel for the performance.
• Play audio files. I’ll test the playback ability for mp3 and wma. I might also test Vorbis and FLAC. This will be yet another test of the sound playback and will test codec decoder ability.
• Play video files. I’ll assemble a collection of video files with different encoding schemes and test them out. This will test the sound playback, smoothness of video rendering, and decoder ability.
• Load Firefox and visit some sites.
  • First I’ll visit homestarrunner.com and watch the intro video. This will be a good test to check for Flash support.
  • Next I’ll go to the Red vs. Blue site and check out their latest video. These videos are much more intense than the Home Star Runner ones and will be a better test of the performance of the Flash video.
  • For the final stop on the Flash tour, I’ll load up Newgrounds.com and play a game. This will be another good Flash performance check.
  • I’ll then log into my Gmail account, and check my messages, send a message, and do some basic browsing around. This should work smoothly, but I’d like to check just to make sure.
  • Gametrailers.com has content in Flash, QuickTime, and Windows Media formats, so I’ll stop there next and check out a game trailer in the different formats. I don’t expect any of the Distros to render QuickTime or Windows Media videos, but I’d love to know if any of them can.
  • A final test is to go to Apple’s Movie Trailers site and try to watch a movie preview in high quality. Once again, I don’t expect any Distro to do this without any tweaking, but I’d love to know if any can.
• SSH into another box. I don’t expect any Distro to have a problem with this, but I’d love to know if any do.
• Load up OpenOffice Writer, add some text, and try to send the file as an email attachment. I’d love to know the different approaches each Distro has to sending an email without a configured Email client.
• Run the Unigine demo as a benchmark for 3D rendering.

Installation Test

After testing out how well the Live Distro works, it’s time to get to business and install to the disk. This process will be done through each Distro’s live installer (except CentOS which will be installed from the DVD due to not having a live installer). This test doesn’t have any steps and each installation will be done with most of the options left as default.

Using the Installed Distro

Unlike my process for the Live Distro, if the task fails, I’ll try my best to make changes that will result in successfully running the task.

• Update all packages.
• Run through all the Live Distro steps again. This will check for any enhancements that came from package upgrades or potential differences between the live and installed versions.
• Connect my USB printer. I’d like to see if any Distro will automatically configure the printer.
  • Install the printer, if necessary.
  • Print out a document with images.
• Connect my camera via USB.
• Install and test Ventrilo.
• Install Wine.
  • Install and test Steam.
  • Install and test Team Fortress 2.
  • Install and test Crimson Editor.
• Get the fancy Compiz/Beryl desktop stuff working. I’m not sure if I will actually use such a desktop, but I would like to see what it takes to get each Distro to do it.

Distros to be Tested

I’ve made a couple of additions to my list. The list of Distros I want to test are as follows:

• CentOS 5.2
• Fedora 9
• Linux Mint
• Mandriva
• openSUSE
• Sabayon
• Ubuntu

Final Thoughts

In the end, I’d like to get a good feeling for what some of the major Linux Distros are capable of and how easy they are to work with. From the results of my tests, I plan to select my favorite and start using it as my primary desktop for a while.

I’ll post my reviews as I progress through the tests.

As I talked about in my Time to Switch to Linux post, I’m ready to finally migrate away from Windows and into Linux. The biggest question for me is which Distro do I want to start with. Me being a lover of lists, I have a few criteria that I want the Distro that I select to meet:

Desired Functionality

• Be relatively easy to install. I don’t need something so easy that my Grandmother could do it, but I also don’t want to be over-burdened with dozens of packages to install just to get the initial installation working properly.
• Have a clean and easy GUI. I think Beryl/Compiz is really slick, but I really don’t think I need to be messing with all that stuff when I just want to start off with something that is functional.
• Be compatible with my current hardware. I know that hardware support in the major Distros has seen vast improvements in recent years. I just want to know that I can still make full use of my Microsoft Natural Ergonomic 4000 keyboard and my Logitech MX 518 mouse. For example, I want to use all of the buttons on my mouse (buttons 4 and 5 should already function for or be easily mapable to forward and back in the browser) and be able to easily map the Windows-specific keys on my keyboard to functions in the UI or key combinations. Just having this functionality alone will do a lot to help me ease my transition.
• Easy hotkey assignments. I love having quick access to programs by hitting some key+key combination. I use AutoHotkey on my Windows systems to hotkey all the programs I use frequently, I’d love to be able to easily mirror the functionality on Linux.
• Have an active and thriving community for support/troubleshooting reasons.
• Be able to support some of my Windows programs if possible. In particular, Team Fortress 2 and Ventrilo as I’m a member of the EGO gaming clan and don’t want to give that up for an operating system. I’m sure I can have some of the Linux-running members help me with getting this set up.

After browsing around DistroWatch for a while, I have found a few Distros have grabbed my attention more than the others.

Ubuntu

I have been an Ubuntu fan for a long time now. I ordered one of the original release disks that they offered years ago and was instantly impressed. My understanding is that they’ve come a long way since then. My only reservation is the lack of support for non-open-source software, such as proprietary nVidia drivers or proprietary codecs. This probably isn’t a big deal as plenty of people have guides about adding such things.

Sabayon

Another one of the Distros I checked out a while back when I first started playing with the idea of running Linux at home, Sabayon is looking good to be my final choice. While I’m not too familiar with Gentoo, the Distro which Sabayon is built off of, Sabayon does have some interesting stuff going for it. I’m most interested in how they try to ease installation and use of many cutting edge features, such as enhanced proprietary driver support, and their unique packaging system Entropy.

openSUSE

I don’t personally have much experience with openSUSE, but I have read many good reviews and recommendations about it. My biggest reservation is I don’t like what I’ve read about Novell and Microsoft and how it relates to openSUSE.

Fedora

Most of my experience with Linux comes from Red Hat EL. Fedora is a Distro that is spun off of the main releases of RHEL, so it will have much in common. Maybe it would be best if I stick with a platform that has a similar foundation which could ease my changeover.

CentOS

My personal webserver runs CentOS, and I really like it, sometimes even more than Red Hat EL. I really like using yum as it makes updates extremely easy. Hopefully the desktop version will be able to hold up to the results I’ve experienced from the server version.

Conclusion

So those are the options that I’m currently looking at. Fortunately, all the distros I’ve decided to explore more have Live Distros available. This will make it easy for me to “try before I buy”. In other words, I can try out a basic version of the Distro without having commit to an installation of it first.

Updates will be coming with my impression of each distro and my final selection.