Over the many years of their coexistence, the terms CGI and Perl have become virtually synonymous. This perception that CGI and Perl are one and the same has contributed to some small degree to the perception that Perl is outdated and an inappropriate language for web programming - unlike its more modern counterparts like Ruby or PHP.
I of course know this to be a complete fallacy. First of all, the company I work for is a devout Perl shop and our products, all written in Perl, are collectively some of the largest and most scalable on the Internet. Few other companies in fact present more widely attended seminars and tutorials about scaling for the Web then ours do at conferences like eTech and OSCON.
But perhaps the more significant fallacy is that CGI and Perl are synonymous. In actuality, they technically have very little to do with one another, and there is technically no reason one should be hampered by the other, despite all evidence to the contrary.
A little history and background
CGI stands for "Common Gateway Interface" and was invented to allow any script, written in any scripting language, to also act as a web based application. In the early days of the Internet this was incredibly helpful to the first Web programmers (a.k.a. system administrators) proficient in sh, bash, csh, tcsh and perl because it allowed them to quickly deploy simple web based automation tools based on scripts and libraries they had already written. Cool.
But the inherent flexibility of language agnosticism is also CGI's greatest liability, and also by association, Perl's as well. You see, CGI is based upon the principle that when the web server receives a request, it does not know what scripting language will interpret that request. Therefore, it defers processing directly to the operative system, and so must do something geeks call "forking and exec'ing" - or in other words, the web server must start up an entirely new process on your server to handle the request. This may not sound like a big deal, but it sometimes can be as each forked process holds the entire Perl interpreter in memory. And that is a big deal. First it can consume a lot of your server's memory, and depending upon the size of your application, it can be slow to initialize. It works - and works well by virtue of working, but it by no means ideal.
More modern languages have been designed to avoid this. Let's take PHP for instance. PHP is a language that was designed for web programming exclusively. Therefore its architects made a critical (perhaps even obvious) decision early on: if the web server is going to handle a lot of PHP scripts, why bother forking a process to determine what scripting language will handle the request (this is expensive) - why not just load the PHP interpreter into memory once and interpret the request within the web server (which is much cheaper)?
So when it comes to CGI vs PHP, it is not really about Perl vs. PHP at all. It is really about understanding two solutions to two different problems - one operating under the assumption that every request will be processed by the same interpreter and the other designed to execute any script via the web.
The solution as it stands today
In the Perl world, there are actually two Apache modules that attempt to do what PHP does inherently: load the Perl interpreter into memory so that you no longer have to spawn a new process each and every time your web server receives a CGI request. Those two modules are mod_perl and mod_fastcgi.
However, these two modules have a critical flaw: they are incredibly complex because they attempt to solve a huge problem set having to do creating a persistent and stateful execution context. The result are two modules that are not only too heavy weight for the average user but also incredibly difficult to install - even for myself.
How to actually fix the problem
The more I have thought of this problem, the more I have come to believe that there is little standing in Perl's way to have all of the benefits that PHP has gained from being a language designed exclusively for the Web.
In theory, one should be able to take the source code of mod_php (the Apache module that dispatches web requests to a PHP interpreter) and swap out the component responsible for dispatching a request to PHP for one that dispatches the request to Perl. In theory it should just work (more or less). The result would be an Apache module that would be easy to install, and be much more efficient in handling and processing requests online.
Granted, this solution would not be stateful and persistent the way mod_perl and mod_fastcgi are, but that is not a problem this solution is engineered to solve.
All of this is a really long-winded setup for what is a very quick conclusion.
I shared this hypothesis with an engineer at work, Aaron Stone, who shares a passion for Perl with me, but who also has a passion for operations. He took on this challenge and devoted part of his 20% time to testing this hypothesis.
The output of his work is called mod_perlite. It is largely derivative of PHP and is capable of processing Perl scripts quickly and efficiently. The next step of the project is to make it compatible with the CGI protocol, which can be done by gutting parts mod_cgi and dropping them into mod_perlite.
So far our results are promising, and it is possible that with a little hacking we may have just made Perl faster on the web and easier to deploy for everyone.
If you are interested in helping or participating in this project please let me know -- we could certainly use the help.