Recently in Technology Category

Recently I received a comment on a post about modperlite alleging that modphp, an Apache module mod_perlite is modeled after, is falling out of favor with hosting providers. That is quite an allegation - one that I initially shrugged off because it seemed preposterous to me. I mean just consider for a moment all of the incredibly popular applications including WordPress, Drupal, PostNuke, phpBB, Mediawiki and many more too numerous to list here let alone count that all rely on PHP exclusively.

Not wanting to completely discount this comment I decided to consult a close acquaintance of mine that works with one of the most popular hosting providers on the Internet. In working with him on a number of projects I have come to respect him on a number of levels, but especially for his knowledge of operations and system architecture. He spends a great deal of his time every day trying to maintain systems running software he didn't write or install on machines run by users he has little or no control over. He is an expert in cleaning up after others. So I asked him if he had an opinion about this statement.

What I learned through our email dialog was enlightening, and I wanted to share it with a larger audience. He permitted me to republish parts of our correspondence, but asked to remain anonymous. (Editorial note: I have taken some of his text and elaborated as best that I could in order to provide a more complete overview of this issue.)

This is a rather multifaceted issue, but to summarize I'd venture that mass virtual hosting on the whole is trending away from mod_php/mod_perl.

He proceeded to back this claim up:

• Real file I/O is impossible or insecure - it is difficult to harden a mod_perl/mod_php application because these Apache modules only run in the permissions context of the web server's user. Software that helps resolve this issue, like suPHP, which allows a script to run in the permissions context of the script's owner, negate the performance benefits of the underlying mod_perl/mod_php module.

• One of the more viable solutions to this permissions dilemma, an Apache module called mpm_perchild, died on the vine years ago. "This was the Apache 2+ intended answer for the security problem and was supposed to (correctly, IMO, given Apache's architecture) have each request thread out based on file ownership, sort of a rolled in suexec."

• "There is growing adoption of code frameworks (rails, django, catalyst, etc etc) that just don't play with mod_php/perl. Building an effective cgi/fastcgi architecture covers bases better."

• "There is a growing adoption of very powerful web servers like lighttpd and nginx whose only interface is cgi/fastcgi. To many developers today Apache really is an anachronism."

• "Another side effect of cgi/fastcgi that I neglected to mention is that it is much more apparent which users' code/processes are monopolizing resources" thereby making it easier for hosting providers to monitor, enforce resource utilization policies and recover from systems under load.

Then y friend also responded to this comment:

@saj - really? You think mod_php is falling out of favor? That is quite a statement... one that I can't imagine is true. PHP is one of the most ubiquitous web scripting languages, second only to Perl. A cannot imagine hosts degrading their support for this language.

With this response:

I think this is the most important misconception to dispel. In an Apache vhosted setup, modphp *is* degraded support. Settings like safemode are generally implemented and users don't have access to edit their own php.ini, relying on .htaccess hackery to eke out the limited php settings customization available.

He concluded, "there is a ton more to the issue and these points really don't do it full justice."

Now lets be real, I don't think my friend is saying that hosting providers will drop support for PHP, not in the foreseeable future, but if hosting providers tend not to prefer mod_perl and mod_php based applications what will this mean for all us developing on top of these languages and deployment platforms?

And ask yourself, when you write a web based application, are you thinking about the company that is going to be hosting your application for your customers/users? And what are you doing for them to make their lives easier?

About five years ago I abandoned the last of my brick-and-mortar financial institutions so that I could manage my money exclusively online. I left Bank of America in favor of eTrade because I believed that as a modern Internet company eTrade stood a much better chance of developing the kind of banking application I would not only value, but also have a modern and savvy user interface. So I switched everything over and bought Quicken, with a rare enthusiasm and excitement for I believed that this would be the new dawn of a new me, a me that actually had his finances ordered and up to date at all times.

Imagine my surprise when I booted up Quicken only to find that it was incompatible with eTrade Bank. Believe it or not, that was in 2003. Three years later when my wife and I began to contemplate buying a new home and I had a renewed interest in trying to gain greater insight into how my wife and I spend money. So I decided to check back with Quicken and see if they had yet added support for eTrade Bank. They had not.

"Shock" does not even begin to describe the feeling I had. I had chosen eTrade because I believed that a company that existed almost exclusively online would work best with products like Quicken that wished to synchronize users' data with financial institutions over the Internet. Seems like a reasonable conclusion to reach, right? Um, yeah.

So in an effort to get a handle on my family's finances I turned to the oldest personal financial application around: Excel. I imported what I could from eTrade and then began the process of manually categorizing all of my income and expenditures over the past three months. It was excruciating and. mind-numbing. But it was also very error prone because my categorization scheme kept adapting and changing as I began to understand my own expenses more and more. During the whole process I kept asking myself, "can't Quicken do this for me? Don't they have enough people's data to know how people generally categorize a payment to Amazon or Grand Lake Theater?"

In the end I toiled over that spreadsheet for about 3-4 hours knowing full well that if I ever needed to update my finances I would need to do it all over again. Talk about poking yourself in the eye with a stick.

I had resigned myself to never finding a tool that would work for me. But just the other day, I stumbled upon a newly released product that in an almost clairvoyant fashion did exactly what I was trying to achieve in Excel in a little under 5 minutes. Plus it gave me the simple, easy to understand personal finance analytics I wanted out of Quicken, and it did it all for free.

So, yes, I am a fan of Mint. I will admit. But it is not perfect. It does a lot of things right, which I talk about below, but there are many things I would like to see it do differently.

Turning Personal Finance into a Game

A number of notable product design folk talk about the use of game mechanics within the tools we use to help fuel our use of and addiction to them. These same game mechanics can also be used to turn otherwise mundane or even anxiety-ridden tasks into ones that are actually and quite possibly fun. The "game" Mint offers lies in encouraging the user to take actions in their personal life in order to manipulate Mint in respondiing in a favorable and desired way. One very simple example of how Mint does this already is with the widget that shows a user's ratio of equity to debt. This is an incredibly reductionist view of your finances, granted, but that is the beauty because it is easy for me to understand how to move the bar left and right. Another good example is the "how do you compare" feature which actually pits me against others users in the system. This I love because it taps into my competitive nature. I hate know for example that I pay more then most others for gas, especially consider how little my wife and I drive.

Mint is only scratching the surface here. The truth is that Mint has lots of fertile ground to help provide additional feedback to the user to help encourage them to develop discipline around their finances, and I hope that Mint begins to define its more of its features with game mechanics specifically in mind. I just love the idea of an application I use enticing me to be better about finances - either overtly or subconsciously. I don't care, because the end result is a real value to me no matter what.

The Brilliance Behind the Product

Omniture and then Google showed the world that advertising can be actually be perceived as valuable by users when that advertising is actually relevant to the content of the page and provides some kind of value to the user. It is my belief that search remains the only context for advertising where the ad serves a purpose and value relatively equal to the content itself. Sure, content targeted advertising on blogs can be relevant, but they are still widely recognized as "ads" in the most pejorative sense of the word, and not immediately perceived as a value.

Mint may very well have discovered the next form of ad unit that provides what consumers will widely perceive as truly valuable. Just click on the "Ways to Save" and you are immediately taken to a page that shows me products and services that will potentially save me money. For example, Mint knows what credit cards I have and their interest rate, so it can present me with credit cards to transfer my balance that will save me on interest payments. Mint also knows who my cell phone provider and can present me with alternative cell phone providers that can save me money on my monthly payment. I honestly hope that one day Mint will also:

• Help me find a better longer term mortgage. What I hope Mint does not do is help me lower my monthly payment. That is a crock. I want Mint to help me build equity in my home, not help others leach equity from it.
• Offer vacation packages for me based upon the cities it sees me spend money. For example, Mint should know that I visit Los Angeles from time to time, and should be able to offer ways to save on my next trip there.
• Help me find a better place to save my money by recommending CDs or Mutual Funds based upon how much of my equity is liquid on average.

A Few Recommendations for Mint

• Make Community More Apparent - The "How do you Compare" feature is awesome, but I think there is greater potential to inform how users can more actively contribute to the system and one another. It would be great to actually show information about each transaction on how others have categorized an item. I am certain their must be something at work on the backend taking into account how others are training the system, but surfacing that information I think will help encourage that behavior in users even more.

• Allow for Category Suggestions - I actually like the constraints you place around categories. While I think a compelling argument can be made to allow people to create their own ontology, I think it is wiser to constrain users, lest they over complicate something that Mint is trying to keep simple. But man, I sure would like a few changes made... perhaps you could let users suggest categories? For example, I would love a category for Home Business (both income and expense). Oh, and I want a "Delivery" option under Food because one thing I am dying to know about myself is how often I "order out."

• Encourage me to Categorize - I think categorization is one of the most important activities a user can engage in. There should be some kind of statistic on the dashboard widget that shows how many uncategorized items I may have. When I click it I should be taken to a "to do list" of sorts that shows all of the uncategorized transactions in my account. As I assign each to a category, my to do list gets smaller and smaller. Ultimately I should categorize very infrequently because over time I should be training the system to do it for me.

• "Labels?" Come on, they're tags. - I wish a more conventional method for applying labels, or let's be honest with one another, tags to my transactions. I would be more diligent about tagging my transactions if the process was more fluid and less cumbersome. Go for a tagging interaction model like Flickr or Vox.

• Spending Trends Improvements - The pie chart is nice, but it is not 66%-of-the-screen-nice. Some of the stats I am really interested in are the less prominent stats, like who are my "favorite" merchants, especially over time. I also want to gain a better understanding of income as well and the ebb and flow of income to expense. Let me blow up spending charts... I want to see more than the top 3 places I spend money within any given category.

The impedance associated with building a network of connections within a social network is a huge risk to innovation on the Internet for two primary reasons:

1. Existing networks have no incentive to innovate because they own the network.
2. New companies struggle to survive in an ecosystem in which the network is monopolized by others.

To put it another way, the best case scenario for a truly innovative and interesting social network that doesn't suck is to attract a group of users approaching at most the size of Twitter (chosen because it is hugely popular, relatively new, and despite its success still seems to have hit a ceiling in growth). And if you are an innovator trying to build a serious business, that is simply not enough people.

The inability for an idea or product to compel its users to rebuild their network of friends will ultimately lead the idea fizzling. No matter how good the idea might be.

The other risk to innovation can be found in the example of MySpace. Here is a product whose soul value lies not in the software, which relative to other products is not truly that innovative, but in the network of people who use it everyday. Companies like this have no incentive to innovate or build a legitimately compelling product because their users are locked in.

But imagine what might happen to MySpace if users could one day wake up and move all of their friends to another service? Or perhaps more accurately keep the network of connections that exist on each service synchronized? All of sudden MySpace might actually have to start building a product that is compelling beyond the network itself.

Now imagine all the ideas out there that might have a leg to stand on if they could have access to a complete and publicly available network graph that they didn't need to compel their users rebuild?

This is why I think the project being incubated by Six Apart is so important. Glue, if successful, will build a completely Free, open and public social networking graph. It, just like OpenID, also invented by Six Apart, will be unencumbered by patents and corporate ownership. It will be given to the world for everyone to benefit from equally.

And the application of this resource extends a great deal beyond social networking. Once a graph has been built and relationships between individuals can be mapped then all sorts of possibilities emerge.

The past several years the Internet has been transformed by a slew of new web based tools and applications. A key success factor in all of these applications is adherence to a development methodology in which product teams release fewer features at a time, but release them at a greater frequency. Adherents to this methodology call this process "iteration."

This methodology allows teams to:

• respond more quickly to customer feedback
• adapt the product based upon actual usage
• become more agile in the marketplace

However, this methodology requires a discipline that is difficult for many to obtain and maintain. That is because it is a radical departure for most software development companies who have deeply ingrained habits and cultures built up around more rigid and waterfall like approaches to software development.

For those that have made the transition to this Agile Development model, what is most intriguing is that while engineering teams and product management have undergone a tremendous process revolution the process of testing software has remained relatively unchanged. And for those companies, I ask you, how agile can your process be when chances are your QA engineers are still testing software the same way they did five years ago and ten years ago?

This is the primary challenge I encountered during Six Apart’s (now a Test Run customer) migration to an Agile Development methodology: no matter how many advances our project managers and software engineers would make in their processes, our QA team was still relying on Excel to manage their test planning process. Excel is a great tool don’t get me wrong, but in this instance Excel made it difficult:

• to gain transparency into what they were testing
• to contribute and collaborate in a meaningful and measurable way
• to audit QA’s progress from release to release

This ultimately resulted in what QA was trying to avoid: more bugs.

On numerous occasions we suffered a regression in overall quality because a QA engineer failed to copy a test case they added in one version of a test plan to another version of the test plan located in a different spreadsheet. Before implementing our Agile development process their process and tools worked just fine. However, their process had failed to keep up with the quickened pace of release cycles. In the past we rarely tested multiple releases concurrently; therefore there as rarely a need to manage the merging and consolidation of test case data throughout a release.

What also made the transition difficult was a turn over in QA staff. As we shuffled resources we lost a lot of knowledge that Excel made difficult to capture. So as new staff members tried to pick up where others had left off, details were missed and bugs surfaced. It took months for those engineers to develop their own internalized knowledge base about our software and be able to operate it with the efficiency of previous team members.

These are some of the challenges that inspired the creation of Test Run. Unlike other test planning tools, Test Run is architected to be iterative, because I believe that test planning is a learning process, just as much as feature development is. In other words, a test case is something that should get better each time your execute it. This is something Test Run helps every QA team achieve in their process, which is something no other test case management tool helps you do. It accomplishes this by:

• utilizing a single, but large repository of test cases to draw upon
• maintaining a test case execution history for you automatically
• allowing users to attach notes and file to a test case that travel with the test case no matter where or when it is executed
• using tags as a means of categorization as opposed to complex and rigid database schemas

In my many years devoted to optimizing the QA process through useful and meaningful software I have learning one thing over and over: your software development process is only as agile as your least agile stage of development. Therefore, if you really want to capitalize on the benefits of the Agile Development Methodology, it is critical for you to look outside the context of software engineering alone and look at your whole process. Ask yourself:

"If my software engineers are exploring more efficient ways of designing and building applications, shouldn’t my QA engineers be exploring more efficient ways of testing those applications?"

Yes. Of course. A million times, yes!

Check out Test Run today!

Reprinted from the Test Run Blog.

Six Apart often speaks of how its technology and contributions to open source help many of the most popular Internet applications scale to unprecedented levels. One of these tools is memcached, yet as important and ubiquitous as it is, it surprisingly lacks the documentation necessary to help beginners in realizing its power and ease of use. Memcached is not for priests and gurus, despite how technical its homepage may appear. Memcache is a tool that can be easily installed and used by almost any developer on almost any platform.

This guide was written in an attempt to give developers out there an introduction to this incredibly powerful tool, and hopefully equip them with enough information to actually get started in building applications on the Internet that are faster and more reliable.

It is amazing how OpenID is really picking up steam. Just in the last 30 days Microsoft, AOL, and Digg have announced or deployed support for OpenID, a simple delegated authentication protocol invented by Six Apart.

But my first reaction when AOL announced support for OpenID was,

Awesome, that's 63 million users... that don't understand OpenID!

I was being cynical of course, because in reality making OpenID ubiquitous is a major milestone in making it accessible to the market.

But adoption by service providers does not necessarily translate to adoption by consumers. I am constantly looking for ways to make OpenID easier to grok by, for example, my mother. If she can use it without me having to explain anything to her, then we seriously have gotten somewhere.

One idea was using email addresses as a mechanism for looking up an OpenID. I like the idea, but it is not without some problems.

But what if I am going about OpenID the wrong way? Let's look at this from a user's perspective. What experience can I offer my Mom such that she could go through OpenID simply and without explanation? In my mind, here is the ideal experience for her:

1. My mom visits http://www.majordojo.com to comment on my latest post to say how funny it is.
2. She sees a prompt: "Login to Vox to comment on majordojo" (where she already has an account - because that is where I post private photos to my family and friends)
3. She clicks the link and is taken to Vox.
4. Vox prompts her to enter her login (her email) and her password.
5. Vox validates the information she entered and redirects her back to majordojo.
6. My mom, now having authenticated, leaves a comment on majordojo.

fin.

So is this experience possible? I think so. Here is how this would work on the back-end:

1. The OpenID client is instructed to authenticate "preemptively" at a URL.
2. The client redirects the user to that URL/OpenID Server.
3. The user authenticates at the OpenID Server.
4. The server returns to the OpenID client the authenticated user's OpenID URL.
5. At this point the regular OpenID protocol takes over.
6. The OpenID client is redirected to the designated OpenID URL.
7. The OpenID client looks up the OpenID Server and redirects the user to that URL.
8. Because the user is already logged in to that URL (they logged in at step #3 above), the OpenID Server simply returns the user to the originated URL where all of this began.

Granted this flow is optimal for hosted service providers like Vox, TypeKey, LiveJournal, Digg and AOL, etc, because the URL to which the user authenticates is fixed. Movable Type blogs for example all live at different URLs, so this experience could not be replicated so easily, but it is possible in a slightly modified form.

In summary, it would be easy for us to proclaim victory with OpenID, but that would be premature. Until we can make OpenID an idea even laggards understand, then OpenID will remain a protocol for the technical elite and adoption by all the services in the world won't make a difference.