Alpha and Beta Testing

In early October I had the honor of speaking to a class of MBA students studying product management at the Haas School of Business. Michael Sippey recommended me as a speaker last year, which in my book is high praise, but considering that I was incoherent at the time I gave my talk, imagine my shock when I was invited to speak again.

Haas School of Business

I believe it was my experience with Agile Development Practices, Product Management and Test Run that I was asked to speak on the topic of Alpha and Beta Testing. It seemed like an easy topic at first, but as I worked on my slide deck and reached the end with only a hand full of slides, I realized I was taking this talk too literally.

I was not considering the bigger picture of test-centric development. A process by which assumptions and ideas are constantly tested, a process in which testing takes place not just at the tail end of the process, but in every step.

The talk was largely influenced by watching the Six Apart product team that built Vox. Vox was released only recently but testing began over a year and a half ago when it was nothing more than some conceptual wireframes. I remember having people come to the office and being led through usability tests with nothing more than a couple of sheets of paper.

It took four to six months from that point for us to be able to use Vox, then code named “Comet.” Once we could, we ate our daily dose of dog food religiously. And we continued to share it with others in usability studies. That systematic process by which we tested our assumptions and the product itself led to an amazing evolution of a piece of software. But perhaps the most important thing to consider is that we weren’t just testing software, we were testing our process.

So many companies fail to become truly agile because they are unable to adapt their processes, and Six Apart is no different, but the mindset they took with Vox allowed them to make and work through the mistakes and problems with the process. This helped the product management team to develop an incredible discipline around deciding what was in and what was out – an essential skill for anyone trying to tighten up their release process.

Alpha Beta Testing PresentationSoon my slide deck went from 10 slides, to over 70 – meant to be delivered in a quick fire fashion. My intent was to build a presentation that had a strong narrative, as Ken Norton’s have. I wanted something that I could leave with the students that was comprehensible without having someone physically presenting. Something more than just a collection of bullet points.

I had a great time with the class this year (not being sick helped a great deal) and I hope Haas will give me the opportunity to speak again next year.

No Comments

Leave a comment

what will you say?


Close