The Quality Chronicles

BugThe recent “failed IPO” of BATS has to be a cautionary tale. This wasn’t just a deal that didn’t price or trade according to plan. A software bug caused it to be withdrawn. You don’t hear that one too often.

A bug killed an IPO?

There is no argument that we live in a world of staggering speed, where competitors race to meet customer needs and time to market matters. Innovation is always factored by the ticking click, who gets the jump and the competitive advantage, when a cost center becomes a profit center. Information compounds on our desktops, the team with analysis paralysis most often loses to the nimble risk takers—but all this means is that in product development, the role of Quality Assurance (QA) has never been more critical.

I have often heard the mantra from development teams: “Better, Faster, Cheaper—we can give you any two and a half.” Believe me, I understand trade-offs. All product development is tempered by tough decisions that incorporate a series of smart and well-balanced quid pro quos. You want to cut the budget, give us more time or expect fewer features. You want to tighten the schedule, give us more capital or reduce the scope of benefits. You want an industry defining product, show us the money or don’t ask for a date.

Surely these threads have become clichés, and as such, they are not without some underlying truth. There are even schools of thought that proclaim speed over accuracy is the game-winning formula, entire companies built on this premise, hugely successful in their own right. Yet when the Decision Maker, whoever that is, makes the call to greenlight a software or product release, another question comes to mind: Is the call transparent or opaque? Said another way, are the risks inherent in the release from staging to live known to the Decision Maker, or is that person flying blind?

If the release is going live with known issues, that becomes a business decision with acknowledged acceptable risk. If a showstopper issue exists but is unknown to anyone, well, I don’t think I have ever seen that case in a going concern. If the release is going live with issues known to others but not the Decision Maker, that is a dysfunctional process, possibly the beginning of the end.

Here is the way I like to think about quality in product development: Quality Assurance is a Process, not a Department.

Like so many of the great lessons I have co-opted in this blog, this first became clear to me in hard-won experience with the magnificent QA Directors with whom I have worked over the years (several of whom reviewed this post in draft prior to publication), and second in Jim McCarthy’s brilliant book Dynamics of Software Development first published in 1995 and still a must read for any of my teams—non-tech even more than tech staff. The most critical constant of which I am aware in delivering great products to market consistently is for Quality to be owned by everyone involved in innovation—from designers to developers to marketers to feedback from end-users.

Of course every great development company will have a final step in the process called Quality Control or Quality Assurance, but it is my sense that the QA formal group is there to be the standard-bearer for Quality and rally the company around it, putting a final go or no-go procedure in place before the world gets its hands on a product, but not accepting proxy status for an otherwise poor process. A QA department is not a dumping ground, not a remote server where code is parked as a step function or convenient checkpoint in a perfunctory release approval, not a cynical target of blame. QA is the proxy for the customer, not management, and as such must have a voice that is shared throughout a company. If a Decision Maker chooses not to listen to either the process or a warning from fully objective and independent QA stewards, you get what you get.

I have always been enamored with QA teams, for their passion, for what they teach me, for how much they care about excellence. When QA is wound into the culture of a company, it is often because of the mutual and shared respect an organization has for the value of Quality as an intrinsic good that will most likely yield extrinsic rewards, but carries reward for itself in the form of realized creativity and pride. It is very hard to fake a love of Quality, and this applies to much more than software. Quality is a path to premium brands cautionary tale and premium prices in a landscape where speed and disseminated knowledge can commoditize just about anything if you let it. Quality is won when it is broadly embraced as a shared value, and then championed by a high energy team that inspires its adoption at the highest levels of management and all through the ranks. If top management does not buy this, Quality is doomed.

If top management at BATS did not know about the bug in their system—a software platform for trading equities like their own on IPO day and beyond —they did not do the hard work that is expected of them and now accept the business consequences. The downside illustrated in this real world example of an incomplete process is about as clear as it can be. To ignore or be ignorant of a showstopper in one’s own product is a reflection of a process that needs to be re-engineered. When you’re working with world-class engineers, it is much easier and far more fruitful to make sure the process is engineered correctly before the products go through it. The material cost of discovering a bug early in development is a tiny fraction of what it can cost you in the hands of the public. Give your engineers a voice and they will save you every time.

Listen to your QA stewards. If you built the right team, they are your first line of offense and your last line of defense. They know of what you speak.

Advertisements

Do I Have To Eat It?

A recent opinion piece in the Wall Street Journal by Jesse Kornbluth, a onetime devoted and inspired employee of America Online, pondered the question of “How AOL—aka Facebook 1.0—Blew Its Lead.” Kornbluth does a good job acknowledging the irony of overlap between the fallen angel and the rising star—the staggering power of community, the seduction of the walled garden, the financial reward of vast momentum—but more importantly, he gets his head around what he believes to be the downward turning point for his former employer. It was not so much the bursting of the bubble, nor even the distractions of failed promise in the historic merger with Time Warner. As a product person, Kornbluth saw the blood start to flow when those who loved product began to be overruled by those who lived by argument. Those arguments were not the healthy tension of developers debating the relative merits of features and benefits. The conflict shifted to initiatives in product strategy that were driven by individuals who had assured themselves their creative ideas would lead to success, even though they did not much have time to embrace and use AOL they way its creators had previously.

When the consultants arrived, strategy was not driven by those who embraced the product and its audience; strategy became a set of theoretical suppositions evidenced by the competitive landscape. There were only two problems: 1) the consultants were no more obsessively using competitive products than those of AOL; and 2) the competitive landscape was crumbling because it was just as inorganic in construct, itself no more than the conclusions of observation. Using a product is not trying it once, it is using it every day and using competitive products to fully internalize how bad becomes good and good becomes great. Data, analysis, reconnaissance, and interpretation are all essential in responding to hyper growth, but if you aren’t eating your own dog food, all bets are off.

Yes, you must Eat Your Own Dog Food.

Alpo Lorne GreeneSome people trace this edict to the television commercials for Alpo in the 1970s and 1980s where Lorne Greene made a point of showing us that he fed the very product he endorsed to his own dogs. No, he didn’t actually eat it himself, but the way he looked at it, you could tell he might be considering it. His dogs were an extension of himself. That love made it clear he would only feed them a product he trusted, and he would only endorse it publicly because he trusted it. I am not saying he was right. I am just noting than his conviction was visceral.

In the software spectrum, the phrase “Eating Our Own Dog Food” is more commonly traced to a 1988 memo from then Microsoft Manager Paul Maritz, encouraging his team to obsess over use of Microsoft’s products. His basic tenet was that to win a category and perfect your work, you had to be the consumer. The memo spread widely throughout Microsoft, over the gate and through the industry. It resonated with many of us, and began being accompanied by such observations as, “If you won’t use the software when it’s free, why should anyone pay you for it?”

Soon after came the dawn of the Dot-com age in the mid 1990s, quickly followed by the implosion of Web 1.0 known as the Dot-bomb era circa 2000. Interesting to note, a few of the companies that survived the turmoil and went onto become the great first generation brands of the Internet like Amazon and eBay made it a point to eat their own dogfood. While third-party consultants poured into corporations to sort out their tanking business models and rationalize their value propositions, far too many of those consultants were busy writing decks and compiling spread sheets. When you asked them what online products and services they loved, they often couldn’t respond, because they were too overwhelmed by time commitments to use the products they would evaluate, let alone love them. For those who had already been through a product development cycle or two, the writing was on the whiteboard.

The absolute necessity of eating your own dog food is anything but limited to software. If you design cars for a living and are not planning to drive your own creation when it comes off the line, how can you attend to every nuance and detail that sets apart your vehicle from the vast number of choices already available for sale? If your team designs a new line of workplace apparel intended to be marketed as more comfortable, durable, and stylish than everything else already hanging on the rack, will you not be planning to wear what you have produced proudly at least a few days each week out of pure joy? When you have the privilege to be creative and innovative in your occupation, you are quickly humbled by the fact that an idea for a new product or service however inspired and brilliant is in fact almost worthless. Customers seldom buy or become loyal to the ideas you pitch. Until a concept is executed expertly and embraced by those who will champion it, it really is just a first draft—perhaps filled with promise, but nonetheless in need of refinement, iteration, and polish. There is a long and winding road from pitch to product, and all along the way details have to be vetted first by those who most love the work, the creators.

Apple long ago coined this notion as Evangelism, and no Apple Evangelist in his or her right mind would try to get you excited about a product they weren’t already using themselves. To be fair, Evangelism is a beginning, not an end, after which customer feedback must become part of the process, but if our goal in social marketing is to engage our community in a supportive and seamless dialogue, then we owe it to them to initiate the dialogue with honesty, commitment, and passion. There will always be pain to share in early releases, but the more defects we extract ahead of release because we already know they are there, the more our customers can trust us to take them seriously in allowing our own needs to be met before we presume to address theirs.

Design is not cynical; its true elegance is purely self-reflective because form and function are easily evaluated in day-to-day use. If something is good enough for your dog, it might be good enough for someone else’s dog. Now imagine if you ensured it was good enough for you before you topped off the can. That would be some seriously tasty dog food. Go on, take a byte.