The Quality Chronicles

BugThe recent “failed IPO” of BATS has to be a cautionary tale. This wasn’t just a deal that didn’t price or trade according to plan. A software bug caused it to be withdrawn. You don’t hear that one too often.

A bug killed an IPO?

There is no argument that we live in a world of staggering speed, where competitors race to meet customer needs and time to market matters. Innovation is always factored by the ticking click, who gets the jump and the competitive advantage, when a cost center becomes a profit center. Information compounds on our desktops, the team with analysis paralysis most often loses to the nimble risk takers—but all this means is that in product development, the role of Quality Assurance (QA) has never been more critical.

I have often heard the mantra from development teams: “Better, Faster, Cheaper—we can give you any two and a half.” Believe me, I understand trade-offs. All product development is tempered by tough decisions that incorporate a series of smart and well-balanced quid pro quos. You want to cut the budget, give us more time or expect fewer features. You want to tighten the schedule, give us more capital or reduce the scope of benefits. You want an industry defining product, show us the money or don’t ask for a date.

Surely these threads have become clichés, and as such, they are not without some underlying truth. There are even schools of thought that proclaim speed over accuracy is the game-winning formula, entire companies built on this premise, hugely successful in their own right. Yet when the Decision Maker, whoever that is, makes the call to greenlight a software or product release, another question comes to mind: Is the call transparent or opaque? Said another way, are the risks inherent in the release from staging to live known to the Decision Maker, or is that person flying blind?

If the release is going live with known issues, that becomes a business decision with acknowledged acceptable risk. If a showstopper issue exists but is unknown to anyone, well, I don’t think I have ever seen that case in a going concern. If the release is going live with issues known to others but not the Decision Maker, that is a dysfunctional process, possibly the beginning of the end.

Here is the way I like to think about quality in product development: Quality Assurance is a Process, not a Department.

Like so many of the great lessons I have co-opted in this blog, this first became clear to me in hard-won experience with the magnificent QA Directors with whom I have worked over the years (several of whom reviewed this post in draft prior to publication), and second in Jim McCarthy’s brilliant book Dynamics of Software Development first published in 1995 and still a must read for any of my teams—non-tech even more than tech staff. The most critical constant of which I am aware in delivering great products to market consistently is for Quality to be owned by everyone involved in innovation—from designers to developers to marketers to feedback from end-users.

Of course every great development company will have a final step in the process called Quality Control or Quality Assurance, but it is my sense that the QA formal group is there to be the standard-bearer for Quality and rally the company around it, putting a final go or no-go procedure in place before the world gets its hands on a product, but not accepting proxy status for an otherwise poor process. A QA department is not a dumping ground, not a remote server where code is parked as a step function or convenient checkpoint in a perfunctory release approval, not a cynical target of blame. QA is the proxy for the customer, not management, and as such must have a voice that is shared throughout a company. If a Decision Maker chooses not to listen to either the process or a warning from fully objective and independent QA stewards, you get what you get.

I have always been enamored with QA teams, for their passion, for what they teach me, for how much they care about excellence. When QA is wound into the culture of a company, it is often because of the mutual and shared respect an organization has for the value of Quality as an intrinsic good that will most likely yield extrinsic rewards, but carries reward for itself in the form of realized creativity and pride. It is very hard to fake a love of Quality, and this applies to much more than software. Quality is a path to premium brands cautionary tale and premium prices in a landscape where speed and disseminated knowledge can commoditize just about anything if you let it. Quality is won when it is broadly embraced as a shared value, and then championed by a high energy team that inspires its adoption at the highest levels of management and all through the ranks. If top management does not buy this, Quality is doomed.

If top management at BATS did not know about the bug in their system—a software platform for trading equities like their own on IPO day and beyond —they did not do the hard work that is expected of them and now accept the business consequences. The downside illustrated in this real world example of an incomplete process is about as clear as it can be. To ignore or be ignorant of a showstopper in one’s own product is a reflection of a process that needs to be re-engineered. When you’re working with world-class engineers, it is much easier and far more fruitful to make sure the process is engineered correctly before the products go through it. The material cost of discovering a bug early in development is a tiny fraction of what it can cost you in the hands of the public. Give your engineers a voice and they will save you every time.

Listen to your QA stewards. If you built the right team, they are your first line of offense and your last line of defense. They know of what you speak.

The Real Lesson of Kodak

KodakIt is hard not to feel at least somewhat sentimental watching Kodak exit the world’s business stage in such a sad state after such a storied run. You have to feel sorry for the employees, especially those likely to lose retirement benefits after long careers of loyal service. It is also hard to feel sorry for the company, particularly its management. Kodak had the solution to its own ills and chose to submarine it. The lesson: if you don’t cannibalize your own business, count on a competitor to do it for you.

A timeline of “Kodak’s Key Moments” recently appeared in the Wall Street Journal, and what is too easily forgotten is that Kodak developed the first digital camera as early as 1975, but chose not to bring it to market for fear of cannibalizing its hugely popular film business. That’s an easy enough Monday morning quarterback call, but how many companies right now know they are on a path to their own obsolescence, have a pretty good idea what the long-term answer to their ills may be, but are ceding alternative paths to their competitors for fear of short-term pain or possibly looking stupid? The answer: more than you think.

In a subsequent article entitled Avoiding Innovation’s Terrible Toll, the Journal further noted that in a study of more than six million firms, only a tiny fraction made it to the ripe old age of 40. The authors of that report, Charles I. Stubbart and Michael B. Knight, reflect that “…despite their size, their vast financial and human resources, average large firms do not ‘live’ as long as ordinary Americans.” We have just seen this of late with the beloved Borders Books, and now we are watching Barnes and Noble try to pull off a comeback around its initiatives with Nook. Other companies like Apple, Johnson & Johnson, IBM, and General Electric have steered their ships across longer journeys. It is possible to go the distance, but it requires an openness to change that is so uncommon in business, you almost have to shake people physically to get them to see how to save themselves. Generally speaking, corporate people don’t like to be shaken, even if it’s good for them.

Creative destruction as most commonly defined by Joseph Schumpeter is real and unavoidable. It is also reasonably easy to argue that despite the pain it causes in transition, it is a positive force of social evolution that drives us forward and replaces inefficient procedures with new technology, updated methodology, and even new financial opportunities for investment and return. My dear friend Kermet Apio, a wonderfully successful standup comedian, captures the essence of Creative Destruction in a 90 second bit where he compares the joy and simplicity today of clicking on a song you might want to hear versus trying to find it on a cassette tape, which might take you so long you’d almost certainly abandon the task unfinished, or worse, try using your pinkie on the internal reels to queue up the precise starting spot. There’s a touch of nostalgia here, and we do find ourselves laughing very hard at what was our norm not so long ago. Click on the link above to see how Kermet tells the tale, the chuckle makes the point.

But no one is laughing at Kodak’s headquarters in Rochester, and no one should be. Kodak had the first digital camera in 1975, and while admittedly neither they nor anyone else knew what to make of this at the time, they had a much more important mandate on their mind: Protect Film. Kodachrome was not only iconic, it was hugely profitable. So was motion picture film processing. So were all their other traditional film developing technologies, not to mention the sale of retail supplies, equipment repairs, and patent licensing. Kodak was a beloved company and a global brand that made the same wrong decision so many other short-life companies make—they worried too much about cannibalism, and not enough about what happens if they don’t cannibalize their own markets.

It doesn’t get any easier to understand than this—if you don’t cannibalize your own markets, someone will do it for you. The choice is that simple: do it to yourself for your own good, or be the victim of outside attack. No form of technology is forever, and any trend you’re surfing is going to break flat on the beach. In Kermet’s bit, he talks about the Sony Walkman. Everyone had one. It was great. Then came the CD, Sony had a piece of that technology, so far so good. Then came Apple with the iPod, not the inventor of portable digital music playback, but the “perfector.” By the time Sony responded, they were on defense instead of offense. Too late. The cannibal is here, it came from elsewhere and did what you feared it would. You knew it would happen, you couldn’t stop it, but you could have been it. That’s the choice. Not will it come, but from where will it come.

That is the real lesson of Kodak: no one can stop the march of innovation because it is inconvenient or upsetting. No company can duck cannibalism by refusing to acknowledge that current markets have to be sacrificed for new markets to be built. If you’re young and just getting into business, get used to this, and get used to your bosses telling you all the reasons why they have to protect what you have today, that the hit to earnings to attack your own hugely successful lines of business with nascent replacement ventures is just too painful. If you’ve been doing this a few decades, remember back on all those long and awful bureaucratic meetings where you wished someone would have pounded the table and screamed, “To Hell with cannibalism, we’re doing this—keep the cannibal in the family!” There were meetings where that happened. Those are the companies with the 100 year brands.

If you are at CES this week wandering the endless aisles of new stuff and you see something that could eat your lunch, ask yourself, why didn’t we think of that? And if we did, would we have had the courage to launch it? Let’s hope this lesson gets easier to recite so we don’t see loyal employees lose their benefits because political correctness forced a gag order or management failed to act when time was on their side. Manage the product life cycle, but don’t be afraid to leave a little money on the table. Get the new products out there before someone does it for you. The real money is in longevity, which means innovation, which means playing offense against yourself.

Are We Thankful Enough?

The following is an edited version of a note I sent to my staff a few years ago.  I started to draft a new version, but then remembered how similar this was in theme:

Each year about this time I like to take a few minutes to share some of my gratitude with colleagues. Given the industry in which we work, it is sometimes hard to separate our business interests in the holiday season from our own more personal sense of human enrichment, but let me try. True enough, the holidays can be seen through the eyes of materialism, and indeed given our dependence and expectations on retail behavior this time of year, it is too easy to allow oneself to “Get Scrooged” without seeing some of the more enlightened generosity that is all around us. Forgive me, Shelley and I attended the annual tour of Trans-Siberian Orchestra this week, so I am in a highly festive and particularly reflective frame of mind. The work we do for our customers and each other is much more than a feeding of the virtual cash register for tabulation by the National Retail Federation. The work we do has meaning because we have chosen to share this time together and infuse it with meaning. It is there if you want to see it, and it is always there for me in each of your own creative contributions and team celebrations.

Let me start with the basics, I am thankful for all of the wonderful people around me each day. As I always say, I have good days and bad days but I never have boring days. The work we do is interesting because the people we share it with are universally interesting. Each day I see your passion expand, your thinking blossom, your communication flourish, and your expectations of yourselves and each other rise to new heights. This isn’t just invigorating for me, it is sustenance. There is reason to come to work each day as long as there is purpose in the day’s activity, and sometimes that purpose is simply rooted in the ability to learn something new. I can honestly share with you that I learn something new from the imagination that surrounds us each day, and I have no sense that has likelihood of disappointing me anytime soon.

I am thankful for the good fortune of being alive at this precise moment in history. To truly appreciate and understand the power of the Internet is to have lived without it for so many years before. I used to say this about the personal computer, that to discover it as an artist’s palette was for me not a continuation of history, but a reinvention of history. Just as many of our parents were born into a world without television, the advancement in democracy of being able to see news from around the world each day was almost a miracle, as was radio before that, and widely available print before that. To be alive today at the inception of the digital age is to me a gift as well as an invitation to have a profound impact on establishing a set of norms that are as evolutionary as they are unknown. Our younger kids see texting and mobile communications and even social networking as quite ordinary, if you were here before them, my sense is you share my awe in the privilege of codifying the extraordinary.

This takes me to my third thank you for the year, appreciation for being able to have even the smallest impact on reaching out to change our world. Our technology has impact, our creativity is unbounded, and our business relationships are honest and crafted around the principle of win-win-win: a win for us is a win for our partners and a win for our customers. You may not always get to work in a culture that embraces notions of empowerment, I certainly have had my own ups and downs over the years in various places I have worked. Yet more than that, we get do fun things like embrace Make-A-Wish kids, give thousands of prize dollars away to families who need it, offer great discounts to families who might not get by without them, help people make the world slightly greener by encouraging them not to drive somewhere if they can shop at home. We also save moms time, lots of time, time that can be better spent with their families enjoying more moments than they might otherwise spend away from home on errands and chores. No, it’s not the work of Mother Theresa, but it is very positive and uplifting, especially when you read all those comments each day from people saying they “love” what we do for them. That’s a powerful word, and each morning I read it in our customer comments, I know we are doing something right.

So I wonder, are we thankful enough? Can we make Thanksgiving something more than a time to power-eat and start charging up our credit cards on the big sales days that follow?  As we enjoy two days away from the office, what is it that we can reflect on that keeps us coming to the office? Thanks for our incomes – I am sure there are varying levels of satisfaction there, but to have a regular income is still unfortunately rare in world of six billion people. Thanks for the people who sit next to us, or in front of us, or in the next room over – again, I am sure there are some around us whom you like more than others, but then again, I am confident that every one of us is within talking distance of at least one or two people we really appreciate, and as I said, don’t take that as a given, it will not always be the case. Encouragement to pursue excellence – OK, I know there are cynics out there who say this is just work-speak, but I promise you it is not, we have created an environment where we expect you to do your best and create work that makes you immensely proud, you’d be missing an important moment if you didn’t embrace and enjoy that, a lot of places it really is just work-speak. And finally, memories and future foundations – the accomplishments we enjoy, the education we give and receive from each other, the stories we are creating to enjoy at a later date, all of that is worth a moment of meditation; time escapes us in precious illusion, and though you are likely to forget this project or that deadline in the years out, if you look around you and thank your colleagues from time to time for even the smallest favor, you just might be making history, as that could become a moment you will share for years to come.

Freedom is such a difficult concept to appreciate because most of us have always known it, it is in the fabric of our society. Yet again, look around, is it the norm or a gift we can cherish? As we keep the women and men who serve us in uniform at the top of our thoughts this time of year, perhaps we can also reflect on just what it means to have the lives that we do, where we can pursue career aspirations and friendships and family and creative contributions to our world all at the same time. As I type these words, it all seems like a pretty big deal to me. I wouldn’t take it for granted. To be thankful is to truly enjoy all that we have, and as I look around our company, I see that we all have so much. I am never sure that I can personally be thankful enough.

I hope you are all enjoying this special time of year, it comes with a lot of work stress and family stress and Scrooge-Stress! Yet the journey is the reward, so let’s do our best to enjoy it and share it and where it makes sense, be thankful. You’d be surprised, it really can be a magical world when you look for the magic in each of the people around you. I see it, so very clearly!

Happy Thanksgiving!

Originally published: 11/22/07

Evolving The Ad Measure

Last week I spent a day and a half at a four-day conference in Los Angeles known as Digital Hollywood. I can remember speaking at this conference a number of times back in the bygone CD-ROM years, when QuickTime v1.0 was all the rage and the notion of getting postage stamp video to play on a PC was gleefully deemed the dawn of FMV (full motion video, which was still about 10 years and many versions of QuickTime to come). Digital Hollywood has been growing steadily since 1990 and is now hosted in multiple locations throughout the year. It has become well-attended and thrives on emerging trends and technologies that carry with them opportunity and hope.

While I can hardly say my tour through the panel discussions at this conference was exhaustive, my experience was that many people were there in search of the question: If I build it, will they pay? The broad desire seemed to be for so many passionate and creative souls, if they put their heart into creating digital content, is there any chance at making even a modest living at it? The big media companies continue to study the little companies, still trying to solve the riddle of how digital pennies can replace analog dollars before the next wave of Creative Destruction breaks on our Company Town shores. The little companies and individual voices remain excited by the notion that self-publishing has zero barriers to entry, and with no constraints on distribution, anyone can be in the communications game through YouTube, a blog, a web site, an email newsletter, a Facebook page, and with just a little bit of push a mobile app. The breadth of creativity screams freedom as well as opportunity, yet when you gather in the halls, creative satisfaction seems to be outpacing financial satisfaction at almost every level of the pyramid.

When asked about business models, studios and individuals alike tend to respond most often with the word “advertising.” There are actually several ways to monetize digital content (subscription, syndication, e-commerce, data mining research), but the approach you hear most is advertising. It continues to strike me as ironic that the greatest and most liberating technologies of our day so often point to something as old school as advertising to fuel them, as I am sure it surprised pioneers from Google to Facebook. We have seen the shift in advertising wreak havoc with print and radio and outdoor, and finally it is beginning to put pressure on television. My question remains, why aren’t traditional television ad budgets under significantly more duress?

A very quick primer on advertising, there are two basic kinds: brand and direct response. Brand marketing attempts to get you to encode a message and take action later, direct response attempts to create instantaneous demand and get you to take action now.

When you watch a network television show in its time slot and a commercial tells you how the floor wax you are seeing in action will get your floors to sparkle, that’s brand advertising. It is meant to get you to remember the brand you saw on the commercial in a positive light when you are in the floor wax section of Safeway. That is achieved with reach (how many individuals see the commercial) and frequency (how many times they see it) which add up to affordable tonnage. If you are my age and can still recite the ingredients in a Big Mac, you have a pretty good idea how much money McDonald’s invested in the reach and frequency for that brand campaign when we were kids, super tonnage to burn into memory that pithy creative construct.

Mad MenBrand advertising is usually measured in terms of broad sales increases in a product line or shifts in competitive market share as a result of the campaign. Skippy buys an ad schedule across TV, print, and radio, spends a certain amount, then measures over a period of time what impact it has on sales of their peanut butter. They may experiment then with new commercials, or add weight to TV and subtract it from print, trying to get the best return on investment possible. You can imagine what an inexact science it is, but if you pay X for your ads and get more than X in increased sales, you at least know your campaign paid for itself, and from there, the sky is the limit. In the 1960s and 1970s with three TV networks, it was really hard to go wrong with the kind of TV buys we now enjoy memorialized in Mad Men.

Direct response advertising used to be the less polished hard sell stuff we saw on late night TV or UHF, where the commercial or infomercial shows you a miracle vacuum cleaner obliterate a thick pile of goo and then gives you an 800 number to respond now and buy it. It is also the kind of advertising that worked well in print and catalogs via mail order. To this day it remains simple and exact to measure because there is little noise in the equation. You run an ad, your switchboard lights up with orders or it doesn’t. You know what you paid for the ad, you know how many orders you got. It’s not very glamorous, but compared to brand advertising, it is easy to evaluate and research is precise.

The glamour factor shifted with the internet from brand to direct, because the economics shifted with the internet at huge scale. On the internet, direct response, or performance based advertising, mostly trumps brand advertising. In the digital world, brand advertising became known as display advertising—in the consumer vernacular, banner ads (industry jargon sometimes calls them dots and spots) where payment is rendered by the advertiser for delivered inventory—insertion order invoiced tonnage just for showing up. The click-through rate on most display ads today is almost not worth measuring, but that does not mean they are not impactful. Most of the ads you see on Facebook are display, lots of reach and frequency, and much better targeted by interest level than ye olde TV.

Yet the glamour business of the internet remains keyword advertising, the logical evolution of direct response advertising, the sponsored links that have been most successful for companies like Google but are also used in comparative shopping sites and similar layouts where the ad buyer does not pay for an ad to be seen, the ad buyer pays for a click on the keyword link, and then counts on a certain number of clickers to follow through to transaction (that means buy something). Here again, the direct response model is more precise than the brand model and can be measured with sophisticated analytics, while the dots and spots—and now with video streams commercial insertions in Hulu and YouTube much like TV only shorter—should be fully intended to contribute to downstream sales activity, but are much harder to evaluate mathematically.

In the world of TV, brand still rules. In the world of internet, direct response still rules. The reason? Performance, also known as Return on Ad Spend (ROAS). You can still drive big sales and shifts in market share via a TV brand campaign, and you can do the same with an internet direct response campaign. So my fundamental question remains: why hasn’t the science of efficacy and research advanced to show how display campaigns online can approach the same sort of massive scale impact on consumers that they have on TV?

There are many things we can measure in both brand and direct response campaigns, some would argue too many. If McDonald’s stopped spending completely on TV and moved all that budget to the internet at better prices, would it have a negative impact on their business? Probably, or they would do it. The question is, how much can they move, and how much more affordable can it be for them to start moving more of it? How can they tie market share gains back to internet display campaigns? Attitude and usage studies—the kinds of email surveys you get asking if you have seen or remember a campaign—aren’t nearly enough to convince them they can move billions of dollars of burgers at the counter without an accompanying TV vehicle. They need digital brand campaigns that sell goods and services at scale and science to attribute the success—and we don’t yet have either.

Going back to our passionate content creators at Digital Hollywood—what do I think they should be worried about for their sustenance? I think their fate will be cast by leapfrog advances in advertising research, and I think those advances will come. With advances in the science of display advertising efficacy in digital platforms whether fixed or mobile, the big brand dollars will have to shift from television to non-television. It is not just a question of eyeballs (mind share, share of voice) shifting with a generation that has grown up digital, it is a question of what works, the reach and frequency and cost efficiency to make or break predictable sales of consumer products. When we have that science to show how digital spending improves on the job of TV, the big brand dollars will shift and content opportunities will flourish.

Almost every graph trend shows that the future for digital media is nothing but bright, but until the research and reporting platforms rescue brand advertisers from the opaque, illusive promise will remain greater than reality. That shouldn’t last much longer. Tell your digital research departments to put those monster TV budgets in their gun sights and keep innovating. It’s really good money if you can get it. And we will.