The 'Hit by a Bus' Social Media ROI MethodThe 'Hit by a Bus' Social Media ROI Method

Like most social media enthusiasts/evangelists in the enterprise, I am constantly asked the inevitable "social media ROI" question. Sometimes with a smug, "gotcha" look, and sometimes with genuine concern. Over the last couple of years, my responses have gotten more sophisticated as I have learned more. The first, more serious part of my response involves a social media capability maturity model (CMM) that I have been developing informally, and which I'll post about in January (stay tuned!). But the second part of my answer has started to rely on something I call "hit by a bus" (HBAB) ROI analysis. Here's how you use it, to reframe ROI discussions that start off with inappropriate assumptions.HBAB ROI AnalysisTo understand how this works, first imagine you are talking to a well-intentioned and sincere person asking about social media ROI. If your interlocutor is a 'gotcha' jerk, give him/her your best cold stare, say "I can recognize bait when I see it" and walk away. Let's call your sincere interlocutor Abe. Here's how the conversation might play out in the best case.Abe: I have to admit, I am concerned about how we'll ever measure the ROI on all this social media stuff.You: Well Abe, would you say email today is business critical? That every business should use it?Abe: Well of course, but...You: Don't worry, I am not making a direct comparison, but how do you measure the ROI of email?Abe: Well, now it is just a cost of doing business since everybody uses it, but I suppose back in the early nineties, the first adopters gained a time-to-market advantage, faster sales and service cycles, stuff like that?You: Agreed. Could you attach an ROI to those forms of value-add, during the period that email was a differentiator?Abe: Yes and no I guess. At one extreme, you could say that partly because of email, a hypothetical Widget Inc. might have launched its 1992 model four months before the competition, and taken away and hung on to 10% additional market share, worth $100 million, even after the competitor caught up. First-mover advantage. So by one measure, the ROI on the investment in email is at least the entire net-present value of the market share they gained. But in another way, this analysis is silly, because email was a necessary, but not sufficient factor in the success of the 1992 model Widget.You: Agreed, so I propose to you that since you can't do controlled experiments of the 1992 Widget product launch with and without email, you have to do some counterfactual reasoning. What if, back in 1992, the "install company-wide email to replace paper memos" evangelist had been hit by a bus before the big meeting where he convinced management? What if you assume that it would have taken them an extra year to get on the email bandwagon, and that only the 1993 Widget model benefited from the accelerated-by-email TTM, by which time, their competitor also had email?Abe: I see where you are going with this. The fact that email was only necessary, but not sufficient, is irrelevant. There isn't much point trying to apportion value realized across multiple individually necessary, and together sufficient differentiators. I guess the ROI for each of the novel differentiators in the success of the 1992 Widget was equal to the whole value realized, so you'd book that 10% increased market share for both the email system and, say, the new chrome finish on the 1992 Widget. It still seems like fuzzy accounting to me though, since it seems like we are double counting.You: See, I don't think that's a problem. We are not talking legal accounting or even managerial budget accounting. We are talking strategic ROI, where the point is to guide decisions, not compliance with accounting practices. A plane with one wing removed doesn't fly half as well. It doesn't fly at all. So it is fair to attribute the flight worthiness of the entire plane to one wing.Abe: Okay, so how do you port the analysis to social media? Email is different, because it replaced an obsolete but obviously mission-critical technology -- paper interoffice memos -- with a better system. And that ended up transforming the nature of the corporation in the medium-is-the-message sense. Enterprise 1.1 say. A lot of social media doesn't do that. My company won't go under in the next quarter if we don't start twittering. It mostly seems like 'plus' activity; augmentation rather than clear substitution. In fact where we've tried to make it a substitute for necessary functions, it usually fails. We are still figuring out why blogging as a replacement for team emails didn't actually work. We do have a network of social media enthusiasts all over our organization, and though I appreciate them, if they all got hit by a bus tomorrow, I don't think we'd go under or not make our next product launch.You: I am glad you said "network of enthusiasts" rather than "community of practice." But that's neither here nor there, just one of my pet peeves. But you made a very important point. 'Social media' is an umbrella term that is really about a set of implicit design principles that have guided the development of each technology within the umbrella. Its impact is diffuse and multifaceted, but overall as dramatic as that of email, I would argue. You made another important point too, that except in very rare cases, viewing social media as workflow substitutes for existing technologies is a bad idea. Individual social media technologies make very bad function-for-function substitutes at the workflow building-block level. They tend to require re-engineering, or rather de-engineering, of the entire process in question along simpler, more ad-hoc lines. Sometimes a de-engineering of the entire company. But that's a whole different conversation. Getting back to the diffuse nature, let me ask you, using the metaphor of the organization as a body, how do you know that the ROI on physical exercise is worthwhile?Abe: I'd like to go down that other bunny trail sometime. But I agree, the impact is diffuse. You can't say specifically that not exercising will cause you to have a heart attack one year earlier, so it isn't like the email-impact-on-TTM argument. But we know that in general, not exercising just makes a whole lot of failure modes a lot more probable, and many opportunities, like climbing Everest at 60, less accessible. So when you add 'em all up, you get a pretty high probability that between two otherwise comparable people, the one who exercises regularly will likely have a longer, healthier and richer life. Whether that means 'better' is for the philosophers to argue.You: I almost agree with your example, but I think you don't have to be quite as cautious as that. You can draw more specific inferences beyond "general company health." Let's say two competing companies are racing to launch the same next generation product. The social media evangelists from company A are all hit by a bus say, van-pooling out to a Wordpress Barcamp to meet Matt Mullenweg. What happens next?Abe: Okay, just to stress-test the counterfactual, let's assume no time-to-market is lost, since the value of improved TTM is practically business gospel these days. So company A's product heads out the door with top-down internally-created design, phone support, a traditional press release, a TV ad campaign and a 1.0 broadcast website. Company B's product has some prosumer design elements, customer community forums, tweeting from the dev team leading up to the release, and so forth.You: There's more. You forgot the internal side. Company A will have waterfall code and incomplete, error-ridden specification documents. Company B will have all its code in Subversion and nice Scrum-based project collateral. By launch date, the sales and marketing launch teams will have been brought to a mind-meld state by internal blogging; most critical product bugs, both hardware and software, would have been ironed out in controlled beta processes. Company B will have better, more current crowd-sourced industry and competitive intelligence informing the product positioning at launch, rather than last year's strategy document, with more room, in the post-launch discourses, to adapt that positioning.Abe: Okay, okay, I get it. So when the products go head to head in the marketplace, whether or not specific individual tactics like twittering have any impact, cumulatively, Company B's product will be a much more robust offering. And one going to a prepared customer partly won-over through participation in design and testing, and more forgiving of remaining issues due to his sense of co-ownership and direct relationships with people deep inside the company. Or should we say, previously invisible people newly moved to the customer-facing skin of the company. You know what this reminds me of? All that talk of network-centric-warfare and shared situation awareness from back when I was in the army in the late 90s.You: That's interesting. Yes, in social media, everything, not just beauty, is only skin deep. It's all skin, you might say. You say the closest analogy you can think of for social media, with all its warm and fuzzy 'sideshow' connotations as opposed to 'serious' business, is the hard-edged world of war? How so?Abe: Your point of the mind-meld between product development and sales/marketing is actually a good specific instance of why the analogy works. If you drop the trekkie-talk, you are basically talking shared situation awareness, or SSA, as the military calls it. The basic hypothesis of network-centric warfare is that better and faster ad-hoc communication leads to better SSA which makes for better operational effectiveness, especially in the reactive and opportunistic parts of warfare. You react to threats and opportunities faster, and with better information. On average, you'll win more.You: So you're making my case for me here. You can actually keep yourself honest about social media ROI, so long as you ask the ROI question in the right way. Not point-for-point ROI comparisons of twitter vs. a press release. That would be silly.Abe: This is still not easy. How do you get this hit-by-a-bus story across to an entire population in a large company? I can't imagine my CEO making a speech about this at the next big communication meeting.You: I do hope you don't even try to get the CEO to make a speech. A CEO blog is nice if the CEO likes the medium, but the job of the network of enthusiasts is to co-opt more people into the network of enthusiasts, one 1:1 conversation at a time, one toe in the water at a time.Abe: Sounds painfully slow.You: Not really. Remember Metcalfe's law and network effects? Word-of-mouth and 1:1 persuasion is ultimately faster than broadcast.Abe: It also sounds vaguely cultish.You: Not really. I am not asking you to believe in the gospel of Social Media. I am just trying to provoke you into trying twitter.Abe: Fine, you got one of my toes in. I'll try twitter today. Next time let's go down that bunny trail of de-engineering processes vs. functional substitutions.You: And if I convince you, you'll try a podcast?Venkatesh G. Rao writes a blog on business and innovation at www.ribbonfarm.com, and is a Web technology researcher at Xerox. The views expressed in this blog are his personal ones and do not represent the views of his employer.