Listing Your Way to the Finish Line
Mar 25, 2010 Statistics, TV and iTV Analytics, Web Analytics
Draft Marketing Analysis Checklist
After reading Atul Gawande’s recent book “Checklist Manifesto”, I was thinking there should be a checklist for marketing analysis. One point that Mr. Gawande makes in his book is that highly-trained specialists shun checklists because in their minds only dummies need lists. However, a majority of surgeons, while rejecting lists for their own use, would want another surgeon to use one if operating on them. This is because they know how easy it is to forget one detail in hundreds.
In marketing analysis, there are a lot of steps and a lot of things to think about, and even a smart person might drop a stitch here or there if they are not following some kind of list. I have included a rough one I dashed off quickly, in hopes that others might offer refinements, altogether better lists, or more specific versions for types of marketing programs. Here it is, have at it!
DEFINE
• SET goals/ hypotheses for program
• SELECT metrics
• CREATE a measurement plan
EXECUTE
• EXECUTE program and measurement plan
• VALIDATE raw data
• PREPARE dataset for analysis
ANALYZE
• VISUALLY EXPLORE dataset for patterns and problems
• SUMMARIZE dataset statistics
• SCORE performance vs. goals/ support for hypotheses
• LIST likely conclusions
• IDENTIFY unexpected or surprising findings
• VALIDATE likely conclusions with numerical/statistical support
• SELECT final findings
COMMUNICATE
• REPORT findings for future activity
• REVIEW findings with user community
• CAPTURE questions &issues from user community
FOLLOW-UP
• INVESTIGATE user-identified questions & issues
• IDENTIFY impact on original findings
• REPORT findings of follow-up analysis
What do you think?
Tags: Analysis, Analytics, Atul Gawande, Check, Checklist, List, marketing, Process
The Cookie vs. The LSO – Should I Care? Should I Worry?
Feb 18, 2010 Industry News, Web Analytics
Here’s a question that savvy web users were being asked by their parents 10 years ago:
What the heck is a cookie, and why do I have them on my computer? Do I need to delete them? How do I delete them?
Don’t be surprised if the question starts to come up again, in a new form:
What the heck is an LSO, and why do I have them on my computer? Do I need to delete them? How do I delete them?
The issue is emerging again because of the people in the business of targeting ads or offers are trying to do their job better, and cookies are not doing the job advertisers want done. So, some web programmers are exploiting a feature of Flash to create “stealth cookies” called LSOs, in hopes that you won’t delete them because you probably don’t know how.
Remind me: What is a cookie again?
A cookie is a small text file that is created via your browser to keep track of session “state” and historic entries and site activity.
What is a cookie for?
The connectionless protocols used by the web do not automatically keep track of any history. If there is no state or history information provided with a page request, then the page will have no idea who you are, even if you just entered that info on a different page in the same site.
What’s so scary about that? Well, people just don’t like their activity being recorded without their permission or awareness. It annoys them. That said, there are useful things that this kind of snooping makes possible:
At the same time, it makes possible:
Cookie Deletion
When many people figured all this out it became a big kerfuffle, and this led to user behavior such that 23% of all cookies are deleted when they are one week old, and that less than half of all cookies (43%) live to be more than eight weeks old (click here to see Microsoft research about cookie deletion). Users can use functionality in their browsers to delete cookies and to control cookie-related policies within the browser.
So who cares? What problems does cookie deletion cause?
If you are an internet advertiser, it adds one more layer of complexity to the already difficult problem of tracking internet ad campaigns. You’ll have tracking pixels in ads to capture views and clicks, but knowing how many times someone has seen an ad during a campaign (frequency) and how many distinct individuals have seen an ad (reach) is pretty critical to understanding what is going on in a campaign, especially as more brand advertising comes online. Measurement is made difficult in internet advertising by these factors:
Net/Net: Bad Measurements
On balance, these issues push the measurements in the direction of overcounting reach and undercounting frequency.
Some of the other deficiencies of cookies from an advertiser point of view are that cookies don’t store very much information (4KB), and there can only be so many cookies related to a given domain (20). Privacy considerations additionally limit how much cross-site behavior can be captured in cookies (and banner campaigns are cross-site, mostly).
LSOs Addess Some of These Shortcomings For Advertisers (Yay!), But Create New Ones for Users (Boo!)
An LSO (Local Storage Object) is a cookie-like file that Flash uses to store information for Flash applications. Except that they are used by clever web programmers for far more than that – they are used by some sites just like really big cookies (as much as 25 times bigger than a cookie) that you don’t know about and so won’t delete. In addition, the same LSOs are accessible from all browsers. Your browser security controls have little or no impact on these things.
You Might Want To Check Your Computer For LSOs Right Now
If you don’t believe me, go to the Macromedia page that lets you see what LSOs are on your machine (it also lets you delete them, enable/disable them, and control their behavior).
It is located here: http://www.macromedia.com/support/documentation/en/flashplayer/help/settings_manager07.html.
While you are there, delete the ones for sites you don’t want your boss to know about.
As for where this is all going, all privacy loopholes on the web are temporary, and there are already browser add-ins that let you control and delete LSOs, and at some point the browsers will absorb that functionality to make it easy for you to use. If I were you, I’d worry more about the things you can’t see: The new keystroke dynamics technique for identifying users announced by Scout Analytics (here) and backend ISP- and CDN- based tracking – all these are fodder for more paranoid posts in the future.
Tags: Analytics, banners, Campaign Analytics, Cookies, Flash Cookies, frequency, Internet Advertising, LSOs, marketing, Media Measurement, Privacy, reach, Security
Opening Weekend for 2010 Winter Olympics: 117 Million US Viewers
Feb 16, 2010 Industry News, TV and iTV Analytics
Check out this article by Robert Seidman on TVbytheNumbers.com In it, he cites Nielsen ratings indicating this year’s opening ceremonies beat the Torino Olympics’ opening weekend by 5 million viewers. The average of 28.6 million viewers over the first weekend beat Torino’s first weekend by 25%.
This beats the 106.5 million viewers last weekend for the Super Bowl (see the prior post in this blog), but that was much more concentrated in time. The Super Bowl got a 68 share while the Olympics first weekend got a 26.
The Olympics also did well on the smaller screens. Three Olympics apps are currently in the top 10 on iTunes, and NBCOlympics.com traffic is 250% higher than it was for Torino. It has only been a few days, but there have already been more unique viewers for NBCOlympics.com during the Vancouver Olympics than than there were for the whole Torino Olympics.
Play With Your TV! (a shameless plug for my #1 client, Ensequence)
If you are watching the Olympics via Dish Networks or Verizon FiOS, then you can access weblike interactive content right on your TV screen alongside your favorite Olympics events. Once you tune to MSNBC, CNBC, or USA, a prompt will pop up (nothing on NBC itself, as far as I know). Clicking the “Select” button on your remote starts an interactive experience that includes Top Stories, Medal Counts, Athlete Bios, and more. Real interactive TV in the wild. Check it out!
Tags: Analytics, audience, Bill Seely, Ensequence, Inteactive Television, Interactive TV, Metrics, Nielsen, Olympics, Olympics Interactive, Practical Marketing Analytics, Robert Seidman, TV and iTV Analytics, TV Ratings, TV Viewers, Viewership
Nielsen Estimates 106.5 Million Viewers for Super Bowl XLIV (aka Beating the Pants Off Elvis)
Feb 8, 2010 Industry News, TV and iTV Analytics
In a story by David Bauder of the Associated Press, A.C. Nielsen went on record estimating that 106.5 million viewers watched Super Bowl XLIV (see at WashingtonPost.com HERE). That is the most heavily viewed event in TV history, bigger than (according to Wikipedia):
– the final episode of M*A*S*H (105.97 million viewers)
– last year’s Super Bowl (98.7 million viewers)
– the Beatles’ first appearance on The Ed Sullivan Show (73 million viewers)
– Elvis’ first appearance on The Ed Sullivan Show (60 million viewers)
To be fair to the shows of yesteryear, the total US population (and the number of households having TVs) has continued to increase since those days. There are about 305 million people currently living in the US, so about 1/3 of the entire population watched the game.
It is remarkable in our modern splinter group society that we could find something that such a huge group of people could watch together, especially when you could rule so many people out right at the starting gate: infants, toddlers, anyone in solitary confinement, anyone unconscious or too sick to care, anyone at work in a job where you can’t watch TV while you work, almost anyone who was in an airplane at the time, and most people who immigrated from countries where a “football” is something spherical.
I don’t know the full importance of this number, but it does suggest that:
1. TV has not been made irrelevant by the Internet, despite Internet entrepreneurs’ claims
2. People will still show up in giant hordes to watch a TV event en masse, if the product they are watching is enticing enough
3. Ed Sullivan really blew it by making them shoot Elvis from the waist up. The least remarkable half of him got 60 million viewers. Who knows what the full Elvis could have scored?
Tags: Analytics, AP, Associated Press, Bill Seely, David Bauder, Elvis Presley, Highest-Viewed TV Show, M*A*S*H, Metrics, Nielsen, Practical Marketing Analytics, Super Bowl, TV Viewers
The December Issue of the Journal of Advertising Research (JAR) has Great Metrics Articles!
Jan 25, 2010 Industry News, Marketing And Advertising Analytics
There are a couple of useful articles this month in the Journal of Advertising Research. They are on a roll over at the JAR, driving some great discussion in the last few months about measurement of marketing, digital and otherwise. Recommended reading in this month’s issue:
“Commentary: Who Owns Metrics? Building a Bill of Rights for Online Advertisers”, by Benjamin Edelman, Harvard Business School Assistant Professor in Negotiation, Organizations & Markets
Ben Edelman, who has written on the role of deception and overcharging in online media (among other topics) is right on target here – he argues that advertisers have a right to know where and when their ads are being shown, delivered in the form of meaningful, itemized billing. He also asserts the advertisers’ ownership of the data that comes from their campaigns, and says they should (for example) be able to use data collected from their Google PPC campaigns to target campaigns on MS AdCenter or Yahoo! This is definitely a controversial area – certainly Google, along with cable and satellite TV operators, would disagree – read it and let me know what you think.
“It’s Personal: Extracting Lifestyle Indicators in Digital Television Advertising“, by George Lekakos, Assistant Professor in e-Business at the University of the Aegean, Greece.
In case you think my comment about TV distributors wanting to own audience data is irrelevant in the context of digital marketing, Lekakos lays out a scheme for using set-top box data to discover and target lifestyle segments that are then used as part of a targeting algorithm. The author lays out an approach by which TV set-top box data can be used to drive very accurate personalization and targeting of ads, but the question of whether the data belongs to the distributors, the programmers, or the advertisers is quite critical to whether this can be implemented. I’d have to say that the question is far from settled.
“Measuring Advertising Quality on Television: Deriving Meaningful Metrics from Audience Retention Data“<by Dan Zigmond, Sundar Dorai-Raj, Yannet Interian, and Igor Naverniouk
The authors explore the use of audience retention metrics captured via TV set-top boxes as a measure of ad quality. They use a “retention score” that purports to isolate the effect of ad creative on audience retention, and link it with future audience response and qualitative measures of ad quality. They assert its usefulness as a relevance measure that could be used to optimize TV ad targeting and placement. Again, we should note that the issue of data ownership needs to be dealt with if this approach is going to be applied widely.
“The Foundations of Quality (FoQ) Initiative: A Five-Part Immersion into the Quality of Online Research“, by Robert Walker, Raymond Petit, and Joel Rubinson
To address both the increasing importance of online research and questions about its validity, the FoQ Initiative has been undertaken to measure the quality of online research. The Online Research Quality Council included large advertisers, ad agencies, academic researchers, and research suppliers in the process. Among the issues they addressed: accuracy, representativeness, and replicability of results, identification and handling of questionable survey-taking behaviors, and the suspicion that small number of “heavy” online respondents are taking most online surveys.
Some of the interesting findings:
Tags: Analytics, Ben Edelman, Bill Seely, Dan Zigmond, FoQ, George Lekakos, Igor Naverniouk, JAR, Joel Rubinson, Journal of Advertising Research, Metrics, Practical Marketing Analytics, Raymond Petit, Robert Walker, Sundar Dorai-Raj, The Foundations of Quality Initiative, Yannet Interian
Search Volume for Analytics Ramping Up Steadily – (More Fun With Google Trends)
Jan 4, 2010 Industry News, Marketing And Advertising Analytics, Statistics, TV and iTV Analytics, Web Analytics
Just for fun, I did another Google Trends search, this time on “analytics” – adding “CRM” and “ERP” as reference points. The result seems to suggest that if you are in the business software market, that you should have an analytics offering. We’ll see, but I predict that the hot growth area in business software in 2010 will be Analytics. Searches for analytics have been steadily ramping up for the last several years, and are now at a higher level than searches for the above-mentioned enterprise business software categories.
I find it very interesting that searches for “ERP” and “CRM” have been flat for so long, but REALLY interesting that the volume of “analytics” searches surpassed them in 2009.
Tags: Analytics, Bill Seely, CRM, ERP, Google Trends, Graph, Keyword Search Frequency, Practical Marketing Analytics