The ugly side of web analytics

Web analytics is a new(er) field. Everyone knows this (right?). The industry still comprises of professionals who are more or less, for a lack of a better word, jacks-of-all trades and master-of-some – out of necessity, if not by design. Web analysts are usually part-developer, part-marketer, part-salesman, part-teacher and unfortunately for too often, part grunt. No one told me that I’d spend hours upon hours overseeing developers (aka herding cats) as they botch up implementation after implementation. Nowhere in the job description does it say that the client sometimes just won’t listen – no matter how many times I explain something to them. Nowhere is it advertised that I would have to have to spend so much time working around so many unbelievably frustrating limitations of analytics packages. Though I was briefed that I would have to wear many hats, I didn’t know I would have to wear a helmet and a cup.

Forget insight, what about hindsight?

So much talk about the information I can extract. So much talk about how to convince hippos. Oh Avinash, why didn’t you warn me about how slow-to-react analytics vendors would be to true business needs? Why don’t their filters work? Why are the reporting abilities limited? What about those missing bounce rate metrics? Don’t get me started about having to crunch numbers in Excel…

The fact of the matter is that as sophisticated as many web analytics solutions are right now, they aren’t that advanced. They lack percentage points, standard deviations, variances, significance and simple forecasting. All these common managerial tools are virtually non-existent. However, they all do give me averages… Forgive me as I rub my temples. Oh, did I mention that a lot of these solutions are slow?  No one told me that I’d have to wait days for some data queries. I’m not a BI analyst per se, but days seems awfully long considering the basic nature of the data. And as most you know, my analyst brothers and sisters, these short-comings have been around for years. Analytics tools come out every quarter – all missing some important piece of the puzzle. It’s like they don’t listen to us. If only Avinash would of warned me – I could of braced myself. *grumble* This situation, while better than 5 years ago, still “sucks”.

Macro-economic theory, take me away

It is said that if one or two companies hold the lion’s share of an industry, it’s because they have an initial technological edge. But overtime, as more entrants enter the market, this advantage fades – leveling the playing field. I can’t wait for that day. But until then, I’ll continue to create work-arounds, I’ll continue to crunch the numbers in Excel, and I’ll continue to double-tag. I’ll suffer through all of this because the rush of helping a client leverage a once unknown piece of intelligence is just too seductive. It feels too satisfying to give up. At least this part of the job Avinash did tell me about it.

Oh, OK. I get it now. Never mind.

Web analysts agree

If there was a common thread throughout the early sessions of eMetrics, it was this: everyone agrees. Lots of nods and “uh huh”s throughout the room. “Start small”, “think about business goals”, “learn to communicate effectively”, “avoid puke” – yes, yes, yes and oh my yes. They were giving out Avinash’s latest book – but I had a strong feeling that a good proportion of the room already owned a copy, I know I did. The entire conference was dancing on the edge of preaching to the choir. “Test, and be prepared to change”. Yes. “It’s not about the tools”. Yes. Web analytics is here and will thrust business into the new world! Hooah! The Spartans of 300 we were. United in thought, united in action.

Sure. There were tidbits of new material. A method of analysis perhaps we didn’t think about. And, of course, there was the contradicting ideas where some say that web analytics is hard and those that felt that web analytics is easy (I’ll weigh in on this debate on day). Though, mostly everyone agreed that web analytics could be way more complicated – just ask the predictive analytics panel.

But overall, unfortunately, for the first two days, eMetrics felt… dare I say, monotonous?

For the first two days, as interested as I was, there was a lack of fire. It felt like each presentation was a slight flavor of the last. Missing was any serious opinion about social media as a marketing channel (other than a way to hold a contest or campaign). Missing was how to tangibly measure engagement. Missing was any talk about pushing the envelope into offline media’s turf. I started to wonder what happened to the visionary spirit of web analytics? Did Peter Pan grow up when I wasn’t looking? When did we stop dreaming and putting out wild ideas out there for people to shoot down? Where was the passion? Perhaps my expectations were set too high. Who knows… at least the beer was cold, I guess.

Thank God it’s Friday

But just as the conference was beginning to feel old shoe, comfortable but ordinary, Friday came to save the day. While the attribution panel was, by far, the most exciting session of the conference for me, it was the final panel that day that made the conference worth coming to. The overarching question posed to the panel of web analytics’ answer to rock-stars: “What is the future of web analytics?”. No one knew. They had ideas, but no one was sure. There was no across the board agreement. There wasn’t a, “I have nothing to add” to be heard. The time flew by and you wanted to hear more. There was a sense that the field come to a fork in the road, and no one had a map.

Would web analytics be drafted into service by business analytics? Would our opinions ever hold weight with the c-suite? Would we live in the basement with the statisticians or in the limelight in front of CEOs? So many questions, so few answers. Yet, being lost never felt so good.

I’ve always felt uneasiness about the future was the cornerstone of any serious profession and that all good ideas are born from a million bad ones. Proper professions always look for ways to be pertinent and meaningful, not only for today but for our children’s tomorrows. Web analytics is just too young to know all the answers already and much too young to know what it should be when it grows up or what not to do.

I left the session pensive with my head full of questions. It couldn’t have ended better.

My take

While, I too cannot predict the future, after some thought, I come up with this: We can do better. We can do more. We must start now.

As web analysts, we have a great opportunity ahead of us. The tortured history of BI does not have to be that of web analytics. We can learn from BI’s mis-steps. We’re the pioneers in a new medium and we’re much more nimble than our predecessors. The old marketing channels are stumbling and are lusting over the potential of the web – we must leverage this. Instead of us aiming to join their aging clubs, we should be paving the way towards a new league of measured media. We should further blur the boundaries of web, print, radio and TV and look beyond the click towards offline. While perhaps web analytics will be part of BI one day, it doesn’t mean it should be a minor player. Web analytics practitioners must not become the modern day number crunchers or Javascript grunts – we’re better than that.

We will make mistakes and that’s OK. Our mistakes will make us better. We will overcome challenges. This will give us confidence. We will make a difference. And this will give us purpose.

Who’s with me? Hooah.

Now, keep in mind, none of this information is necessarily new or game-changing. In fact, much of it I’ve heard and read before from blogs and forums. But to hear it live, alongside peers, definitely gives it a new twist. And having the opportunity to hear stories about implementations gone bad and victories long forgotten, definitely adds that needed colour that the worlds of analysts often lack – helping us feel like we’re not alone in our data-crunching fortresses of solitude. Here are my top three take-ways thus far:

Don’t measure what you are not prepared to change – paraphrased from John Blackmore, IBM

I think this one will be a tough one to follow, but in a way I think there is some weight to this suggestion. I say this is tough because it is so “easy” to capture so much data about web interactions. Why not capture this event or that event? Why not capture this action or that action? Why not capture all of it… just in case? Half of me is this just in case kind of guy.I’ll be the first guy to admit that I’ve definitely forgotten a thing or two during my career, so having captured some data “just in case’ is like having a safety net. So I completely understand that view. However, how I understand this advice is that we shouldn’t “be lazy”.

Here me out. As consultants or analysts, if we simply capture data, just in case, we stop being part of the solution and revert to simply being data-collectors and *shiver* report generators. As professionals looking to find root problems we need to always be applying the scientific method, testing hypotheses, creating issue trees and generating insight. Our data collection, ideally, should be purposeful and targeted. Furthermore, there is another wonderful nuance to this bit of advice, being “prepared to change”. This implies there is buy-in to act on insight. This implies that there is an impetus for change. This means that what is measured matters in some tangible way to something that matters.  Brilliant.

The web is just another marketing medium – again, paraphrased from John Blackmore, IBM

This is something that I preach all the time to whomever will listen. As much as I’d love to think that the web channel is a different animal, in the end, it’s just another marketing medium with its own intricacies. It won’t be long until web analytics reports will be lumped in with other campaign results. Also, speaking towards that last point, as Andrea Bertone from 360Training.com alluded to, to make analytics reports better accepted by upper management, take a cue from the reports from the other lines of business and/or channels. The web is a viable communications and revenue channel and, as such, it should stand shoulder to shoulder with the rest of the business channels. Having analytics reports in line with those of other business channels in terms of look, language and types of insights, will go a long way towards gaining web analytics the respect it deserves. This reporting discipline is something I strongly suggest that all web analytics professionals take because it’s only a matter of time before it’s forced upon us anyways.

People’s behaviours are similar regardless of the channel – paraphrased from  Paul Tyndall, RBC

There is this notion that people behave differently depending on the touch point or channel a user is interacting with. While it’s true that the individual actions may differ – the behaviour doesn’t necessarily change. For instance, if I was an overly aggressive person, I would slam doors and yell on the phone. Different actions, yet the same behaviour: aggressiveness. While I appreciate it will take some time to categorize web behaviours according to matching offline behaviours, it definitely is something to work towards and it is something worth investigating and leveraging.

Despite my having page after page of illegible scribbles for what I like to call notes, I think these three messages stuck with me most so far and I’m hoping to learn more tomorrow. Until then… hmmm, I forgot what I forgot what I was saying – I guess it’s time for ….zzzzzzzzz

A snippet from my life

One day, as I was sitting at a company meeting, they handed us these little remote controls with buttons labeled one through 9 (it could of been A through I, but that’s besides the point). After the speaker finished his presentation, he quizzed the audience to see if the main points of his presentation were absorbed. “Neat”, I thought. That was pretty much it for my thinking that day, honestly. But the story gets better, swear.

Wouldn't it be nice if you could skip pre-rolls?

A few minutes later, after I got my Habs fill, I left the TSN site. At that moment,  a friend walked by. He, too, was a Habs fan and I wanted to show him the video I just viewed. I called him over and I returned to TSN.ca and was AGAIN presented with the pre-roll. I was instantly annoyed. I thought to myself, “Hello? I just saw this – why are you showing this to me again. Yes, I KNOW. Samsung. I got it. Thank you…. 10 seconds to go? *insert swear words here*”.

Enter: lightbulb, stage right.

Why pre-roll?

I thought (I do a lot of thinking as you can probably tell), what are the points of these pre-roll ads? Obviously:

• To expose me to a brand…
• To call me to action…
• To place a product at the top of my mind…
• Ensure I got the message, etc.

All marketers want to do is have users recognize the brand they are pushing and hopefully a message as well – but let’s not get carried away here. Wouldn’t it be great if marketers could actually tell that the user either recognized the brand  advertised or remembered the commercial? Wouldn’t it be great for users if they could skip the pre-roll?

A possible solution: The quiz-skip (I’m working on a better name…)

Here’s my (half-baked) idea. While the pre-roll is playing, have an overlay appear asking the user a simple question, “What brand is being advertised in this video?” (or something along those lines). If the user remembers and types the correct answer in, he benefits from skipping the video and the advertiser benefits from having a meaningful metric that a user acknowledged their brand. If the wrong answer is entered, a hint could be displayed, or different video, etc. Apart from getting information about brand recognition, other meaningful metrics could be collected as well:

• Misspellings – this would tell the marketer that perhaps work must be done in emphasizing a product name.
• Incorrect answers – this could indicate to the marketer that the ad is too similar to a competitor’s

The possibilities, while not endless, are plentiful.

What are your thoughts on pre-rolls? Is there any benefit to the quiz-skip?

I can’t help but think the days of the web analytics professional are numbered.

Once upon a time, TV and radio were new. Direct mail was once new. And even print was novel. It’s not far-fetched to think that a day will come when websites are no longer “new media” – if that day hasn’t come to pass already that is. The web will just be another channel and the methods to measure its effectiveness will be old hat. Because really, deep down, how different is web analytics as compared to other metrics analysis? Once you have the handful of principles, it really isn’t that hard. At least not in my opinion. Quite frankly, marketing students today are being trained in web analytics in many schools as we speak – and with their marketing backgrounds are in better positions to act as compared to today’s web analysts. Soon web analytics will just be part of being a business analyst – and web analytics professionals will now be competing with the likes of McKinsey and Bain.

Some will say, but Rommil, there is more to analytics than analysis, there’s the implementation knowledge and deep understanding of how websites work. I won’t disagree with this argument, but I will say that this situation is transient. Implementation can be outsourced to IT workers, domestic and overseas. Best practices will emerge and solutions will converge eventually – probably within the next 5 years or so. So while some implementations can be very challenging, I would say they would be short work for a Computer Science graduate. And who knows? Now that Adobe owns Omniture, SiteCatalyst could be incorporated into Dreamweaver or Flash as an option. Click, click , click, OK. Publish. Done.

So what is a Web Analytics Association Certification worth?

In the not-to-distant future? Not a whole lot in my opinion. I think it will be just as valuable as a web designer certification from 1999 –  kinda important in the now, but not super important ten years from now. The standards will change very quickly over the next few years as techniques advance, forcing regular certifications renewals, and millions of books to be written and re-written. But can the WAA keep up with the ever changing world of web analytics? That’s a pretty tall order for a volunteer group – no matter how dedicated they are.

So what should we do?

I am in no way saying to switch out of analytics. Analytics is hot right now – at least in general. Measured media is the new black – as it should be. The days of “spend it and they will come” marketing strategies will be behind us very shortly. More than ever, web analysts have to expand their horizons. Marketers and business-types with spanking new MBAs are entering the market with strong and proven analytics skills. Not only will they be able to trend (sorry, perform ocular regression), they can perform linear regressions, apply strategic frameworks and perform statistical process control using six-sigma. They won’t have the technical knowledge of how to define s_accounts or fire _trackPageviews – nor will they care to. Simply because they’ll just hire a computer science intern to do that for them.

Know this: an analyst’s true worth is their business sense. Period. The rest can be duplicated. Easily. So unless a web analytics certification has a business component, while it doesn’t hurt to get, in the long run, it is a temporary measure. Remember, just around the corner – younger, brighter and more agile professionals are coming… and they’re hungry to impress.

Web analysts, prepare yourselves. Commit to expanding your minds and your boundaries. And please, think business – not click-through.

Learn to play your instrument before getting on stage

1. You do a Google search, and see an AdWord campaign (those ads at the top and the right sidebar).
2. You click on one of them.
3. You arrive at a site.
4. You leave the site.
5. For the next 6 months, the original click you made on the that AdWord campaign will receive “credit” for your consequent visits regardless if you visit the site directly or via an AdWord campaign.

I watched as consipiracy theorists Tweeted ad nausium about Google hiding this little bit of information, and trying to manipulate the system in their favor. All I can say is, “Oh please”.

Firstly, the 6-month window is a default. I’m pretty sure people would complain no matter what length they chose. (One could also say that XiTi’s window is too short as well) Secondly, one should always familiarize themselves with the intricacies of an analytics tool before relying on its stats. Otherwise, it’s the same as trying to measure something with a ruler without the units marked on it.

Just because Google Analytics is free, doesn’t mean one shouldn’t take the time to learn about the system and perhaps get some training in it. Similarly, just because some analytics solutions are expensive, doesn’t mean it will do everything out of the box.

We’re living in a time of fiscal responsibility (though I do find the phrase “tough economic times” overused). Meaning, that ad spend should be monitored, and justified. So if everyone else is being responsible, why are people dropping the ball when it comes to their web analysis?

As the saying goes, “It’s a poor musician who blames his instrument”. So let’s stop pointing fingers, and learn to play properly and better.

By the way, Google offers free online tutorials on Google analytics here. Knowing’s half the battle, eh?