Sunday, February 24, 2008

Monster Guest Post: Michael Hinman



Ratings

Tuesday night, as the second episode of "Jericho" was airing on CBS, I decided to go to bed early. It's deadline day in my real job, and I was exhausted. So why not let the DVR do what it's designed to do, and allow me to watch one of my favorite shows when I'm more awake?

Convenience is something society has longed for, and DVRs (and its predecessor, the VCR -- you can find those in the Smithsonian) are the poster child of that kind of society. Nothing wrong with that, in my book, that is until I see the overnight ratings the next day and find that the "Jericho" audience is eroding.

I have to ask myself, it is my fault? Did I cost "Jericho" a big set of eyeballs that could've helped boost it in the Jewish and overweight male 18-49 demographic? Should I have called my mom in Pennsylvania just before the start of the show to make sure she was tuned in?

While it's important to make sure that people ARE watching the show, unless you're a Nielsen Family (and trust me, you'd know if you're a Nielsen Family), you could make sure everyone in your town was gathered in your living room to watch the show, and it wouldn't matter a single iota.

Believe it or not, Nielsen isn't tracking every single household, every single viewer. In fact, for many decades, Nielsen has depended primarily on statistical sampling to provide audience measurement. That means choosing households in different markets at random (believe it or not, families are NOT picked based on demographics ahead of time), and then tracked and categorized based on the demographics they represent. But what the heck does all that mean?

During each reporting period, Nielsen releases a pair of numbers they say represent the audience of a particular show. For example, a recent overnight rating gave "Jericho" a 3.9/8. By itself, those numbers are practically meaningless, but they do say a lot. Without having to look at anything else, you can tell that a little over 4 million households turned on "Jericho" the night before. That's because the 3.9 is a household rating. Each rating point is a percentage of the 111.4 million households in the United States that has a television, so that means each ratings point represents 1.1 million households. Remember, however, that this number is different from the total number of viewers, because depending on the show, a household could have anywhere between 1 and 100 people who could be watching a show at any particular time. More commonly, however, it's about 2 people in every household, and more if it appeals to family audiences.

It's easier to track the number of households watching a particular show than it is the number of viewers, although viewership numbers are becoming easier to determine thanks to advances in technology spearheaded by Nielsen. More and more Nielsen Families are using a device known as a People Meter, which looks like a small cable box and a remote control. The box records data on what the family is watching, and sends it each night to Nielsen Media Research's Oldsmar, Fla. facility (which just happens to be a dozen or so miles from where I live). Each family member has their own assigned number that they press on the remote control when they watch TV so Nielsen knows if mom and dad are watching one thing, and the kids are watching another. However, a majority of families still use paper diaries, meaning each person in the house documents what they were watching, how long they watched it for, and who all watched it with them.

It's a rather complex process, but is the lifeblood of helping to determine who is watching what.

The second number found in the 3.9/8 example is what is known as the "share." If you looked at "Jericho's" household numbers, you will quickly discover that only 3.9 percent of the total TV households in America were tuned in, which doesn't seem like a lot. But what if many of those families weren't home? What if they were like me and decided to go to bed early? Should that count against shows like "Jericho"?

That's where share comes in. While 3.9 represents the total percentage of households tuned in to a show, the share represents the total percentage of households that had their TV on that were tuned in. That means 8 percent of all television sets turned on were on CBS during that time period.

Do the math, and and you may panic that 92 percent of households that had their TV turned on weren't watching the show. But please, sit down and drink a glass of water. Even the best performing shows are lucky to get 20 percent of households with TVs turned on to be watching their shows. There are a lot of channels out there for people to watch, and getting a share of that audience is difficult at best for anyone. Not even "American Idol" can pull in better than 25 percent of those televisions.

Now that the numbers make more sense, how can we use ratings to determine the fate of our shows? Well, that's where it gets very complicated.

Nielsen has no interest in what shows survive and what shows die. Contrary to popular belief, networks don't ask for ratings in order to see what survives and what doesn't. They pay for these ratings for one primary goal: to know how much they can charge advertisers who buy commercials for their shows.

Obviously, a show that pulls in 7 million households will demand more money per 30 seconds of commercial airtime than a show that has 5 million households. But overall numbers isn't how everything works. There are so many substrata of numbers that networks look at, like demographics. It's not just about black, white, Hispanic, or Asian, or male or female, or even age ... it's a number of audience subsets created based on nothing more than their buying power. Adults 18 to 49 is one of the key demographics because they have disposable income. Although the gay and lesbian community reportedly only represents 10 percent of the total population, they also are a key demographic because without children to pay for, they have a bit more money to spend (and boy, do they like to shop!)

Seriously, though, a question I have received a lot over the last 10 years running SyFy Portal is why did Show X, with an average rating of 4.5 get cancelled while Show Y, with an average rating of 3.5 stay on the air? What's the magic number?

There is no overall magic number. Each show costs a different amount to produce. "Jericho," obviously, is more expensive to produce than "Boston Legal," but cheaper to produce than "Battlestar Galactica." Going into any project, a network determines how much money they need to make, to not only pay for the show, but also to make money as well (this is showBUSINESS after all). I don't know what the target numbers are for "Jericho," but for example's sake, CBS could sit down and say that "Jericho" needs a household rating of 4.5 for it to earn enough advertising dollars to pay for itself and for them to break even. However, stockholders who invest in CBS Corp. want to see big profit margins, so the network needs to hit a 5.5 in order to make the minimum profit margin demanded for the show, because it can charge higher advertising rates for a 5.5 than it can for a 4.5.

But then "Jericho" comes in with a 4.7 consistently. Does that mean "Jericho" will automatically be cancelled? Not exactly. There are a number of other factors involved. Does "Jericho" have a fanbase outside its live viewers who tune in right when the show airs? Is it making a lot of money with rebroadcasts in other markets, and overseas? Are other distribution nodes popular like downloads on CBS.com or iTunes? Is it projected to make a lot on DVD? Could CBS replace "Jericho" with something they think will better meet profit margins or ratings expectations?

It's never a black and white answer, and many times comes down to nothing more than a judgment call by a network exec. Over the past decade. SyFy Portal has had a tremendous track record in predicting shows that would stay or go, mostly because of our detailed understanding on the ratings that matter to networks, and because of long-term relationships we've maintained with sources who are part of the bean-counting pool. In fact, one of the rare occasions where we were actually wrong was with "Jericho." The numbers, while down from the first half of the season, were still pretty strong by network standards, and we were told by our sources at CBS that it looked almost certain the show was going to be greenlighted, something that has since been confirmed by network officials. But at the last second, CBS had a change of heart, and suddenly "Jericho" was no more.

While there is a tremendous amount of science backing decisions on whether shows live or die by the Nielsen sword, it still comes down to human judgment in the end. And you can't fault the network heads. They have all the variables you read about above and many, many more swirling around in their heads, and they have to make the best decision they can while trying to satisfy as many people as possible. They know that decisions to cancel shows caused them to be vilified, but we have to respect the fact that they know what they are doing, and if we disagree we, we do it the "Jericho" way -- we are polite, we speak in terms that networks understand (helped by having a detailed knowledge about how they made the decisions they do), and we send tons and tons of nuts just to make sure they don't forget about us.

"Jericho" fans proved that can work, and I still have a special place in my heart for all of the Rangers who helped bring my show back to the network.

Michael Hinman is the founder and site coordinator for SyFy Portal, a leading independent science-fiction and fantasy news Web site run by fans for fans. He resides in Tampa, Fla., and is author of the popular weekly column, SyFriday. He covered the "Jericho" campaign practically from the beginning, being the first Web site to interview the company handling distribution of nuts, and was one of the first to announce the show's return a month later.

17 comments:

terocious said...

Excellent article Michael. My thanks to you and others like you who have studied this and share your knowledge.

Rich said...

Good one Michael,

Except, um, we all know the life blood is tainted.

Consider Friday Night Lights, which is one show critics are saying has the best chance to keep going despite ratings around 6.2 million people, with only 2.2 million, ages 18-49, supposedly the key demographic. So while ratings certainly look good on paper as you present them, networks still back the programs they want, and then use any numbers they want to back up their decision.

So why the nod anyway? It takes less money to produce FNL than some of the other shows being axed. More on that when I have a chance to write it.

The other thing to consider is that about one percent of households, even less off sweeps, is actually counted. So the math doesn't really hold up. There isn't a pollster alive, other than Nielsen and the people who track it, who would consider that credible. And in all honesty, the networks aren't really looking at Nielsen as much. The reason: advertisers aren't looking at it, which is why Ford can buy into a Knight Rider, sight unseen.

Best,
Rich

Jericho Returns said...

Thanks Tero and Rich.

Rich, that makes sense. Numbers can be made to support whatever one wishes.I could say 100% of Jericho's fans visit this blog and that could be true because I had a sample of one.
Maybe the advertisers can turn the tide on this foolishness.

Michael said...

But that's not accurate, Rich.

Not every show requires the same rating threshold to survive as others. Some shows may have a lower overall rating, but could have a demographic that has an interest from advertisers willing to pay more to reach ... say a program that attracts a lot of 60+ adults, and advertisers looking to reach them more directly.

It's not a black and white science, and it isn't about keeping the shows that network people like. They see this all as a business, as a way to get the best return on their investment.

Sometimes they will hold on to a poorly performing show if they see a good future for it in alternative media, or they don't have anything to put there they feel will do better in the timeslot, or a variety of other reasons.

Networks don't NEED to use numbers to back up their decisions. They have the ability to choose what stays and go for whatever reasons they want. There is no rule or law that says they have to provide a ratings justification for any decision they make. They can pull the plug on any single program they want at any single time for no other reason than the fact that some big wig somewhere doesn't like it.

And I am starting to grow weary about people who talk to me about statistics and keep talking about 1 percent of the audience being counted. It's not like they are just taking 1 percent, and going by what they say. That 1 percent is broken down into hundreds of different categories and viewing habits, and that is applied all over its numbers, to bring a representative calculation.

This is exactly how presidential polling works. The thing that we're seeing right now, however, is that the primaries are unpredictable not because the science is bad -- where they actually survey less than 2 percent of the total expected voters -- but because the "undecided" are much higher, and have swung some of these primaries to a particular candidate, sometimes the one not leading in the polls. That messes up numbers because you don't go to the polls and select "undecided." By the time you get there, you will have decided, or at least by the time you pull the crank lever (yes, I know, those don't exist anymore).

And Rich, please show me where networks have stated they are no longer putting a lot of weight into Nielsen. Please show me where advertisers are saying this. Because it seems like they are investing in Nielsen far more than ever, and if they weren't considering it a primary way to measure audience, why would they spend so much money on it?

As far as "Knight Rider," many of these programs are sold to advertisers "sight unseen." What happens is that the networks GUARANTEE a specific rating. Say, NBC said "Knight Rider" would earn a 7.0 HH overall, and that is what they base the advertising spot costs on. Let's say "Knight Rider" instead earned a 6.5 HH overall ... the networks then provide what is called a "make-good," which is basically free or reduced additional advertising to help them reach the guaranteed audience whether it be with the same program, or somewhere else in the time slot.

The same is true with advertising campaigns on Web sites. If my advertising company sells a traffic count that is too high for what we normally pull in, and we don't achieve those numbers during the campaign, we will run their ads a bit a longer, or offer an additional campaign that will run longer, until the desired audience and more is reached.

Please understand that ratings are far more complicated than just "here is the number, now decide whether to cancel or renew."

Rich said...

Michael,

I've written pretty extensively about how networks are looking at alternative measures, and addressed some your comment on my own blog. Also, just today, The New York Times ran a piece on how Nielsen is trying to change to keep up. Consider...

"Nielsen faces competitors in every medium. In television, T.N.S. is offering new types of ratings information, like viewing by some of DirecTV’s satellite subscribers. On the Internet, comScore fiercely competes with Nielsen to provide counts of page views, video streams and monthly visitors to Web sites. And on cellphones, companies like M:Metrics track activity." — NYT

You are right that one percent is broken down into hundreds of different categories and viewing habits, which makes the validity even less. It's also skewed urban and toward minorities by Nielsen's own definition.

Likewise, I'm glad you mentioned political polling. Political candidates don't really believe polls Michael. I've worked on three congressional campaigns, two gubernatorial campaigns, and several state legislative campaigns.

Sure, the jump on them when they are favorable and dump them when they are not, but the bottom line is political polls are about as reliable as Nielsen. That's makes these polls more about public relations as opposed to factual analysis.

The same goes with networks. Sure, they can say anything they want, but the mood of the consumer is different. Consumers want justification. So if a network says ... we had to cancel it because of low ratings, then it eases the blow in most cases because consumers didn't know any better. Now, they are starting to know.

One of my favorite recent examples is Journeyman. Everybody else said it was about ratings, but the truth was NBC cancelled the show for financial reasons. Why why did everyone, including yourself report is was ratings? It seems to be reality even though it really wasn't about that. Nielsen makes it easy for critics guess at network decisions when I know for a fact that not all network decisions are based on ratings. They only say that because it's easy.

Michael, again, you're a good guy and I have nothing but respect for you. But the simple truth is that networks would not be making moves toward new media if ratings were the only game it town.

Best,
Rich

Michael said...

Hiya, Rich!

"Nielsen faces competitors in every medium. In television, T.N.S. is offering new types of ratings information, like viewing by some of DirecTV’s satellite subscribers.

The only problem with that is DirecTV represents only a fraction of the audience, just as TiVO represents an even smaller fraction of the audience. Plus, it wouldn't be true sampling, where Nielsen will measure you no matter what cable service (or lack of cable service) you have.

On the Internet, comScore fiercely competes with Nielsen to provide counts of page views, video streams and monthly visitors to Web sites. And on cellphones, companies like M:Metrics track activity." — NYT

And that actually applies more to NetRatings, a subsidiary of Nielsen, rather than Nielsen itself.

I think competition is good!

You are right that one percent is broken down into hundreds of different categories and viewing habits, which makes the validity even less. It's also skewed urban and toward minorities by Nielsen's own definition.

Only because it has to be to balance the sample. It's not skewed so much as to taint it, however, although sampling is not a perfect science.

Likewise, I'm glad you mentioned political polling. Political candidates don't really believe polls Michael. I've worked on three congressional campaigns, two gubernatorial campaigns, and several state legislative campaigns.

No, they only believe in them when they say what they want. Political polls are actually studied quite extensively, and it's not always who people are going to choose over who ... many times, they utilize polling on issues that are affecting potential voters, or how they might respond to particular messages. So to say they don't believe in them broadstroke ...

Sure, the jump on them when they are favorable and dump them when they are not, but the bottom line is political polls are about as reliable as Nielsen.

Well, I would think so, considering they use the same form of statistical sampling.

That's makes these polls more about public relations as opposed to factual analysis.

It is never meant to be an exact science, but instead to identify trends. In every respect.

The same goes with networks. Sure, they can say anything they want, but the mood of the consumer is different. Consumers want justification. So if a network says ... we had to cancel it because of low ratings, then it eases the blow in most cases because consumers didn't know any better. Now, they are starting to know.

What are they starting to know? The primary reason a show gets cancelled is because it is not generating the return on investment it is expected to, and that said network feels something different in that timeslot would generate better returns.

Those returns are based on advertising revenue, and advertising revenue rates are based on Nielsen ratings. What am I missing here?

One of my favorite recent examples is Journeyman. Everybody else said it was about ratings, but the truth was NBC cancelled the show for financial reasons.

Ummm .. how are ratings and financial mutually exclusive?

The ratings for "Journeyman" were terrible. I mean, terrible. "Bionic Woman" was cancelled despite having far better ratings.

I am not sure how you can separate the financials from the Nielsens when it's Nielsens that determines ad revenue rates, thus producing the financial return networks are looking for.

Why why did everyone, including yourself report is was ratings?

Because it was.

It seems to be reality even though it really wasn't about that.

In what way? Are you telling me that "Journeyman" had great ratings? Or do you have information that proves the network went opposite of a ratings determination?

Nielsen makes it easy for critics guess at network decisions when I know for a fact that not all network decisions are based on ratings. They only say that because it's easy.

They're not just based on Nielsen ratings. As you read in this very piece, there are a number of variables that are considered when deciding the fate of a show, and ratings are just one of them. There is no dispute there.

Michael, again, you're a good guy and I have nothing but respect for you.

Same here, Rich! :) You know I'm a fan of your blog!

But the simple truth is that networks would not be making moves toward new media if ratings were the only game it town.

New media has nothing to do with ratings. Networks are making the move to new media because the consumer base is asking for it. Just like how networks started to do heavy product placement because consumer bases were enjoying time-shifting devices such as VCRs to push through traditional commercials that used to be watched live.

I'm just not following your logic. So you're saying that networks putting shows on DVD also is a way to tell Nielsen to shove it?

Networks are simply trying to diversify revenue streams, like ALL good businesses do. Just because they are diversifying revenue streams doesn't mean they are giving the boot to tools in their toolbox, like Nielsen.

Yes, they are lessening their dependence on advertising revenue, and giving viewers more of a direct voice which can be exercised by paying $1.99 to download an episode, or to sit and watch a streaming version live with a commercial or two, or to shell out $80 for a season of the show.

While such diversification is good news for consumers with disposable income, it doesn't mean that Nielsen itself is becoming obsolete, because advertising-supported programming is going to be around for a long time to come.

Rich said...

Michael,

I cannot respond in depth at the moment, so let me share what everyone has been emailing me all day while I've been prepping for back-to-backs ...

"Quincy Smith, who runs CBS' new media division, says the size of the audiences on Veoh are getting close to what some shows are receiving on broadcast television. While he declined to cite specifics, he said the post-apocalyptic drama Jericho has a bigger online audience than it does on traditional TV." – USA Today

Um, that would be double the audience or more.

Told you so. (Meant in the nicest way of course).

Best,
Rich

Michael said...

One problem with Quincy Smith's comments:

He provides no numbers to back it up. And he doesn't identify WHICH network shows he's comparing it to. If he's comparing it to, say, just about anything on The CW, that's not saying very much.

Michael said...

Remember:

Before you start taking network people or anyone at their word, you need to ask for substantiation. You need to ask what they are building numbers on.

I could say that a small Web site of mine tripled its audience! That would be enough to get people excited and talking ... "Michael Hinman tripled his audience in a single day! Wow! What a genius!"

The problem is, the number of visitors in the past was 6 people, and I tripled it to 18.

Makes a big difference on how substantiation works.

Not saying that Quincy is wrong ... I would like to see what he is comparing it to, and I would like to see the numbers from Veoh that he is alluding to.

Jericho Returns said...

"Jericho has a bigger online audience than it does on traditional TV."

Interesting stuff gentlemen.

Since we know what Jericho's traditional numbers on TV are then it follows that online audience is bigger than that which, to me, says Nielsen is ignoring the majority of those.

Balceroregontr said...
This comment has been removed by the author.
Balceroregontr said...

check out this article for facts about the new media needing to be included in the ratings game.
http://www.usatoday.com/tech/products/2008-02-26-veoh_N.htm
Debby

Michael said...

There's a problem about including new media:

Unless there's an advertising component involved, it makes no sense.

Remember: audience measurement is done for ADVERTISERS, and for no other reason.

Period.

Jericho Returns said...

That's why it was so nice last week to hear that the advertisers are looking to place ads online since TV numbers are down so low. They finally realize not everybody watches "live."

AdAge

Balceroregontr said...

Actually Michael the time they are a changing to quote song lyrics. I hope that one thing that changes is how television programing is decided. I buy a book because it is good not because someone places an advertisement in it. The same for movies and dvds. Television can find a way to be profitable and have quality programming if the networks don't change they will eventually find themselves out of the equation. Smaller niche cable works are gaining viewers. I know that we watched more cable during the strike. In the early days of television programing was put on because it was good not because it sold ads. As viewers move to new media adveertisers will have to follow.
Debby

Rich said...

Michael,

Thank you. I understand your perspective perfectly.

Suffice to say that we will agree to disagree. Much like many folks told me last year that old television shows would never be resurrected online and broadcast would never play to the space. Ho hum.

Wish I could write more, but it's been a long day and I'm off to check out an episode of the original Star Trek. Oh, that would be online, though I suppose it makes no sense ... the embedded ads must be a public service on their part. ;)

Best,
Rich

dona said...

Journeyman is excellent tv show. i have some collection of some sessions of this show. Now I am searching more TV Downloads of this show.