Detailing Details: Ensuring Quality with Big Data

It has been said that “the devil is in the details,” but I’d like to say also that the saints can be found there as well. And return path TV information is all about the details. So in this blog, I’m going to delve into some operational processes that are unique about Rentrak, particularly our Set-top Box Lab.

Rentrak has developed and productized best-in-class tools and processes for handling return path TVs. Rentrak uses a combination of automated processes and Rentrak-designed tools to enable our team to manage a massive amount of inputs. The “virtuous circle” is shown below.

STB Lab

Let’s highlight this process moving counter clockwise. There is a team of people and automated processes to align tunes to schedules and to maintain accuracy of schedule content. This includes over 100,000 reported “streams” (channel locations) from our operator partners. This involves tracking items such as:

  • Networks/Stations
  • Markets
  • Content/Affiliation Changes
  • Call Sign Changes
  • Breaking News
  • Sports Overruns
  • Last minute schedule changes

In regard to data integrity and data recovery (which are mirror images of each other), another team works seven days a week to ensure that data is processing as expected. Through automation and marketplace knowledge, the team asks the following:

  • Does the data make sense?
  • Are data trends following reasonable patterns?
  • Are thresholds being exceeded?
  • Can anomalies be explained?
  • Is behavior consistent across MVPDs, markets, networks, telecasts, and time periods?
  • If there is a issue, what needs to be fixed?

To do this, the following is examined:

  • 700+ networks analyzed daily
  • 3,500+ stations analyzed daily
  • 150,000+ telecasts on any given day
  • 6 MVPD data feeds monitored
  • 1 primary 3rd party schedule data feed, supplementary feeds from other sources
  • 600+ clients served

Turning to STB Lab testing—this adds an important control over both internal and external changes. Testing helps detect impact of new features and functionality with STBs. Testing detects issues with data changes. Testing detects issues with code changes and internal business rules.

Rentrak has a number of full-time Operations QC personnel. The group uses automated testing software, replacing the most time consuming and error prone parts of the testing process with accurate and scalable automation. This has enabled a massive ramp up in the expansion of our lab’s testing throughput, as shown in the chart below.

Tests vs. STBs

Rentrak’s Lab tests 80 percent of STB models deployed “in the field” by our operator partners which comprises 70+ individual STBs. The STBs are physically located across the U.S.; Those outside of Portland are “slingboxed” into for testing. We maintain a STB “library” of models, features, functions, etc. This library is evaluated multiple times per year to ensure it is current.

The lab runs a battery of tuning events for each STB model on a weekly basis and reconciles the results to verify that the data from the box is fully understood, imported correctly and processed as expected all the way through our system.

Bottom line, Rentrak maintains the largest known STB testing lab as part of our overall process to ensure the viewing data is of the highest quality possible. You should be proud of what we do for the industry—we are.

In case you don’t know, I am Bruce Goerlich, Chief Research Officer at Rentrak, the global standard in movie measurement and your TV Everywhere measurement and Research Company. I have been in the research end of the marketing business for more than 30 years primarily on the ad agency side, with my last stint prior to Rentrak in the role of President, Strategic Resources Zenith Optimedia North America. Somewhere along the way I morphed from young Turk to old fogey. Now that I have grey hair and am horizontally-challenged, I can speak with some authority on advertising and research issues – which I will do from time-to-time on this blog.