top of page
← All Case Studies

Connected TV Incrementality Test for a Global Talent Marketplace

Incrementality Testing • Path Analysis • Audience Analytics • Paid Media Audit

Online Talent Marketplace

Service: Paid Media Mix Audit, from $5,500

 

Key Results

$356K

MEDIA AUDITED ACROSS FIVE TEST CELLS

41%

OF CONVERSIONS FROM 8% OF SPEND

36%

OF SPEND ON PATHS WITH NO MEASURABLE LIFT

 

At a glance. A large online talent marketplace (anonymized) tested connected TV and an advertising network's video, display and native inventory to drive job postings. I was the measurement and analytics lead. I audited the campaigns, worked directly with the client's agency and the platform's team, and told them what to change.

 

The Challenge

On the platform's own reporting, the program looked like a success: more than 250,000 attributed actions on about $356,000 in media and 15.4 million impressions, at roughly $1.61 per action. My job was to answer a harder question. Would those households have posted a job anyway?

 

The Strategy

My audit looked at the program in three layers.

 

The first was a holdout test. Matched control groups received no ads. In the initial study, exposed households acted at about 1.9% and the control group at about 3.7%. In the reach-expansion cell, the figures were about 1.8% exposed and 2.3% control. The attributed actions were real, but the media was not what caused them.

 

The second was a path analysis across 28 combinations of ad formats. Display banners and full-episode video placements, alone or together, took about 8% of the spend and drove 41% of the conversions. Their cost per conversion ran from the teens to the mid-twenties, against an average of about $90. Standalone connected TV, run-of-network and the platform's own media placements took about 36% of the spend and showed negative incremental lift.

 

The third was an audience match. Exposed households were matched to third-party offline data and compared with the people who actually posted jobs. The media over-indexed on households under $30,000 in income (1.44x). Converters over-indexed above $250,000 (1.21x). Converters also clustered in large markets the ads were under-reaching: Miami indexed at 1.44 for converters and 0.91 for exposure, and the San Francisco Bay Area at 1.13 for converters and 0.69 for exposure.

 

The Results

I took these findings to the agency and the platform team with a clear direction. Move budget toward the display and full-episode video placements that showed real lift. Retarget toward the higher-income, large-market households who were actually converting. Judge the program on incrementality, not on the platform's reported cost per action.

 

What It Meant

Platform-reported CPA is the platform grading its own homework. Without a holdout, this program would have been scaled on a $1.61 number that a control group did not support. A low reported cost per action and real incremental value are different things, and only a test design shows you which one you have.

 

Not sure your media is driving results the dashboard takes credit for? Let's talk about a paid media mix audit.

 

Related services: Paid Media Mix Audit | Marketing Mix Modeling & Analytics | Marketing Audits

Related case studies: Independent validation of a health plan marketing mix model | How expert-led creative outperformed brand advertising

Featured In Leading Media & Journals

Quoted in national media, published in peer-reviewed journals, and the author of five business books.

Forbes logo, linking to the Forbes article quoting Dr. Linda Orr on solo marketers
Entrepreneur logo, linking to Dr. Linda Orr’s articles on Entrepreneur.com
Yahoo logo
The Daily Upside logo, linking to the Daily Upside article quoting Dr. Linda Orr on AI marketing and brand risk
business.com logo, linking to the business.com article featuring Dr. Linda Orr on offline marketing tactics
Glossy logo, linking to the Glossy Fashion Briefing article quoting Dr. Linda Orr
Journal of the Academy of Marketing Science logo, linking to Dr. Linda Orr’s article in the journal
Journal of Business Research logo, linking to Dr. Linda Orr’s article in the journal
Psychology & Marketing logo, linking to Dr. Linda Orr’s article in the journal
Journal of Business Ethics logo, linking to Dr. Linda Orr’s article in the journal
Marketing Management Journal logo, linking to Dr. Linda Orr’s article A Re-Inquiry of Hofstede’s Cultural Dimensions

What Clients Say

Most engagements are under NDA. Here is what clients say, plus our reviews on Google.

★★★★★

“Dr. Orr's exceptional leadership and advanced knowledge in sales and marketing were invaluable. She provided practical tools our team could use immediately to perform with confidence.”

B. MacDonald

B2B Sales Enablement · Sales Training

★★★★★

“Linda was a pleasure to work with and extremely thorough. She led a market research project for our 3D dental imaging launch, and the insights directly shaped our positioning and go-to-market strategy.”

Midmark Dental

Healthcare · Market Research · Launch Strategy

★★★★★

“Working with Linda was an absolute pleasure, and the results she delivered were beyond our expectations! ... She interviewed over 20 key players in the yachting industry, ranging from yacht charter company owners to marketing managers, extracting valuable insights that we wouldn't have been able to gather on our own... The final report she presented was top-notch. It was thorough, well-structured, and packed with actionable recommendations that demonstrated her deep understanding of the industry.”

Founder, Yacht Charter Marketing Agency

Market Research · Market Entry Strategy

SELECTED CLIENTS AND COMPANIES

Selected Orr Consulting clients: PepsiCo, Upwork, McKesson Foster and Thrive, Midmark, FedEx Custom Critical, Arm and Hammer, TheraBreath, Mavrik Dental Systems, medmix, Elos, GFOA, Indeed, Magenta Inc, PreVu Dental, SMRTFT, Sparks, The Critical Thinking Institute, Tahoe Modern, Mayberry Law, Ollie Pediatrics, Therapevo, Mind On Photography, Artemis, Disc It
Selected Orr Consulting clients: PepsiCo, Upwork, McKesson Foster and Thrive, Midmark, FedEx Custom Critical, Arm and Hammer, TheraBreath, Mavrik Dental Systems, medmix, Elos, GFOA, Indeed, Magenta Inc, PreVu Dental, SMRTFT, Sparks, The Critical Thinking Institute, Tahoe Modern, Mayberry Law, Ollie Pediatrics, Therapevo, Mind On Photography, Artemis, Disc It (continued)

Let's Discuss Your Next Growth Challenge

Whether you need fractional CMO leadership, an independent marketing assessment, or research to support a major business decision, let's discuss your objectives and determine the right approach.

bottom of page