Scraping Through Wrappers: Getintent Researches Header Bidding Solutions

Scraping Through Wrappers: Getintent Researches Header Bidding Solutions

Photo credit: ntr23 via / CC BY-NC-SA

The year of 2016 was marked by the rise of header bidding over the digital landscape. Media have been brimming with headlines dubbing the technology a driveway to the promised land for publishers. The latter one by one have been reporting maximized yields from adopting header bidding.

Curious about what figures back up such claims, Getintent, a New York based ad tech company developing programmatic solutions, embarked on its own research.


“We wrote a PhantomJS scraper that is able to identify header bidding technology used on the website and see what demand sources the page addresses when loading,” said Vladimir Klimontovich, CTO at Getintent.

“For the sake of research, we scraped through Alexa top 1,000 websites to analyze how many publishers have shifted to header bidding and what solutions — specifically, wrappers and adapters — they are implementing,” Klimontovich explained.

In a nutshell, omitting the details of how header bidding works, wrappers, or containers, help publishers manage multiple header bidding solutions. A wrapper gathers bids from demand sources and implants them directly into a publisher’s ad server to compete with the bids from direct sales partners. Inside the wrapper, there are adapters that translate bids into a common key value for the ad server.

As a result of the scraping process, header bidding solutions were spotted on 121 unique web pages out of Alexa top 1,000 websites, which makes 12%.


The research revealed that among these 121 publishers there are 164 cases of header bidding solution usage. The most commonly used header bidding solutions are those provided by Criteo (in 69 cases, which is approx. 42.1%) and Amazon (in 40 cases, which makes up 24.4%). In 28 cases (17.1%) publishers use the open-source and free Prebid.js container. Not without reason, some publishers take advantage of using several wrappers together.

According to Getintent, if the scraper observed page calls for adapters but was not able to identify the wrapper, it’s highly probable that the publisher is using some custom solution. Thus, in the rest 27 cases (16.5%) publishers fall back on self-made containers.

“Some publishers make their own header bidding containers,” commented Maxim Kraynov, Lead Software Engineer at Getintent responsible for the research. “Sometimes it’s a fully proprietary technology, sometimes it is a wrapper built upon some already existing container,” he specified.

“Interestingly, the recognized leaders in the header bidding domain, Amazon and Criteo, are not wrappers per se,” Kraynov added. “Our scraper managed to detect the Amazon technology as a container, but the company itself doesn’t label its header bidding product a wrapper, because it’s a cloud-based solution. Similarly, the Criteo technology is not a container as such. Nevertheless, for simplicity reasons we call such an approach header bidding.”


As part of the research Getintent was able to review what bidder adapters are implemented by the publishers for Prebid.js.

“It should be noted that, a cloud-based solution, Amazon has no visible adapters,” Maxim Kraynov highlighted. “Neither does Criteo, as in this case the publisher embeds the code directly onto the page, so there’s no need for an adapter.”

The scraper spotted 126 cases of adapter usage on the selected 121 websites, which verifies that some publishers leverage technologies of multiple vendors simultaneously.

As for the tech providers, it turned out that the AppNexus solution is most common (approx. 24%), followed with the Rubicon, Sonobi, and Sovrn adapters (approx. 10% each).


“The research shows that the header bidding market has become more diverse and competitive with vendors coming up with browser-based, server-side, and cloud products,” summed up the analysis Vladimir Klimontovich. “The Criteo and Amazon solutions (that aren’t actually wrappers) are currently most favored with publishers. Yet, we can’t say for sure which solutions will gain mass adoption,” he commented.

The Getintent analysis has also shown that despite the reportedly growing number of publishers adopting header bidding, most publishers (more than 80% of Alexa top 1,000 websites) are still to discover the game-changing technology.

“The scraper is not a faultless tool though, and the findings don’t claim a monopoly of absolute accuracy,” Klimontovich remarked. “Nevertheless, the Getintent research results are not a far cry from the data reported by similar studies.”