How relevant are Reddit's ads?

I recently had a discussion with a friend regarding the relatively high relevance of Instagram Sponsored posts versus what seems like a lack of relevance in Reddits’ ads. Reddit allows advertisers to pay for promoted content to appear as interstitial posts within the main Reddit post feed (note that this is separate from Google and Amazon display ads which appear on the side rails of Reddit’s webpage). We had noticed a few weeks ago while browsing (in incognito mode on a new VPN, with cookies and tracking scripts disabled) that it seemed that the online medicine delivery company LemonAid was paying for Erectile Dysfunction treatment ads to appear on the subreddit r/WallStreetBets (note: it seems that in the intervening weeks, Reddit completely removed ads from this subreddit).

While we found it amusing that an advertiser might find a rich target audience for male performance enhancement services within the confines of this flippant investment focused forum, we were also curious as to whether Reddit has a sufficient diversity of advertisers on its platform to deliver a relevant ad experience to its patrons. Additionally, we were curious if we could assess how good Reddit’s ad platform is at showing contextually-relevant ads when a user is browsing a specific subreddit.

To this end, I built a custom chrome extension that records Reddit ads when we browse the website on our home and work desktops. A group of friends and I logged several thousand ads over the course of a few weeks. To try to enforce some analytical consistency, we only used the extension when browsing in incognito mode on a VPN (to avoid biasing our analysis with our behavioral or demographic data), and only recorded ads on the top 125 safe for work (SFW) subreddits (by subscriber count)  listed on the Reddit List.

During this time, we saw a total of 7,877 total ads, of which there were 209 unique ads. The occurrence of these ads had a long-tail distribution - heavily skewed by a small number of advertisements which appeared numerous times.

7,877 promoted Reddit ads seen across 125 subreddits

Most frequently seen Reddit ads across our sample of 7,877 ads

Why are there no Reddit ads posted on r/BlackPeopleTwitter ?

For the top 125 subreddits we checked, the number of promoted reddit ads was relatively consistent, within one order of magnitude. However, we noticed that there were four subreddits for which we saw no ads - r/BlackPeopleTwitter, r/dankmemes, r/instant_regret, and r/PublicFreakout. Of the 4 subreddits with no ads, r/BlackPeopleTwitter had the highest number of subscribers, and it was the only subreddit with over 5 million subscribers that had no ads on it.

There are three possible explanations for this. First, Reddit controls which subreddits ads can appear on, so it’s possible that r/BlackPeople has ads disabled on it intentionally or at the request of the subreddit’s moderators.

Secondly, there have been documented cases of advertisers ‘blacklisting’ certain keywords or digital contexts from being allowed to host their ad content. For example, advertisers using Google Display ads can specify that they do not want their ads to appear on websites which contain keywords about Covid19 or violence. In June, Vice Media documented that the use of third-party brand safety vendors to avoid specific keywords in the content environment in which digital ads appear led to news articles about Black Lives Matter generating lower amounts of ad revenue for publishers. It is possible that advertisers (or their media agencies) are directly or indirectly preventing their ads from appearing on subreddits like r/BlackPeopleTwitter. I checked a number of subreddits to see if I could see a similar lack of ads. r/BlackLivesMatter has no ads on it as well, but r/blackladies and r/blackpeoplegifs both have ads on them. It was also interesting to note that r/creepy, r/politics, r/Tinder, and r/AmItheAsshole (all in the top 125) did have ads appearing on them.

Thirdly, we could simply have not been in the target audience for ads on this subreddit (perhaps due to our IP addresses), an observational bias introduced in this experiment.

Which subreddits are showing ads for Erectile Dysfunction Drugs?

While r/WallStreetBets no longer seems to be showing any ads and we did not see any additional ads for LemonAid’s erectile dysfunction treatments, we did see other ED advertisements circulating on a number of subreddits. ForHims, GetMeGiddy, and Vault Health are running ad campaigns for ED treatments. We observed GetMeGiddy’s ads 95 times across 23 subreddits, ForHims’ ads appeared 11 times across 6 subreddits, and Vault Health’s 6 times on 2 subreddits. We were somewhat surprised by the context in which these ads appeared. For example, we saw the ED ads appear a lot more frequently on r/TwoXChromosomes (a subreddit “intended for women’s perspectives”) than a subreddit dedicated to dating - r/Tinder.

The top 10 subreddits with most ED drug reddit ad occurrences in our sample

Based on this analysis, we could not infer any context specific targeting of ED ads. This could be because the sponsors of these ads decided not to target their ads based on subreddit content (i.e. male health or romance related), or because Reddit’s algorithms are not classifying the respective subreddits the way we expected.

We did however observe a temporal affinity in the ED ad targeting - namely it appeared that the ads appeared more frequently at night time for viewers in their respective time zones.

On how many different subreddits does a given ad appear?

In an effort to generalize past the ED treatment analysis, we looked at the other unique ads we saw on Reddit and tried to calculate a ‘ad enrichment score’ - a measure of how often a given ad occurs on a small number of subreddits. A low enrichment score indicates that an ad is equally likely to appear on any subreddit, without much contextual targeting. A high enrichment score suggests the ad sponsor is intentionally targeting a small number of subreddits with their ads, presumably because they think that they will find their target high-intent audience on those subreddits.

For each of the 209 unique ads we saw, I divided the number of times we saw the ad by the number of distinct subreddits it appeared on. I then ranked each ad by its enrichment score. A handful of advertisers did a great job of targeting their ads to specific subreddits. For example, u/StadiaOfficial posted ads for Google’s cloud gaming service Stadia only on three highly relevant subreddits - r/Games, r/pcmasterrace, and r/gaming. Interview KickStart, an interview prep service for software engineers, only placed ads on r/programming.

Reddit ads which only appear on a very limited, targeted group of subreddits. Enrichment score indicates how 'concentrated' a given ad's appearances are on a few subreddits, suggesting intentional contextual targeting

On the other hand, a large number of advertisers were not very selective with their ad placements. For the ads which had at least 20 appearances, 75% of them had an enrichment score less than 2, meaning that if the ad was seen 22 times, it appeared on 11 different subreddits. In some cases, this could be intentional - BagelBites.com or Samsung have an equally likely to find a potential customer in virtually any subreddit.

In other cases though, the lack of contextual relevance was a bit more perplexing. For example, Atlassian ran an ad for Jira (a “Software development tool used by agile teams”), which we observed 67 times across 42 different subreddits, including r/Tinder, r/AmItheAsshole, r/NintendoSwitch, and r/Parenting. It is possible that Atlassian was intentionally running this ad campaign on a wide breadth of subreddits in order to gather some initial data before narrowing down their target customer segments.

Reddit ads which appear on a large number of different subreddits. Low enrichment score suggest that ads are not being intentionally targeted to a specific subset of subreddits.

Overall, it seemed as though most advertisers were not using some kind of refined contextual targeting, which could help explain our initial question about the seeming lack of relevance in reddit ads we had been thinking about. A 2016 study of Reddit’s ad system found that marketing tools did a poor job of suggesting which subreddits to target for a given ad, with many “unrelated or useless” recommendations.

Potential lack of diversity in reddit’s advertising sponsors

Other ad platforms such as instagram do a good job of attracting a wide diversity of businesses, both in terms of size and industry. This diversity of ad content allows their algorithms to fine tune what to show users in response to a given users’ interests. While our analysis is by no means scientifically rigorous and our sample set is relatively small, I decided to categorize the 209 unique ads our group saw via the Interactive Advertising Bureau’s Content Taxonomy to determine if the ads had a bias towards a limited number of industries.

Categorizing the 209 unique ads showed 26 different IAB categories represented in our sample. Of these, “Technology & Computing” ads were the most common, while “Video Gaming” were the second most common. The technology ads also have a disproportionate share of the 7,871 ads we saw.

IAB categories for 209 unique advertisers seen across the 125 top SFW subreddits (by subscriber count)

IAB categories for 7,871 total ads seen across the 125 top SFW subreddits (by subscriber count)

The over representation of technology advertisements, such as for Cloud-computing services, is not surprising. According to data from Comscore, 38% of Reddit users consider themselves technology lovers (more than on Facebook or Instagram). Similarly, the relative abundance of electronic game ads is consistent with a survey that found 47% of redditors have purchased consumer electronics within the last 6 months. Overall though, we can state that reddit does have a minimal threshold diversity in ad content, though we would need more data points as well as finer content category resolution to make a more rigorous comparison with Instagram.

Conclusion

In our data collection exercise, we focused on the top 125 subreddits and avoided using our personal profiles to avoid behavioral or demographic targeting. It is possible (and likely) that a sampling of more niche subreddits with lower numbers of subscribers would reveal a different composition of advertisers. While Reddit did have a decent diversity of advertisers on its platform, the distribution of ad impressions was heavily skewed by technology ads and certain ads that were shown many times over. It seems that Reddit does not currently participate in the Ad Choices program, so there is no way for users to indicate (at least on the Desktop interface) that they are not interested in seeing the same ad anymore. Separately from our observations in this experiment, we noticed that we often see the exact same ad dozens of times within a short time span, which suggests that even if a user is signed into their Reddit profile, Reddit’s ad platform does not seem to perform frequency capping in many cases. I could not say whether this was a technical limitation of the ad platform, an intentional strategy by the advertisers to test out a marketing strategy or to maximize the reach and number of impressions for their ads, or an unintended consequence of loosely configured ad campaign parameters.

Previous studies have shown that advertisers can struggle to achieve marketing objectives with Reddit Ads, and problems with the platform’s targeting tools were named as one the issues. In other cases, Reddit ads have been shown to produce positive business outcomes, but are extremely context-sensitive. An ex-AWS engineer self-published an e-book on AWS infrastructure and advertised it on a number of programming-related reddits. While ads placed on the topically focused r/AWS drove positive ROI, the more generic r/programming did not. This highlights the importance of fine tuning reddit ads to the demographic idiosyncrasies of each subreddit, which effectively function as unique communities.

It is in Reddit’s self-interest to improve contextual targeting (or teach its clients how to use it more readily). In the past, Reddit’s executives have expressed concern about revenue growth being not up to their own expectations. Irrelevant or out-of-context ads harm users by wasting their time and data, as well as the sponsors paying for the ads. Additionally, irrelevant ads have been shown to lower consumers’ opinion of a brand and lead to an increase in the adoption of ad blockers. Lastly, giving users the ability to opt-out of certain ads and understand why they are being shown an ad may improve the platform’s performance.

Take away points

1. The Reddit ad experience could be improved for both users and advertisers by improved contextual targeting, frequency capping, and Reddit’s adoption of Ad Choices to give users more transparency and control over which ads they see

2. Technology related ads are over represented in the top subreddits

3. The majority of Reddit’s most active advertisers are not being very specific with their contextual targeting, at least for anonymous users of the platform