We removed advertising cookies and here’s what happened

This is not another abstract post about what the ramifications of what a cookieless future might be; Sentry actually removed cookies from our website a few months ago. It has impacted us positively and negatively, in both expected and unexpected ways. I hope this can serve as a guide or inspire others who are considering making this change.

Why remove cookies?

The thought of going completely cookieless seems daunting and borderline ludicrous to many of the marketers that I talk to. If you were to remove all cookies today, as we did at Sentry in July 2023, it would radically change the functionality of your martech stack. This is covered thoroughly in articles like this, but in brief, things like attribution, remarketing, or Account Based Marketing (ABM) would be rendered much less effective and it would almost certainly lead to a material loss in revenue.

Not to mention If performance marketing is a key driver of revenue for your business, levers like targeting, bidding, reporting, and procurement become extremely challenging. However, with Google’s act of removing third-party cookies for some users in Q1 2024 and plan to completely remove 3P cookies from Chrome sometime in 2025 looming, you basically have three approaches you can take:

1. Ignore that this is happening and wait until you are forced to make changes around Q2/Q3 this year (not recommended).
2. Become an early adopter as ad platforms continue to optimize and introduce newer solutions for cookie removal like Enhanced Conversions, GA4, Conversions API, Conversion Lift, etc. so as to not cause any drastic shake-ups until the platforms are ready for the shift.
3. Completely remove cookies sooner rather than later, and start directing your time and energy towards future models that respect your users’ privacy.

We chose the third route despite Google sending us an entire deck to tell us why we shouldn’t. We took it a step further by removing all user tracking, period. Was it worth it? In this post, I’ll share some early results and all of the pitfalls and triumphs that we’ve faced ever since removal of every advertising cookie and user tracking in July 2023. I hope that this can give others a dos and don’ts list and best practices for the inevitable shift from cookies in 2024.

Why go cookieless & remove user tracking now when I can just wait?

The way I see it, there are a handful of reasons a marketer might decide to gut their website of cookies and traditional user tracking in general:

• Cookie consent banners, App Tracking Transparency (ATT), and consent management platforms (CMPs) for GDPR/CCPA compliance have already obfuscated tracking data enough to make you question if the data you have is reliable.
• Your audience is probably privacy-conscious. Aren’t we all learning to be as companies misuse our data? Ultimately you want your brand to align with your audience’s desires and demands. And your audience will appreciate one less banner to dismiss when they just want to load your page.
• Your time and effort could be better spent driving towards learning new cookieless tools and techniques while also building sugar-free (bad cookie joke) solutions for acquiring customers, understanding behaviors, reporting, and measurement.
• This shift will happen later this year anyway and you’re likely already struggling with increasingly non-traceable user journeys, dark social channels, etc. Being reactive instead of proactive will put you behind while other companies (see Apple) move forward and build moats of differentiation.

At Sentry, the idea to strip all advertising cookies was born out of respecting our privacy-conscious audience, and our privacy by default values. We market to developers who notoriously do not like being marketed to (we should know; we are a developer-led company and Sentry users ourselves), so the idea of removing ad cookies instantly intrigued us. And, if given the choice, why not build for tomorrow, today? That decision has had ancillary benefits along our journey that we’ll get into, one being which it gave us an excuse to do away with the annoying cookie consent banners which have come to define modern web browsing.

That sounds like good-enough reasoning, right? Despite the momentum internally to do this, to be honest, as a growth marketer I was excited at the prospect of it, but then I started thinking about all of the things that are going to break. The initial response I had was to raise concerns for why we shouldn’t do it or to delay it. The following breakpoints stuck out:

• Our attribution models (first and multi-touch in our case)
• Our marketing reports in our business insights tool
• Google Analytics
• All of our SEO reporting (built on GA and GA4)
• Google Ads smart bidding
• Procurement of onboarding new tools
• Remarketing

…And there’s an admittedly large list of stuff we didn’t think through that caused problems when we removed ALL user tracking on our site:

• YouTube video embeds on our website used cookies
• Our GCLID from Google Ads passing onto our site*
• Our bot blocking tool for ads relying on both a pixel and GCLID
• Lookalike modeling not being an option b/c of data sharing with ad platforms*
• Not being able to use Salesforce x Google Ads integration*
• 4xing our website’s 404s and redirects on accident (whoops)
• Completely losing signups from display marketing and YouTube
• Losing referrer data for content analysis

*Some of these were more due to extreme measures we took by removing user tracking and the cookie banner as a whole, but most were due to the removal of cookies

We’ll cover some of these and share how to avoid the same mistakes we made in the next section.

To state the obvious, performance marketing is critical for many businesses; Gartner says digital spending accounted for over half of marketing budgets in 2022. Given the importance of this channel and the challenges laid out above, how can you take on the risk of running performance monitoring without cookies?

I don’t have all of the answers for every industry and audience, but I’ll offer you insights from our first eight months of going completely without cookies and without user tracking.

How to run performance marketing completely cookieless

Let’s break this down into four sections: targeting, bidding, tracking/reporting, and procurement (don’t skip this!). Each section will cover considerations to be thinking about when going cookieless, the issues that we ran into, and the pivots that we made so that you can help prepare for the future.

Targeting without cookies

The current state of targeting has already been severely impacted by various changes in the last few years. Remember the havoc Apple wreaked on the entire digital advertising ecosystem when they launched ATT? At the time, Facebook (now Meta) estimated a loss of 10-13 billion ad revenue dollars related to the change, likely a direct reflection of advertisers shifting their budget away from Meta because they lost ROI. And that was just one in a string of privacy changes that likely impacted digital marketing performance.

If that wasn’t enough of a triggering thought, how much less of a remarketing pool did you have across ad platforms when people started opting out of the consent banner on your site, a 30% reduction? 50%… or more? On top of that, you were already dealing with ad blockers, and probably depend heavily on walled gardens to give you targeting options that work for your specific audience.

When third-party cookies are fully deprecated this year, there will undoubtedly be more struggles for performance marketers. Without traditional pixels or conversion signals, Google (the largest ad platform in the world) struggles to find the intent of web visitors to purchase. Shifting now, this made our targeting on Google search much worse, but actually still tolerable because you can use keyword intent and negative search terms as a guiding light. However YouTube, Display, Shopping, Demand Gen, or PMAX campaigns depend on these purchase intent signals paired with black box contextual audiences that Google gives you which, at times, in our case were scattershot at best. For example, one custom intent audience that utilized search terms/history was showing us billions of impressions (more than the current population of the world) in the estimated audience size, even though it utilized long tail search terms. Not exactly precise targeting to reach software developers in our case.

What happened to our targeting at Sentry?

Our Google rep, our ad agency, and I all knew that Google’s lack of signals would cut deep, so when we went cookieless, we treated it as an opportunity to learn how we would pivot to find our audience, rather than a doomsday event.

Display and other channels were severely hampered after cookie removal; our traditional retargeting motion died off pretty quickly as we couldn’t use GA4’s audiences or our Google Ads pixel. We had to pivot our strategy fast.

We decided to rely on ad engagement retargeting (rather than traditional retargeting) on most of our ad channels which isn’t the same, but still gives us a semblance of a funnel. We tailored our ads that are focused on the middle of funnel (MOF) and bottom of funnel (BOF) to this engaged audience. This is a not-so-bad patch for a broken retargeting effort as long as you have engaging content and the budget to promote it. Engagement retargeting is offered on Meta, LinkedIn, YouTube, and Reddit, and we pair video ads at the top of the funnel to drive engagement.

At the same time we were figuring out remarketing, we also lost confidence in prospecting with Display and decided to migrate the budget to sponsorships and publishers that we instinctively knew had our core audience. This may produce less flashy conversion counts depending on your business, but it gave us a place to tell our story to our audience. It also forced us to focus more on content marketing which speaks to the potential customer’s jobs to be done; it also builds trust and goodwill instead of trying to get vanity signups or [insert your BOF KPI] with a cold prospecting audience.

When deciding to remove cookies or user tracking in general, these are your top three targeting considerations IMO:

1. How you will segment funnel stages in creative ways
2. Will you still have faith in your current channels to continue to reach your audience or do you need to explore other channels?
3. Which bidding model you will shift to and what potential impacts to targeting might you see
4. BONUS: How will you prepare the stakeholders in your company for the inevitable impact on traceable conversions? This could be its own post, so we’ll keep this one about the nuts and bolts, but you should start thinking about this now. Getting cross-functional buy-in (and crucially, that of leadership) is the best first step, so feel free to share this post with whomever needs to see it

Bidding

Most ad platforms rely on pixels and cookies to train the bidding algorithm on which type of users convert for your business and which will hit specific KPIs or cost per actions (CPAs) that you’re going after.

In our case, most of our ad platforms were set up with pixels that sent conversion signals to the ad platforms when a signup or a request for a demo happened on our website. We were also piping in conversions with offline uploading to get down-funnel events to train Google to find the right user intent. This can be done with Google, Meta conversions API (CAPI), and LinkedIn (with offline conversion upload.) For digital marketers, this is a way to improve CPAs and level up from 101 digital marketing, but it also involves either using cookies or sending user information back to an ad platform. Quick nuance: for certain tracking technology like hashed offline passbacks—although it isn’t technically a cookie—it may still be subject to “cookie laws” and you may need to keep your screen-eating cookie banner. So in addition to cookie removal, we had to do away with GCLID passback for example, because we wanted to remove our cookie banner and because our users deserve their privacy. Be sure to discuss with your legal team how this might apply if you want to remove your banner.

For us, our bidding models had the largest impact on revenue; because Google is one of our largest ad platforms and is now no longer trained on clicks/intention to purchase, just that the user clicked. There was about a two to three-week delay in performance declines after we went cookieless where Google still had data we used to train it on and our campaigns continued to perform well.

However when comparing period over period from before we removed user tracking and cookies on our site to after, we saw around a 30% increase in our cost per click (CPCs) in Google search. It varied between brand and non-brand searches, but this shows the impact of what you’re up against. Bottom line: your ad campaigns could get that much worse post-cookie removal if you don’t adopt hashed pass backs or offline signals/APIs and if you’re utilizing smart strategies like Target CPA bidding, Maximize Conversions, etc.

Our options became pretty limited without these conversion signals fed into our ad platforms because of our hard line drawn at total cookie banner removal and 0 user tracking:

For Google Search: We shifted back to Enhanced CPC and Max Clicks which optimizes for less important factors than our old model (definitely a downgrade). We stay on top of our negative keywords and regional data so much more now as a proxy for using a bidding model to sort through the intent of users.

For YouTube: We shifted to Target CPV for any video viewer or engagement and Target CPM for awareness. We had to remove video action campaigns here so we focused more on reach. Instead of a drastic loss, we ended up 13xing our views on our videos period over period, and are getting more quality site visitors than we used to. We’re still getting conversions from this channel and our benchmarks for view rate are right at our industry average (and above the benchmark from Google) so this ended up being a successful shift. We also saw over 100% growth in folks mentioning they heard us from YouTube on our onboarding survey quarter over quarter. Shifting bidding strategies here could be worth trying, regardless of your decision with cookies.

For Display: We shifted to viewable CPM. As a result, our CPM was nearly cut in half and we saw a surprisingly large increase in impressions (north of 20%) month over month after making this shift. However our conversions tailed off and then dropped off of a cliff with the shift in bidding models.

Note: We kept our bidding model on smart bidding for a while after the cookie change in hopes Google would utilize historical conversion data, it worked for a little but was a sharp decline”

These conversions weren’t converting through our funnel to paying customers at a high enough rate anyway (even with a long term view), so our shift in bidding strategy for display was paired with a shift to executing more brand and awareness marketing with display rather than direct response which wasn’t a great fit for the channel anyway.

For most other channels: We removed pixels and relied on UTMs paired with manual bidding. More on this in the section below.

Cookieless tracking, attribution, & reporting

Consider the challenges you probably already face with regards to reporting:

• Most users on paid social channels are on mobile devices which introduces cross-device tracking as well as view-through targeting challenges.
• Ad blockers were already plaguing your ability to see a large portion of your audience. 42.7% of internet users worldwide use ad blockers.
• Hidden user journey moments like a shared linkedIn post or an executive hearing about you on a podcast or at an event are already influencing deals.
• If you’re using an attribution model, it’s probably not accounting for the above touches and you are probably favoring certain channels too much given how complex a buyer journey is in today’s age.
• Matching up spending to down-funnel events for ROI/ROAS calculations is a big undertaking. Drilling down to the keyword or audience to understand your distribution areas and where to spend your budget is time-consuming.
• Even if you have everything orchestrated and deduping out of one ad server like CM360, you’re still dealing with the cookie banner and ad blockers removing a chunk of your vision and signals.

Now take away cookies and user tracking and you’re left with…?

Reporting and attribution became a huge lift for us when we went cookieless. Our Business Insights (BI) team flagged that our attribution data was going to break and we needed to migrate to a new underlying table. On top of this, we had to switch attribution models from First and Multi-touch (depending on what we were measuring) to Last-click because we would only be able to track the previous page visited by a user before conversion without storing data on the user’s browser. We were lucky to have a competent BI team and alignment top-down to be able to pivot and get a new attribution model and a new table of underlying data.

This took a TON of back and forth, basically building logic that an out-of-the-box attribution solution already has in SQL, but we finally got to a place where we could salvage around 50% of attribution data. This was a huge win for us, to directionally understand the best channels that lead to business outcomes. Because we lost so much data, we also instituted a self-reported attribution “how did you hear about us” survey. Eight months in, we’ve actually found new channels and learned a lot about our old ones. Ironically, a lack of data led to new insights.

My advice here is to figure out where your attribution is done (CRM, Data Warehouse, 3rd party tool?) and if you’re serious about removing cookies, start discussing the lift it would take to move to a simpler model.

After more than half a year of cookieless attribution, my takeaway is that UTMs and referrer data, when done right, directionally do the trick for understanding which ads, publishers, campaigns, and audiences are performing. The self-reported attribution survey we prioritized now helps remove our bias and uncover new expansion areas. We’re also working on understanding blended data. All of these are excellent muscles to develop for a cookieless future.

Procurement

Going cookieless doesn’t mean you shouldn’t actively be hunting for new solutions to help improve performance. In the Marketing Technology Landscape, published by chiefmartec.com and its editor Scott Brinker, the amount of technology options for marketing grew from 150 in 2011 to 11,038 in 2023. There are more options now than ever before, and much more innovation and disruption happening.

We’ve recently met with companies that do Attribution, Account Based Marketing, Demand Sided Platforms (DSPs), Connected TV, and Analytics tools, and every conversation sputters out as soon as we mention privacy and our lack of cookies. Our marketing team would leave demo calls with serious concerns that our new privacy policy would render the tool not worth the sticker price, even if we could use it in a limited fashion.

Despite this challenge, there are going to be more and more tools out there that can navigate user tracking and cookieless well. Here are a few questions that I’ve found to be useful to ask current vendors up for renewal, or prospective tools:

• Does your software comply with data protection and privacy regulations, such as GDPR or CCPA? If so, can we get access to any documentation that you have?
  • Data Processing Agreement
  • Terms of service
  • BONUS: While you’re here asking for docs, you may have a security/compliance team that will ask for these anyway:
    • ISO/SOC2 certifications
    • Prepopulated security questionnaire like the CAIQ
• Does it require a tracking script to run?
  • If the script picks up analytics data, e.g., conversion data, then it could violate your cookie policy.
• Can you explain the different types of cookies your software utilizes and their specific functions?
• How does your software collect and store data related to user interactions?
• Does your software share user data with third parties, and if so, how is this handled?

Often a tool will claim that they are cookieless and still do some form of user tracking. For example, we switched from GA4 to Plausible and needed to know the nuances of how they use IP addresses. Know that if you want to remove your cookie banner, even if a tool is “cookieless”, you still may not be able to use it because cookie laws can apply broadly to tracking technology. Make sure to get on the same page about this with your legal team early on, before you make any procurement decisions.

Cookie removal & the future

Going cookieless might seem like a big undertaking with complex cross-functional challenges, so I can see the temptation to put it off. But the cookieless future is coming, whether you like it or not. Shifting away from cookies is a burden, and although you’ll have to sacrifice, you may gain a new perspective, and learn how to market to your users while respecting their privacy.

To summarize:

• Remember the lack of visibility and data issues you’re already facing and how this will step-function degrade as we speed towards H2 2024
• Completely cookieless performance marketing is achievable
• It takes cross-functional alignment from all stakeholders: business insights, website, legal, leadership, etc.
• There will be expected outcomes, and unexpected outcomes
• Going cookieless forces you to get creative and to start re-imagining the way you do targeting, bidding, tracking, and procurement.

We didn’t cover every challenge that we faced in this post, and we still (and will continue to) measure the impacts on KPIs eight months in.

Bottom line: Would we do this all over again?

Because Sentry values privacy and the future of the open internet, yes. We took a firm stance on this despite knowing that paid channels would get a major shake-up. But if you work somewhere that will hold on tightly to their cookie banner and rapidly diminishing tracking, at minimum we would suggest the following:

1. Institute self-reported attribution somewhere in your conversion flow.
2. Move all paid channels to utilize conversion APIs, hashed passbacks, offline uploads, etc. (because of the downsides we experienced firsthand without being able to use them.)
3. Have conversations about testing new channels that might not have as clear of an ROI as demand capture channels like paid search

Even if visitors to your site notice nothing else, they’ll at least have one less cookie banner to dismiss, and that is a win for the entire web.

---

Postscript

For clarification, Sentry has removed all cookies, other than essential cookies that do not require site visitor consent. Work with your legal team to better understand which of your cookies qualify as essential cookies under the laws that apply to you.