In early 2018, several advocacy groups noticed a drop in open rates for subscribers with Gmail domains. They said this had a negative impact on calls to action and donations. 

Using data sent to us by three of these advocacy groups as well as Change.org, a for-profit company that hosts petitions for political causes, we confirmed there was a lasting decrease in open rates unique to subscribers using Gmail. 

Throughout 2017, all the organizations in the chart above had significantly higher email open rates for the Gmail subscribers than those that used other providers. Gmail open rates for Change.org (represented by the red trend line above) were, on average, 66 percent higher compared to those of non-Gmail subscribers. Between January and May 2018 (the blue rectangle in the graph), open rates for Gmail subscribers across all four organizations dropped by at least 47 percent compared to open rates for non-Gmail subscribers. Change.org experienced the greatest drop, 79 percent.

After May 2018, the ratio of Gmail to non-Gmail open rates mostly sat below 1.0 for the advocacy organizations. (We do not have data for CREDO Action past May 2018.) This signifies higher open rates for non-Gmail subscribers than Gmail subscribers. We agreed not to share raw open rates publicly, but we looked at them, and there was not a statistically significant increase in open rates for non-Gmail users between January and May 2018 for any of the four groups we analyzed.

In interviews, the advocacy groups we spoke to suspected that the plunge in open rates was due to a change in Gmail’s automated mailroom that was placing their emails in the “promotions” inbox rather than users’ primary inbox. The promotions inbox is one of five algorithmically labeled tabs added to Gmail in 2013. 

To begin to understand how Gmail classifies and sorts political emails, we designed an experiment with a new email address. We subscribed to receive emails from more than 200 political groups: from presidential candidates, House representatives approaching reelection in battleground states, think tanks, and advocacy organizations that either organized or supported political issues. Not everyone sent us emails. 

After four months of data collection, we analyzed 5,134 emails from 171 groups and politicians.

• Across the political spectrum, Gmail diverted nearly 90 percent of political emails from the primary inbox in our experiment. Most went to the “promotions” folder (50 percent), a feature enabled by default, or the spam folder (40 percent). About 11 percent of all emails were delivered to the primary inbox.
• Gmail blocked almost half of the political senders in our sample from the primary inbox entirely—42 percent of senders failed to get a single email into the primary inbox. (This statistic excludes senders with fewer than two emails in our sample.)
• Gmail categorized presidential candidates’ emails as promotions 64 percent of the time, which is more than it did for other categories of political email. It sent more than 90 percent of Bernie Sanders’s and Amy Klobuchar’s emails to promotions.
• Gmail directed less than one percent of campaign emails from House of Representative incumbents in battleground states to the primary inbox.
• There were some outliers: Gmail’s algorithm sent the majority of emails from some senders, including the American Enterprise Institute (99 percent), Democratic Socialists of America (75 percent), and Pete Buttigieg (63 percent), into the primary inbox.

Our findings are based on a four-month study of one model email address and a list of political candidates, advocacy groups and think tanks that was robust but by no means exhaustive. 

Our data show how Gmail handled one particular inbox with no user input; it’s how the default algorithm worked in this case without individual user participation. 

Gmail says its algorithms respond to each user’s interactions to adjust categorization in the aggregate and for individual users. If so, email categorization will vary from person to person. It’s unclear to what extent this behavior would affect the categorization of political mail for any given person, but it’s important to note that our findings would not necessarily match where emails end up for a particular Gmail user.

We did not determine why Gmail categorized certain emails or certain senders’ emails the way that it did. 

Gmail does not have an explicit label in the email metadata for the primary inbox. To estimate the emails that wound up in the primary inbox, we subtracted emails that went to other default tabs (promotions and social) and spam. 

Gmail also labels emails with sorting for other tabs it offers, such as “updates,” but we did not analyze those categories because those tabs are turned off by default. Separating those emails would have further reduced the number of emails in the primary inbox—by 540 emails for the “updates” tab alone.

Organizations and campaigns send email from different addresses and domains, so we used a distinct tracking alias for each group and campaign (i.e., email+dnc@gmail.com) to identify the sender. All emails sent to that address were attributed to that sender. As a result, some emails may have been attributed to a sender incorrectly due to sharing or selling of email subscriber lists. 

To discern how often that might happen, we checked a sample, 400 emails (100 from each sender category), and found only four of them, one percent, were from another sender. Of those, two emails that we attributed to End the Border Crisis Now were instead sent by Chip Roy, who funds that website. The other two emails that came from a sender other than the alias were from the TJ Cox campaign. They were sent to the email address we had set up for Harley Rouda’s campaign. It appears that Rouda shared his subscriber list with others.

When we checked all the presidential candidate emails, the only mislabeled email was one from the GOP that used an alias we’d created for Trump events.

Of the 231 campaigns and groups we subscribed to, 60 did not send us any email, including, notably, DonaldJTrump.com. 

We created a Gmail account using a new mobile number on Oct. 15, 2019. We used a new phone number to avoid having Google associate it with an existing Google user profile. Google disclosed to us that user activity influences email classification.

We used our new email address to subscribe to email communication from 231 entities: presidential candidates (N=17), House representatives in battleground states (N=141), and advocacy organizations and think tanks (N=73). The list of political advocacy groups and think tanks is neither comprehensive nor scientific, but we sought representation from organizations across the political spectrum. We made sure to include the groups that initially reported the drop in open rates in 2018 (CREDO Action, Democracy for America, SumOfUs, Color Of Change, Demand Progress, Courage Campaign, Presente.org, Social Security Works) as well as Change.org, which noticed the drop independently.

Some groups required a zip code for email sign-ups. In those cases, we used 33601—Tampa, Fla. 

We created a custom email alias for each sign-up, such as email+dnc@gmail.com, for bookkeeping. This was essential for tracing the emails we received back to the initial entities we shared it with. The complete list of the email senders we subscribed to can be found in the appendix. We did not receive emails from every group.

We collected 5,417 emails in our inbox from Oct. 16, 2019 to Feb. 12, 2020, from 171 out of 231 of the entities we signed up for. During this time frame, we periodically exported the inbox to an mbox file in Google takeout. Mbox is a standardized format of storing an email inbox as a single semi-structured text file. Regular exports were essential, as emails labeled by Gmail as spam disappeared after 30 days. 

We did not open, move, or label the emails in the inbox. 

We preprocessed the mbox files by extracting relevant metadata fields, filtering out 44 emails sent from Google and 235 emails from email addresses not in our study sample. 

These 235 emails are from nonpolitical groups (like the Girl Scouts and Veterans Affairs) that we had subscribed to at the start of the experiment but have since removed from our study. They also include emails that were not sent to a tracking alias, which may have occurred because the email sign-up form rejected the alias or the sender removed the alias.

There were no emails in the trash or sent folders, which we would have removed from the study. 

Our final dataset consisted of 5,134 emails.

We referenced Gmail’s own email labels, found in the metadata field X-Gmail-Labels, to establish a set of classifications for the emails we received. 

Primary Inbox are emails that do not contain “category promotions,” “spam,” or “trash” in X-Gmail-Labels.

Promotions are those labeled by Gmail as category promotions in X-Gmail-Labels but not as spam. Although emails can receive both labels by Gmail, the presence of the spam label directs the email to the spam folder in the user interface. In our sample, 69.22 percent of the emails that Gmail labeled as spam were also labeled as category promotions. We marked these emails as spam and not promotions.

Spam is any email labeled by Gmail as spam in X-Gmail-Labels.

We only analyzed data for email placement in spam and tabs that Gmail enables by default: primary, promotions, and social. Gmail offers another two tabs, but those must be activated by the user.

We used the following distinctions when deciding which entities to sign up for:

For the full list of entities we signed up for, how many emails received from each, and the dates for the earliest and latest emails, refer to the appendix.

Overall, Gmail diverted almost all emails from the political groups and candidates we signed up to receive—89.37 percent—to promotions or spam. It delivered only 10.64 percent to the primary inbox. For many senders, Gmail did not send a single email to the primary inbox. 

Of the entities with at least two emails in our sample, Gmail blocked nearly half of them from the primary inbox—41.61 percent didn’t get a single email into primary in our test. Gmail’s email classification varied from sender to sender and category to category. 

This table shows variances in how Gmail treated email from different categories of political entities in our experiment. We collected data from Oct. 16, 2019, to Feb. 12, 2020. 

For House campaigns, Gmail delivered less than one percent of emails to the primary inbox, diverting half the emails to promotions. Gmail treated House official emails differently, allowing almost half of those emails to the primary inbox—far more than other types of senders. Still, Gmail diverted one-fourth of the House official emails to promotions, even though “.gov” addresses cannot legally be used for campaigning.

Presidential candidates had the second lowest amount of emails sent to the primary inbox, 6.40 percent. Gmail sent fewer presidential candidates’ emails to spam than it did for other groups but sent the largest share to promotions, 63.81 percent.

Zooming in on each presidential candidate, two were outliers, with Gmail sending a high rate of their mail to the primary inbox: Pete Buttigieg and Andrew Yang, with 62.79 percent and 46.38 percent, respectively, far more than any other candidate. Gmail diverted less than 9 percent of their emails to promotions.

Compare this to Elizabeth Warren, Joe Biden, Kamala Harris, Joe Walsh and Beto O’Rourke: Gmail directed none of their emails to the primary inbox in our experiment. Gmail categorized approximately 85 percent of Warren’s and Biden’s emails as promotions.

For Sanders and Klobuchar, Gmail sent more than 90 percent of their emails to the promotions tab—the most among presidential candidates. It delivered only 1.8 percent and 0.96 percent of their emails, respectively, to the primary inbox.

This table shows how Gmail categorized emails from each presidential candidate in our experiment. We collected data from Oct. 16, 2019, to Feb. 12, 2020, except for Gabbard, whose emails we started collecting Nov. 8, 2019; and Bloomberg and Steyer, whose emails were gathered from Feb. 6, 2020, to Feb. 12, 2020. 

Although we subscribed to email updates on DonaldJTrump.com, we did not receive any emails from the Trump campaign. We also signed up for a Trump event and received one email sent from contact@action.gop.com, which landed in spam.

For the four groups that said they were harmed by decreases in their Gmail open rates starting in 2018, Gmail sent between 60.23 percent and 90.24 percent of their emails to promotions in our experiment. This was higher than the average for all advocacy groups and think tanks we signed up for, which was 46.38 percent.

This table shows how Gmail categorized emails from select advocacy groups in our experiment. These groups reported a sudden drop in open rates two years ago. The data were collected from Oct. 16, 2019, to Feb. 12, 2020.

For two of the groups, Gmail didn’t send any emails to the primary inbox in our experiment. One of these groups, CREDO Action, shut down in January 2020.

Just as some presidential candidates’ emails were outliers in how often Gmail sent them to the primary inbox, the same was true of some advocacy groups and think tanks, like the American Enterprise Institute, the Claremont Institute, and Democratic Socialists of America, at 98.86, 82.35, and 75 percent, respectively (see appendix).

Google communications manager Katie Wattie said in an email that the company’s categories “help users organize their email.” 

“Mail classifications automatically adjust to match users’ preferences and actions,” she said. “Users really like the tab organization.”

Wattie declined to say whether most users keep the tabs, but an email deliverability firm said about 34 percent of respondents to a 2016 survey said they use them. 

In response to concerns that Gmail’s tabs and inbox ads would turn into a “Facebook-style news feed where you have to pay for placement,” Wattie replied: “What you describe is not on our roadmap for Gmail.”

The advocacy groups whose open rates for subscribers using Gmail dropped in 2018 said that, in a phone call that year, a Google representative suggested they buy ads in the promotions tab to improve their access to Gmail users.

Wattie did not respond directly to questions about the call but rather wrote in an email that Gmail has not allowed “political content” in ads since 2016 and that those would include issue advocacy and fund-raising.

We found that Gmail’s algorithm sent few emails from politicians and political groups to the primary inbox in our experiment. Half of political emails wound up in Gmail’s promotions tab, which the company says is for sales and marketing.  

We also found that Gmail’s categorization of emails was uneven across organizations and politicians. For instance, Gmail did not send a single one of Warren’s 66 emails to the primary inbox in our experiment, yet it did send 63 percent of Buttigieg’s emails to the primary inbox, the highest-profile location in Gmail’s default tabbed inbox. What occurred to Warren is not out of the ordinary: For 42 percent of email senders with at least two emails in our sample, Gmail didn’t deliver a single one of their emails to the primary inbox.

We were unable to discern from the data we gathered why Gmail treated emails from different political entities differently. The company said its algorithm decides where to place emails based on individual and group behavior. 

The spreadsheet of political entities we signed up for, their website, our email alias, as well as the category we assigned to them can be found here.

A list of political entities that did not send us any emails can be found here.

Period of experiment 10/16/19 to 2/12/2020

Period of experiment 10/16/19 to 2/12/2020

 Period of experiment 10/16/19 to 2/12/2020

 Period of experiment 10/16/19 to 2/12/2020

Sorted by percentage of emails sent to primary inbox.  Period of experiment 10/16/19 to 2/12/2020

Sorted by percentage of emails sent to primary inbox.  Period of experiment 10/16/19 to 2/12/2020

Sorted by percentage of emails sent to primary inbox.  Period of experiment 10/16/19 to 2/12/2020