Excess Media Volatility

Why politicians misperceive immigration attitudes

A common argument against taking voter preferences on immigration seriously goes like this: public opinion is too volatile to be a reliable guide for policy. “The body of a drowned asylum seeker washes up on a Mediterranean beach, and everyone is for open borders. A few days later, an asylum seeker commits a terrorist attack, and everyone wants to abolish the asylum system.” Following such narratives, attitudes are often assumed to swing wildly with the news cycle. Hence, politicians would be foolish —irresponsible, even — to chase them. This argument is often deployed by politicians defending unpopular positions and by commentators dismissing survey research in particular or public opinion. It provides an intellectual justification for what I have elsewhere called representation gaps.

However, this argument is empirically wrong. In this article, I will show that immigration attitudes are actually stable and provide a potential explanation for why so many important stakeholders do not know this.

Immigration attitudes are stable

Figure 1: Share of Americans wanting less immigration. Note: Own calculation based on data from Gallup polls. Americans were asked, “Should immigration be kept at its present level, increased, or decreased?” I plot the share of respondents who choose “decreased”, ignoring those who select “keep at the present level”.

“In your view, should immigration be kept at its present level, increased, or decreased?” Gallup has asked Americans this question for many years. Figure 1 plots the share of those who want to reduce immigration among those who want to increase or decrease it (for consistency with other polls, to be discussed below, I ignore the neutral option, but the results are qualitatively the same when I include them). There is a consistent tendency among US Americans to want less immigration. This tendency has become a bit weaker over time, but public opinion does not fluctuate wildly.

Figure 1 plots one specific item and focuses on the US. Kustov, Laaker, and Reller (2021) have shown this pattern rigorously using nine panel datasets. Their finding: immigration attitudes are remarkably stable over time and robust to major economic and political shocks. This stability is not only present at the society level. When you track the same individuals across years, their views barely change either.

A potential reason for the volatility myth

If immigration attitudes are indeed stable, why do many people, for whom this information is important, think that attitudes fluctuate wildly? I think one major reason is that a naïve reading of polls cited in the media evokes the perception of volatile attitudes.

The argument that I hear from politicians and political commentators goes like this: “Just after an asylum seeker commits a terrorist attack, polling suggests that people are very anti-immigrant. A week later, an asylum seeker drowns at the border, and suddenly, polls show that the vast majority is pro-immigration. Hence, public opinion is volatile and easy to influence. People who argue like this fail to realize that the corresponding polls usually ask very different things that cannot be compared to infer opinion change. Hence, the opinion change they think they see is just an artifact of selective poll use by media outlets.

Immigration attitudes can be elicited via different items. For instance, one could ask Americans whether they are for or against a law that allows Christian refugees to come to the USA. Alternatively, one could ask them whether greater efforts should be undertaken to protect the southern border against illegal immigration. Agreement with the former statement may be read as pro immigration, while agreement with the latter item could be seen as anti-immigrant. But it would be naïve to assume an opinion change if a majority of Americans agrees with the first and then, a week later, also with the second item because they ask different questions. The first item is framed such that most people will choose the pro-immigration option, while the second is formulated in a way that makes many people choose the anti-immigration option. The same individual can very well agree with both of them.

As I will document below, media companies use very different items. They may partly do that to portray the public as more aligned with their own leaning. Maybe they do so to generate the impression of opinion change since “massive right-shift in opinions” makes a better headline than “no change at all”. Another reason that comes to mind for why media outlets do not use comparable items is laziness. It requires effort and coordination to ensure comparability. Most journalists likely do not care about comparability and just want to throw in a few numbers that support their story.

I do not assume or claim that the media is lying or even misrepresenting public opinion. All survey results they present may be true and reveal important insights into different facets of immigration attitudes. However, a reader who falsely treats these numbers as measuring the same thing may perceive wild swings in opinion that do not exist in any single consistent measure. Consider the example given above, taking the media into account. If a terrorist attack dominates the news cycle, the media will add to their articles questions tailored to the moment: Should immigrants suspected of terrorism be deported? Such a question will produce much approval under any circumstance. A naïve reader will read that as “the public is very anti-immigrant right now”. A week later, an asylum seeker dies at the border. Now the polls ask whether refugees' lives should be protected. This specific item will also produce much approval at any time. The naïve reader now thinks, “wow, the public is super pro-immigrants now. It has changed a lot.” These polls do not contradict each other. People can want to deport immigrants who pose a security threat, and believe that genuine refugees should be protected. But a reader who maps them onto a single anti-versus-pro-immigration dimension will conclude that the public just reversed its position in a week.

This made me wonder: If a reader naively takes media-cited poll results at face value, treating every number as if it measured the same underlying attitude, they should perceive far more volatility in immigration opinion than actually exists. The next section tests exactly that empirically.

Measurement

The analysis heavily relied on the usage of Claude Code, which collected and analysed all data. I supervised Claude Code and checked its approach and output for consistency. Throughout, I focus on the United States. To compare actual opinion and its media portrayal, I (i.e., Claude Code) collected 704 articles about immigration opinion from 31 U.S. outlets across the political spectrum, covering 2000 to 2025. The outlets include Fox News, Breitbart, Washington Times, National Review, and Daily Wire on the right; CNN, Washington Post, NPR, MSNBC, and HuffPost on the left; and CBS News, NBC News, ABC News, PBS NewsHour, AP, Reuters, and The Hill in the center. I searched for articles that explicitly cite polls or claim to describe what Americans think about immigration.

From each article, I extracted every poll citation using the LLM Sonnet 4.6, which read the full text of each article and recorded which survey was cited, what question was asked, what number was reported, and in which direction. A set of twelve filters excluded numbers that do not reflect current population-level opinion: historical comparisons, partisan or demographic subgroup breakdowns, politician approval ratings, non-opinion statistics, salience measures, and several others. After filtering, the extraction produced 1,522 poll citations across 389 articles. Of these, 534 citations reported both the anti- and pro-immigration share, which is required to compute a standardized score. Aggregated to the article level — one observation per article, equal to the mean of its valid citations — this yields 214 observations.

To make media-cited polls comparable to Gallup, I converted all results to a binary anti-immigration share: anti_pct / (anti_pct + pro_pct). This drops Gallup’s “present level” middle option and, for media-cited polls with a neutral category, recomputes the share after stripping the middle.

That step is worth pausing on. 86% of the polls cited in media articles do not offer a middle option. No “keep things as they are.” Just for or against, support or oppose, favor or disfavor. Yet 35–40% of Americans consistently choose Gallup’s middle option when it is available. These are people whose actual position is “things are roughly fine” but who get forced into a camp when a binary poll removes that choice. This mechanically inflates the appearance of polarization in the media’s portrayal of public opinion.

Notably, Claude Code did do a pretty good job at performing the whole data collection effort + analysis in one go after a 30min discussion during which I explained the project. It did make some critical mistakes and I had to monitor closely what it does a few times to arrive at the final result shown here. For instance, it did not include reasonable filters for poll numbers on its own, even though this should have been clear from the context. Hence, I had to validate the extraction of some articles manually and improve the prompt a few times. Still, it took me maybe 8 hours to perform all data collection and analysis (including prompting), which is a huge improvement over any alternative. I am not even sure I could have done this at a similar quality without LLMs. Read this post for a general discussion of LLMs in social science research.

Excess Media Volatility

Figure 2: Media portrayal vs. actual opinion on immigration. Note: The blue line shows Gallup’s anti-immigration share (% “decreased” / (% “decreased” + % “increased”)), excluding the “present level” option). Each orange dot represents one media article’s average anti-immigration share, converted to the same binary.

Figure 2 shows what happens when you overlay the media-generated picture onto Gallup’s consistent baseline. The Gallup baseline fluctuates within a narrow band. The media-cited polls scatter across the entire range, from 13% to 90% anti-immigration, often within the same year.

Three things stand out. First, media portrayal (i.e., the impression a reader gets about public opinion by naively perceiving different items as identical) is more volatile. The standard deviation of article-level anti-immigration shares is 1.5 times that of Gallup’s baseline (permutation test, p = 0.005). The plurality switch rate makes this concrete: how often do consecutive observations cross the 50% line, making it look like the majority position just flipped? For Gallup, this happens 8% of the time. For media articles, it happens 45% of the time — nearly every other article paints the opposite picture of whether Americans are majority anti- or pro-immigration. That is 5.6 times more frequent than in the Gallup baseline.

Second, the year-matched correlation between the two series is 0.17. Hence, an observer who takes media-cited polls seriously will not be able to infer the direction of public opinion change with much precision.

Third, media portrayal is shifted downward. Since 2000, Gallup’s binary baseline averages 66% anti-immigration. The media average over the same period is 51%. A naïve reading of conservative outlets comes closest to Gallup at 58%. Liberal outlets average 45%, center outlets 46%. All three are significantly below Gallup (p < 0.004 in each case). A news consumer reading across the political spectrum would therefore come away thinking the country is roughly split on immigration. This stands in contrast to how the Gallup item paints public opinion, which is consistent with my general understanding of current public opinion in the USA and other Western countries: the plurality wants less, a large middle is content with the status quo, and a smaller share wants more.

This excess volatility is not an artifact of comparing across outlets. It exists within individual outlets. Fox News articles are 1.6 times more variable than Gallup (p = 0.024). The Hill: 1.8 times (p = 0.008). Washington Times: 1.5 times (p = 0.014). Newsweek: 2 times (p = 0.027). Even a single outlet, read consistently over time, generates a picture of public opinion that is more volatile than reality.

Where the Excess Volatility comes from

To understand the mechanics of excess volatility, I classified all 415 extracted poll citations into three categories. The first category is what I call trackable items: neutral questions about immigration levels or enforcement strictness that are repeated over time with the same wording. Gallup’s “should immigration be increased, decreased, or kept at its present level?” is the gold standard, but similar questions exist — “should the number of legal immigrants be increased, decreased, or kept the same?” or “are immigration laws too strict, about right, or not strict enough?” These items are most useful to measure and track attitudes over time: they are neutrally framed, and they offer balanced response options.

The second category is general sentiment items: questions about whether immigration is “a good thing” or “a bad thing” for the country, whether immigrants “strengthen” or “burden” the economy, or whether they “help” or “hurt.” These are less neutral. Survey respondents tend to agree with any statement given to them, such that “Is immigration good for the country?” will overestimate pro-immigration attitudes, while “Is immigration bad for the country?” will underestimate it.

The third category is policy-specific items: questions about particular proposals like building a wall, deporting undocumented immigrants, the travel ban, DACA, sanctuary cities, and asylum policy. These make up the vast majority of what media outlets cite. They are also, from a measurement perspective, the least reliable. Survey questions need validation: testing whether they actually measure what they claim to measure. Established tracking questions like Gallup’s have known properties developed over decades of use. A question invented by a cable news pollster for this week’s controversy likely does not.

Figure 3 summarizes key information by category:

Figure 3: Poll types cited in media articles, compared to Gallup baseline. Note: Mean level = average anti-immigration share. Volatility ratio = standard deviation relative to Gallup. Switch rate = share of consecutive article-level observations that cross the 50% line. Percentages in subtitle show each category’s share of all media citations. Own analysis, 214 article-level observations from 31 outlets, 2000–2025.

Three patterns stand out. First, only 10% of media-cited polls ask the kind of question that would allow meaningful comparison over time. Gallup’s own question is cited in 5% of all extracted poll numbers.

Second, trackable items are the closest to Gallup in both level (62% vs. 66% anti-immigration) and volatility (1.1×). General sentiment items, by contrast, average 35% anti-immigration, almost half of Gallup’s figure.

Third, the excess volatility is concentrated in policy-specific items: 1.5 times Gallup’s variance and a 49% switch rate. Nearly every other article that cites a policy-specific poll flips the apparent majority. This finding provides a potential explanation for what is going on: media outlets mainly (81%) cite timely, unvalidated, and policy-specific items. These items, because they are unvalidated and all differ widely, paint a very different picture of immigration attitudes. They may actually reveal important information. But because they differ so much, comparisons between them do not allow inference on opinion change. Yet, this seems to be what many politicians and observers are doing, at least this is what many told me themselves.

Another type of Excess Volatility

For readers familiar with financial economics, there is an interesting analogy. Robert Shiller received the Nobel Prize partly for demonstrating that stock prices are more volatile than the underlying dividends and earnings would justify. His key insight was methodological: you do not need to know the “correct” price to show that volatility is excessive. You just compare the variance of the price to the variance of the fundamental it is supposed to track. If the price moves more than the fundamental, the excess variance must come from something other than information — from narratives, sentiment, or noise.

If one wanted to transfer this logic to the current exercise, one could reason like this: Public opinion is the fundamental. Media-cited polls are the price. The media portrayal is significantly more variable than the underlying opinion it claims to measure — and the excess variance comes from the fragmented polling landscape and selective citation practices.

Robustness and caveats

My analysis rests on the assumption that Gallup’s question — “should immigration be kept at its present level, increased, or decreased?” — is the right baseline. This is not self-evident. This item has the advantage that it is very established and has been asked consistently, with the same wording, for many years. Moreover, it asks directly how people want policy to change, which is arguably the most relevant measure for understanding political preferences. There do exist other items that fulfill these criteria. However, as Kustov, Laaker, and Reller (2021) show, these survey items usually yield the same result documented here. In that sense, the item I used can be seen as exemplary for a more robust finding. Similarly, the degree of anti-immigration attitudes, i.e., the level of the blue curve in either figure, depends on the survey item chosen. However, it is a very well-established finding that the public in the USA and most other Western countries desires less immigration when asked in a neutral, balanced way.

Several robustness checks support the main results. Imputing the missing side of one-sided citations under a binary assumption, i.e., if an article reports “41% want immigration decreased” without the pro share, assuming the rest is pro, adds 114 articles and yields a variance ratio of 1.4× and a switch rate of 48%, consistent with the main analysis. Restricting the sample to major outlets with at least five articles each produces nearly identical results.

Still, several caveats are in order. This analysis is a proof of concept, not a definitive study. Most of the data processing — article collection, poll extraction, and classification — was performed by an AI system. I spot-checked the extraction against original article texts and found it to be accurate, but I did not hand-code all 704 articles. A more rigorous version of this analysis would validate a larger set of articles using human coders, a pre-registered coding protocol, and a more systematic article selection procedure. The article sample was collected through targeted searches for immigration opinion coverage, not through a complete census of all immigration articles from each outlet. This means the sample may overrepresent articles that prominently cite polls and underrepresent those that mention polling in passing. That said, I think that the results are unlikely to change dramatically with a more rigorous approach.

Implications

The “volatile preferences” argument provides intellectual cover for one of the most consequential features of contemporary democracy: the persistent gap between what voters want and what politicians deliver on immigration. If attitudes really did swing wildly with every news cycle, then politicians who ignore polls would be acting prudently. But attitudes do not swing wildly. They are stable, slow-moving, and consistent when measured properly. What swings is the media’s portrayal — driven not by fabrication, but by a fragmented polling landscape where every outlet can find a number that fits its narrative.

This finding has important downstream consequences. I have argued that large representation gaps on immigration — where politicians are systematically more liberal than their voters — are a major driver of right-wing populism in Europe, and John Burn-Murdoch has extended this analysis to the United States with similar results. If politicians, academics, and commentators systematically overestimate the volatility of immigration opinion, they will underestimate how durable the demand for restriction is. They will treat each anti-immigration poll as a transient spike rather than a persistent signal. And they will continue to be surprised when populist parties win elections by promising what majorities have wanted for decades. The irony is that many politicians believe public opinion to be easily biased by the media, while, in fact, the media may bias their own view of public opinion.