Misleading Data (Surveys)
The best financial data are descriptive of a prevalent investment concept and predictive of a key performance indicator. Misleading data tend to fall well short of one or both requirements, which leads to a distorted understanding of the present and/or an inaccurate portrayal of the future.
Survey data very often fall short of the predictive requirements for useful data, especially surveys that deal with sentiment or projections.
One particularly nefarious use of surveys is to power headlines. News outlets prefer particularly outrageous survey results, whether they are conducted properly or not. Are respondents led to respond in a certain way due to social pressure or sloppy survey language?
Another more modern challenge to surveys is that sampling is getting more difficult. People are much less likely to answer the phone, and online surveys are more difficult to administer. Although still the gold standard of survey data, the BLS has seen falling survey response rates for some time now.
Other surveys have lost so much predictive power from response rate challenges that they’ve been supplanted by innovative solutions. Election surveys were far less useful than betting markets (like Polymarket) at predicting the 2024 election results. Nate Silver wrote about some of the artificial ways that pollsters tried to keep up, to no avail.
https://www.yahoo.com/news/nate-silver-cheating-pollsters-putting-163354984.html
And a final challenge comes in the form of sampling bias. We know that respondents will report wildly different economic expectations based on who is in the White House, despite a very small correlation between political parties and economic success.
These mood swings are okay when dampened via balanced sampling, but sometimes the sample is biased in one way or another. For example, the NFIB Small Business Association has a majority of Republican members. Their expressed sentiment can differ wildly from their reported activity during Democrat presidencies.
The logistical challenges to trusting any given survey data are insurmountable. Further, markets care very little about survey data. They care about policy and economic outcomes, both of which are invariably tied to hard economic data.
The pessimistic sentiment data from surveys between 2020-2024 were a trap for investors. I’ve seen a lot of investors remain too pessimistic because survey data indicated a poor economy. The hard economic data have shown otherwise. My suggestion, just try to ignore almost all surveys!