Can you trust the numbers the U.S. government reports daily for the number of confirmed COVID-19 cases? Can you trust China's or Italy's figures? How about the case counts reported by Russia or other nations?

2020 has been a bad year for many people around the world, mainly because of the coronavirus pandemic and many governments' response to it, which has almost made COVID-19 as much a political condition as a viral infection. Among the factors that make it a political condition is the apparent motives of political leaders to justify their policies in responding to the pandemic, which raises questions of whether they are honestly reporting the number of cases their nations are experiencing.

Telling whether they are or not is where Benford's Law might be used. Benford's Law describes the frequency by which leading digits appear in sets of data where exponential growth is observed, as shown in the chart above. The expected pattern that emerges in data showing exponential growth over time according to Benford's Law is strong enough that significant deviations from it can be taken as evidence that non-natural forces, such as fraud or manipulation for political purposes are at play.

Economists Christoffer Koch and Ken Okamura wondered if the data being reported by China, Italy and the United States for their respective numbers of reported cases was trustworthy and turned to Benford's Law to find out. We won't keep you in suspense, they found that the growth of each nation's daily COVID-19 case counts prior to their imposing 'lockdown' restrictions were consistent with the expectations of Benford's Law, leading them to reject the potential for the data having been manipulated to benefit the interests of their political leaders. Here's the chart illustrating their findings from their recently published report:

But that's only three countries. Are there any nations whose leaders have significantly manipulated their data?

A preprint study by Anran Wei and Andre Eccle Vellwock also found no evidence of manipulation in COVID-19 case data by China, Italy and the U.S., and extends the list of countries with trustworthy data to include Brazil, India, Peru, South Africa, Colombia, Mexico, Spain, Argentina, Chile, France, Saudia Arabia, and the United Kingdom. However, when they evaluated COVID-19 case data for Russia, they found cause for concern:

Results suggest high possibility of data manipulations for Russia's data. Figure 1e illustrates the lack of Benfordness for the total confirmed cases. The pattern resembles a random distribution: if we calculate the RMSE related to a constant proability of 1/9 for all first digits, it shows that the RMSE is 20.5%, a value lower than the one related to the Benford distribution (49.2%).

Wei and Vollock also find issues with Russia's COVID-19 data for daily reported cases and deaths. Here is their chart summarizing the results for total confirmed COVID-19 cases for each of the nations whose data they reviewed:

They also found issues with Iran's daily confirmed cases and deaths, but not enough to verify the nation's figures have been manipulated.

Previously on Political Calculations

• Prime Numbers and Benford's Law
• Using the F-Score to Detect Accounting Fraud

References

Koch, Christopher and Okamura, Ken. Benford's Law and COVID-19 Reporting. Economics Letters. Volume 196, November 2020, 209573. DOI: 10.1016/j.econlet.2020.109573.

Wei, Anran and Vellwock, Andre Eccel. Is the COVID-19 data reliable? A statistical analysis with Benford's Law. [Preprint PDF Document]. September 2020. DOI: 10.13140/RG.2.2.31321.75365.

With COVID-19 coronavirus testing results continuing to dominate the news, many Americans are getting a crash course in the limits of statistical analysis, whether they wanted it or not.

For example, a study by Stanford University researchers recently made a large splash in the news because it found that "between 48,000 and 81,000 people in Santa Clara County alone may already have been infected by the coronavirus by early April — that's 50 to 85 times more than the number of official cases at that date."

Those numbers are alarming because through Saturday, 18 April 2020, 28,963 people in the entire state of California had been officially tested positive for having been infected by the SARS-CoV-2 coronavirus. If double that many people had already been infected in just Santa Clara County alone, which is consistent with the lower end estimate of the Stanford study, that result would directly affect how medical resources are being allocated within the state.

For example, with such a high number of previous infections compared to the number of deaths attributed to COVID-19, it would be evidence the coronavirus is far less deadly than previous analysis indicated, which could lead public health officials to dial back their efforts to limit the rate of growth of new infections.

But if the newer, headline-grabbing analysis is incorrect, that would be a very wrong action to take. If the incidence of COVID-19 infections is really much less, with the same death toll, more serious action would be needed to mitigate the potentially fatal infection.

That leads to the question of how the Stanford researchers collected their data and did their analysis. Here's an excerpt from their preprint paper:

On 4/3-4/4, 2020, we tested county residents for antibodies to SARS-CoV-2 using a lateral flow immunoassay. Participants were recruited using Facebook ads targeting a representative sample of the county by demographic and geographic characteristics. We report the prevalence of antibodies to SARS-CoV-2 in a sample of 3,330 people, adjusting for zip code, sex, and race/ethnicity. We also adjust for test performance characteristics using 3 different estimates: (i) the test manufacturer's data, (ii) a sample of 37 positive and 30 negative controls tested at Stanford, and (iii) a combination of both. Results The unadjusted prevalence of antibodies to SARS-CoV-2 in Santa Clara County was 1.5% (exact binomial 95CI 1.11-1.97%), and the population-weighted prevalence was 2.81% (95CI 2.24-3.37%).

We're going to focus on the unadjusted infection rate of 1.5%, since that's the figure they directly determined from their 3,330 solicited-via-Facebook-ad sample of Santa Clara County's 1.9 million population, in which 50 tested positive for having SARS-CoV-2 antibodies.

That's the researchers' reported rate of COVID-19 infections in Santa Clara County, but what's the true rate?

Estimating that value requires knowing what the testing's rate of both false positives (people who aren't really infected but who have test results indicating they are), and false negatives (people who really are infected, but whose test results gave them an 'all-clear'). Alex Tabarrok explains how to do the math to estimate the true infection rate if you know those figures:

The authors assume a false positive rate of just .005 and a false negative rate of ~.8. Thus, if you test 1000 individuals ~5 will show up as having antibodies when they actually don’t and x*.8 will show up as having antibodies when they actually do and since (5+x*.8)/1000=.015 then x=12.5 so the true rate is 12.5/1000=1.25%, thus the reported rate is pretty close to the true rate. (The authors then inflate their numbers up for population weighting which I am ignoring). On the other hand, suppose that the false positive rate is .015 which is still very low and not implausible then we can easily have ~15/1000=1.5% showing up as having antibodies to COVID when none of them in fact do, i.e. all of the result could be due to test error.

In other words, when the event is rare the potential error in the test can easily dominate the results of the test.

To illustrate that point, we've built the following tool to do that math, where we've standardized the various rates to be expressed as percentages of the sampled population, which we've also standardized at 1,000 individuals. If you're accessing this article on a site that republishes our RSS news feed, please click through to our site to access a working version of the tool.

Considering Alex' two scenarios, the tool confirms the results of a true infection rate of 1.25% for the default values, while increasing the false positive rate from 0.5% to 1.5% returns a true incidence rate of 0.00%, which indicates that all the reported positives could indeed be the result of false positive test results within the sampled population.

The ACSH's Chuck Dinerstein reviewed the study and discusses how the rate of false positives among the raw sample would affect the statistical validity of the report's population-adjusted findings:

The test for antibodies, seropositivity, is new, and the sensitivity and specificity of the test are not well calibrated. The researchers tested the test against know positive and negative serum (based on a more accurate nucleic acid amplification test – the swab in the nose or collected before the COVID-19 outbreak) among their patients and made use of the sensitivity and specificity studies done by the manufacturer on similar clinically confirmed COVID-19 patients and controls. While the specificity, false positives, with high and very similar for both test conditions, the sensitivity, the false negatives, were much higher in the local serum as compared to the data from the manufacturer. The researchers used blended false positives and negatives in their calculations.

• Fewer false positives lowered the calculated incidence of COVID-19 to 2.49%.
• Higher false positives increased the calculation to 4.16%.
• Their blended value gave the 50-fold increase being reported.

In that thoughtful "peer-review," it is pointed out that the false-positive rate may be higher, it depends on how it is calculated. As that false positive rate increases, given their sample size, the conclusion may become lost in statistical uncertainty. In other terms, in the 50 positive tests, there might be 16 to as many as 40 false-positive results.

Before we go any further in this discussion, we should note the Stanford team's results are being torn apart by statisticians, including Luis Pedro Coelho and Andrew Gelman, the latter of whom is demanding the study's authors provide an apology for wasting everybody's time with what he describes in detail as avoidable statistical errors. Gelman also has this to say about Stanford University's credibility:

The authors of this article put in a lot of work because they are concerned about public health and want to contribute to useful decision making. The study got attention and credibility in part because of the reputation of Stanford. Fair enough: Stanford's a great institution. Amazing things are done at Stanford. But Stanford has also paid a small price for publicizing this work, because people will remember that "the Stanford study" was hyped but it had issues. So there is a cost here. The next study out of Stanford will have a little less of that credibility bank to borrow from. If I were a Stanford professor, I'd be kind of annoyed. So I think the authors of the study owe an apology not just to us, but to Stanford.

Before we close, take a closer look at the image we featured at the top of this article, particularly the solution to the algebra problem that appears on the whiteboard. This image was snapped from Stanford University's promotional materials for its graduate school of education. It's not just rushed studies that Stanford professors have to be annoyed about coming out from the university these days.

Back in 2009, we wrote about Hauser's Law, which at the time, we described as "one of the stranger phenomenons in economic data". The law itself was proposed by W. Kurt Hauser in 1993, who observed:

No matter what the tax rates have been, in postwar America tax revenues have remained at about 19.5% of GDP.

In 2009, we found total tax collections the U.S. government averaged 17.8% of GDP in the years from 1946 through 2008, with a standard deviation of 1.2% of GDP. Hauser's Law had held up to scrutiny in principal, although the average was less than what Hauser originally documented in 1993 due to the nation's historic GDP having been revised higher during the intervening years.

We're revisiting the question now because some members of the new Democrat party-led majority in the House of Representatives has proposed increasing the nation's top marginal income tax rate up to 70%, nearly doubling today's 37% top federal income tax rate levied upon individuals. Since their stated purpose of increasing income tax rates to higher levels is to provide additional revenue to the U.S. Treasury to fund their "Green New Deal", if Hauser's Law continues to hold, they can expect to have their dreams of dramatically higher tax revenues to fund their political initiatives crushed on the rocks of reality.

Meanwhile, the U.S. Bureau of Economic Analysis completed a comprehensive revision to historic GDP figures in 2013, which significantly altered (increased) past estimates of the size of the nation's Gross Domestic Product.

The following chart shows what we found when we updated our analysis of Hauser's Law in action for the years from 1946 through 2018, where we're using preliminary estimates for the just-completed year's tax collections and GDP to make it as current as possible.

From 1946 through 2018, the top marginal income tax rate has ranged from a high of 92% (1952-1953) to a low of 28% (1988-1990), where in 2018, it has recently decreased from 39.6% to 37% because of the passage of the Tax Cuts and Jobs Act of 2017.

Despite all those changes, we find that the U.S. government's total tax collections have averaged 16.8% of GDP, with a standard deviation of 1.2% of GDP. Applying long-established techniques from the field of statistical process control, that gives us an expected range of 13.2% to 20.5% of GDP for where we should expect to see 99.7% of all the observations for tax collections as a percent share of GDP.

And that's exactly what we do see. The next chart zooms in on the total tax collections as a percent share of GDP data from the first chart, and adds the data for individual income tax collections as a percent share of GDP below it.

What we find is that the federal government's tax collections from both personal income taxes and all sources of tax revenue are remarkably stable over time as a percentage share of annual GDP, regardless of the level to which marginal personal income tax rates have been set. The biggest deviations we see from the mean trend to be associated with severe recessions, when tax collections have tended to decline somewhat more than the nation's GDP during periods of economic distress.

We also confirm that the variation in total and personal income tax receipts over time is well described by a normal distribution. We calculate that personal income tax collections as a percentage share of GDP from 1946 through 2018 has a mean of 7.6%, with a standard deviation of 0.8%.

For both levels of tax collections, if Hauser's Law holds, we would then expect any given year's tax collections as a percent of GDP to fall within one standard deviation of the mean 68% of the time, within two standard deviations 95% of the time, and within three standard deviations 99.7% of the time. And that is pretty close to what we observe with the reported data from 1946 through 2018.

As for high tax revenue aspirations, we can find only three periods where tax collections rose more than one standard deviation above the mean level.

1. In 1968, the Democratic U.S. Congress and President Lyndon Johnson passed a 10% income surtax that took effect in mid-year, which suddenly raised the top tax rate from 70% to 77% (which increased the amount collected from top income tax earners by 10%.) Coupled with a spike in inflation, for which personal income taxes were not adjusted to compensate, this tax hike led to outsize income tax collections in that year.
2. The sustained high inflation of 1978 (7.62%), 1979 (11.22%), 1980 (13.58%) and 1981 (10.35%) led to higher tax collections through bracket creep, as income tax brackets in the U.S. were not adjusted for inflation until 1985 as part of President Ronald Reagan's first term Economic Recovery Tax Act.
3. Beginning in April 1997, the Dot Com Stock Market Bubble minted a large number of new millionaires as investors swarmed to participate in Internet and "tech" company initial public offerings or private capital ventures, which in turn, inflated personal income tax collections. Unfortunately, like the vaporware produced by many of the companies that sprang up to exploit the investor buying frenzy, the illusion of prosperity could not be sustained and tax collections crashed with the incomes of the Internet titans in the bursting of the bubble, leading to the recession that followed.

Now, what about those other taxes? Zubin Jelveh looked at the data back in 2008 and found that as corporate income taxes have declined over time, social insurance taxes (the payroll taxes collected to support Social Security and Medicare) have increased to sustain the margin between personal income tax receipts and total tax receipts. This makes sense given the matching taxes paid by employers to these programs, as these taxes have largely offset a good portion of corporate income taxes as a source of tax revenue from U.S. businesses. We also note that federal excise taxes have risen from 1946 through the present, which also has contributed to filling the gap and keeping the overall level of tax receipts as a percentage share of GDP stable over time.

Looking at the preliminary data for 2018, which saw both personal and corporate income tax rates decline with the passage of the Tax Cuts and Jobs Act of 2017, we see that total tax receipts as a percent of GDP dipped below the mean, but still falls within one standard deviation of it, just as in over two-thirds of previous years. Tax receipts from individual income taxes however rose slightly, despite the income tax cuts that took effect in 2018, staying above the mean but still falling within one standard deviation of it.

Hauser's Law appears to have held up surprisingly well over time.

Will it continue? Only time will tell, but given what we've observed, it would take more than simple changes in marginal income tax rates to boost the U.S. government's tax revenues above the historical range that characterizes the strange phenomenon that is Hauser's law.

References

Bradford Tax Institute. History of Federal Income Tax Rates: 1913-2019. [Online Text]. Accessed 13 January 2019.

Tax Foundation. Federal Individual Income Tax Rates History. [PDF Document]. 17 October 2013.

U.S. Department of the Treasury. September 2018 Monthly Treasury Statement. [PDF Document]. 17 October 2018.

U.S. Bureau of Economic Analysis. National income and Product Accounts Tables. Table 1.1.5. Gross Domestic Product. [Online Database]. Last Revised: 21 December 2018. Accessed: 14 January 2019.

U.S. White House Office of Management and Budget. Budget of the United States Government. Historical Tables. Table 1.1. Summary of Receipts, Outlays, and Surpluses or Deficits (-): 1789-2023. Table 2.1. Receipts by Source: 1934-2023. [PDF Document]. 12 February 2018.

In 2017, U.S. soybean producers sent an estimated 1.32 billion bushels of their crop to China, the second-most on record. The record for U.S-to-China soybean exports came a year earlier, when U.S. soybean producers exported an estimated 1.53 billion bushels of their crop that year to China, which was a dramatic increase over the 1.15 billion bushels they sent to China in the year before.

2016 would appear to be have been the most successful year to date for U.S. soybean exporters, but there's a lot more to that story.

Although the year saw optimal growing conditions for soybeans in the U.S., which resulted in a bumper crop, one of the main contributors to the success of U.S. soybean producers that year came about as a result of a severe drought in Brazil, the world's top soybean exporting nation.

Brazil's drought created a unique opportunity for U.S. soybean producers seeking to claim a larger share of the world market in 2016. Since Brazil's annual harvest peaks in the second quarter of each year, thanks to its Southern hemisphere geography that puts its growing seasons six months ahead of the U.S., the news that Brazil's 2016 soybean crop and exports would be reduced because of drought conditions provided U.S. growers with the advance warning they would need to respond to what, for them, would be an opportunity.

So they took it. U.S. soybean producers planted seed varieties that would optimize the yield for their crops, which helped contribute to 2016's bumper crop in the United States. They then aggressively harvested the crop to satisfy China's domestic demand for soybeans, where China was buying up as many bushels of soybeans from the U.S. as they could that year.

But there was a dark side to that success, which is now becoming increasing apparent. In choosing seeds that would maximize crop yields, U.S. soybean producers sacrificed the protein content of their crop, effectively reducing the quality of their product. In 2017, that meant having to compete with higher quality soybeans grown in Brazil as that nation's crops have rebounded from 2016's drought conditions.

U.S. soybean growers are losing market share in the all-important China market because the race to grow higher-yielding crops has robbed their most prized nutrient: protein.

Declining protein levels make soybeans less valuable to the $400 billion industry that produces feed for cattle, pigs, chickens and fish. And the problem is a key factor driving soybean buyers from the U.S. to Brazil, where warmer weather helps offset the impact of higher crop yields on protein levels....

Soybeans are by far the most valuable U.S. agricultural export, with $22.8 billion in shipments in 2016. Declining protein levels and market share pose another vexing problem for soy farmers already reeling from a global grains glut and years of depressed prices.

The quality problems of U.S. soybean producers go beyond that however. In their race to export as many soybeans as they could to China in 2016, they also got sloppy in their harvesting and processing practices, where an excessive amount of foreign material was being included within the industry's soybean shipments.

China's response to that problem was to impose stricter import specifications on U.S. soybean exports at the end of 2017, which is expected to negatively impact up to 50% of the nation's soybean exports in 2018. That impact will come in the form of higher costs for U.S. soybean producers, who will have to take steps to reduce the amount of non-soybean material that will be shipped to China.

Half of U.S. soybeans exported to China this year would not meet Chinese rules for routine delivery in 2018, according to shipping data reviewed by Reuters, signaling new hurdles in the $14-billion-a-year business.

More stringent quality rules, which take effect on Jan. 1, could require additional processing of the U.S. oilseeds at Chinese ports to remove impurities. This could raise costs and reduce sales to the world’s largest soybean importer, according to U.S. farmers and traders.

Half of the 473 vessel shipments in 2017 and half the total 27.5 million tonnes of U.S. soybeans exported to China this year contained more than 1 percent of foreign material, exceeding a new standard for speedy delivery, according to U.S. Department of Agriculture (USDA) data compiled by grain broker McDonald Pelz Global Commodities LLC.

In the short run, the choice to sacrifice quality to pursue additional revenue and higher profits made a lot of sense to U.S. soybean producers. In the long run, that choice could very well leave them worse off than if they hadn't taken that path. What choice would you have made in 2016 if you were playing the soybean export game?

When a scientist realizes that they've made a fundamental error in their research that has the potential to invalidate their findings, they are often confronted with an ethical dilemma in determining what course of action that they might take to address the situation. Richard Mann, a young researcher from Uppsala University, lived through that scenario back in 2012, when a colleague contacted him right before he presented a lecture based on the results of his research that he had a problem that called his results into question.

When he gave his seminar, Mann marked the slides displaying his questionable results with the words "caution, possibly invalid". But he was still not convinced that a full retraction of his paper, published in Plos Computational Biology, was necessary, and he spent the next few weeks debating whether he could simply correct his mistake with a new analysis rather than retract the paper.

But after about a month, he came to see that a full retraction was the better option as it was going to take him at least six months to wade through the mess that the faulty analysis had created. However, it had occurred to him that there was a third option: to keep quiet about his mistake and hope that no one noticed it.

After numerous sleepless nights grappling with the ethics of such silence, he eventually plumped for retraction. And looking back, it is easy to say that he made the right choice, he remarks. "But I would be amazed if people in that situation genuinely do not have thoughts about [keeping quiet]. I had first, second and third thoughts." It was his longing to be able to sleep properly again that convinced him to stay on the ethical path, he adds.

Mann's case represents a success story for ethics in science, where his choices to demonstrate personal integrity and to provide transparency regarding the errors he had made through the retraction of his work proved to have no impact on his professional career, though he may have feared it. Such are the rewards of integrity and transparency in science, where the honest pursuit of truth outweighs both personal reputation and professional standing.

Still, an 2017 anonymous straw poll of 220 scientists indicated that 5% would choose to do nothing if they detected an error in their own work after it had been published in a high-impact journal, where they would hope that none of their peers would ever notice, while another 9% would only retract a paper if another researcher had specifically identified their error.

According to Nature, only a tiny fraction of published papers are ever retracted, even though a considerably higher percentage of scientists have admitted to knowing of issues that would potentially invalidate their published results in confidential surveys.

The reasons behind the rise in retractions are still unclear. "I don't think that there is suddenly a boom in the production of fraudulent or erroneous work," says John Ioannidis, a professor of health policy at Stanford University School of Medicine in California, who has spent much of his career tracking how medical science produces flawed results.

In surveys, around 1–2% of scientists admit to having fabricated, falsified or modified data or results at least once (D. Fanelli PLoS ONE 4, e5738; 2009). But over the past decade, retraction notices for published papers have increased from 0.001% of the total to only about 0.02%. And, Ioannidis says, that subset of papers is "the tip of the iceberg" — too small and fragmentary for any useful conclusions to be drawn about the overall rates of sloppiness or misconduct.

There is, of course, a difference between errors resulting from what Ioannidis calles "sloppiness", which can run the gamut from data measurement errors to the use of less-than-optimal analytical methods, which can all happen to honest researchers, and those that get baked into research findings through knowing misconduct.

The good news is that for honest scientists who act to disclose errors in their work, there is no career penalty. And why should there be? They are making science work the way that it should, where they are contributing to the advancement of their field where the communication of what works and what doesn't work has value. As serial entrepreneur James Altucher has said, "honesty is the fastest way to prevent a mistake from turning into a failure."

The bigger problem is posed by those individuals who put other goals ahead of honesty. The ones who choose to remain silent when they know their findings will fail to stand up to serious scrutiny. Or worse, the ones who choose to engage in irrational, hateful attacks against the individuals who detect and report their scientific misconduct as a means to distract attention away from it, which is another form of refusing to acknowledge the errors in their work.

The latter population are known as pseudoscientists. Fortunately, they're a very small minority, but unfortunately, they create outsized problems within their fields of study, where they can continue to do damage until they're exposed and isolated.

Previously on Political Calculations

• The Examples of Junk Science Series
• The Rewards of Integrity and Transparency

One week ago, the Wall Street Journal broke the news that the World Bank had a serious problem with one of its most popular and useful products, its annual Doing Business index.

The World Bank repeatedly changed the methodology of one of its flagship economic reports over several years in ways it now says were unfair and misleading.

The World Bank’s chief economist, Paul Romer, told The Wall Street Journal on Friday he would correct and recalculate national rankings of business competitiveness in the report called “Doing Business” going back at least four years.

The revisions could be particularly relevant to Chile, whose standings have been volatile in recent years—and potentially tainted by political motivations of World Bank staff, Mr. Romer said.

The report is one of the most visible World Bank initiatives, ranking countries around the world by the competitiveness of their business environment. Countries compete against each other to improve their standings, and the report draws extensive international media coverage.

In the days since, Romer has clarified that he doesn't believe that the World Bank staff engaged in a politically-motivated strategy aimed at disadvantaging Chile's position within its annual rankings, but instead failed to adequately explain how changes that the World Bank's staff made in updating their methodology of its Doing Business index affected Chile's position within the rankings from year to year.

Looking at the controversy from the outside, we can see why a political bias on the part of the World Bank's staff might be suspected, where positive and negative changes in Chile's position within the annual rankings coincided with changes in the occupancy of Chile's presidential palace.

That's why we appreciate Romer's willingness to openly discuss the methods and issues with communication, including his own, that have contributed to the situation. In time, thanks to the demonstrations of integrity and transparency that Romer is providing today as the staff of the World Bank works to resolve the issues that have been raised, the people who look to the Doing Business product will be able to have confidence in its quality. That will be the reward of demonstrating integrity and providing full transparency during a pretty mild version of an international public relations crisis.

Not every matter involving correcting the record has the worldwide visibility of what is happening at the World Bank. We can find similar demonstrations of the benefits of integrity and transparency at a much smaller scale, where we only have to go back a couple of weeks to find an example. Economist John Cochrane made a logical mistake in arguing that property taxes are progressive, which is to say that people who earn higher annual incomes pay higher property taxes than people who earn lower annual incomes.

In search of support for his argument, he requested pointers to data indicating property taxes paid by income level from his readers, who responded with results that directly contradicted his argument. How he handled the contradiction demonstrates considerable integrity.

Every now again in writing a blog one puts down an idea that is not only wrong, but pretty obviously wrong if one had stopped to think about 10 minutes about it. So it is with the idea I floated on my last post that property taxes are progressive.

Morris Davis sends along the following data from the current population survey.

No, Martha (John) property taxes are not progressive, and they're not even flat, and not even in California where there is such a thing as a $10 million dollar house. (In other states you might be pressed to spend that much money even if you could.) People with lower incomes spend a larger fraction of income on housing, and so pay more property taxes as a function of income. Mo says this fact is not commonly recognized when assessing the progressivity of taxes.

Not only is Cochrane providing insight into the error in his thinking, his transparency in addressing why it was incorrect is helping to advance the general knowledge of his readers.

In both these cases, we have examples of problems that could have been simply swept under a rug and virtually ignored with nobody being the wiser, but where the ethical standards of the people involved wouldn't let them do that. Even in making mistakes while addressing the mistakes they made or discovered, they made the problems known as they worked to resolve them, and because they did so, we can have greater confidence in trusting the overall quality of their work.

In the real world where people value honesty, admitting or acknowledging errors is no penalty. In fact, there are solid examples where scientists have retracted papers because of errors they made that, ultimately, had zero impact on their careers. Which in some cases, involved going on to be awarded Nobel prizes in their fields.

Retracting a paper is supposed to be a kiss of death to a career in science, right? Not if you think that winning a Nobel Prize is a mark of achievement, which pretty much everyone does.

Just ask Michael Rosbash, who shared the 2017 Nobel Prize in physiology or medicine for his work on circadian rhythms, aka the body's internal clock. Rosbash, of Brandeis University and the Howard Hughes Medical Institute, retracted a paper in 2016 because the results couldn't be replicated. The researcher who couldn't replicate them? Michael Young, who shared the 2017 Nobel with Rosbash.

This wasn't a first. Harvard's Jack Szostak retracted a paper in 2009. Months later, he got that early morning call from the Nobel committee for his work. And he hasn't been afraid to correct the record since, either. In 2016, Szostak and his colleagues published a paper in Nature Chemistry that offered potentially breakthrough clues for how RNA might have preceded DNA as the key chemical of life on Earth - a possibility that has captivated and frustrated biologists for half a century. But when Tivoli Olsen, a researcher in Szostak's lab, repeated the experiments last year, she couldn't get the same results. The scientists had made a mistake interpreting their initial data. Once that realization settled in, they retracted the paper - a turn of events Szostak described as "definitely embarrassing."

What isn’t absurd is the idea that admitting mistakes shouldn’t be an indelible mark of Cain that kills your career.

Indeed it shouldn't. And for honest people with high standards of ethical integrity and a willingness to be transparent about the mistakes that they have made or that have occurred on their watch, it isn't.

One of the more surprisingly popular posts that we've featured this year was "A Centrist's Guide to Media Bias and Usefulness", in which we featured a chart that ranks general news reporting web sites by the information quality of their reporting and a more accurate indication of their relative position on today's political spectrum than an earlier version that had gone viral.

So we were intrigued when we found that RealClearScience's Ross Pomeroy and Tom Hartsfield had joined with the American Council on Science and Health's Alex Berezow (formerly of RealClearScience) to create a similar chart for science news web sites.

Although here, they adapted the measure of information quality to focus more on how strongly evidence-based the stories covered by the science news sites they evaluated were, which they then assessed against a non-political spectrum that assesses how accessible or compelling each sites' stories would be to a general audience. Here's the chart they developed:

Overall, their rankings are pretty solid, though we might tweak the placement of some of the sources (we would rank Wired slightly higher on the "compelling" measure for example, although that's attributable to the kinds of science news stories that we find interesting.

As for what sites we'd like to see added to the rankings, we would put Phys.org on the list, although it has considerable overlap with LiveScience as a news aggregator, it covers more fields of science. Its downside is that a lot of the stories it picks up read like press releases, so for a lot of stories, it would only fall in the "sometimes compelling" categorization.

We'd also look to add more maths into the mix, which though it falls outside the focus on evidence-based science, where mathematics news is concerned, there are considerably fewer sites that generate anything close to what might make for compelling reading by a general audience. And really, we'd recommend just three for that kind of consumption: Quanta and +Plus, for written word reporting, and BBC Radio 4's More or Less, which is presented in audio format.

Previously on Political Calculations

• A Centrist's Guide to Media Bias and Usefulness
• The Examples of Junk Science Series
• The Map of Mathematics
• The Biggest Math Story of 2016
• The Biggest Math Story of 2015
• The Biggest Math Story of the Year (2014)

Previously, when we considered the psychological profile of pseudoscientists, we wondered if there wasn't an alternative checklist to the one we use for detecting junk science, but one that if followed, would lead to the opposite of junk science.

It occurred to us that there just might be and that it might be found in Jeremy Kun's "Habits of highly mathematical people"! Here's his story of how it came about as a response to a very common question heard by math teachers everywhere, along with the basic list.

The most common question students have about mathematics is "when will I ever use this?" Many math teachers would probably struggle to give a coherent answer, beyond being very good at following precise directions. They will say "critical thinking" but not much else concrete. Meanwhile, the same teachers must, with a straight face, tell their students that the derivative of arccosine is important. (It goes beyond calculus, in case you were wondering)

So here is my list. The concrete, unambiguous skills that students of mathematics, when properly taught, will practice and that will come in handy in their lives outside of mathematics. Some of these are technical, the techniques that mathematicians use every day to reason about complex, multi-faceted problems. Others are social, the kinds of emotional intelligence one needs to succeed in a field where you spend almost all of your time understanding nothing. All of them are studied in their purest form in mathematics. The ones I came up with are,

1. Discussing definitions
2. Coming up with counterexamples
3. Being wrong often and admitting it
4. Evaluating many possible consequences of a claim
5. Teasing apart the assumptions underlying an argument
6. Scaling the ladder of abstraction

Kun's essay is very much a RTWT, and for our money, the section on "being wrong often and admitting it" is perhaps the most valuable aspect of it. Long time readers of Political Calculations know that we actually enjoy acknowledging when we've been wrong or have made errors in our analysis, because we learn more as part of the process of getting to the truth, which is where we really want to get in doing what we do. That's why we take time to describe how to replicate the mistakes we've made and also why we even thank those who identify our errors for the improved insights that result from correcting the mistakes!

True story. Earlier this year, in replicating an analysis that Mark Perry did, we identified a problem with the data he used that meant that it could not support his findings. After we alerted him to the issue, he was extraordinarily gracious and in fact, went the extra mile to consult with others to identify if there might be other issues with the data. He then updated his original post where he had presented his findings to identify and describe each the problems with the data that had been found.

That's a key difference between science and pseudoscience. Even though a mistake had been made, Perry took that mistake as a challenge to improve, and to do so in a way that would benefit others, which we would describe as model scholarly behavior. That contrasts with the actions of pseudoscientists who, when mistakes are found in their findings, often react by seeking to distract attention away from their mistakes, and who often resort to incredibly unscholarly behavior in doing so.

Mistakes are a fact of human life. It's how you recover from them that determines the kind of person you are.

Here's an excerpt of that portion of the essay:

Being wrong often and admitting it

Two mathematicians, Isabel and Griffin, are discussing a mathematical claim in front of a blackboard. Isabel thinks the claim is true, and she is vigorously arguing with Griffin, who thinks it is false. Ten minutes later they have completely switched sides, and now Isabel thinks it's false while Griffin thinks it's true.

I witness scenarios like this all the time, but only in the context of mathematics. The only reason it can happen is because both mathematicians, regardless of who is actually right, is not only willing to accept they're wrong, but eager enough to radically switch sides when they see the potential for a flaw in their argument.

Sometimes I will be in a group of four or five people, all discussing a claim, and I'll be the only one who disagrees with the majority. If I provide a good argument, everyone immediately accepts they were wrong without remorse or bad feelings. More often I'm in the majority, being forced to retrace, revise, and polish my beliefs.

Having to do this so often-foster doubt, be wrong, admit it, and start over-distinguishes mathematical discourse even from much praised scientific discourse. There's no p-value hacking or lobbying agenda, and there's very little fame outside of the immediate group of people you're talking to. There's just the search for insight and truth. The mathematical habit is putting your personal pride or embarrassment aside for the sake of insight.

It's such a contradiction to the kind of behavior that seems part and parcel with that of the peddlers of junk science. Please do RTWT - it's well worth your time!

Following up our discussion of the priorities of the TSA (a.k.a. the Transportation Security Administration), we thought it might be worth looking at the priorities of the U.S. Department of Homeland Security (DHS) as measured by how it directed the money it was authorized to spend over the past 10 years.

The chart below visualizes what we found in the DHS' own Budget in Brief documents for each year from the U.S. government's 2007 fiscal year (beginning on 1 October 2007) through the projected end of the current 2016 fiscal year (ending 30 September 2016). We've ranked the spending by major organization within the DHS from lowest to highest as of FY2016. The values in the chart are given in thousands of U.S. dollars.

Since Fiscal Year 2007, the annual budgets of two organizations within the U.S. Department of Homeland Security have more than tripled: the Federal Emergency Management Agency (FEMA) and the National Protection and Programs Directorate (also called the Office of Infrastructure Protection).

Meanwhile, five have budgets that are over 148% larger in FY2016 than they were in FY2007: Management and Operations (177%), Customs and Border Protection (171%), Office of Inspector General (164%), Citizenship and Immigration Services (163%) and the Secret Service (148%).

Meanwhile, the remaining seven major organizations have budgets in FY2016 that are all within 31% of what they were in FY2007 - some higher and some lower: Immigration and Customs Enforcement (131%), Coast Guard (128%), Transportation Security Administration (118%), Federal Law Enforcement Training Center (97%), Analysis and Operations (88%), Science and Technology Directorate (81%) and the Chemical, Biological, Nuclear and Explosives Office (72%).

This analysis omits the DHS' Office of Health Analysis, for which the Obama administration appears to have not requested funding in FY2017, and also the FEMA grants to U.S. states and local governments that are managed through the DHS, but which really represent spending by state and local governments that is funded by the federal government.

The TSA's budget peaked at $7.84 billion in FY2012, but was cut by 8.3% in FY2013. Since then, it has slowly risen back to be 94.9% of its peak budget value, which at $7.44 billion in FY2016, is also 118% of what the TSA spent to perform its mission in FY2007.

In FY2007, the TSA spent $6.33 billion. Adjusted for inflation, that $6.33 billion in 2007 U.S. dollars is the equivalent of $7.33 billion in 2016 U.S. dollars. Since the TSA will spend $7.44 billion in 2016, its spending has kept ahead of inflation.

What that means is that the TSA has all the money it would take to provide a similar level of service in 2016 as it did in 2007, if only it spent money the same way. Because it is spending its money differently, the problems the TSA is now having in causing significant delays for travelers at U.S. airports may be entirely attributed to the decisions of the DHS and its own management for how its available funds have actually been spent.

That's why the TSA's security chief, Kelley Hoggan, has been put on administrative leave, pending a new assignment (no, he hasn't has his federal government employment terminated as erroneously reported by several mainstream news outlets - that would take a system of real accountability for managerial job performance that the civilian branches of the U.S. federal government haven't had since 2008 - Hoggan is still receiving his full paycheck.)

Where does the buck stop again?

Data Sources

U.S. Department of Homeland Security. Budget in Brief: Fiscal Year 2017. [PDF Document].
Accessed 25 May 2016.

U.S. Department of Homeland Security. Budget in Brief: Fiscal Year 2008 through Fiscal Year 2014. [Links to PDF Documents]. Accessed 25 May 2016.