“Free lunch fraud runs rampant” blared the headline of a recent article about a government audit in Indiana, but the facts prove that headline wrong. The article estimates that one third of Indiana students receiving free or reduced-price school lunches do not qualify for the program, citing as the source audits performed by the Indiana Department of Education.
This led Indiana state Rep. Eric Turner, (R-Cicero), to opine, “If a third [of those students receiving free or reduced price school meals] are misrepresenting their income, that’s a lot of dollars that are flowing from Indiana taxpayers.”
Indeed, if one third of the students getting government-paid lunches were not eligible for that benefit, that would be a lot of cheating. But there is no evidence that cheating is happening on such a widespread scale; in fact, the evidence shows that less than one half of one percent of Indiana students getting government-paid meals submitted invalid applications.
How did Rep. Turner come to make such an exaggeration?
Further down, the article explains “Indiana school corporations must audit 3 percent of the families of free or reduced-price meal recipients. The Department of Education found that 33 percent of those audited do not meet the eligibility requirements and are therefore committing fraud.”
Apparently Rep. Turner (and perhaps the author of the hysterically-titled news article as well) believed that the audited 3% of meal applications represented a random sample, and that therefore the 33% error rate found in the sample could be extended to the entire population of students receiving free or reduced price lunch.
But that is not the case. The errors were found in a sampling that targeted the applications that are most likely to contain errors or misstatements – not a random or representative sampling of all free lunch recipients.
The USDA, which administers the school meal program, specifically directs schools to audit “Three percent (3%) of all applications approved by the LEA [Local Education Agency] for the school year, as of October 1 of the school year, selected from error prone applications” [emphasis mine]. That “error prone” group, as defined by the USDA, is comprised of families whose income is within $100 per month of the cutoff for eligibility for government-paid school meals (which in 2012-13 was $3554 monthly for a family of 4 in the 48 contiguous states, slightly higher in Alaska and Hawaii.)
In other words, of the relatively small group of applications considered most likely to contain an error, the state audit found that 33% did indeed contain an error. But that doesn’t mean that 33% of all other applications also contain errors.
If 33% of the group of people with the highest risk factors for cancer eventually do develop cancer, can we assume that 33% of all people will develop cancer? Of course not, because those results are not based on random sampling.
In a true random sample, each member of the entire population has an equal chance of being selected. When some members or groups within the population are excluded from selection, the results are called a “biased sample.” Results from a biased sample cannot be extended to the entire population – that’s just Statistics 101.
Auditing a 3% sample drawn only from applications most likely to contain errors yields a biased sample, because applications more likely to be accurate were excluded. Just because 33% of the applications already designated as “error prone” did in fact turn out to contain errors, does not mean that a similar error rate would be found in the entire “population” of applicants.
So just how widespread was the “fraud” that the state audit uncovered?
Answering that question requires an understanding of just how many applications made up the total “population”. Ironically, due to the potential for inaccuracies when income information is self-reported, and to the cost of verifying that information, the USDA has recently been moving away from the use of applications to identify students eligible for government-paid school meals.
Instead, schools are encouraged to identify eligible students through a method called “direct certification”, which uses a student’s enrollment in other government assistance programs as an automatic qualifier for school meals. This means that students whose families get SNAP benefits (formerly called food stamps), or Temporary Assistance for Needy Families (TANF), can get free school meals without submitting an application at all. Students who are migrant, homeless, runaway, or in foster care also get free school meals without an application.
The move to direct certification has been so successful that in 2011-12, 11.6 million (about half) of the 22.77 million students nationwide who got free school lunch did so based on their qualification for SNAP alone. Many more qualified through other programs like TANF or foster care.
The 33% error rate found in the 3% of applications examined by the Indiana audit cannot be applied to this large group of students who qualified by direct certification, because their families did not fill out an application, having already passed a far more rigorous means-tested screening to qualify for other government assistance.
With so many students being directly certified, how many applications were audited? The Indiana Department of Education has not posted that number, but it can be estimated using available data.
In 2012, about 532,000 Indiana students qualified for free or reduced price meals. Subtract from that the estimated 50% (reflective of the national average) who qualified just from SNAP alone, leaving 266,000 students.
It is impossible to tell how many of the more than 32,000 Indiana children receiving TANF benefits in 2012 were both school age and not already directly certified via SNAP, so let’s not include any of them (although likely there are at least some students who do meet both criteria.) Likewise, there is no way to know how many of the 19,000 Indiana students who were homeless, migrant, or in foster care, also qualified via SNAP, so for now let’s leave them out too.
Based on all of the 266,000 students not identified via SNAP for direct certification, the 3% of audited applications equals 7,980 applications. An error rate of 33% within those 7,980 means 2,633 applications contained an error (or “fraud”, as the headline trumpets.) That’s 2,633 applications actually found to contain an error, out of about 532,000 students receiving free meals – or an error rate of about half of one percent.
To be fair, some of those 2,633 “fraudulent” applications may have covered more than one child, since families can list all of their school age children on one application. But then we didn’t exclude from the 266,000 any students who had no application, because they were directly certified via TANF, migrant, homeless, runaway or foster care programs, either, so those numbers likely balance each other out. We are back to a proven error rate of only about half of one percent.
In what universe does a one half of one percent error rate justify a claim that “one third” of students on free lunch don’t qualify for the program, or a headline of “Free lunch fraud runs rampant”?
If this whole “school lunch program fraud” storyline sounds familiar, maybe it is because a similar scenario took place in New Jersey just a week before the Indiana allegations surfaced. In New Jersey, the “fraud” claims focused on school employees, including some school board members, who wrongfully obtained free meals for their children.
Those people should be ashamed of themselves, but the highly publicized New Jersey flap may not have really been about school lunch, or scummy school board members, at all. A New Jersey education blogger who goes by the name of Jersey Jazzman explains what the ruckus in his home state was really all about.
Citing a section of the NJ Comptroller’s “fraud” report that focuses on the fact that NJ schools with higher rates of students getting free meals also receive greater amounts of state funding, Jersey Jazzman writes,
“And so we see what this jihad against free school lunches is really about: undermining school funding equity. And to do that, Comptroller Matthew Boxer takes an enormous leap: since there is evidence of extremely limited fraud in the school lunch program, free/reduced price lunch statistics must be inaccurate measures of district poverty.
“Sorry, but that’s absolutely unwarranted. Just because Boxer found a tiny fraction of cases that were fraudulent, it doesn’t automatically follow that free lunch statistics hugely overstate relative poverty rates in school districts.” [emphasis by Jersey Jazzman]
Indiana also provides extra money to schools with higher numbers of low income students, but unlike NJ, Indiana has already switched from using the school meal program data as a proxy for low-income. Going forward, Indiana will determine students’ low-income status using enrollment in its free textbook program, which requires a higher level of means-testing than school meals.
Remember what Indiana state Rep. Eric Turner said about the supposed “fraud” in his state?
“If a third [of those students receiving free or reduced price school meals] are misrepresenting their income, that’s a lot of dollars that are flowing from Indiana taxpayers.”
Since it is the federal government (meaning all of us, not just Indiana taxpayers) funding the school meal program, and since the state is already moving away from using free lunch enrollment to determine state school spending, maybe Rep. Turner should back away from his posturing about potential massive fraud in the school lunch program.
Local school funding varies so widely from state to state that improprieties in one place don’t necessarily imply similar improprieties elsewhere. But the structure of the school lunch program is universal, and one state lawmaker crying “fraud” can trigger an avalanche of similar claims nationwide. That’s why it’s vital that such audits be fully understood and accurately explained by those seeking to publicize their results. Instead, what we are seeing, in both New Jersey and Indiana, is politicians misrepresenting the findings to declare “massive fraud” where no such thing exists.
I’m pretty sure we have not heard the last of this issue. Based on the New Jersey and Indiana claims both hitting the media within a week of each other, likely there will be other states examining other audits, and other clueless politicians (and newspaper headlines) making similar unsubstantiated claim of “massive fraud.”
The moral of this story is: Whenever you see a “school lunch fraud” claim, remember to do the math, because the numbers may not add up.Archive