By Nick Cooney
January 15, 2013
This is Part 2 of a blog post on the impact of college leafleting. To read Part 1, scroll up to the blog post above or click here.
Additional Results
What else did the study tell us?
For one thing, those who were getting a leaflet for the first time ever reported almost twice as much diet change as those who had received one before. (“Diet change” here means number of animals spared by diet change, calculated as described below in the “How the Impact was Calculated” section). Once you account for social desirability bias, that difference becomes even larger. Those who received a leaflet for the first time were at least twice as likely to make a change in diet, and they could possibly have been as much as four times as likely. (There’s no way to know for sure where the number falls on that range since we don’t know the exact level of social desirability bias.)
College juniors reported more dietary change than other class levels, followed at a distance by sophomores, then seniors, with freshmen reporting the least amount. This was true despite the fact that freshmen were far more likely to be receiving a leaflet for the first time (which should have led them to report the most dietary change).
Most likely juniors reported more dietary change due simply to the margins of error in the chicken consumption category because that represents nearly all of the difference between juniors and other grades. However, sophomores and juniors were also more likely than freshmen and seniors to say they now ate “a lot less” animal products, which plays a part as well.
It’s possible therefore that sophomores and juniors may be the grades most likely to spare the greatest number of animals after getting a leaflet, even after accounting for the fact that many of them have received a leaflet before.
On that note, the survey also found that about 57% of those leafleted had already received a leaflet before. The likelihood of having received a leaflet generally increased with grade level.
10% said they looked at the leaflet less than 10 seconds; 30% viewed it for 10 seconds to a minute; 45% for 1 to 5 minutes; and 15% looked at it for more than 5 minutes.
How the Impact was Calculated
To view a copy of the actual survey that students filled out, click here.
To calculate the number of animals impacted (i.e., spared), we used rounded versions of these estimates of the number of animals impacted by the average American meat-eater each year plus this data to factor in dairy and egg consumption. We declared the average meat-eater to impact 28 chickens, 2 egg industry hens, 1/8 beef cow, 1/2 pig, 1 turkey, and 1/30 dairy cow each year. Others may prefer to translate the number of animals spared into days of suffering and level of suffering spared per year, but we translated the data into number of animals impacted per year.
We assigned values to each category in the survey as follows: “I eat more” of a product was calculated as a 30% increase in consumption of that product; “I eat a little less” was calculated as a 10% decrease; “I eat a lot less” was calculated as a 40% decrease; “I stopped eating this product” was calculated as a 100% decrease; “I eat the same amount” and “I did not eat this product to begin with” were calculated as no change.
If you’re interested, you can check out the graphs of reported dietary change for each product category here: chicken – red meat – fish – eggs – dairy. Statistics lovers can download the raw data from the study here.
Accounting for Bias
This survey had no “non-response bias” because students were approached randomly on campus, and they did not know what the survey was about prior to agreeing to take it.
Non-response bias is when people who decide to respond to a survey are more likely to respond in a certain way. If the survey were emailed to potential respondents, we would have expected to see a large non-response bias. Those who had changed their diet would probably be more inclined to fill out the survey.
While this survey did not have non-response bias, we do need to watch out for people giving inaccurate answers. Numerous studies have found that people will often over report consumption of things they think they are supposed eat more of (such as fruits and vegetables) and under report how much they eat things they are supposed to eat less off (like red meat.) Researchers call this “social desirability bias.”
Students in this study may have guessed that the survey takers wanted them to answer that they had eaten less meat. As a result, some of those who said they changed their diet probably did not. Others may have over reported how much meat they cut out of their diet. This bias was probably highest among those who said they ate “a little less” of an animal product.
Thankfully, biased answers don’t account for all of the change reported in this study. How do we know? Because different groups reported very different rates of change. For example, people who were getting a leaflet for the first time reported almost 90% more change than those who had received leaflets in previous semesters. (Change here is indicated as number of animals spared by dietary change.)
Both groups should answer with about the same level of social desirability bias. So if all of the reported change was simply due to bias, then the dietary change reported should have been roughly the same whether or not people had received a leaflet before.
However, there were major differences in reported change between those who received leaflets before and those who hadn’t, between those who received one leaflet versus the other, and between students of different grade levels. Out of nine subgroups measured, the groups with the lowest amount of reported change spared 88 and 101 animals respectively. Even if we assume that 100% of that reported change is fraudulent and is merely the result of social desirability bias, it would suggest that for the other subgroups measured, the amount of actual change was the total number of animals helped minus approximately 90 animals (90 animals being the amount of animals not actually helped but reported due to social desirability bias). The remaining subgroups helped between approximately 115 and 250 animals per 100 leaflets (before accounting for social desirability bias).
The raw data, as a whole, once adjusted to be equally representative of all grade levels, suggests that 141 farm animals will be spared for every 100 leaflets distributed on a college campus. Considering that, and considering our estimation of a maximum social desirability bias of around 90 animals spared per 100 leaflets distributed, we can make a conservative estimate that, at a minimum, about 50 of those 141 animals reported to be spared were actually spared. In summary, for every 100 leaflets distributed, we can conservatively estimate that approximately 50 farm animals are spared each year from a lifetime of misery.
The actual number is almost surely higher. And the conservative estimate is significantly higher among certain subgroups. For example, the conservative estimate for students who never before received a leaflet and who now received a Something Better leaflet is roughly 150 animals spared for every 100 leaflets distributed after accounting for social desirability bias. And again, all of the numbers become dramatically higher once you include the multiple years a person maintains their change, the ripple effects as they spread their dietary change to others, and the number of wild fish spared.
One other area for possible inaccuracy in the results stems from the fact that the survey chose not to define what “more,” “a little less,” or “a lot less” meant. Rather than have respondents indicate a percentage change in consumption of each product, we simply assigned estimated values to each change. While we attempted to be conservative in our estimates of what each change meant, it’s possible that our estimates were out of line with what respondents meant.
If we discard all partial changes (a conservative step, since at least six times as many people report reducing each product as report increasing it), look only at those who stopped eating products entirely, and account for social desirability bias, we can still estimate that for every 100 leaflets distributed a conservative minimum of 20 farm animals are spared per year from a lifetime of misery by individuals who have completely removed a product from their diet. The actual number is surely higher than this, and it would be inaccurate to use this as a best estimate, but we can view this as a minimum bound for the amount of change produced just by those who have completely eliminated one or more products: 1 farm animal spared per year for every 5 leaflets distributed.
For more details, additional spreadsheets of results, questions about methodology, or anything else, feel free to email me at ncooney@farmsanctuary.org .
Want to receive blog updates twice a month? Join the Compassionate Communities Campaign to get them delivered straight to your inbox.
Imagine you’re standing in a dining room before a massive table set with 100 plates. Spread on the plates are all the chickens, cows, and pigs an average American consumes in one year. Americans eat a lot of meat, so the plates are piled high with animal flesh. 
