Surveyitis is a disease that afflicts people who stay indoors too long poring over data. It can be alleviated by fresh air and meeting people. Symptoms include credulity about the accuracy of survey responses and morbid attachment to outdated questions. It is particularly dangerous in relation to religion.
A recent upsurge in interest in "nones" suggests a new outbreak of surveyitis. Nones are those who declare on surveys that they have no religion or belief. The fact that their number has been rising is cited by humanists and other there-is-no-God-botherers as proof of the demise of religion. A new report from the thinktank Theos points out that the "nones" encompass "nevers" who don't participate, "atheists" who don't believe, and the "non-religious" who don't belong to a particular religion, and that a significant percentage retain some religious beliefs and practices. The report concludes that we are dealing with shades of grey rather than black or white religion or secularity. Fair enough. But only surveyitis could have led to the idea that a bald religion question could tell us anything useful in the first place.
When it comes to surveys, the simpler, more concrete and less emotionally freighted a question the better. "Did you eat an egg for breakfast?" is good and "What party did you just vote for?" is OK, so long as it's anonymous. The problem with religion is that there aren't many questions like this. "Did you go to church last Sunday?" might seem fairly straightforward, but when American researchers counted cars in church parking lots they found that conservative Christians massively exaggerated their actual attendance. That's because it mattered to them – and the more an issue means to people, the more difficult it is to get simple and reliable answers.
It's even worse when questions are also vague and contested. Anything with the word "religion" in it falls into this category. No one has ever been able to agree a definition of religion, nor will they. So it's amazing to expect that survey respondents will miraculously understand the word, and mean the same thing. Even the smallest of changes to the wording of a religion question can make a significant difference to the responses. The Office for National Statistics demonstrated this beautifully when it experimented with small changes to the question on religion in preparation for the 2011 census. A new report from the Centre for Longitudinal Studies suggests with some exasperation that "a quarter of responses to any question on religion are unreliable".
A further difficulty is that religion is becoming a toxic word. It suggests something dogmatic and ritualistic. Even conservative Christians often disassociate from it and prefer the word "spiritual". This annoys "alternative" people who abandoned religion for spirituality a long time ago, and now have to find another word.
But religion is ever-changing, and it is now something quite different from when most survey questions about it were devised. As with politics, few people now take over a total identity and set of beliefs from their parents. Identities and commitments mix many elements, both religious and secular. Surveyitis concludes that people have become muddled, when the reality is that surveys are working with zombie categories. There is an urgent need to devise new ways of capturing people's life-shaping commitments in ways that take account of at least four dimensions: what they believe, what collectives and causes they identify with, what they do, what they consider good and evil.
The other issue that needs to be faced is that people are not always honest, not always consistent, not always transparent to themselves, and not always certain. When I was involved in designing a national survey we expanded the question on belief by asking not just if people believed in God, a spirit, life force, or none of the above, but also whether they believed "there is something there" or "I don't really know what to think". 23% said yes to the former, and 12% to the latter.
Doubt, subtlety, uncertainly and cognitive modesty are important and appropriate in relation to the big questions of life. To believe otherwise is to be seduced by a fantasy rational man with clear and distinct ideas. He lurks behind many survey designs and is undoubtedly behind the idea that asking people whether they are religious or not is a useful exercise. That's surveyitis masquerading as hard science. But in truth, good social science is harder still.