Randomness, Sex and the Survey

A little understood issue is randomness in determining either public opinion or treatment effects.

The India Today, November 13 issue "Secret Desires" uses an invalid method to generalize the survey research for their section on male sexuality "Men in a Muddle." If one goes by the method stated on their webpage, they use street-corner sampling which is basically, convenience sampling and not random sampling. Only random sampling can give rigorous generalizibility. Random sampling would imply that everyone in the target population has to have an equal chance of being selected to participate in the survey or at least a known chance of being selected. Even if we think of their technique as purposive sampling, it still is not generalizable.because they form nonrandom/nonprobabilistic survey methods.

For a succint understanding of why convenience sampling or purposive sampling wont be appropriate read Populations and Samples.

Sample a couple of passages in the article.

In attitudes and values, we stumble across the great hypocrisy reef of India. The modern Indian man-as many as 80 per cent in Hyderabad and Chennai-expects the woman he marries to be a virgin. ... The survey probes further, trying to understand where the sex appeal of a woman lies. The answers go beyond the predictable notion of beauty and identify other factors like intelligence and sexual skills. Chennai and Hyderabad produce a smacker with 12 per cent identifying sex appeal with submissiveness. Small deviations can be seen in Ahmedabad and Ludhiana, where 32 per cent and 40 per cent, respectively, are ready to date or marry a much older woman. ... Most men confess to some homosexual experience-37 per cent have had at least one such experience-but rarely talk about it. ... Most are happy with their sexual experiences but feel upset about sex when it gets repetitive with the same woman or if the woman shows emotional expectations. A large percentage feel they can impose their need for sex regardless of their girlfriend's mood. Men are content with penis size and insist that a majority of their women have orgasms. A large fraction seems to have engaged in group sex and 11 per cent seem bisexual, with Hyderabad revealing a high of 24 per cent. Men prefer long foreplay with different towns itemising different priorities. A happy unity in diversity, not really kinky, but may be a bit too content about performance.

One cannot generalize the results to the target population (the young urban Indian male or even the young urban Indian males for the particular city) based on their convenience sampling method. I am not sure if there are any statisticians at India Today or reporters who understand statistics or can understand what ORG-MARG-AC-Nielsen has been briefing them. Hence the possible mismatch between the performed research and the narrative theme. This is exploratory research at best. Even if they employed highly rigorous methods to make their street-sampling random, all their estimates need caveats like response rate, sampling error and confidence intervals of the stated percentages.

Definitely convenience or purposive sampling is hugely cost-effective but no way generalizable, not one of those stated percentages in the article. The article provides extremely biased information to its readers because it uses non-random sampling methods to provide generalizable findings. Admittedly we are still in an infancy stage in terms of high-quality conducting and reporting survey research but a respected mag like India Today can take the lead on these issues.