Q: How should I respond to a reviewer's comment regarding selection bias?
I have received a reviewer's comment that states: "Only 750 subjects out of 23,188 subjects remained. Why did so few people answer the questions? There may be a huge selection bias here."
How should I respond to this? Is it enough if I explain why there were so many people who didn't answer the questions? Or should I show that there was no selection bias, and if so, is there a statistical method that I can represent to prove there was no selection bias?
A response to any methodological comment by the reviewer should be considered serious and responded with adequate clarity. Considering the reviewer has raised a suspicion towards the selection bias, we suggest you first explain the reason behind people not answering the questionnaire. If the reason in itself is strong and scientifically sound, it will not be necessary to conduct a statistical analysis to overcome the selection bias. However, if you believe that your reason is not strong, a statistical test like Mann-Whitney U test can be used. It is important to consider the data distribution in your study for deciding the correct statistical test.
Related reading:
This content belongs to the Journal submission & peer review Stage