5 Things you must not forget when reporting psychological experiments

Reporting in experimental psychology has considerably advanced in the last two decades, especially with the publication of the American Psychological Association’s Reporting Standards for Research in Psychology.

Below, we offer some suggestions on aspects of experimental research that are often under-reported or inadequately described, yet very important, especially from the perspective of making the study reproducible.

1. Report whether your participants were left- or right-handed, neurologically normal, taking any antipsychotic medications, and had normal or corrected-to-normal vision. Depending on the experimental protocol, one or more of these factors can affect your results, so it is a good idea to inform your readers about them beforehand.

2. Report how participants were assigned to the experimental and control groups (randomization, matching, etc.).

3. If trials were conducted in blocks, give the number of trials in each block as well as the total number of blocks. Make sure you state which blocks were training blocks and which were actual test blocks. Report both the interstimulus interval and intertrial interval, and if the order of trials was counterbalanced across different participant groups, describe the counterbalancing method (e.g., Latin square design).

4. Describe the apparatus used to present stimuli in complete detail (including model and supplier information). For instance, if stimuli are visually presented, report screen dimensions, luminance, refresh rate, distance from participant to screen, and visual angle.

5. Specify how data were summarized. For example, response/reaction times could be summarized as both means (standard deviations) and medians (interquartile ranges), so make sure your readers understand your data analysis procedures, including how you dealt with outliers if any.

Of course, depending on the type of paper you are submitting (e.g., single-experiment vs. multi-experiment, brief report vs. full paper), you may find it difficult to include all the above information within the journal’s word limit. In such cases, it’s a good idea to check previously published papers to see the amount of detail reported. Further, journals like Psychological Science and the Journal of Experimental Social Psychology allow some methodological details to be reported in the supplementary materials section, which is published online only. But in general, when reporting methodology, it is better to provide too much information, rather than too little.

You might also be interested in reading about Publication and reporting biases and how they impact publication of research.