Q: In my pilot for assessing the influence of different instructional methods on learning, can I switch the questions from Test A to Test B?
I am researching the influence of different instructional methods on the effectiveness, efficacy, and transfer of learning. I have created material for learning chess based on two instructional approaches. To test which one is better, I have created two versions of the test (A and B) and made them as similar as possible. Both have the same number of tactical positions; even the number of figures on the board are quite similar. In the pilot, I found that A is a bit easier. If I switch five questions from A to B, they will be practically identical. Am I allowed to do this without conducting one more pilot?
Your field (you come across as a chess player/professional) and your research are both interesting. Also, thanks for all the details. To answer your question, if the questions are to be practically the same for the two tests in terms of content and ordering, it wouldn’t help to conduct an additional pilot. If however, the ordering and some details are different, then you may consider conducting an additional pilot to substantiate your conjecture.
Hope that helps. In case you are seeking help to an even more professional extent, you may also wish to learn about our Experimental Design and Statistical Review services as below.
This content belongs to the Manuscript Writing Stage