The extent to which a study's design, execution, and analysis prevents bias. Scientific validity allows researchers to judge how much confidence they have in their research findings being true.
What is validity?
Validity is the extent to which a study’s design, execution, and analysis prevents bias. Scientific validity allows researchers to judge the level of confidence they have in their research findings being true. There are three types of validity to consider:
- Internal validity – achieved through methodological rigor when designing and conducting the research project. The internal validity of a study can be affected by systematic error and random error. Items like randomization and blinding decrease systematic error and can increase internal validity. Adequate sample sizes can reduce random error increase precision of a study’s results
- External validity – achieved when the study findings can be replicated a different time or onto other populations (i.e. other ages, sex, species of animals), or by other researchers, in another lab.
- Construct/Translational validity – achieved when the animal disease model accurately reflects the clinical disease that it is meant to represent.
Who should concern themselves with validity?
All researchers should consider all three types of validity in their studies.
When should validity be taken into consideration?
Validity should be considered throughout the study, from the design phase to the analysis. Methods used to address validity should be planned in advance. Additionally, validity should be considered when reporting a study’s findings.
Why is validity important?
Without considering validity throughout the study, researchers are at risk of systematically biasing the research findings and may result in the model lacking clinical impact, thereby contributing to the current reproducibility crisis.
To achieve good internal validity:
Refer to the NIH preclinical reporting guidelines to ensure that all 7 items are met within the study design. For example, appropriate randomization and blinding techniques are planned.
Implement randomization in your experimental design.
Implement blinding in your experimental design.
Do a sample size calculation to help ensure your study is adequately powered.
Account for biological and technical replicates for each outcome.
Record all procedure details and protocol deviations.
More information on how to implement these methods to increase internal validity can be found on this website, along with the ARRIVE (Animal Research: Reporting of In Vivo Experiments) guidelines, and the SYRCLE Risk of Bias tool.
To achieve suboptimal internal validity:
- Do not implement procedures such as randomization, blinding, and sample size calculations into the design of your experiment
- Fail to account for and report protocol deviations
Perform the experiment multiple times, but only publishing results from the studies that show favorable outcomes
Resources and Tools
Publications outlining importance:
- Annette M. O’Connor, Jan M. Sargeant, Critical Appraisal of Studies Using Laboratory Animal Models, ILAR Journal, Volume 55, Issue 3, 2014, Pages 405–417.
- Henderson, V. C., Kimmelman, J., Fergusson, D., Grimshaw, J. M., & Hackam, D. G. (2013). Threats to validity in the design and conduct of preclinical efficacy studies: a systematic review of guidelines for in vivo animal experiments. PLoS medicine, 10(7), e1001489.