This is the last in this series of short guides which we hope provide some guidance in relation to statistical issues researchers may encounter when conducting research, audit or indeed quality improvement projects. Here we focus on an issue that is often overlooked within statistical textbooks, with a couple of notable exceptions [1,2].
When you perform an analysis within a statistical package, the output can be bewildering, and it is difficult to know what of this output should be included within a manuscript. Fortunately, there are now publication standards to assist researchers in making such decisions; CONSORT for clinical trials, PRISMA for meta-analyses and STARD for diagnostic accuracy studies being some of the better known standards. These are housed within the EQUATOR network – a resource which all new to research are encouraged to visit [3]. It is a library of publication standards which are freely available and are updated over time.
New guidelines are added as time passes and researchers can use the search facility to identify whether there are publication standards for whatever type of study is being conducted. We would strongly advise visiting these in advance of conducting your study – at the study design stage, finding out you have not conducted an essential item after you have done the study is too late. If you visit these pages and search for statistics, you will come across the SAMPL guidelines [4]. This is a readable paper which gives very clear guidance as to what should be reported in relation to any statistical analysis conducted.
The first principle behind these guidelines is that you describe the methods you have used in sufficient detail to allow a knowledgeable reader to verify the reported results.
Indeed, journals nowadays may request that you submit a dataset along with a paper to allow replication of results reported. The second principle is that you should report enough information to allow your results to be incorporated into other analyses. This is particularly important since meta-analyses have become more common and a valid meta-analysis will require core data to be reported in a paper. A systematic review, for example, whose primary analysis is a random effect estimate of a mean difference may require means and standard deviations to be reported for all papers which contribute to the meta-analysis. It is not uncommon to find such information missing and whilst efforts are made to obtain information from authors these are often not fruitful.
The SAMPL guidelines cover general principles such as reporting the total sample and group sizes for each analysis (In Ophthalmology this would infer, the total number of patients and eyes and the number of patients with one or more eyes) to more advanced statistical methods such as time to event analyses (confirming that assumptions are adhered to) and Bayesian analyses (how were priors determined and what software has been used).
Two textbooks which are very helpful in relation to guiding researchers as to what to report are those by Altman and Peacock [1,2]. Altman includes a section on presentation within each chapter and Peacock illustrates output from three commonly used statistical packages (R, SPSS and Stata) and shows what to extract for a report.
“Always remember the SAMPL guidelines when writing your research – a simple way to keep the statistical reviewer happy”
Whilst it can be helpful to review previous papers to assess what should and should not be included within a manuscript, it is important to note that the SAMPL guidelines were first published in 2015 and that these were developed in response (in part) to evidence of poor statistical reporting. Even papers previously reported in high impact journals may not adhere to these guiding principles. An example of such is the study by Wakefield et al. which led in part to the reduction in the uptake of the MMR vaccine. The numbers of children included in the study reported is unclear despite its publicity [5].
A systematic review of trials (unpublished) highlighted that often it was not possible to determine whether papers were reporting results relating to single eyes from single patients or both eyes from all patients or a combination of these. Without such information it is not possible to determine whether or not the statistical analysis has been conducted appropriately and thus these results should be treated with caution.
Speaking to trainees, the importance of research seems to generate different perspectives. Some find research a burden. Sayings such as ‘publish or perish’ do not help the general mood and it seems research is something required for the CV. What must be agreed is that all research should be carried out at the highest standards. If you are not interested in research, an understanding of research methodology is nevertheless important. Much of career progression beyond medical school will rely upon assessing publications and knowing the good from the less good.
There are many other potential roles in research such as critical appraisal of literature both independently and as a journal reviewer. We would encourage all researchers to look at the EQUATOR website. The toolkits section is useful, providing practical help on issues such as writing research, selecting the appropriate reports, peer reviewing research etc [6]. To supplement your understanding of statistics ‘The essential concepts of statistics’ is a summary of concepts relevant to all [7].
Previous learning
- We hope you have learnt about how to approach the subject from the series. A brief summary of these articles is summarised below.
– Part 1: An introduction to data – how do we classify it and why does it matter?
We established there are different ways of classifying data and that these can guide future statistical testing.
– Part 2: How to summarise your data and why it’s a good idea to do so.
In this article we showed how you start bringing your data to life. Deciding if it is quantitative or categorical can identify which descriptive techniques are best suited.
– Part 3: Which statistical test should I use (if any)?
How to choose and use statistical tests. Drawing attention to the earlier articles.
Current learning
- Even if you are not interested in research, an understanding of research methodology is nevertheless important to allow your practice to incorporate innovations which are evidence based.
USEFUL RESOURCES
-
Presenting Medical Statistics’ book website: http://medical-statistics.info
-
NIHR Statistics Group: https://statistics-group.nihr.ac.uk
References
1. Altman DG: Practical Statistics for Medical Research. Chapman & Hall/CRC; 2006.
2. Peacock J, Kerry SM, Balise RR: Presenting Medical Statistics from Proposal to Publication. Second Edition. Oxford University Press; 2017.
3. Bunce C. Avoiding experimental death: the EQUATOR network (a valuable resource for research). Ophthalmic Physiol Opt 2017;37(4):367-9.
4. Lang TA, Altman DG. Basic statistical reporting for articles published in biomedical journals: the “Statistical Analyses and Methods in the Published Literature” or the SAMPL Guidelines. Int J Nurs Stud 2015;52(1):5-9.
5. Wakefield AJ, Murch SH, Anthony A, et al. Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet 1998;351(9103):637-41. Erratum in: Lancet 2004;363(9411):750. Retraction in: Lancet 2010;375(9713):445.
6. EQUATOR network.
http://www.equator-network.org/toolkits/
Last accessed August 2019.
7. Motulsky HJ. GraphPad Statistics Guide 2016
http://www.graphpad.com/guides/
prism/7/statistics/index.htm
Last accessed August 2019.
COMMENTS ARE WELCOME
