In a new series focussing on research integrity, two of our experts will dig into the issues and challenges of this incredibly important aspect of research. In this first instalment, Dr Catherine Winchester explores the importance of the pre-submission review…
We all appreciate that peer review is an invaluable process for evaluating the quality, validity and context of research articles. Yet we continue to see papers having expressions of concern, being corrected or even retracted with platforms such as PubPeer being used to critique them. Why is this?
That’s a complex question to answer. While analysis of data in the Retraction Watch database suggests that deliberate misconduct accounts for approximately 60% of retractions, in fact honest mistakes, errors and problems with data reproducibility are often found to be the cause. This is where research integrity experts can really help. As guardian of research integrity at the CRUK Beatson Institute I review manuscripts prior to submission to journals.
Catching the mistakes
So what, you might ask, is pre-submission manuscript review? And why is it necessary?
The first thing to say is that it is not peer review, neither is it about checking a particular journal’s formatting instructions or editorial policies. The goal is to raise the standards of scientific communication and help publish accurate research articles, containing data that are trustworthy and reproducible. I minimise errors as well as guarding against misconduct.
This is no small task, in 2021 I conducted pre-submission integrity reviews on 71 research articles and checked 35 reviews and 16 PhD theses for text plagiarism.
“Essentially, I am looking for transparent reporting of the entire research process so that papers can be understood and evaluated by non-specialist readers as well as researchers working in a particular field.”
Some of this work involves checking things that are peripheral to the actual research itself. I review research manuscripts for text plagiarism, check authors’ affiliations, funding acknowledgements, authors’ contributions and data availability statements. You might not think that these necessarily fall under the remit of a manuscript integrity review, but they are intrinsic to the core values of research integrity of openness, accountability, care and respect, as defined by the concordat to support research integrity.
However, the crux of my pre-submission review is to evaluate the integrity of the research and to encourage best practises in open and transparent reporting. I check manuscripts for accuracy, adherence to standards, reproducibility and objective reporting and, where necessary, advise and support researchers on how to achieve these.
I read the abstract and introduction for clarity and coherency, then check the methods for detail and specificity, before scrutinising the results text and checking it aligns with the figures and figure legends. Finally, I inspect the figures for style consistency, inappropriate manipulation or duplication. I tend not to read the discussion as I think contextualising the findings of a manuscript is better served by the authors and peer reviewers. The key to any review process is consistency. I use an integrity framework based on common issues in biomedical research, journal reporting requirements, current best practices and field standards.
Do researchers actually find this useful?
Martin Bushell, Deputy Director and Senior Group Leader at the CRUK Beatson Institute, certainly thinks so:
“The internal review of manuscripts pre-submission within the Beatson Institute is an amazing process allowing an independent set of eyes to examine the manuscript as a whole.
The process is conducted in a non-judgemental way, which is critical and truly constructive feedback is always given. Importantly this process allows interception at an early stage of problems within any manuscript before submission, which have accidentally been incorporated over time and to which the authors have become blind to.
This process adds an important extra layer of assessment which strengthens all manuscripts that are taken through this process.”
What do I look for in a manuscript?
Essentially, I am looking for transparent reporting of the entire research process so that papers can be understood and evaluated by non-specialist readers as well as researchers working in a particular field.
When it comes to integrity and reproducibility, I specifically look for data and figure descriptions that enable readers to determine exactly what data are being presented – are they technical replicates, biological replicates or data from independent experiments? At the Beatson Institute, we encourage authors to present their data transparently using graphs such as SuperPlots rather than condensing information in bar graphs. I check that methods are comprehensive and complete, and include information on reagents, computer code and statistics.
Statistics alone are, of course, not sufficient. During my review I question which statistical tests have been employed and tie this in with statements on significance and impact, so that authors consider biological significance as well as statistical significance. Overall, the aim is to make sure descriptions are clear, unambiguous and not too biased.
Reporting back to researchers is important and I try to do this in a sensitive, informative and instructive manner. I am not here to criticise authors’ research or their manuscripts, instead my aim is to support them in publishing to the highest standards of scientific rigour.
It is often said that I am picky, but this attention to detail can be useful as an objective fresh pair of eyes can spot mistakes or inconsistencies that an author immersed in a paper can no longer see.
– Remember integrity professionals are not the ‘science police’! This is all about helping researchers do the best research they can.
– The more robust you can make your paper prior to submission, the more likely it is to have a smoother peer-review process at the journal you submit it to.
– Be wise with your choice of statistical analysis.
– If you don’t have one, consider a research integrity toolbox. At the Beatson Institute we created a ‘Publishing at the Beatson’ page of the research integrity toolbox on our intranet for extra guidance, which includes a publishing checklist.
– If your laboratory or institute has access to research integrity experts – remember to use them! We are here to help you and I for one am always happy to discuss manuscripts with researchers at the Beatson.
In the next instalment of this series, Dr Andrew Porter gets into some of the integrity pitfalls you can stumble into when writing your methods section – and how to avoid them.
Dr Catherine Winchester is senior research adviser – grants and research integrity at the Cancer Research UK Beatson Institute
There are no tags