{"id":28511,"date":"2022-12-07T12:29:49","date_gmt":"2022-12-07T12:29:49","guid":{"rendered":"https:\/\/news.cancerresearchuk.org\/?p=25015"},"modified":"2024-11-03T15:13:11","modified_gmt":"2024-11-03T15:13:11","slug":"research-with-integrity-the-madness-of-short-methods","status":"publish","type":"post","link":"https:\/\/ogkologos.com\/en\/research-with-integrity-the-madness-of-short-methods\/","title":{"rendered":"Research with Integrity \u2013 the madness of short methods"},"content":{"rendered":"
\n

The next in our series focussing on research integrity finds Dr Andrew Porter exploring the dangers of publishing with a limited methods section, asking how we can reverse this trend and, incredibly, taking a lesson from the Great British Bake Off\u2026<\/em><\/p>\n<\/div>\n

<\/div>\n

This may strike you as an odd way to start a column about research integrity \u2013 but I think there is something interesting to learn about communication of research methods from The Great British Bake Off\u2026<\/p>\n

The competition features a round called the \u2018technical challenge\u2019. The contestants are given all the ingredients they need to produce a classic bakery product, with the twist that many of the steps have been removed from the recipe. A typical instruction is \u201cmake the dough\u201d. It\u2019s not how recipes normally work \u2013 removing this information tests the baker\u2019s knowledge and instinct, and they often do amazingly well. The details are hidden for entertainment \u2013 in real life you\u2019d be very frustrated if you picked up a recipe book so light on detail.<\/p>\n

But that\u2019s not dissimilar to how many scientists write the method sections of their research papers. Long, complex and technically challenging processes are frequently reduced to short statements. \u201cSamples were prepared according to standard procedures\u201d is a common one \u2013 or \u201cImages were acquired using a confocal microscope\u201d. These statements \u2013 while true \u2013 provide very little help to anyone looking to either understand what the authors have done or to attempt the same experiments themselves.<\/p>\n

\n

\u201cTo write out an experiment as it actually occurred<\/em> almost feels like breaking a taboo.\u201d<\/p>\n<\/blockquote>\n

So why are method sections so often like this? Why are they the equivalent of \u201cmake the dough\u201d?<\/p>\n

I think there are several reasons, and it\u2019s not generally because scientists are trying to hide anything.<\/p>\n

Precedent and space<\/h2>\n

Journals often have strict guidelines in terms of space and word limits. Historically these are linked to printing processes and the cost of distributing and storing information in physical form. While publications have largely moved online, requirements for authors \u2013 such as maximum word limits for method sections \u2013 often remain. This hangover from an age of paper has a restrictive effect on how much detail can be added.<\/p>\n

And that simple issue has an important knock-on effect. Scientists learn to be authors by reading what has gone before and acquainting themselves with the literature. It\u2019s clear then, that researchers pick up the unspoken message that method sections should be short, and minimal on detail.<\/p>\n

Authors often assume that others know their work as well as they do, so adding in lots of detail can feel like stating the obvious. And this is reinforced by reading traditionally short methods in the literature. To write out an experiment as it actually occurred<\/em> almost feels like breaking a taboo.<\/p>\n

Another fine mess<\/h2>\n

Another issue is that science is messy. Sometimes in a Bake Off technical challenge a process has gone so badly that the baker scraps their first attempt and starts again. It\u2019s often the same in research, but papers often present the perfect scenario in their methods, and leave out the trial-and-error. Details of repeats are often excluded because they didn\u2019t work out, or information revealing the degree of uncertainty in the analysis process is scant.<\/p>\n

Again, if researchers don\u2019t see these in the literature, it\u2019s a very reasonable assumption that they are not supposed to be there. Researchers do share this information, but usually in informal spaces like conferences or scientific meetings, through social media or by personal communications. However, these tend to be hidden from the wider scientific community.<\/p>\n

This all has consequences. In the recent reports from the Cancer Reproducibility Project<\/a> in eLife, a major barrier to even attempting<\/em> to replicate a published experiment was the lack of detail in the original publications. This multi-year project attempted to repeat 193 experiments selected from 53 highly cited papers published between 2010 and 2012. At the end of the project, the team was only able to repeat 50 experiments from 23 papers, and many could not even be started due to a lack of information. When approached, in around half the cases researchers were often able to add important \u2013 and sometimes essential \u2013 information not included in the published manuscript.<\/p>\n

Method in the madness<\/h2>\n

How might we begin to address this issue? During the pre-submission checks at the CRUK Manchester Institute \u2013 very similar to those Catherine described last month<\/a> at the CRUK Beatson Institute \u2013 I read the methods of each paper in detail, and encourage the authors to include more relevant information wherever possible.<\/p>\n

What about page limits and space restrictions? There are other ways to include more detail, such as including supplemental methods in an online-only part of the paper. Here the authors are free to add as much detail as they wish, and even if the journal doesn\u2019t explicitly ask for it, this is a great way to increase the reproducibility and transparency of the work. It can even include videos of a process \u2013 capturing the steps for preparing a sample using a smartphone, for instance.<\/p>\n

\n

\u201cIf you read someone\u2019s work, and there\u2019s something you don\u2019t understand about their process, ask them to add some clarification.\u201d<\/p>\n<\/blockquote>\n

If this isn\u2019t an option, platforms such as Protocols.io<\/a> can help. Full, detailed methods can be published on the platform, stored securely in a freely accessible way, and given a DOI (digital online identifier) allowing them to be cited. Versions of the protocol can be updated and tracked over time, and I\u2019d argue giving the method its own space can add more visibility to your particular approach.<\/p>\n

Finally, everyone can play a part in normalising the inclusion of longer, more detailed methods. If a colleague asks you to look over a manuscript, give them feedback on whether or not you\u2019d be confident repeating their experiments. If you read someone\u2019s work, and there\u2019s something you don\u2019t understand about their process, ask them to add some clarification.<\/p>\n

And don\u2019t be afraid to include the gory details of your method \u2013 there will undoubtedly be someone out there who will thank you for them!<\/p>\n

\n

Takeaway tips<\/h2>\n