{"id":27429,"date":"2021-09-08T08:30:14","date_gmt":"2021-09-08T08:30:14","guid":{"rendered":"https:\/\/news.cancerresearchuk.org\/?p=23518"},"modified":"2024-11-03T15:14:30","modified_gmt":"2024-11-03T15:14:30","slug":"speaking-a-thousand-words-how-a-cancer-image-collection-is-set-to-improve-ai-diagnosis","status":"publish","type":"post","link":"https:\/\/ogkologos.com\/en\/speaking-a-thousand-words-how-a-cancer-image-collection-is-set-to-improve-ai-diagnosis\/","title":{"rendered":"Speaking a thousand words \u2013 how a cancer image collection is set to improve AI diagnosis"},"content":{"rendered":"
\n

Artificial intelligence is set to revolutionise many aspects of cancer diagnosis and treatment \u2013 but an AI is only as good as the data used to train it. Here we find out how the OPTIMAM project\u2019s mammogram collection is heralding a new dawn of machine learning based cancer detection<\/em><\/strong><\/p>\n<\/div>\n

<\/div>\n

Artificial intelligence (AI) is on the brink of profoundly changing cancer diagnostics. Algorithms now exist that can detect abnormalities in X-rays, PET scans and histological samples as well as an expert human can. Such software holds the potential to speed processing times, cut medical costs and reduce human errors.<\/p>\n

And if algorithms emerge that can detect cancers earlier than these malignancies are perceptible to human observers, they could make screening efforts more effective.<\/p>\n

\n

\u201cWhilst the computer science behind these innovations is impressive when it comes to medical AI, getting hold of the data needed to develop such algorithms can often be a major challenge.\u201d<\/em><\/p>\n<\/blockquote>\n

Whilst the computer science behind these innovations is impressive when it comes to medical AI, getting hold of the data needed to develop such algorithms can often be a major challenge.<\/p>\n

In 2008, Cancer Research UK funded a project \u2013 OPTIMAM \u2013 aiming to improve the accuracy and efficiency of the UK\u2019s already very successful NHS Breast Screening Programme (NHSBSP).<\/p>\n

Over the initial five-year grant period, the OPTIMAM team, led by Professor Kenneth C. Young, collated a library of X-ray images with which to run studies that would establish the best protocols for acquiring mammograms and to identify some of the pitfalls in their interpretation. All of this to help the performance of human<\/em> radiologists.<\/p>\n

\u201cAI wasn\u2019t on the cards at all,\u201d says Professor Mark Halling-Brown, Head of Scientific Computing at the Royal Surrey NHS Foundation Trust, who now jointly manages the OPTIMAM database with Young.<\/p>\n

<\/p>\n

Dr Mark Halling-Brown is Head of Scientific Computing at the Royal Surrey County Hospital<\/p>\n<\/div>\n

As this work proceeded, however, it became increasingly clear that the future of breast cancer screening \u2013 and of diagnostics more broadly \u2013 would be transformed by AI technologies. Hence, Halling-Brown explains, when the OPTIMAM team applied for further support from CRUK in 2013, their application centred on creating a large database of mammogram images with which they could facilitate the development of such technologies.<\/p>\n

Fast forward eight years and the OPTIMAM database<\/a> is used by over 40 academic and commercial collaborators, including many of the world\u2019s leading medical AI companies.<\/p>\n

\u201cWhat makes OPTIMAM great is the team behind it,\u201d says Sarah Kerruish, Chief Strategy Officer of London-based Kheiron Medical Technologies<\/a>. \u201cIt\u2019s a very strong team of people who are really familiar with the nuances of what it takes to develop AI.\u201d The OPTIMAM team has also greatly benefited from the participation of leading UK breast radiologists who ensure the accuracy of the clinical data and its interpretation.<\/p>\n

<\/p>\n

Sarah Kerruish is Chief Strategy Officer of Kheiron Medical.<\/p>\n<\/div>\n

In fact, the database has proved so successful, OPTIMAM has inspired and guided the creation of a national database of chest X-rays<\/a> relating to COVID-19 and will likely shape the development of further nationwide databases.<\/p>\n

All of which highlights that researchers and clinicians should appreciate the value of the data they generate \u2013 for while the science of AI is astounding, it works best when it\u2019s built from well-designed data sets. Joe Day, business development executive for big data and AI at CRUK is keen that the research community not only recognise that, but begin to think about how they might act on it too. \u201cMany cancer researchers \u2013 and indeed those in the wider life science fields \u2013 will be collecting vast amounts of data,\u201d he says. \u201cBut it\u2019s often the case that teams generating the data won\u2019t realise just how useful it could be to other research efforts, both academic and commercial. The OPTIMAM project is a perfect example of how such data can be utilised by others, and I\u2019d urge any researchers who think their data could be useful to talk to CRUK\u2019s Commercial Partnerships team about how we can help maximise the impact of their research data.\u201d<\/p>\n

Why breast screening?<\/h2>\n

Early detection of breast cancers by screening is proven to reduce overall breast cancer mortality. For example, the NHSBSP \u2013 which invites all UK-based women aged 50-70 to attend for screening every three years \u2013 is estimated to save one woman\u2019s life for every 400 screened, cutting breast cancer deaths by 25%.<\/p>\n

Such screening programmes represent an obvious platform for AI applications. Mammograms \u2013 ie the algorithm\u2019s input data \u2013 have a fairly standardised format. And the initial decision a radiologist makes \u2013 ie the output an algorithm must replicate, and potentially improve upon \u2013 is a relatively straightforward binary call: does this person need to undergo further assessment?<\/p>\n

\n

\u201cBreast cancer detection is one of the hardest detection tasks, because breast tissue has many features that are confounding to the human eye, in terms of what maybe looks like a tumour but actually isn\u2019t.\u201d<\/em><\/p>\n<\/blockquote>\n

Moreover, programmes such as the NHSBSP generate huge volumes of data \u2013 currently, mammograms from over 2 million women, attending roughly 100 local screening sites, annually. And large volumes of data are what researchers need to train, then test, machine learning systems.<\/p>\n

Finally, to execute a large-scale screen as effectively as the NHS does demands significant time, money and resources. \u201cBreast cancer detection is one of the hardest detection tasks,\u201d says Kerruish, \u201cbecause breast tissue has many features that are confounding to the human eye, in terms of what maybe looks like a tumour but actually isn\u2019t. Only when you have a lot of experience as a radiologist or radiographer, would you be able to interpret that image.\u201d<\/p>\n

In the UK \u2013 to achieve maximal accuracy \u2013 every mammogram is assessed by two expert radiologists or specially trained clinical staff. Only, if they agree, is their decision actioned; if they don\u2019t, the mammogram is sent to two further radiologists for arbitration. There is clear potential then for automated systems to cut costs and processing times by standing in for human radiologists.<\/p>\n

Such technology is exactly what Kheiron is developing. In the UK, Kerruish says, \u201cit could automate up to 45% of that workflow. It\u2019s a combination of human plus machine intelligence that we think is going to deliver the best solution for patients. It\u2019s never about replacing radiologists; it\u2019s about giving them better tools.\u201d<\/p>\n

In addition, new computational tools may aid screening programmes in other ways. For instance, New Zealand-based Volpara Health<\/a> has developed software that processes mammograms to compute a volumetric breast density score. With breast density being a major risk factor for breast cancer, further Volpara algorithms will incorporate this score with other types of patient information \u2013 such as age and family history \u2013 in models to estimate an individual\u2019s breast cancer risk.<\/p>\n

<\/p>\n

Ralph Highnam is Chief Executive Officer and Executive Director of Voplara Health.<\/p>\n<\/div>\n

Volpara\u2019s breast density scoring software was pivotal to the DENSE trial \u2013 a ten-year randomised control study<\/a> done by researchers in the Netherlands \u2013 which showed that if woman with high density breast tissue were referred for further MRI scans, breast cancer detection rates increased significantly.<\/p>\n

\u201cAs we\u2019ve gone along,\u201d says CEO and founder Ralph Highnam, \u201cwe\u2019ve realised the impact of mammography quality. If you can put good images in, you get good density scores out.\u201d<\/p>\n

His team are now working to develop automated methods for ensuring high quality mammogram acquisition processes and real-time methods for assessing if individual mammograms are of sufficiently high quality or whether they should be re-taken. They\u2019re also seeking to develop better systems for tracking women\u2019s breasts over time, in order to determine if this allows better prediction of cancer risk. And for both projects they collaborate with OPTIMAM.<\/p>\n

From project to database<\/h2>\n

All of these types of software were merely distant pipe dreams when the OPTIMAM project was instigated in 2008 by Professor Kenneth Young, a medical physicist and Head of Research at the National Coordinating Centre for the Physics of Mammography, based at The Royal Surrey. \u201cIt was more about the fundamental physics of the image formation process,\u201d says Halling-Brown about the original plan. \u201cThings like image processing, image display, lighting conditions, reading conditions, different manufacturers, different detectors, how X ray dose affects the image formation and how these factors make different types of cancers appear differently \u2013 and whether these affect long term outcomes for women.\u201d<\/p>\n

<\/p>\n

Professor Kenneth C. Young is Head of Research in the Medical Physics Department of the Royal Surrey County Hospital.<\/p>\n<\/div>\n

Doing this was necessary because despite the NHSBSP being a national programme, each screening centre has certain freedoms to decide exactly how they operated. Getting to grips with how these factors affected human radiologists\u2019 ability to detect cancers helped refine screening methods \u2013 but it also critically informed the process of pivoting OPTIMAM from being a research project to being a database that could underpin the development of AI image analysis technologies.<\/p>\n

To create an algorithm that can detect cancerous growths on X-rays of breast tissue, developers need mammograms for two purposes. First, their machine learning algorithms must be exposed to many, many examples of the data they are to analyse \u2013 here, a mixture of mammograms with and without cancers. Second, once developers have honed their software, it needs to be tested on a set of mammograms it\u2019s never seen before. Its success rate \u2013 in terms of both false-positives and false-negatives \u2013 can then be compared to radiologists\u2019 performances.<\/p>\n

The database Young, Halling-Brown and team envisaged was a high-quality repository of images from multiple screening sites, with each mammogram annotated as fully as possible with clinical and technical details. The person from whom each mammogram came would be completely de-identified at the site of acquisition. And it would be a live database \u2013 new images being constantly added, including repeat mammograms for the same individual, whether from the next routine screening visit or owing to that individual developing breast cancer in the interim.<\/p>\n

Another key feature was that OPTIMAM would make its data available to anyone who had a sound reason for using them. \u201cWe have a \u2018mini-grant application\u2019 data access process,\u201d says Halling-Brown, with interested parties outlining their credentials and what they will do with the data. Academic collaborators pay a small administrative fee, while commercial parties pay what\u2019s deemed a fair price \u2013 a process that\u2019s overseen by CRUK\u2019s legal and commercialisation team. A proportion of this income is shared with the centres which provide data.<\/p>\n

The way OPTIMAM designed their database made them an attractive proposition to work with. The way they shared it gave OPTIMAM a unique \u2013 and important \u2013 position in the Wild West of medical data sourcing by AI companies.<\/p>\n

An enviable dataset<\/h2>\n

At first, OPTIMAM gathered data from three centres, meaning its always incorporated data acquired according to the nuances of those three sites. But it has been expanding and diversifying across the UK since 2019. It now collects data from five sites, with four new sites currently joining and four more invited to do so. At the last publicly disclosed count, the OPTIMAM data set contained over 3.3 million images taken from nearly 170,000 clients. Over 150,000 of those clients were cancer free, roughly 8,500 had malignant tumours, nearly 5,000 had benign growths and about 2,000 developed cancers between screens.<\/p>\n

\n

\u201cThe OPTIMAM database<\/a> is used by over 40 academic and commercial collaborators, including many of the world\u2019s leading medical AI companies.\u201d<\/em><\/p>\n<\/blockquote>\n

But it\u2019s not just the size that makes the database so attractive. \u201cIf you look at the other data sets people are publishing on in mammography,\u201d says Lester Litchfield, a Data and Science Manager at Volpara, \u201cthey\u2019re often: one, not as large, and, two, quite old and quite limited in their availability of annotation.\u201d<\/p>\n

\u201cThe OPTIMAM data set is nicely curated and controlled, and they have a lot of the clinical data recorded along with the images\u201d adds Volpara\u2019s Highnam. He also praises the fact that OPTIMAM collects the raw imaging data and not just processed images, which can vary significantly according to choices made by device manufacturers and radiologists.<\/p>\n

For Volpara\u2019s goal of developing technologies that track breast health over time, Litchfield stresses the value of OPTIMAM\u2019s continuous collection of images permitting them to gather more and more evolving cases \u2013 including ones where women develop cancer during the time she\u2019s been screened. \u201cMost academic data sets are frozen snapshots in time,\u201d he says.<\/p>\n

Kheiron\u2019s Kerruish also applauds the structure of the OPTIMAM dataset but emphasises too its diversity.<\/p>\n

Diversity is essential for the ultimate effectiveness of the end product \u2013 small, homogeneous data sets can present serious problems because they can result in algorithms that won\u2019t function across the general population.<\/p>\n

Automated readers, Halling-Brown says, are considerably more prone to being tripped up by variations between sets of data than human readers are. \u201cAnd an algorithm doesn\u2019t necessarily know when it doesn\u2019t know,\u201d he says.<\/p>\n

Consequently, he explains: \u201cIf you rely on AI vendors going out and trying to find their own data from individual hospitals, all they will do is produce an algorithm which is really good at detecting cancers in that hospital\u2019s population.\u201d<\/p>\n

The challenge of generalisability affects all of AI research but in medical diagnostics it will risk lives. Systematic differences likely to bias a screening algorithm in fundamental ways can result from all those technical factors that the initial phase of the OPTIMAM project investigated \u2013 X-ray machine manufacturer, radiation dose, image processing, etc. But they can also arise from different hospitals serving different demographics.<\/p>\n

It\u2019s now well-recognised that ethnic groups vary in terms of breast size, density and other subtle features which influence algorithm performance. Consequently, an algorithm trained mainly on one group may perform poorly in another.<\/p>\n

This is why Kerruish is happy that OPTIMAM includes data from different types of hardware and from the different patient populations screened by its various contributors. \u201cThose are the characteristics we look for in any data set for machine learning,\u201d she says<\/p>\n

This generalisation challenge isn\u2019t, however, the only issue that arises from AI vendors seeking their own data.<\/p>\n

Levelling the playing field<\/h2>\n

When AI companies acquire medical data by forging commercial relationships with individual hospitals, they, typically, favour deals that give them exclusive access to patient data, which can be problematic on two fronts \u2013 both of which OPTIMAM has been mindful of.<\/p>\n

First, the public are often justifiably uncomfortable withompanies gaining direct access to patient data \u2013 especially if there is a risk of that data not being securely anonymised. OPTIMAM serves as a go between, meaning companies do not directly interact with hospitals. And vitally, OPTIMAM has worked extensively to ensure rigorous anonymisation procedures, so that mammograms are de-identified before they even leave the screening centre.<\/p>\n

The second issue that can result from companiesurcing data concerns transparency. \u201cIn machine learning, we have a fairly severe reproducibility issue,\u201d says Litchfield. Often, when companies or academic groups publish new methods based on proprietary data sets, he explains, it prevents others in the field from attempting to replicate their work.<\/p>\n

\n

\u201cIt\u2019s now well-recognised that ethnic groups vary in terms of breast size, density and other subtle features which influence algorithm performance. Consequently, an algorithm trained mainly on one group may perform poorly in another.\u201d<\/em><\/p>\n<\/blockquote>\n

\u201cIt\u2019s a real problem for people who want to enter the market with new products,\u201d Lichtfield says, \u201cbecause there\u2019s no way to validate that your performance is better than somebody else and to show that it\u2019s verifiably good.\u201d<\/p>\n

OPTIMAM\u2019s data set being available to anyone who wants to put it to good use immediately by-passes some of these issues.<\/p>\n

Yet the OPTIMAM team also goes one step further, often acting as a semi-independent quality controller. If companies come to them looking for training data, OPTIMAM will supply them with a subset of its database, holding back a fraction of the data that they can then supply for algorithm testing. By keeping a tight rein on the data, it ensures that end users are held to the highest community standards.<\/p>\n

In fact, Halling-Brown believes there are genuine problems with the present level of validation required for AI systems to get marketing approval. Currently, a vendor can market products that have only been shown to work on a small, company-selected data set. \u201cAnd that\u2019s just 100% not enough,\u201d he says. \u201cYou can buy AI tools for breast screening that in independent validations are not doing very well because of these massive generalisable problems.\u201d<\/p>\n

Young, Halling-Brown and team are working with health authorities to explore how to create standardised validation for any product seeking to contribute to national screening programmes in the UK.<\/p>\n

On the horizon<\/h2>\n

CRUK now solely supports the upkeep of the mature database, rather than funding its further development. The arrangement works well, says Halling-Brown with CRUK\u2019s legal and administrative expertise smoothly arranging new partnerships and collaborations that not only allow the data to be used for new purposes, but also help grow and diversify the dataset. This is a mindset which builds on an initial collaboration with Google Health and Imperial College London in 2017 which enabled the OPTIMAM group to collect many additional cancer-free mammogram images to help make the dataset more representative of the general population.<\/p>\n

As OPTIMAM expands by welcoming new sites, Halling-Brown says that this process is now relatively straightforward. This is thanks to the development of a SMART box, which is installed at each new site to automate the collection and annotation of images, the de-identification of participants and the secure uploading of data to the cloud.<\/p>\n

\u201cThat box has taken us eight years to perfect: a huge amount of knowledge has gone into that thing, and it is a very intelligent piece of software,\u201d he says. And that know-how, he says, will be very useful in other contexts where health systems are looking to set up secure, large scale databases.<\/p>\n

Meanwhile, Kheiron have just completed a major study of their latest breast cancer detection algorithms in which they performed as well as human readers. This follows a study<\/a> by Google Health, published in January 2020, using the OPTIMAM data, that made similar claims.<\/p>\n

Kerruish says the software \u2013 christened Mia \u2013 annotates images so that human colleagues can see what features of the breast tissue it is responding to. The aim, of course, being to increase the speed and robustness of human decisions. \u201cWe all know very well the difference in prognosis if a woman\u2019s breast cancer is detected early versus late,\u201d says Kerruish.<\/p>\n

This, along with the many other exciting AI approaches being trained on the OPTIMAM dataset, illustrates just what is at stake.<\/p>\n

\u201cYou think about it in terms of an image,\u201d says Kerruish, \u201cbut this is a woman\u2019s life.\u201d<\/p>\n

\n
\n

If you\u2019re working with AI in cancer or want to increase the reach of your research data, get in touch with us at Commercial@cancer.org.uk<\/a><\/p>\n

Learn more about OPTIMAM here<\/a><\/p>\n<\/div>\n<\/div>\n

Author:<\/em><\/p>\n

Liam Drew is a writer and journalist covering biology and medicine. In 2020, he received the Association of British Science Writers\u2019 Award for Best Engineering and Technology Reporting.<\/em><\/p>\n

More on this topic<\/h3>\n<\/p>\n

Source<\/a><\/p>","protected":false},"excerpt":{"rendered":"

Artificial intelligence is set to revolutionise many aspects of cancer diagnosis and treatment \u2013 but an AI is only as good as the data used to train it. Here we find out how the OPTIMAM project\u2019s mammogram collection is heralding a new dawn of machine learning based cancer detection Artificial intelligence (AI) is on the […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"footnotes":""},"categories":[5137],"tags":[],"class_list":["post-27429","post","type-post","status-publish","format-standard","hentry","category-world-news"],"_links":{"self":[{"href":"https:\/\/ogkologos.com\/en\/wp-json\/wp\/v2\/posts\/27429","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ogkologos.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ogkologos.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ogkologos.com\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ogkologos.com\/en\/wp-json\/wp\/v2\/comments?post=27429"}],"version-history":[{"count":1,"href":"https:\/\/ogkologos.com\/en\/wp-json\/wp\/v2\/posts\/27429\/revisions"}],"predecessor-version":[{"id":43634,"href":"https:\/\/ogkologos.com\/en\/wp-json\/wp\/v2\/posts\/27429\/revisions\/43634"}],"wp:attachment":[{"href":"https:\/\/ogkologos.com\/en\/wp-json\/wp\/v2\/media?parent=27429"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ogkologos.com\/en\/wp-json\/wp\/v2\/categories?post=27429"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ogkologos.com\/en\/wp-json\/wp\/v2\/tags?post=27429"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}