Baidu AI researchers create new cancer detection algorithm

Artificial intelligence researchers at the Chinese internet giant Baidu Inc. today released details of a new deep learning algorithm that they claim can help pathologist identify tumors more accurately.

Research scientists Yi Li and Wei Ping from Baidu’s Silicon Valley AI Lab said the new algorithm, which they call a neural conditional random field or NCRF, improves upon current biopsy image analysis using deep learning.

Deep learning is a burgeoning field of machine learning using artificial neural networks that seeks to emulate roughly how the brain works. It’s responsible for big improvements in recent years in image and speech recognition, self-driving cars and other applications.

Current deep learning methods are limited to examining small patches of very large, gigabyte-sized pathology slides. “No one neural network can handle that,” Li said. And because small groups of tumor cells called micrometastases that appear as small as 1000 pixels in diameter (above), it’s a tedious and time-consuming process to analyze them, “literally like finding a needle in a haystack.”

What’s more, even breaking up the images into smaller ones creates a problem, because it’s often tough to predict whether a patch is a tumor or normal. That’s because the surrounding images, especially the regions between tumors and normal cells, make a difference in detection, so the inability of current convolutional neural networks or CNNs to take into account those surroundings often causes false positives.

Baidu’s NCRF algorithm allows the neural network to extract some of the features of the image and then use a “probabilistic graphical model” called a conditional random field — the CRF in the acronym — to aggregate information from nearby images. “We allow the algorithm to zoom out as well,” Li said. “So the algorithm (below) can examine the context of the image.”

Although the absolute improvement in cancer prediction over existing methods is only 1 to 2 percent, Li said the algorithm outperforms a professional pathologist on a standard measure of sensitivity. Some AI technology such as this has started to surpass the ability of human reviewers, but Li views this technology as an aid rather than a replacement for people. “Think of it as a web browser for the pathologist,” he said.

The public data sets Baidu used were chiefly for breast cancer, but Li said the algorithm is a general technique that would work on other types. The limiting factor for now is access to data sets, which the company hopes to obtain mostly from hospitals in China.

The new capability would be added to existing machines that analyze pathology slides. Li said that will required following regulatory rules, but there aren’t very concrete guidelines, so it could take some time to be used in a clinical setting.

Baidu is releasing the algorithm into open source on GitHub, and the researchers will present the paper at a conference in July.

Images: Baidu