Crowdsourcing project predicts progression of neurodegenerat
Source: Kelly Servick
Using data from old clinical trials, two groups of researchers have found a better way to predict how amyotrophic lateral sclerosis (ALS) progresses in different patients. The winning algorithms―designed by non-ALS experts―outperformed the judgments of a group of ALS clinicians given the same data. The advances could make it easier to test whether new drugs can slow the fatal neurodegenerative disease.
The new work was inspired by the so-called ALS Prediction Prize, a joint effort by the ALS-focused nonprofit Prize4Life and Dialogue for Reverse Engineering Assessments and Methods, a computational biology project whose sponsors include IBM, Columbia University, and the New York Academy of Sciences. Announced in 2012, the $50,000 award was designed to bring in experts from outside the ALS field to tackle the notoriously unpredictable illness.
Liuxia Wang, a data analyst at the marketing company Sentrana in Washington, D.C., was used to helping companies make business decisions based on big data sets, such as information about consumer choices, but says she “didn’t know too much about this life science thing” until she got an unusual query from a client. One of the senior managers she worked with revealed that her son had just been diagnosed with ALS and wondered if Sentrana’s analytics could apply to patient data, too. When Wang set out to investigate, she found the ALS Prediction Prize. The next step, she said, was to learn something about ALS.
The disease destroys the neurons that control muscle movement, causing gradual paralysis and eventually killing about half of patients within 3 years of diagnosis. But the speed of its progression varies widely. About 10% of patients live a decade or more after being diagnosed. That makes it hard for doctors to answer patients’ questions about the future, and it’s a big problem for testing new ALS treatments. To measure if a drug is working, a clinical trial has to compare disease progression in people receiving a new treatment with their likely rate of decline if they hadn’t received it―often represented by a control group. A disease as variable as ALS requires costly clinical trials to make this comparison using a large control group.
To crowdsource a better predictive tool, Prize4Life first assembled a patient database from completed clinical trials. Now freely available for investigators, the PRO-ACT database contains information such as age, medical history, vital signs, and other lab tests from 17 clinical trials including more than 8000 patients. For the competition, participants were given just a slice of this data set, collected over 3 months, and asked to design an algorithm to predict how patients would fare in the subsequent 9 months, according to a standard functional scale that measures their ability to move and care for themselves.
The challenge received 37 submissions, most from people who knew nothing about ALS, says Neta Zach, Prize4Life’s scientific director in Tel Aviv, Israel, and an author on the new paper, published online on 2 November in Nature Biotechnology. “It was really like a merger of two groups that had never met before,” she says. Among the participants was the Sentrana team, which tied for first place with a pair of researchers from Stanford University―a lawyer and a statistics professor. The two groups each won $20,000, and a runner-up received $10,000.
When predictions from the two winning algorithms were combined, they outperformed estimates solicited from a dozen ALS clinicians who pored over the same data, the authors report. They estimate that using these algorithms to predict outcomes could allow a drug sponsor to reduce the size of the trial by at least 20% and save as much as $6 million in a large phase III trial.
The paper also breaks down the clinical features that seemed to predict faster or slower decline. Of the 15 features that at least two of the teams identified in their top 30 predictors, several have been reported before. Age, time since onset, amount of weight lost, and forced vital capacity (a measure of lung function) are all thought to be related to disease progression. Other measurements now being investigated, such as levels of the compound creatinine in the blood, got new support from the challenge results. And the algorithms identified a few previously unidentified predictors. Higher blood pressure and heart rate, for example, seem to accompany a faster progression in some patients.
“This is just fascinating,” says neurologist Hiroshi Mitsumoto of Columbia University about the results. The new predictive features that emerged may give researchers new clues about the mechanisms of the disease, he says. “This kind of information never comes unless we have a large number of patients.” He also points out that for a rare disease like ALS, where patient populations are often limited, reducing the number of participants in one clinical trial frees up participants to test another drug.
Predictive algorithms have been developed for other diseases, notes Dan Moore, a biostatistician at the California Pacific Medical Center in San Francisco, but the crowdsourcing approach yielded some very sophisticated statistical solutions. He says that the clinical trial strategy the authors propose―comparing treated patients against predictions―is “a great idea,” although it has yet to win the blessing of the U.S. Food and Drug Administration. As for the strength of the algorithms themselves, Moore is more cautious with his praise. “You’d be able to do better than just throwing darts,” he says of the individual predictions in the paper. The improvement in accuracy could prove very powerful in a large clinical trial, he says, but it’s not reliable enough to provide a prognosis for a single patient.
Meanwhile, Wang says several pharmaceutical companies have expressed interest in the new tool, and Sentrana has formed a spinoff company called Origent Data Sciences to continue the project. The original algorithm is now freely available upon request, but Origent hopes to market programs that can be tailored to specific clinical trial designs or even used to help doctors make more personalized predictions for their patients.
Prize4Life is planning another challenge, this time to seek new ways to classify patients according to disease characteristics, such as genetic features that may correlate with cognitive deficits. The ALS Stratification Prize, supported by a crowdfunding campaign, will launch in the spring of 2015.
| }
|