Revolution, interrupted: Why AI has failed to live up to the hype in drug development

In 2019, Deep Genomics announced it had discovered a treatment for a rare condition called Wilson disease, which is fatal if not addressed. For the afflicted, copper accumulates in their bodies, particularly in the liver, brain and cornea, where the metal can appear as a brownish ring. The Toronto-based company said in a news release that its molecule was the “first ever AI-discovered therapeutic candidate.”

It wasn’t the only one Deep Genomics would divine through artificial intelligence. Founded in 2015, the company’s AI models would go on to test more than 200 million molecules for their ability to treat disease. By 2021, Deep Genomics had zeroed in on 10 drug candidates for preclinical study and aimed to have four undergoing human trials within a couple of years.

Today, Deep Genomics has zero drugs in clinical trials and many of its plans have blown up. The company halted its Wilson disease program, ditched dozens of its machine learning models, appointed a new chief executive and is pursuing a different approach to using AI. It’s also open to a sale.

“AI has really let us all down in the last decade when it comes to drug discovery,” said Deep Genomics founder Brendan Frey. “We’ve just seen failure after failure.”

The blunt assessment is shocking coming from Mr. Frey, an influential figure in Canada’s AI community. He has been studying machine learning since the 1990s, and he is the co-author of more than 200 papers and a co-founder of the Vector Institute, a nerve centre for AI research in Canada.

Deep Genomics has raised US$238-million and it is one of the only domestic biotech companies backed by the Canada Pension Plan Investment Board.

AI was supposed to revolutionize drug discovery. It hasn’t. Not yet, anyway. Machine learning promised to speed up a lengthy and fraught process, and achieve breakthroughs beyond the capabilities of the human mind. But there are no drugs solely designed by AI in the market today, and companies that have used AI to assist with development have suffered setbacks.

An eczema drug developed by U.K. biotech BenevolentAI S.A. failed to reduce itch and inflammation in trials last year, while Exscientia PLC, another British firm, shuttered trials of its cancer treatment. Recursion Pharmaceuticals Inc. in the United States said that it would find 100 clinical candidates in its first 10 years. A decade later, it has five in trials. In August, Recursion and Exscientia announced a merger, after years of cratering share prices. More consolidation could be on the way.

“If you rewind the clock 10 years, there was a lot of hubris, a lot of hype and a lot of big claims that were made that have perhaps not played out in reality,” said Daniel Cohen, co-founder of Valence Discovery, which was spun out of Mila, Quebec’s AI institute, and acquired by Recursion last year. “But I think that’s necessary in creating a new way of doing things.”

Drug discovery is notoriously hard, and some 90 per cent of candidates fail clinical trials. There are intractable realities of bringing a new treatment to market that AI cannot change – namely, the expensive and time-consuming animal and human trials crucial to proving a compound is effective and safe enough to satisfy regulators.

Some experts quibble with just how much time AI can save during the development process. “I’m not doubting that it can get us there a little bit quicker,” said Brian Bloom, CEO of Bloom Burton & Co., a Toronto investment bank focused on health care. “But in the 10-year journey of drug development, you may be saving five months.”

The setbacks in drug development could hold lessons for other industries hoping that AI will unlock massive productivity gains. Molecular biology is immensely more complicated than building a chatbot, of course, but new technology is anything but straightforward. There are unwelcome surprises, inflated expectations and dashed hopes.

Even so, Mr. Frey is not giving up. AI may not yet be a miracle cure for the pains of drug development, but it is a powerful tool. A new trend in AI has emerged, in fact, and Deep Genomics has remade itself to take advantage. “This is the sea change,” he said. To understand why, you have to understand what went wrong.

Last September, Mr. Frey stepped down as Deep Genomics CEO, a role he had held from the start, and he is now chief innovation officer. (Brian O’Callaghan, a seasoned biotech executive, was appointed CEO.)

Mr. Frey’s role brings him a lot closer to science, which he much prefers. At age 55, he sports grey hair and a grey beard, favours boldly patterned shirts and speaks in the sonorous tones of a university professor, which is not surprising, given he’s taught at the University of Toronto for more than 20 years.

He enrolled in the 1990s to study machine learning and completed a PhD under Geoffrey Hinton, a towering figure in the field. Mr. Frey’s research focused on computer vision and speech recognition, but when he and his wife were expecting a child in 2002, a geneticist delivered a bombshell. Their child was carrying a genetic mutation that could be benign, or a disaster.

Around this time, scientists were decoding the human genome, translating the building blocks of our DNA into a string of letters (A, C, G and T) billions of characters long. That sequence determines our fate – our appearance, how we will grow and whether our bodies can resist disease or succumb to it.

The problem was scientists did not know how to read it or figure out whether a small change in that long string, a mutation, foretold a serious condition or nothing at all. So, Mr. Frey and his wife faced a void of uncertainty about their unborn child. (He declined to comment on the outcome.)

But a genetic sequence is just data. And Mr. Frey was an expert in using machine learning to surface insights from data. He dropped his research to focus on using AI to understand our genetic codes, and predict the consequences of a mutation.

His lab at U of T produced results, but Mr. Frey was frustrated that others were not acting on the findings. He also wanted to move faster. By 2015, he had spun out his research to found Deep Genomics.

He was not the only one, and more advanced machine learning techniques were starting to take off. Atomwise Inc., another U of T spinoff, started even earlier, in 2012, using AI to screen drug candidates faster than traditional methods. In 2018, Google DeepMind built the first version of AlphaFold to predict the three-dimensional structure of proteins, which can take on a mind-boggling number of configurations. Knowing the shape of a protein helps scientists identify drug molecules that can bind to it, like a key fitting into a lock, to treat disease.

Deep Genomics developed its own leads, and one example that Mr. Frey has shared involves spinal muscular atrophy, a condition that causes muscles to shrivel. While a treatment called Spinraza is available, it took 15 years to develop. Deep Genomics identified the region of the genome to target and designed its therapy on a computer in one afternoon.

But a drug candidate is just that – a potential winner or loser. When Deep Genomics moved ahead with its Wilson disease treatment, for example, there was a problem: The company’s machine learning model predicted a smaller patient sample size than was needed to show the drug would be effective. “The AI got the biology right. We’re confident about that,” Mr. Frey said.

The company had a more fundamental issue. Deep Genomics had developed around 40 different AI models that would each work on some aspect of drug discovery, such as identifying genes to treat, predicting the toxicity of molecules and tracking down new patient populations.

“We used to say this proudly,” Mr. Frey said, “that a lot of companies are only focused on one of those things you need to get right for the molecule to work, and Deep Genomics is focused on many different ones.”

In practice, the situation was a mess. The company had to engineer 40 different complex models, maintain 40 different training data sets and rely on multiple computing platforms to power them. Typically, AI models can be improved by cramming more data into them and firing up more graphics processing units. Deep Genomics couldn’t easily do that because, well, it would have to do it for 40 different models. “That whole thing was not scalable,” Mr. Frey said.

Separate models grinding away on narrow slices of the same problem may not work well together, either.

“Sometimes the predictions may contradict each other. Or each individual method may just not work well enough, because they haven’t seen the other parts of the story,” said Bo Wang, chief AI scientist for Toronto’s University Health Network and an adviser to Deep Genomics.

Dion Madsen, co-founder of Montreal-based Amplitude Ventures, was an early believer in the power of AI to improve drug discovery, and the early stage life sciences financing firm first invested in Deep Genomics in 2020. “We haven’t made as much progress as we had planned,” he acknowledged.

The company didn’t have experienced drug developers on staff to validate predictions in the lab and provide feedback on the problems the AI models should be directed toward, and how to improve them. “Our hiccups really came not from the technology, but from not having the right quality of people on the validating side,” Mr. Madsen said, adding that the situation is now improved.

Cracking biology is also just a very, very hard problem. “Biological systems are super complicated. They cannot be reduced to a set of simple rules,” said Jerel Davis, a managing director in Vancouver for Versant Ventures, a biotech investment firm based in San Francisco. “There is a lot we don’t know,” he said.

Companies have been hamstrung by data issues, too. Every AI model needs information to learn from. Today’s large language models, such as OpenAI’s GPT-4, have hoovered up a massive quantity of text from the internet.

But a company that wants to use AI to model the three-dimensional structure of protein pockets can’t pull reams and reams of that kind of thing from the web. Some companies generate their own data by running experiments in a lab, but that can be time-consuming and expensive.

Data that does exist can be of poor quality and needs a lot of curating to be useful for AI models. “It’s always garbage in, garbage out,” Mr. Davis said. “The quantity of data required is often vast.”

True expertise in drug discovery is also scarce. Relatively few people have toiled away in labs and navigated the regulatory process to have a new drug approved. “They know exactly where the difficulties lie, and what to look out for,” said Clarissa Desjardins, founder of Congruence Therapeutics in Montreal, another Amplitude-backed drug developer, and a veteran biotech entrepreneur.

Experts in AI are typically not experts in pharmaceuticals either. “When I heard that Big Pharma was hiring people from SkipTheDishes for AI? Oh my god,” she said, referring to the food delivery service. “It takes four years just to speak scientific language, let alone understand how to develop a drug.”

Companies that have overcome these issues and brought drugs developed with the assistance of AI into clinical trials are grappling with another sobering reality: AI does not guarantee success.

Take BenevolentAI, whose eczema drug failed to alleviate itch and inflammation during human trials last year. The company then cut staff, reduced its lab footprint and axed its eczema program. Similarly, a schizophrenia treatment by Sumitomo Pharma in Japan, which was developed in part with the aid of AI, did no better than a placebo in two expanded efficacy trials last year.

Deep Genomics, meanwhile, could have kept going down this road. But then one of its research scientists decided to run an experiment one weekend.

Albi Celaj doesn’t have much time to peruse scientific papers during the week, when the demands of his job at Deep Genomics consume the hours, so he uses the weekend to catch up.

In 2021, bigger, more powerful AI models were coming online, and to him, it seemed like some puzzle pieces were falling into place. Instead of dozens of machine learning models, maybe Deep Genomics could get by with one to do it all.

On a weekend in November, he built a little proof of concept and pledged to return to it the following Monday. “It was really fast from that point to get to the prototype,” he recalled.

The realization put the company on a new path to focus on foundation models – large, powerful systems that can be used for a range of tasks, even if not specifically trained to do so.

Foundation models such as OpenAI’s GPT-4 are the basis for today’s wave of generative AI, fueled by more sophisticated algorithms and computer chips, and benefitting from a greater availability of data. For Deep Genomics, one for biology could simplify operations by replacing its unwieldy hydra of AI models, and scale more easily to produce better results.

Last September, Deep Genomics announced BigRNA, its first foundation model. It’s a ChatGPT for biology, in a way. A researcher can feed it a DNA or RNA sequence, and the model can then predict all kinds of things crucial to understanding biology and developing new drugs.

“BigRNA is connecting the dots between a DNA sequence, and all of the molecular biology that actually leads to disease,” Mr. Frey said.

The model can predict the mechanisms that regulate which genes are turned on in different tissues in our bodies, identify potential protein binding sites and examine what are called non-coding variants. These are changes in DNA that can flip on a gene at the wrong place or time, or affect the production of an important protein, potentially leading to disease.

“It’s not perfect,” said Mr. Celaj, who was the team lead on the project. “It’s really about, how do we make it better?” BigRNA could speed up the development process and potentially produce more effective drugs because of the information the model provides. “You can understand why it works,” he said. “You’re not just doing these blind screens.”

Handol Kim also jumped on foundation models. He managed the AI team at quantum computer developer D-Wave Quantum Inc. until the Burnaby, B.C., company shut down the division in 2019, and he found himself out of a job.

He and some of his colleagues started Variational AI in Vancouver to bring their AI expertise to drug discovery. Mr. Kim has no background in the field, and studied English literature in university. “We are immigrants to biotech,” he said.

Variational AI has built a foundation model for small molecules, which it calls Enki. Instead of language, Enki deals purely with chemistry. A user can input the desired properties of a molecule, and Enki will spit out structures that meet the criteria. A drug developer can take those molecular structures to investigate them further and potentially turn them into real drugs.

“We can deliver these really good molecular structures very quickly,” Mr. Kim said, who serves as CEO.

In January, Variational announced that drug giant Merck had signed on as an early-access user. As untraditional as Variational’s approach to drug development is, so too is its business model. Typically a startup partnering with a company such as Merck would ask for milestone and royalty payments should one of its molecules pass the regulatory hurdles required to turn into a successful drug.

But Variational takes no royalties and sells molecules wholesale, for relatively cheap – hundreds of thousands of dollars to get results within weeks, as opposed to potentially tens or hundreds of millions of dollars received over years under typical drug development commercial arrangements.

“Our economics are ridiculous,” Mr. Kim said. “We can provide these structures at a small fraction of the cost and still make a lot of margin.” (That also raises questions about just how much its rivals will be able to earn, when Variational is charging so little.)

But is a molecule produced by generative AI any more likely to succeed in clinical trials? “Once you are in the clinic, it’s anyone’s guess,” Mr. Kim said, though he said the company’s molecules are high-quality.

Still, the cost savings and ease of constructing new molecules could allow drug companies to take more candidates into clinical trials, increasing the odds of finding a winner. “The more shots on goal they take, the higher probability of success,” he said.

Some of the philosophical questions about generative AI are relevant in drug discovery, too. Does ChatGPT really produce anything new? Or does it just remix the material in its vast repository of training data? The same can be asked of molecules. Last year, a paper in the journal Medicinal Chemistry Letters said that the technology still needs to be improved to produce “truly novel molecular structures” that have a strong intellectual property case.

“Everyone is kind of using the same data,” Mr. Kim said of training generative AI models. But he doesn’t necessarily see that as a problem. What will make the difference is the calibre of a company’s AI skills to coax results from that corpus of data.

Other companies are using AI in much more targeted ways, including to deal with one of the biggest challenges in drug discovery: finding a “hit.”

A drug molecule needs to bind to a protein in order to treat disease, and finding a match is a ridiculously complicated problem, in part because proteins can assume different shapes and structures. They might also have multiple binding sites, each with different shapes, sizes and chemical properties. AI is of little help.

“It’s impossible because you don’t have training data,” said Marcelo Bigal, CEO of Ventus Therapeutics, which operates out of Montreal and the Boston area.

His company has a workaround. Proteins reside inside cells, which are full of water. That water sloshes around, bumping up against proteins. So, Ventus instead uses AI to predict the behaviour of water inside the binding sites of proteins, which provides the company with a blueprint of sorts. That allows the company to more efficiently identify molecules that might bind.

Instead of testing thousands of molecules in a lab, Ventus will test just dozens. “You dramatically decrease the time and increase the probability of success,” Mr. Bigal said.

For other companies, AI is just one more tool for a complex job. “It’s a hammer,” said Clarissa Desjardins at Congruence Therapeutics. “But eventually it will be a steam engine that will change the way drug discovery happens.”

That day could be a long way off, however. “We are just at the beginning,” she said.

If that’s the case, then Brendan Frey still has a long road ahead. He has not lost the faith. “What’s going on right now with these foundation models is a really big deal. It’s going to change things,” he said.

Whether Deep Genomics will see that through as an independent company is up in the air. Given setbacks in the industry, he expects more consolidation, and that could include his own company. He’s open to all possibilities – partnerships, buying a smaller firm or getting acquired.

“We’re not running out of cash. We have well over $100-million in the bank,” he said. “If there’s a big pharmaceutical company that sees a lot of value in an acquisition of Deep Genomics … that would be on the table.”

The road at Deep Genomics hasn’t been easy, he acknowledged, but he has learned a lot. He comes from academia, after all, where it’s always assumed a hypothesis could be wrong and every failure is dissected for lessons to build toward a deeper understanding. Sometimes the true impact of a discovery isn’t realized until much later.

Few people could have anticipated that some of the work he did decades ago with Geoffrey Hinton, who himself was dismissed as being on the fringes of AI for years, would prove so influential in the development of the technology today.

The work at Deep Genomics may prove to be as seismic. But that’s something no one – and no AI model – can predict.