Catching virtual cheaters
As experienced practitioners of fragment-based
lead discovery will know, the best way to avoid being misled by artifacts is to
combine multiple methods. (Please vote on which methods you use if you haven’t
already done so.) Normally this advice is for physical methods, but what’s true
in real life also applies to virtual reality, as demonstrated in a recent J.
Med. Chem. paper by Brian Shoichet and collaborators at University of California
San Francisco, Schrödinger, and University of Michigan Ann Arbor.
The Shoichet group has been pushing
the limits of computational screening using ever larger libraries. Five years ago
they reported screens of more than 100 million molecules, and today multi-billion compound libraries are becoming routine. But as more compounds are screened, an
unusual type of artifact is emerging: molecules that seem to “cheat” the
scoring function and appear to be virtual winners but are completely inactive when
actually tested. Although rare, as screens increase in size these artifacts can
make up an increasingly large fraction of hits.
Reasoning that these types of
artifacts may be peculiar to a given scoring function, the researchers decided
to rescore the top hits using a different approach to see whether the cheaters
could be caught. They started with a previous screen in which 1.71 billion
molecules had been docked against the antibacterial target AmpC β-lactamase using
DOCK3.8, and more than 1400 hits were synthesized and tested. These were rescreened
using a different scoring approach called FACTS (fast analytical continuum
treatment of solvation). Plotting the scores against each other revealed a bimodal
distribution, with most of the true hits clustering together. Of the 268
molecules that lay outside of this cluster, 262 showed no activity against AmpC
even at 200 µM.
Thus encouraged, the researchers
turned to other studies in which between 32 and 537 compounds had been
experimentally tested. The top 165,000 to 500,000 scoring hits were tested
using FACTS, and 7-19% of the initial DOCK hits showed up as outliers and thus likely
cheaters. For six of the targets, none of these outliers were strong hits. For each
of the other three, a single potent ligand had been flagged as a potential cheater.
To evaluate whether this “cross-filtering”
approach would work prospectively as well as retrospectively, the researchers focused
on 128 very high scoring hits from their previous AmpC virtual screen that had
not already been experimentally tested. These were categorized as outliers
(possible cheaters) or not and then synthesized and tested. Of the 39 outliers,
none were active at 200 µM. But of the other 89, more than half (51) showed
inhibition at 200 µM, and 19 of these gave Ki values < 50 µM. As we
noted back in 2009, AmpC is particularly susceptible to aggregation artifacts, so
the researchers tested the ten most potent inhibitors and found that only one
formed detectable aggregates.
In addition to FACTS, the researchers
also used two other computational methods to look for cheaters: AB-FEP
(absolute binding free energy perturbation) and GBMV (generalized Born using molecular
volume), both of which are more computationally intensive than either FACTS or
DOCK. Interestingly, GBMV performed worse than FACTS, finding at best only 24
cheaters but also falsely flagging 9 true binders. AB-FEP was better, finding
37 cheaters while not flagging any of the experimentally validated hits.
This is an important paper,
particularly as virtual screens of multi-billion compound libraries become
increasingly common. Indeed, the researchers note that “as our libraries grow toward
trillions of molecules… there may be hundreds of thousands of cheating
artifacts.”
And although the researchers acknowledge
that their cross-filtering aproach has only been tested for DOCK, it seems likely
to apply to other computational methods too. I look forward to seeing the
results of these studies.