Large Scale Paired Antibody Language Models
Author Information
Author(s): Kenlay Henry, Dreyer Frédéric A., Kovaltsuk Aleksandr, Miketa Dom, Pires Douglas, Deane Charlotte M.
Primary Institution: Exscientia, Oxford Science Park, Oxford, United Kingdom
Hypothesis
Can specialized language models improve the performance of antibody sequence recovery and predictive tasks?
Conclusion
The IgBert and IgT5 models significantly enhance the potential for antibody design and therapeutic development by outperforming existing models in key tasks.
Supporting Evidence
- The models were trained on a dataset of over two billion unpaired antibody sequences.
- IgBert and IgT5 outperformed existing models in sequence recovery and binding affinity predictions.
- The trained models are publicly available for further research in antibody engineering.
- Fine-tuning on paired data significantly improved model performance.
- Both models leverage machine learning and high-performance computing for antibody design.
Takeaway
Scientists created two smart computer programs that help design better medicines by understanding how antibodies work. They used a lot of data to teach these programs, and now they can predict how well antibodies will work.
Methodology
The study involved training two language models, IgBert and IgT5, on over two billion unpaired antibody sequences and fine-tuning them on paired sequences to improve performance in sequence recovery and predictive tasks.
Limitations
The study does not address the optimal amount of paired data needed for performance improvements and the potential for catastrophic forgetting during fine-tuning.
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website