Efficient structural information integration into protein language models with implicit mixture-of-experts ProBEiT nature.com

Researchers at Shanghai Jiao Tong University introduced ProBEiT, a 400-million-parameter protein model that combines amino-acid sequences with tokenized structural data using a parameter-efficient mixture-of-experts design. Trained on 1.7 million AlphaFoldDB proteins, it outperformed larger models on six fitness and function tasks and identified beneficial variants in an experimental enzyme-stability study.