NovoBench: Benchmark $de$ $novo$ peptide sequencing algorithms

📚 Introduction

NovoBench provides a unified framework for de novo peptide sequencing, focusing on four key aspects:

• 💥 Datasets (diverse MS/MS spectrum data)

  • ✨ Multiple types of spectrum data
  • ✨ Standardized data splits (ensures fair and reproducible evaluation)

• 💥 Models (integrates state-of-the-art methods)

  • ✨ Included models (DeepNovo, PointNovo, Casanovo, InstaNovo, AdaNovo, $\pi$-HelixNovo)

• 💥 Influencing factors (evaluates model robustness)

  • ✨ Key factors (peptide length, noise peaks, missing fragment ratio)

• 💥 Evaluation metrics (comprehensive performance measures)

  • ✨ Comprehensive metrics (amino acid-level and peptide-level precision/recall, PTM identification, efficiency, confidence)

NovoBench abstracts de novo peptide sequencing into well-defined challenges, providing standardized datasets, integrated models, and evaluation metrics to facilitate large-scale comparative studies and drive future advancements in proteomics.

📑 Please see more details in our NeurIPS 2024 paper.

📦 Installation

This project has provided an environment setting file of conda, users can easily reproduce the environment by the following commands:

conda env create -f novobench.yaml
conda activate novobench

🍩 Dataset & Checkpoint download

• All the necessary data files can be downloaded from the link.

• The InstaNovo pretrain weight can be downloaded from the Nine-species, HC-PT.

🚀 Getting Started with NovoBench

Train a New Model

To train a model from scratch, run:

python tests/casanovo.py --mode train --data_path parquet_path --model_path ckpt_path  --config_path config_path

Sequence Mass Spectra

To sequence the mass spectra with NovoBench, use the following command:

python tests/casanovo.py --mode seq --data_path parquet_path --model_path ckpt_path --denovo_output_path csv_path --config_path config_path

⚠️ Note

• DeepNovo and PointNovo need more cpu to process the dataset.
• The unified config file for DeepNovo and PointNovo is in progress. Currently, other models can run using a single config file.

🔗 Citation

@misc{zhou2024novobenchbenchmarkingdeeplearningbased,
      title={NovoBench: Benchmarking Deep Learning-based De Novo Peptide Sequencing Methods in Proteomics}, 
      author={Jingbo Zhou and Shaorong Chen and Jun Xia and Sizhe Liu and Tianze Ling and Wenjie Du and Yue Liu and Jianwei Yin and Stan Z. Li},
      year={2024},
      eprint={2406.11906},
      archivePrefix={arXiv},
      primaryClass={q-bio.QM},
      url={https://arxiv.org/abs/2406.11906}, 
}