AlphaFold 3 for RNAs benchmark

This repository is the source code for the visualisations of AlphaFold 3 benchmark for RNA
It also contains the structures for each prediction for the
five test sets: RNA_PUZZLES, CASP_RNA, RNASOLO, RNA3DB_0 and RNA3DB_LONG.

Folder

You will find the following folders:

• data: contains the data used for the visualisation
  • info: contains metadata computed for the predictions or datasets. We advise not to compute it again (the command to run code to compute it is available in the Makefile)
  • orphans: contains original structures with or without context for the RNA3DB_0 dataset. It can be downloaded at this link
  • output: contains the metrics computed for each prediction
  • paper: contains the figures used in the paper (main and supplementary)
  • plots: contains the plots generated for the visualisation. This is where are saved the outputs of the visualisation code available in this repository.
  • pdb: contains the structures predicted by AlphaFold 3 and the benchmarked models. Should be downloaded at this link.
• src: contains the source code for the visualisations
• Makefile: file with the main commands summarized
• environment.yml: file with the different python requirements to run the code

Installation

To install the required libraries, you can use the following command:

conda env create -f environment.yml
conda activate af3

Data

To get all the .pdb files, you have to unzip the pdb folder. If you need the original structures for the RNA3DB_0 dataset (with the context), you can download them here.

Note that to run the visualisations, you do not need to have the original .pdb files. Visualisations are made from either metadata or directly computed scores (available in the data/output folder).

Visualisations

Raw visualisations

To reproduce the visualisation available in data/plots, you just need to use:

make viz_alphafold

or its python equivalent:

python -m src.viz_alphafold

It will save the plots in the data/plots folder.

You will find some of the following plots:

Bar chart	nWC plot	Scatter plot	Scatter mean

To further see which code is used to generate the visualisations, you can check the src/viz_alphafold.py file.

Particularly, you can have a look at the run method:

    def run(self):
        df = self.get_merged_model_df()
        table_helper = TableHelper(df, self.save_dir)
        table_helper.run()
        table_helper.show_context_vs_no_context()
        ComputePValue(df).run()
        nwc_helper = NWCHelper(df, self.save_dir)
        nwc_helper.get_viz()
        scatter_helper = ScatterHelper(df, self.save_dir)
        scatter_helper.get_viz()
        scatter_helper.get_viz_mean()
        scatter_helper.get_viz_diff()
        BarHelper.get_viz()

Each class corresponds to a different type of plot, which are available in the src/viz folder.

Paper visualisations

The clean visualisations of the paper are available in the data/paper folder.

Optional code

Metrics computation

If you want to reproduce the metrics computation, we used the RNAdvisor tool, using:

make compute_metrics

Otherwise, the scores are already computed and available in the data/output folder.

Statistics

To get the statistics of the datasets, you can use the following command:

make stats

Interactions

First, you would need to install the RNA_Assessment library.

To install it, I have made a fork of the original repository with the necessary changes:

make install_rna_assessment

Then, you can compute the interactions for each prediction, using:

make count_interactions

Note that you will need to have a linux system to run the code (because of the MC-Annotate tool).

License

This project is licensed under the Apache 2.0 License.

Citation

If you use this code, please cite the following paper:

Clément Bernard, Guillaume Postic, Sahar Ghannay, Fariza Tahi.
"Has AlphaFold3 achieved success for RNA?"
Acta Crystallographica Section D. 2025. Vol. 81, No. 2.
https://doi.org/10.1107/S2059798325000592

Or using the bibtex format:

@article{af3_for_rnas,
  author = "Bernard, Cl{\'{e}}ment and Postic, Guillaume and Ghannay, Sahar and Tahi, Fariza",
  title = "{Has {\it AlphaFold}3 achieved success for RNA?}",
  journal = "Acta Crystallographica Section D",
  year = "2025",
  volume = "81",
  number = "2",
  pages = "",
  month = "Feb",
  doi = {10.1107/S2059798325000592},
  url = {https://doi.org/10.1107/S2059798325000592},
}

Authors

• Clément Bernard
• Guillaume Postic
• Sahar Ghannay
• Fariza Tahi