dataset: DISEASES

[tristan-f-r]

From #39.

• Depends on feat: scaffolding, caching, EGFR #65.

[tristan-f-r]

OUTSTANDING TODO: The Snakemake file is up-to-date, but there are three files that are no longer needed (ensg-ensp.tsv, HumanDO.tsv, and HumanDO.tsv.metadata). I removed them from the Snakemake fetch() commands, but (a) are these also somewhere else? This is another use case for local vs. global files - how do I know whether these files are being used by other data collections? How do I remove existing files from outdated pipelines? Etc.

I talk about this in more detail in a comment under #65, but for local files, you can remove them outright. For global files, that is harder to track, but one can quickly search if they are being used by using the (for lack of a better word, 'query tuple') ("BioMart", "ensg-ensp.tsv") across the codebase for any other uses of it.

QUESTION: The Snakefile currently only has two prize and GS files required (the two are arbitrarily chosen). Should those stand in for all 40-ish disease files? We don't want the Snakefile to be dependent on the outputs of the files their rules generate...this seems very circular. If one of those two files is missing, then it should be fine to regenerate all disease input files.

Yes the snakefile should create all of the 40ish disease files. So it should also be dependent on all 40ish diseases not the 2 arbitrary ones picked.

We'll have to think about how to do this. I don't think we can use snakemake to generate the files and then have the last rule be "re-generate this Snakemake file."

We'll have to use a Snakemake checkpoint. I can add this, but I'm also wary of breaking anything in this pipeline, so as @ntalluri suggested some time ago, I'll upload the processed files to Google Drive, and make sure that they don't change with my new changes.

[annaritz]

The committed change now includes using the full DISEASES datasets. We now have 121 disease datasets, though we could filter this (either by increasing the minimum number of high-confidence gold standard genes in the disease set, or by requiring a certain number of prize nodes).

[annaritz]

QUESTION: We currently don't require that the gold standard disease-gene pairs have ENSP IDs in the interactome (STRING); we do require this for the TIGA GWAS inputs. Should we also ensure that all genes are in STRINGDB in the gold standard, or is that caught in a downstream process?

We should be trimming the gold standard to be the ones in the interactome as well. I think there is code for that in #65.

Does that mean that the diseases code should be trimming, or does that happen elsewhere? That might change the new numbers, if we require at least 10 gold standard disease-gene pairs that appear in STRING.

[ntalluri]

The snakemake file only mentions 121 diseases

[annaritz]

There are 134 diseases that have at least 10 high-confidence disease-gene pairs from DISEASES. Some of these don't have enough TIGA evidence to be used - that's in the next step. At this point, there are 134 diseases that COULD be used.

[ntalluri]

This site is down right now and so I can't run the code to process the diseases data. Is there a way to download this data and have a static version we use?

[annaritz]

I put in a help ticket and will wait to hear back. Once it's back up, we should retain a static version.

If it doesn't come back online, I will look up the original data sources (MONDO and EFO) and get cross-reference files from those databases.