Package ubic.gemma.core.loader.expression.geo

Class GeoSingleCellDetectorTest

java.lang.Object

org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests

ubic.gemma.core.util.test.BaseTest

ubic.gemma.core.loader.expression.geo.GeoSingleCellDetectorTest

All Implemented Interfaces:

org.springframework.beans.factory.Aware, org.springframework.context.ApplicationContextAware

@ContextConfiguration
@NetworkAvailable(url="ftp://ftp.ncbi.nlm.nih.gov/geo/series/")
public class GeoSingleCellDetectorTest
extends BaseTest

TODO: move SOFT files in test resources and mock FTP downloads

Field Summary

Fields

Modifier and Type	Field	Description
final NetworkAvailableRule	networkAvailableRule

Fields inherited from class org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests

applicationContext, logger

Constructor Summary

Constructors

Constructor	Description
GeoSingleCellDetectorTest()

Method Summary

Modifier and Type	Method	Description
void	setUp()
void	testDownloadSingleCellDataInCellXGene()
void	testDownloadSingleCellDataInCellXGeneWithoutACollectionId()
void	testGSE109774()	This series has numerous TAR archives with numerous small files within.
void	testGSE125708()	This series has samples lacking data.
void	testGSE132188()	This is an AnnData dataset that uses the .h5ad.h5 extension.
void	testGSE147495()	This is a case of a MEX dataset where files are located in the series.
void	testGSE148611()
void	testGSE155499()	This example has some additional supplementary materials that are inside a TAR archive that contains MEX data.
void	testGSE155695()	This dataset has technical replicates, this is not supported.
void	testGSE158184()	This is a curious example of a SOFT file that has NONE indicators in its supplementary materials.
void	testGSE159416()	This is a Loom dataset with a single file in the series.
void	testGSE162631()	This is a pathologic case with nested zip and gzipped files.
void	testGSE162807()	This is a custom H5 attachment that is unsupported.
void	testGSE165635()	The SRA metadata does not use the "TRANSCRIPTOMIC SINGLE CELL" library source, but keywords in the protocol indicate it is single-cell.
void	testGSE169285()
void	testGSE174574()	Interesting case because it uses genes.tsv instead of features.tsv.
void	testGSE178226()	MEX dataset stored in ZIP archives.
void	testGSE179516()	This is a Loom dataset with files in individual samples
void	testGSE184506()	This series has complete MEX files at the sample-level, but also provide MEX files in the series.
void	testGSE185737()	This dataset as a mix of scRNA-Seq and ATAC-Seq
void	testGSE193884()
void	testGSE196516()	This file has a large TAR attachment in its ATAC-seq samples that should be ignored.
void	testGSE199762()	This one uses non-standard file names.
void	testGSE200202()	This files produces "java.util.zip.ZipException: invalid block type" apparently...
void	testGSE200218()	This dataset does not use 'single-cell transcriptomic' as a library source, so we have to rely on keywords.
void	testGSE201032()
void	testGSE201263()	Similar case here with large TAR attachments, but in this case it has a lot of samples to skip.
void	testGSE201814()	This is a case of a MEX dataset where files are bundled in a per-sample TAR archive.
void	testGSE201814DownloadAll()
void	testGSE202051()	This dataset has multiple .h5ad supplementary files.
void	testGSE208742()	This dataset has MEX data at both series and sample-levels.
void	testGSE218621()	This is a MEX dataset with a barcode_metadata.tsv.gz file.
void	testGSE221522()
void	testGSE221593()	This AnnData file has invalid columns and numerical categorical arrays and uses a dense matrix.
void	testGSE224438()	Typical MEX format: 3 supplementary files per sample
void	testGSE225158()	AnnData (and also Seurat Disk, but the former is preferred)
void	testGSE228370()
void	testGSE235314()	This series contains large TAR attachment that should be skipped when inspecting for MEX.
void	testGSE237862()	This dataset has a mixture of Loom and MEX, the latter should take precedence because Loom is not supported yet.
void	testGSE242423()	This is a case of a MEX dataset that has matrix and barcodes at the sample-level (as we support it!), but genes at the series level.
void	testGSE244451()	This dataset as an array of ENUM, a raw.X and needs to be transposed.
void	testGSE251724()
void	testGSE254569()
void	testGSE256105()
void	testGSE274772()
void	testGSE71585()
void	testGSM4282408()	This sample has duplicated cell IDs.
void	testHasSingleCellDataInCellXGene()
void	testHasSingleCellDataInSra()
void	testIsSingleNuclei()

Methods inherited from class org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests

setApplicationContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

networkAvailableRule

public final NetworkAvailableRule networkAvailableRule

Constructor Details

GeoSingleCellDetectorTest

public GeoSingleCellDetectorTest()

Method Details

setUp

public void setUp()
           throws IOException

Throws:

IOException

testGSE225158

public void testGSE225158()
                   throws IOException,
NoSingleCellDataFoundException

AnnData (and also Seurat Disk, but the former is preferred)

Throws:

IOException

NoSingleCellDataFoundException

testGSE221593

public void testGSE221593()
                   throws IOException,
NoSingleCellDataFoundException

This AnnData file has invalid columns and numerical categorical arrays and uses a dense matrix.

Throws:

IOException

NoSingleCellDataFoundException

testGSE221522

public void testGSE221522()
                   throws IOException,
NoSingleCellDataFoundException

Throws:

IOException

NoSingleCellDataFoundException

testGSE254569

public void testGSE254569()
                   throws IOException,
NoSingleCellDataFoundException

Throws:

IOException

NoSingleCellDataFoundException

testGSE224438

public void testGSE224438()
                   throws IOException,
NoSingleCellDataFoundException

Typical MEX format: 3 supplementary files per sample

Throws:

IOException

NoSingleCellDataFoundException

testGSE201814

public void testGSE201814()
                   throws IOException,
NoSingleCellDataFoundException

This is a case of a MEX dataset where files are bundled in a per-sample TAR archive.

Throws:

IOException

NoSingleCellDataFoundException

testGSE201814DownloadAll

public void testGSE201814DownloadAll()
                              throws IOException,
NoSingleCellDataFoundException

Throws:

IOException

NoSingleCellDataFoundException

testGSE228370

public void testGSE228370()
                   throws IOException,
NoSingleCellDataFoundException

Throws:

IOException

NoSingleCellDataFoundException

testGSE174574

public void testGSE174574()
                   throws IOException,
NoSingleCellDataFoundException

Interesting case because it uses genes.tsv instead of features.tsv.

We need to download it to make sure that the file is properly stored.

Throws:

IOException

NoSingleCellDataFoundException

testGSE202051

public void testGSE202051()
                   throws IOException

This dataset has multiple .h5ad supplementary files. The solution is to manually pick one of them.

Throws:

IOException

testGSE200218

public void testGSE200218()
                   throws IOException,
NoSingleCellDataFoundException

This dataset does not use 'single-cell transcriptomic' as a library source, so we have to rely on keywords.

It als has a bunch of additional supplementary files.

Throws:

IOException

NoSingleCellDataFoundException

testGSE162631

public void testGSE162631()
                   throws IOException

This is a pathologic case with nested zip and gzipped files.

Throws:

IOException

testGSE148611

public void testGSE148611()
                   throws IOException

Throws:

IOException

testGSE193884

public void testGSE193884()
                   throws IOException,
NoSingleCellDataFoundException

Throws:

IOException

NoSingleCellDataFoundException

testGSE147495

public void testGSE147495()
                   throws IOException,
NoSingleCellDataFoundException

This is a case of a MEX dataset where files are located in the series. We don't support downloading those, but we want to make sure we're producing a useful error message.

Throws:

IOException

NoSingleCellDataFoundException

testGSE242423

public void testGSE242423()
                   throws IOException,
NoSingleCellDataFoundException

This is a case of a MEX dataset that has matrix and barcodes at the sample-level (as we support it!), but genes at the series level.

Throws:

IOException

NoSingleCellDataFoundException

testGSE132188

public void testGSE132188()
                   throws IOException,
NoSingleCellDataFoundException

This is an AnnData dataset that uses the .h5ad.h5 extension.

Throws:

IOException

NoSingleCellDataFoundException

testGSE162807

public void testGSE162807()
                   throws IOException

This is a custom H5 attachment that is unsupported.

Throws:

IOException

testGSE158184

public void testGSE158184()
                   throws IOException

This is a curious example of a SOFT file that has NONE indicators in its supplementary materials.

Throws:

IOException

testGSE185737

public void testGSE185737()
                   throws IOException,
NoSingleCellDataFoundException

This dataset as a mix of scRNA-Seq and ATAC-Seq

Throws:

IOException

NoSingleCellDataFoundException

testGSE196516

public void testGSE196516()
                   throws IOException,
NoSingleCellDataFoundException

This file has a large TAR attachment in its ATAC-seq samples that should be ignored.

Throws:

IOException

NoSingleCellDataFoundException

testGSE235314

public void testGSE235314()
                   throws IOException

This series contains large TAR attachment that should be skipped when inspecting for MEX.

Throws:

IOException

testGSE201263

public void testGSE201263()
                   throws IOException,
NoSingleCellDataFoundException

Similar case here with large TAR attachments, but in this case it has a lot of samples to skip.

Throws:

IOException

NoSingleCellDataFoundException

testGSE200202

public void testGSE200202()
                   throws IOException,
NoSingleCellDataFoundException

This files produces "java.util.zip.ZipException: invalid block type" apparently...

Throws:

IOException

NoSingleCellDataFoundException

testGSE155499

public void testGSE155499()
                   throws IOException,
NoSingleCellDataFoundException

This example has some additional supplementary materials that are inside a TAR archive that contains MEX data.

Throws:

IOException

NoSingleCellDataFoundException

testGSE201032

public void testGSE201032()
                   throws IOException

Throws:

IOException

testGSE218621

public void testGSE218621()
                   throws NoSingleCellDataFoundException,
IOException

This is a MEX dataset with a barcode_metadata.tsv.gz file.

Throws:

NoSingleCellDataFoundException

IOException

testGSE237862

public void testGSE237862()
                   throws IOException,
NoSingleCellDataFoundException

This dataset has a mixture of Loom and MEX, the latter should take precedence because Loom is not supported yet.

Throws:

IOException

NoSingleCellDataFoundException

testGSE179516

public void testGSE179516()
                   throws IOException,
NoSingleCellDataFoundException

This is a Loom dataset with files in individual samples

Throws:

IOException

NoSingleCellDataFoundException

testGSE159416

public void testGSE159416()
                   throws IOException,
NoSingleCellDataFoundException

This is a Loom dataset with a single file in the series. We support detection and download, but not loading.

Throws:

IOException

NoSingleCellDataFoundException

testGSE178226

public void testGSE178226()
                   throws IOException,
NoSingleCellDataFoundException

MEX dataset stored in ZIP archives.

Throws:

IOException

NoSingleCellDataFoundException

testGSE199762

public void testGSE199762()
                   throws IOException,
NoSingleCellDataFoundException

This one uses non-standard file names.

Throws:

IOException

NoSingleCellDataFoundException

testGSE155695

public void testGSE155695()
                   throws IOException,
NoSingleCellDataFoundException

This dataset has technical replicates, this is not supported.

Throws:

IOException

NoSingleCellDataFoundException

testGSM4282408

public void testGSM4282408()
                    throws NoSingleCellDataFoundException,
IOException

This sample has duplicated cell IDs.

Throws:

NoSingleCellDataFoundException

IOException

testGSE109774

public void testGSE109774()
                   throws IOException

This series has numerous TAR archives with numerous small files within. We have a policy of skipping these if we skip enough entries within the archive.

Throws:

IOException

testGSE208742

public void testGSE208742()
                   throws IOException,
NoSingleCellDataFoundException

This dataset has MEX data at both series and sample-levels. TODO: add support for this, see #1232

Throws:

IOException

NoSingleCellDataFoundException

testGSE244451

public void testGSE244451()
                   throws IOException,
NoSingleCellDataFoundException

This dataset as an array of ENUM, a raw.X and needs to be transposed.

Throws:

IOException

NoSingleCellDataFoundException

testGSE184506

public void testGSE184506()
                   throws NoSingleCellDataFoundException,
IOException

This series has complete MEX files at the sample-level, but also provide MEX files in the series.

Throws:

NoSingleCellDataFoundException

IOException

testGSE125708

public void testGSE125708()
                   throws NoSingleCellDataFoundException,
IOException

This series has samples lacking data.

Example: GSM3580744

Throws:

NoSingleCellDataFoundException

IOException

testGSE274772

public void testGSE274772()
                   throws IOException

Throws:

IOException

testGSE165635

public void testGSE165635()
                   throws IOException

The SRA metadata does not use the "TRANSCRIPTOMIC SINGLE CELL" library source, but keywords in the protocol indicate it is single-cell.

Throws:

IOException

testGSE169285

public void testGSE169285()
                   throws IOException

Throws:

IOException

testGSE256105

public void testGSE256105()
                   throws IOException

Throws:

IOException

testGSE251724

public void testGSE251724()
                   throws IOException

Throws:

IOException

testHasSingleCellDataInSra

public void testHasSingleCellDataInSra()
                                throws IOException

Throws:

IOException

testHasSingleCellDataInCellXGene

public void testHasSingleCellDataInCellXGene()
                                      throws IOException,
NoSingleCellDataFoundException

Throws:

IOException

NoSingleCellDataFoundException

testDownloadSingleCellDataInCellXGene

public void testDownloadSingleCellDataInCellXGene()
                                           throws IOException,
NoSingleCellDataFoundException

Throws:

IOException

NoSingleCellDataFoundException

testDownloadSingleCellDataInCellXGeneWithoutACollectionId

public void testDownloadSingleCellDataInCellXGeneWithoutACollectionId()
                                                               throws IOException,
NoSingleCellDataFoundException

Throws:

IOException

NoSingleCellDataFoundException

testGSE71585

public void testGSE71585()
                  throws IOException

Throws:

IOException

testIsSingleNuclei

public void testIsSingleNuclei()
                        throws IOException

Throws:

IOException