Package ubic.gemma.core.analysis.service

Class ExpressionDataFileServiceImpl

java.lang.Object

ubic.gemma.core.analysis.service.ExpressionDataFileServiceImpl

All Implemented Interfaces:

ExpressionDataFileService

@Service
@Transactional(propagation=NEVER)
public class ExpressionDataFileServiceImpl
extends Object
implements ExpressionDataFileService

Supports the creation and location of 'flat file' versions of data in the system, for download by users. Files are cached on the filesystem and reused if possible, rather than recreating them every time.

Never use this service in a Transactional context, it uses locks for files and spends significant time writing to disk.

Author:

paul

Constructor Summary

Constructors

Constructor	Description
ExpressionDataFileServiceImpl()

Method Summary

Modifier and Type	Method	Description
Path	copyMetadataFile(ExpressionExperiment ee, Path existingFile, String filename, boolean forceWrite)	Copy a generic metadata file.
Path	copyMetadataFile(ExpressionExperiment ee, Path existingFile, ExpressionExperimentMetaFileType type, boolean forceWrite)	Copy a metadata file to the location of a given metadata type.
Path	copyMultiQCReport(ExpressionExperiment ee, Path existingFile, ExpressionExperimentMetaFileType fileType, boolean forceWrite)	Copy a MultiQC report.
int	deleteAllAnalysisFiles(ExpressionExperiment ee)	Delete all analyses files for a given experiment.
int	deleteAllAnnotatedFiles(ExpressionExperiment ee)	Delete all files that contain platform annotations for a given experiment.
int	deleteAllDataFiles(ExpressionExperiment ee, QuantitationType qt)	Delete all data files for a given QT.
int	deleteAllFiles(ExpressionExperiment ee)	Delete any existing design, coexpression, data, or differential expression data files.
int	deleteAllProcessedDataFiles(ExpressionExperiment ee)	Delete all the processed files for a given experiment.
boolean	deleteCoexpressionDataFile(ExpressionExperiment ee)	Delete a coexpression data file if it exists.
boolean	deleteDataFile(ExpressionExperiment ee, boolean filtered, ExpressionExperimentDataFileType type)	Delete a processed data file if it exists.
boolean	deleteDataFile(ExpressionExperiment ee, QuantitationType qt, ExpressionExperimentDataFileType type)	Delete a raw or single-cell data file if it exists.
boolean	deleteDesignFile(ExpressionExperiment ee)	Delete the experimental design file for a given experiment.
boolean	deleteDiffExArchiveFile(DifferentialExpressionAnalysis analysis)	Delete a diff.
boolean	deleteMetadataFile(ExpressionExperiment ee, ExpressionExperimentMetaFileType type)	Delete a metadata file.
LockedPath	getDataFile(String filename, boolean exclusive)	Locate any data file in the data directory.
LockedPath	getDataFile(String filename, boolean exclusive, long timeout, TimeUnit timeUnit)
LockedPath	getDataFile(ExpressionExperiment ee, QuantitationType qt, ExpressionExperimentDataFileType type, boolean exclusive, long timeout, TimeUnit timeUnit)
LockedPath	getMetadataFile(ExpressionExperiment ee, String filename, boolean exclusive)
LockedPath	getMetadataFile(ExpressionExperiment ee, String filename, boolean exclusive, long timeout, TimeUnit timeUnit)
Optional<LockedPath>	getMetadataFile(ExpressionExperiment ee, ExpressionExperimentMetaFileType type, boolean exclusive)	Locate a metadata file.
int	writeCellBrowserSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, boolean useBioAssayIds, boolean useRawColumnNames, int fetchSize, boolean useCursorFetchIfSupported, Writer writer, boolean autoFlush)
void	writeDesignMatrix(ExpressionExperiment ee, Writer writer, boolean autoFlush)	Writes out the experimental design for the given experiment.
void	writeDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis analysis, OutputStream stream)	Write a differential expression analysis archive file for a given analysis to an output stream.
void	writeDiffExAnalysisArchiveFileById(Long analysisId, OutputStream outputStream)
Collection<Path>	writeDiffExAnalysisArchiveFiles(Collection<DifferentialExpressionAnalysis> analyses, Path outputDir, boolean forceWrite)
Collection<Path>	writeDiffExAnalysisArchiveFiles(ExpressionExperiment ee, Path outputDir, boolean forceWrite)	Write all the differential expression data files for a given experiment to a particular directory.
int	writeMexSingleCellExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, ScaleType scaleType, boolean useEnsemblIds, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite, Path destDir, boolean autoFlush)
int	writeMexSingleCellExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, ScaleType scaleType, boolean useEnsemblIds, OutputStream stream)
int	writeMexSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, boolean useEnsemblIds, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite, Path destDir, boolean autoFlush)	Write single-cell expression data to a given output stream for a given quantitation type.
int	writeMexSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, boolean useEnsemblIds, OutputStream stream)	Write single-cell expression data to a given output stream for a given quantitation type.
LockedPath	writeOrLocateCoexpressionDataFile(ExpressionExperiment ee, boolean forceWrite)	Write or located the coexpression data file for a given experiment
Optional<LockedPath>	writeOrLocateDesignFile(ExpressionExperiment ee, boolean forceWrite)	Locate or create an experimental design file for a given experiment.
Optional<LockedPath>	writeOrLocateDesignFile(ExpressionExperiment ee, boolean forceWrite, long timeout, TimeUnit timeUnit)
LockedPath	writeOrLocateDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis analysis, boolean forceCreate)	Locate or create the differential expression archive file for a given analysis.
LockedPath	writeOrLocateDiffExAnalysisArchiveFileById(Long analysisId, boolean forceCreate)	Locate or create the differential expression archive file for a given analysis ID.
Collection<Path>	writeOrLocateDiffExAnalysisArchiveFiles(ExpressionExperiment ee, boolean forceWrite)	Locate or create the differential expression archive file(s) for a given experiment.
Optional<LockedPath>	writeOrLocateJSONProcessedExpressionDataFile(ExpressionExperiment ee, boolean filtered, boolean forceWrite)
LockedPath	writeOrLocateJSONRawExpressionDataFile(ExpressionExperiment ee, QuantitationType type, boolean forceWrite)
LockedPath	writeOrLocateMexSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite)	Write single-cell expression data to a standard location for a given quantitation type.
Future<Path>	writeOrLocateMexSingleCellExpressionDataAsync(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite)
Optional<LockedPath>	writeOrLocateProcessedDataFile(ExpressionExperiment ee, boolean filtered, boolean forceWrite)	Locate or create a data file containing the 'preferred and masked' expression data matrix, with filtering for low expression applied (currently supports default settings only).
Optional<LockedPath>	writeOrLocateProcessedDataFile(ExpressionExperiment ee, boolean filtered, boolean forceWrite, long timeout, TimeUnit timeUnit)
LockedPath	writeOrLocateRawExpressionDataFile(ExpressionExperiment ee, QuantitationType type, boolean forceWrite)	Locate or create a new data file for the given quantitation type.
LockedPath	writeOrLocateRawExpressionDataFile(ExpressionExperiment ee, QuantitationType type, boolean forceWrite, long timeout, TimeUnit timeUnit)
LockedPath	writeOrLocateTabularSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite)	Write single-cell expression data to a standard location for a given quantitation type in tabular format.
Future<Path>	writeOrLocateTabularSingleCellExpressionDataAsync(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite)
int	writeProcessedExpressionData(ExpressionExperiment ee, boolean filtered, ScaleType scaleType, boolean excludeSampleIdentifiers, boolean useBioAssayIds, boolean useRawColumnNames, Writer writer, boolean autoFlush)	Write processed expression data to a given writer for a given quantitation type.
int	writeProcessedExpressionData(ExpressionExperiment ee, List<BioAssay> samples, boolean filtered, ScaleType scaleType, boolean excludeSampleIdentifiers, boolean useBioAssayIds, boolean useRawColumnNames, Writer writer, boolean autoFlush)
int	writeRawExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, ScaleType scaleType, boolean excludeSampleIdentifiers, boolean useBioAssayIds, boolean useRawColumnNames, Writer writer, boolean autoFlush)
int	writeRawExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, boolean excludeSampleIdentifiers, boolean useBioAssayIds, boolean useRawColumnNames, Writer writer, boolean autoFlush)	Write raw expression data to a given writer for a given quantitation type.
int	writeTabularSingleCellExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, ScaleType scaleType, boolean useBioAssayIds, boolean useRawColumnNames, int fetchSize, boolean useCursorFetchIfSupported, Writer writer, boolean autoFlush)
int	writeTabularSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, boolean useBioAssayIds, boolean useRawColumnNames, int fetchSize, boolean useCursorFetchIfSupported, Writer writer, boolean autoFlush)	Write single-cell expression data to a given writer for a given quantitation type in tabular format.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ExpressionDataFileServiceImpl

public ExpressionDataFileServiceImpl()

Method Details

deleteAllFiles

public int deleteAllFiles(ExpressionExperiment ee)

Description copied from interface: ExpressionDataFileService

Delete any existing design, coexpression, data, or differential expression data files.

Experiment metadata are not deleted, use ExpressionDataFileService.deleteMetadataFile(ExpressionExperiment, ExpressionExperimentMetaFileType) for that purpose.

Specified by:

deleteAllFiles in interface ExpressionDataFileService

See Also:

• ExpressionDataFileService.deleteDesignFile(ExpressionExperiment)
• ExpressionDataFileService.deleteAllDataFiles(ExpressionExperiment, QuantitationType)
• ExpressionDataFileService.deleteAllProcessedDataFiles(ExpressionExperiment)
• ExpressionDataFileService.deleteAllAnnotatedFiles(ExpressionExperiment)

deleteDesignFile

public boolean deleteDesignFile(ExpressionExperiment ee)

Description copied from interface: ExpressionDataFileService

Delete the experimental design file for a given experiment.

Specified by:

deleteDesignFile in interface ExpressionDataFileService

deleteAllAnnotatedFiles

public int deleteAllAnnotatedFiles(ExpressionExperiment ee)

Description copied from interface: ExpressionDataFileService

Delete all files that contain platform annotations for a given experiment.

This includes all the data and analysis files.

Specified by:

deleteAllAnnotatedFiles in interface ExpressionDataFileService

See Also:

• ExpressionDataFileService.deleteAllDataFiles(ExpressionExperiment, QuantitationType)
• ExpressionDataFileService.deleteAllProcessedDataFiles(ExpressionExperiment)
• ExpressionDataFileService.deleteDiffExArchiveFile(DifferentialExpressionAnalysis)

deleteAllDataFiles

public int deleteAllDataFiles(ExpressionExperiment ee,
 QuantitationType qt)

Description copied from interface: ExpressionDataFileService

Delete all data files for a given QT.

This includes all the possible file types enumerated in ExpressionExperimentDataFileType.

Specified by:

deleteAllDataFiles in interface ExpressionDataFileService

deleteAllProcessedDataFiles

public int deleteAllProcessedDataFiles(ExpressionExperiment ee)

Description copied from interface: ExpressionDataFileService

Delete all the processed files for a given experiment.

This includes all the possible file types enumerated in ExpressionExperimentDataFileType.

Specified by:

deleteAllProcessedDataFiles in interface ExpressionDataFileService

deleteAllAnalysisFiles

public int deleteAllAnalysisFiles(ExpressionExperiment ee)

Description copied from interface: ExpressionDataFileService

Delete all analyses files for a given experiment.

Specified by:

deleteAllAnalysisFiles in interface ExpressionDataFileService

See Also:

• ExpressionDataFileService.deleteDiffExArchiveFile(DifferentialExpressionAnalysis)
• ExpressionDataFileService.deleteCoexpressionDataFile(ExpressionExperiment)

deleteDiffExArchiveFile

public boolean deleteDiffExArchiveFile(DifferentialExpressionAnalysis analysis)

Description copied from interface: ExpressionDataFileService

Delete a diff. ex. analysis archive file for a given analysis.

Specified by:

deleteDiffExArchiveFile in interface ExpressionDataFileService

Returns:

true if any file was deleted

deleteCoexpressionDataFile

public boolean deleteCoexpressionDataFile(ExpressionExperiment ee)

Description copied from interface: ExpressionDataFileService

Delete a coexpression data file if it exists.

Specified by:

deleteCoexpressionDataFile in interface ExpressionDataFileService

getMetadataFile

public Optional<LockedPath> getMetadataFile(ExpressionExperiment ee,
 ExpressionExperimentMetaFileType type,
 boolean exclusive)
                                     throws IOException

Description copied from interface: ExpressionDataFileService

Locate a metadata file. If the metadata file is represented by a directory (i.e. ExpressionExperimentMetaFileType.isDirectory() is true), then the latest file in the directory is returned. If no such file exists. Optional.empty() is returned.

If the data file is being written (i.e. with ExpressionDataFileService.copyMetadataFile(ExpressionExperiment, Path, ExpressionExperimentMetaFileType, boolean), this method will block until it is completed.

Specified by:

getMetadataFile in interface ExpressionDataFileService

Returns:

the metadata file or empty if none is present in the case of a directory-structured metadata

Throws:

IOException

getMetadataFile

public LockedPath getMetadataFile(ExpressionExperiment ee,
 String filename,
 boolean exclusive)
                           throws IOException

Specified by:

getMetadataFile in interface ExpressionDataFileService

Throws:

IOException

getMetadataFile

public LockedPath getMetadataFile(ExpressionExperiment ee,
 String filename,
 boolean exclusive,
 long timeout,
 TimeUnit timeUnit)
                           throws InterruptedException,
TimeoutException,
IOException

Specified by:

getMetadataFile in interface ExpressionDataFileService

Throws:

InterruptedException

TimeoutException

IOException

copyMetadataFile

public Path copyMetadataFile(ExpressionExperiment ee,
 Path existingFile,
 ExpressionExperimentMetaFileType type,
 boolean forceWrite)
                      throws IOException

Description copied from interface: ExpressionDataFileService

Copy a metadata file to the location of a given metadata type.

Writing to a directory (i.e. ExpressionExperimentMetaFileType.isDirectory() is true) is not supported.

If the metadata file is in use, this method will block until it is released.

Specified by:

copyMetadataFile in interface ExpressionDataFileService

Parameters:

existingFile - file to copy, must exist

forceWrite - override any existing metadata file

Returns:

the resulting metadata file, which can also be retrieved with ExpressionDataFileService.getMetadataFile(ExpressionExperiment, ExpressionExperimentMetaFileType, boolean)

Throws:

IOException

copyMetadataFile

public Path copyMetadataFile(ExpressionExperiment ee,
 Path existingFile,
 String filename,
 boolean forceWrite)
                      throws IOException

Description copied from interface: ExpressionDataFileService

Copy a generic metadata file.

This can only be used for metadata files that are not listed in ExpressionExperimentMetaFileType; using any of these reserved filenames will result in an exception. In addition, CHANGELOG.md is also reserved, use ExpressionMetadataChangelogFileService to manipulate it.

Specified by:

copyMetadataFile in interface ExpressionDataFileService

Throws:

IOException

copyMultiQCReport

public Path copyMultiQCReport(ExpressionExperiment ee,
 Path existingFile,
 ExpressionExperimentMetaFileType fileType,
 boolean forceWrite)
                       throws IOException

Description copied from interface: ExpressionDataFileService

Copy a MultiQC report.

This is a helper based on ExpressionDataFileService.copyMetadataFile(ExpressionExperiment, Path, String, boolean) that will automatically copy the main HTML report file as well as the log and JSON data files if they exist.:w

Specified by:

copyMultiQCReport in interface ExpressionDataFileService

Parameters:

fileType - must be a MultiQC report (i.e. ExpressionExperimentMetaFileType.RNASEQ_PIPELINE_REPORT or ExpressionExperimentMetaFileType.CELL_TYPE_ANNOTATION_PIPELINE_REPORT)

Throws:

IOException

deleteMetadataFile

public boolean deleteMetadataFile(ExpressionExperiment ee,
 ExpressionExperimentMetaFileType type)
                           throws IOException

Description copied from interface: ExpressionDataFileService

Delete a metadata file.

If the metadata file is organized as a directory, it is deleted recursively.

If the metadata file is in use, this method will block until it is released.

Specified by:

deleteMetadataFile in interface ExpressionDataFileService

Parameters:

type - the type of metadata file to delete

Returns:

true if a metadata file was deleted

Throws:

IOException

getDataFile

public LockedPath getDataFile(String filename,
 boolean exclusive)
                       throws IOException

Description copied from interface: ExpressionDataFileService

Locate any data file in the data directory.

Specified by:

getDataFile in interface ExpressionDataFileService

Parameters:

exclusive - if true, acquire an exclusive lock on the file

Throws:

IOException

getDataFile

public LockedPath getDataFile(String filename,
 boolean exclusive,
 long timeout,
 TimeUnit timeUnit)
                       throws IOException,
InterruptedException,
TimeoutException

Specified by:

getDataFile in interface ExpressionDataFileService

Throws:

IOException

InterruptedException

TimeoutException

getDataFile

public LockedPath getDataFile(ExpressionExperiment ee,
 QuantitationType qt,
 ExpressionExperimentDataFileType type,
 boolean exclusive,
 long timeout,
 TimeUnit timeUnit)
                       throws InterruptedException,
TimeoutException,
IOException

Specified by:

getDataFile in interface ExpressionDataFileService

Throws:

InterruptedException

TimeoutException

IOException

deleteDataFile

public boolean deleteDataFile(ExpressionExperiment ee,
 QuantitationType qt,
 ExpressionExperimentDataFileType type)
                       throws IOException

Description copied from interface: ExpressionDataFileService

Delete a raw or single-cell data file if it exists.

If the data file is in use, this method will block until it is released.

Specified by:

deleteDataFile in interface ExpressionDataFileService

Returns:

true if the file was deleted, false if it did not exist

Throws:

IOException

deleteDataFile

public boolean deleteDataFile(ExpressionExperiment ee,
 boolean filtered,
 ExpressionExperimentDataFileType type)
                       throws IOException

Description copied from interface: ExpressionDataFileService

Delete a processed data file if it exists.

If the data file is in use, this method will block until it is released.

Specified by:

deleteDataFile in interface ExpressionDataFileService

Returns:

true if the file was deleted, false if it did not exist

Throws:

IOException

writeTabularSingleCellExpressionData

public int writeTabularSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 Writer writer,
 boolean autoFlush)
                                         throws IOException

Description copied from interface: ExpressionDataFileService

Write single-cell expression data to a given writer for a given quantitation type in tabular format.

Specified by:

writeTabularSingleCellExpressionData in interface ExpressionDataFileService

Parameters:

ee - the experiment to use

qt - the quantitation type to retrieve

scaleType - a scale type to use or null to leave the data untransformed

useBioAssayIds - whether to use bioassay and biomaterial IDs instead of names or short names

useRawColumnNames - whether to use raw column names instead of R-friendly ones

fetchSize - retrieve data in a streaming fashion

useCursorFetchIfSupported - use cursor fetching if supported by the database. It is not recommended to use this for public-facing operations because it may require a lot of memory

Throws:

IOException

See Also:

• TabularMatrixWriter

writeTabularSingleCellExpressionData

public int writeTabularSingleCellExpressionData(ExpressionExperiment ee,
 List<BioAssay> samples,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 Writer writer,
 boolean autoFlush)
                                         throws IOException

Specified by:

writeTabularSingleCellExpressionData in interface ExpressionDataFileService

Throws:

IOException

writeOrLocateTabularSingleCellExpressionData

public LockedPath writeOrLocateTabularSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite)
                                                        throws IOException

Description copied from interface: ExpressionDataFileService

Write single-cell expression data to a standard location for a given quantitation type in tabular format.

Specified by:

writeOrLocateTabularSingleCellExpressionData in interface ExpressionDataFileService

Returns:

a path where the vectors were written

Throws:

IOException

See Also:

• ExpressionDataFileService.writeTabularSingleCellExpressionData(ExpressionExperiment, QuantitationType, ScaleType, boolean, boolean, int, boolean, Writer, boolean)
• TabularMatrixWriter

writeOrLocateTabularSingleCellExpressionDataAsync

public Future<Path> writeOrLocateTabularSingleCellExpressionDataAsync(ExpressionExperiment ee,
 QuantitationType qt,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite)

Specified by:

writeOrLocateTabularSingleCellExpressionDataAsync in interface ExpressionDataFileService

See Also:

• ExpressionDataFileService.writeOrLocateTabularSingleCellExpressionData(ExpressionExperiment, QuantitationType, int, boolean, boolean)

writeCellBrowserSingleCellExpressionData

public int writeCellBrowserSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 Writer writer,
 boolean autoFlush)
                                             throws IOException

Specified by:

writeCellBrowserSingleCellExpressionData in interface ExpressionDataFileService

Throws:

IOException

writeMexSingleCellExpressionData

public int writeMexSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useEnsemblIds,
 OutputStream stream)
                                     throws IOException

Description copied from interface: ExpressionDataFileService

Write single-cell expression data to a given output stream for a given quantitation type.

Note: this method is memory intensive because the whole matrix needs to be in-memory to write each individual sample matrices. Thus, no streaming is possible.

Specified by:

writeMexSingleCellExpressionData in interface ExpressionDataFileService

Parameters:

ee - the experiment to use

qt - the quantitation type to retrieve

scaleType - a scale type to use or null to leave the data untransformed

useEnsemblIds - use Ensembl IDs instead of official gene symbols

Throws:

IOException

See Also:

• MexMatrixWriter

writeMexSingleCellExpressionData

public int writeMexSingleCellExpressionData(ExpressionExperiment ee,
 List<BioAssay> samples,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useEnsemblIds,
 OutputStream stream)
                                     throws IOException

Specified by:

writeMexSingleCellExpressionData in interface ExpressionDataFileService

Throws:

IOException

writeMexSingleCellExpressionData

public int writeMexSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useEnsemblIds,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite,
 Path destDir,
 boolean autoFlush)
                                     throws IOException

Description copied from interface: ExpressionDataFileService

Write single-cell expression data to a given output stream for a given quantitation type.

Specified by:

writeMexSingleCellExpressionData in interface ExpressionDataFileService

Parameters:

ee - the experiment to use

qt - the quantitation type to retrieve

scaleType - a scale type to use or null to leave the data untransformed

useEnsemblIds - use Ensembl IDs instead of official gene symbols

fetchSize - fetch size to use for streaming, or load everything in memory of zero or less

useCursorFetchIfSupported - use cursor fetching if supported by the database. It is not recommended to use this for public-facing operations because it may require a lot of memory

forceWrite - whether to force write and ignore any pre-existing directory

destDir - the destination directory to write the data to. It is now allowed to write under the ${gemma.appdata.home}/dateFiles directory using this method, use ExpressionDataFileService.writeOrLocateMexSingleCellExpressionData(ExpressionExperiment, QuantitationType, int, boolean, boolean) instead.

Throws:

IOException

See Also:

• MexMatrixWriter

writeMexSingleCellExpressionData

public int writeMexSingleCellExpressionData(ExpressionExperiment ee,
 List<BioAssay> samples,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useEnsemblIds,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite,
 Path destDir,
 boolean autoFlush)
                                     throws IOException

Specified by:

writeMexSingleCellExpressionData in interface ExpressionDataFileService

Throws:

IOException

writeOrLocateMexSingleCellExpressionData

public LockedPath writeOrLocateMexSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite)
                                                    throws IOException

Description copied from interface: ExpressionDataFileService

Write single-cell expression data to a standard location for a given quantitation type.

Specified by:

writeOrLocateMexSingleCellExpressionData in interface ExpressionDataFileService

Returns:

a path where the vectors were written

Throws:

IOException

See Also:

• ExpressionDataFileService.writeMexSingleCellExpressionData(ExpressionExperiment, QuantitationType, ScaleType, boolean, int, boolean, boolean, Path, boolean)
• MexMatrixWriter

writeOrLocateMexSingleCellExpressionDataAsync

public Future<Path> writeOrLocateMexSingleCellExpressionDataAsync(ExpressionExperiment ee,
 QuantitationType qt,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite)

Specified by:

writeOrLocateMexSingleCellExpressionDataAsync in interface ExpressionDataFileService

See Also:

• ExpressionDataFileService.writeOrLocateMexSingleCellExpressionData(ExpressionExperiment, QuantitationType, int, boolean, boolean)

writeRawExpressionData

public int writeRawExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean excludeSampleIdentifiers,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 Writer writer,
 boolean autoFlush)
                           throws IOException

Description copied from interface: ExpressionDataFileService

Write raw expression data to a given writer for a given quantitation type.

Note: For compression, wrap a GZIPOutputStream with a OutputStreamWriter. To write to a string, consider using StringWriter.

Specified by:

writeRawExpressionData in interface ExpressionDataFileService

Parameters:

ee - the expression experiment

qt - a quantitation type to use

scaleType - a scale type to use or null to leave the data untransformed

excludeSampleIdentifiers -

useBioAssayIds - whether to use bioassay and biomaterial IDs instead of names or short names

useRawColumnNames - whether to use raw column names instead of R-friendly ones

writer - the destination for the raw expression data

Throws:

IOException - if operations with the writer fails

writeRawExpressionData

public int writeRawExpressionData(ExpressionExperiment ee,
 List<BioAssay> samples,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean excludeSampleIdentifiers,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 Writer writer,
 boolean autoFlush)
                           throws IOException

Specified by:

writeRawExpressionData in interface ExpressionDataFileService

Throws:

IOException

writeDesignMatrix

public void writeDesignMatrix(ExpressionExperiment ee,
 Writer writer,
 boolean autoFlush)
                       throws IOException

Description copied from interface: ExpressionDataFileService

Writes out the experimental design for the given experiment.

The bioassays (col 0) matches the header row of the data matrix printed out by the MatrixWriter.

Specified by:

writeDesignMatrix in interface ExpressionDataFileService

Throws:

IOException

See Also:

• ExperimentalDesignWriter

writeProcessedExpressionData

public int writeProcessedExpressionData(ExpressionExperiment ee,
 boolean filtered,
 @Nullable
 ScaleType scaleType,
 boolean excludeSampleIdentifiers,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 Writer writer,
 boolean autoFlush)
                                 throws FilteringException,
IOException

Description copied from interface: ExpressionDataFileService

Write processed expression data to a given writer for a given quantitation type.

Note: For compression, wrap a GZIPOutputStream with a OutputStreamWriter. To write to a string, consider using StringWriter.

Specified by:

writeProcessedExpressionData in interface ExpressionDataFileService

Parameters:

ee - the expression experiment

excludeSampleIdentifiers -

useBioAssayIds - whether to use bioassay and biomaterial IDs instead of names or short names

useRawColumnNames - whether to use raw column names instead of R-friendly ones

writer - the destination for the raw expression data

Throws:

IOException - if operations with the writer fails

FilteringException

writeProcessedExpressionData

public int writeProcessedExpressionData(ExpressionExperiment ee,
 List<BioAssay> samples,
 boolean filtered,
 @Nullable
 ScaleType scaleType,
 boolean excludeSampleIdentifiers,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 Writer writer,
 boolean autoFlush)
                                 throws FilteringException,
IOException

Specified by:

writeProcessedExpressionData in interface ExpressionDataFileService

Throws:

FilteringException

IOException

writeOrLocateCoexpressionDataFile

public LockedPath writeOrLocateCoexpressionDataFile(ExpressionExperiment ee,
 boolean forceWrite)
                                             throws IOException

Description copied from interface: ExpressionDataFileService

Write or located the coexpression data file for a given experiment

Specified by:

writeOrLocateCoexpressionDataFile in interface ExpressionDataFileService

Parameters:

ee - the experiment

forceWrite - whether to force write

Returns:

file

Throws:

IOException

writeOrLocateProcessedDataFile

public Optional<LockedPath> writeOrLocateProcessedDataFile(ExpressionExperiment ee,
 boolean filtered,
 boolean forceWrite)
                                                    throws IOException,
FilteringException

Description copied from interface: ExpressionDataFileService

Locate or create a data file containing the 'preferred and masked' expression data matrix, with filtering for low expression applied (currently supports default settings only). It will be gzip-compressed. The file will be regenerated even if one already exists if the forceWrite parameter is true, or if there was a recent change (more recent than the last modified date of the existing file) to any of the experiments platforms.

Specified by:

writeOrLocateProcessedDataFile in interface ExpressionDataFileService

Parameters:

ee - the experiment

filtered - filtered

forceWrite - force re-write even if file already exists and is up to date.

Returns:

file, or empty if the experiment has no processed vectors

Throws:

IOException

FilteringException

writeOrLocateProcessedDataFile

public Optional<LockedPath> writeOrLocateProcessedDataFile(ExpressionExperiment ee,
 boolean filtered,
 boolean forceWrite,
 long timeout,
 TimeUnit timeUnit)
                                                    throws TimeoutException,
IOException,
InterruptedException,
FilteringException

Specified by:

writeOrLocateProcessedDataFile in interface ExpressionDataFileService

Throws:

TimeoutException

IOException

InterruptedException

FilteringException

writeOrLocateRawExpressionDataFile

public LockedPath writeOrLocateRawExpressionDataFile(ExpressionExperiment ee,
 QuantitationType type,
 boolean forceWrite)
                                              throws IOException

Description copied from interface: ExpressionDataFileService

Locate or create a new data file for the given quantitation type. The output will include gene information if it can be located from its own file.

Specified by:

writeOrLocateRawExpressionDataFile in interface ExpressionDataFileService

Parameters:

type - the quantitation type

forceWrite - To not return the existing file, but create it anew.

Returns:

file

Throws:

IOException

writeOrLocateRawExpressionDataFile

public LockedPath writeOrLocateRawExpressionDataFile(ExpressionExperiment ee,
 QuantitationType type,
 boolean forceWrite,
 long timeout,
 TimeUnit timeUnit)
                                              throws TimeoutException,
IOException,
InterruptedException

Specified by:

writeOrLocateRawExpressionDataFile in interface ExpressionDataFileService

Throws:

TimeoutException

IOException

InterruptedException

writeOrLocateDesignFile

public Optional<LockedPath> writeOrLocateDesignFile(ExpressionExperiment ee,
 boolean forceWrite)
                                             throws IOException

Description copied from interface: ExpressionDataFileService

Locate or create an experimental design file for a given experiment. The file will be regenerated even if one already exists if the forceWrite parameter is true, or if there was a recent change (more recent than the last modified date of the existing file) to any of the experiments platforms.

Specified by:

writeOrLocateDesignFile in interface ExpressionDataFileService

Parameters:

ee - the experiment

forceWrite - force re-write even if file already exists and is up to date

Returns:

a file or empty if the experiment does not have a design

Throws:

IOException

See Also:

• ExpressionDataFileService.writeDesignMatrix(ExpressionExperiment, Writer, boolean)

writeOrLocateDesignFile

public Optional<LockedPath> writeOrLocateDesignFile(ExpressionExperiment ee,
 boolean forceWrite,
 long timeout,
 TimeUnit timeUnit)
                                             throws TimeoutException,
IOException,
InterruptedException

Specified by:

writeOrLocateDesignFile in interface ExpressionDataFileService

Throws:

TimeoutException

IOException

InterruptedException

writeOrLocateJSONProcessedExpressionDataFile

public Optional<LockedPath> writeOrLocateJSONProcessedExpressionDataFile(ExpressionExperiment ee,
 boolean filtered,
 boolean forceWrite)
                                                                  throws FilteringException,
IOException

Specified by:

writeOrLocateJSONProcessedExpressionDataFile in interface ExpressionDataFileService

Throws:

FilteringException

IOException

See Also:

• ExpressionDataFileService.writeOrLocateProcessedDataFile(ExpressionExperiment, boolean, boolean)

writeOrLocateJSONRawExpressionDataFile

public LockedPath writeOrLocateJSONRawExpressionDataFile(ExpressionExperiment ee,
 QuantitationType type,
 boolean forceWrite)
                                                  throws IOException

Specified by:

writeOrLocateJSONRawExpressionDataFile in interface ExpressionDataFileService

Throws:

IOException

See Also:

• ExpressionDataFileService.writeOrLocateRawExpressionDataFile(ExpressionExperiment, QuantitationType, boolean)

writeOrLocateDiffExAnalysisArchiveFiles

public Collection<Path> writeOrLocateDiffExAnalysisArchiveFiles(ExpressionExperiment ee,
 boolean forceWrite)
                                                         throws IOException

Description copied from interface: ExpressionDataFileService

Locate or create the differential expression archive file(s) for a given experiment.

Specified by:

writeOrLocateDiffExAnalysisArchiveFiles in interface ExpressionDataFileService

Parameters:

ee - the experiment

forceWrite - whether to force write

Returns:

collection of files, one per analysis.

Throws:

IOException

See Also:

• ExpressionDataFileService.writeOrLocateDiffExAnalysisArchiveFiles(ExpressionExperiment, boolean)

writeOrLocateDiffExAnalysisArchiveFileById

public LockedPath writeOrLocateDiffExAnalysisArchiveFileById(Long analysisId,
 boolean forceCreate)
                                                      throws IOException

Description copied from interface: ExpressionDataFileService

Locate or create the differential expression archive file for a given analysis ID.

Specified by:

writeOrLocateDiffExAnalysisArchiveFileById in interface ExpressionDataFileService

Throws:

IOException

See Also:

• ExpressionDataFileService.writeOrLocateDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis, boolean)

writeOrLocateDiffExAnalysisArchiveFile

public LockedPath writeOrLocateDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis analysis,
 boolean forceCreate)
                                                  throws IOException

Description copied from interface: ExpressionDataFileService

Locate or create the differential expression archive file for a given analysis.

We generate an archive that contains following files: - differential expression analysis file (q-values per factor) - file for each result set with contrasts info (such as fold change for each factor value).

Specified by:

writeOrLocateDiffExAnalysisArchiveFile in interface ExpressionDataFileService

Parameters:

analysis - analysis

forceCreate - whether to force creation

Returns:

a locked path to the archive file, which must be released after use

Throws:

IOException

writeDiffExAnalysisArchiveFiles

public Collection<Path> writeDiffExAnalysisArchiveFiles(ExpressionExperiment ee,
 Path outputDir,
 boolean forceWrite)
                                                 throws IOException

Description copied from interface: ExpressionDataFileService

Write all the differential expression data files for a given experiment to a particular directory.

Specified by:

writeDiffExAnalysisArchiveFiles in interface ExpressionDataFileService

Throws:

IOException

writeDiffExAnalysisArchiveFiles

public Collection<Path> writeDiffExAnalysisArchiveFiles(Collection<DifferentialExpressionAnalysis> analyses,
 Path outputDir,
 boolean forceWrite)
                                                 throws IOException

Specified by:

writeDiffExAnalysisArchiveFiles in interface ExpressionDataFileService

Throws:

IOException

writeDiffExAnalysisArchiveFileById

public void writeDiffExAnalysisArchiveFileById(Long analysisId,
 OutputStream outputStream)
                                        throws IOException

Specified by:

writeDiffExAnalysisArchiveFileById in interface ExpressionDataFileService

Throws:

IOException

See Also:

• ExpressionDataFileService.writeDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis, OutputStream)

writeDiffExAnalysisArchiveFile

public void writeDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis analysis,
 OutputStream stream)
                                    throws IOException

Description copied from interface: ExpressionDataFileService

Write a differential expression analysis archive file for a given analysis to an output stream.

Specified by:

writeDiffExAnalysisArchiveFile in interface ExpressionDataFileService

Throws:

IOException