Interface ExpressionDataFileService

All Known Implementing Classes:

ExpressionDataFileServiceImpl

public interface ExpressionDataFileService

Author:

paul

Method Summary

Modifier and Type	Method	Description
Path	copyMetadataFile(ExpressionExperiment ee, Path existingFile, String filename, boolean forceWrite)	Copy a generic metadata file.
Path	copyMetadataFile(ExpressionExperiment ee, Path existingFile, ExpressionExperimentMetaFileType type, boolean forceWrite)	Copy a metadata file to the location of a given metadata type.
Path	copyMultiQCReport(ExpressionExperiment ee, Path existingFile, ExpressionExperimentMetaFileType type, boolean forceWrite)	Copy a MultiQC report.
int	deleteAllAnalysisFiles(ExpressionExperiment ee)	Delete all analyses files for a given experiment.
int	deleteAllAnnotatedFiles(ExpressionExperiment ee)	Delete all files that contain platform annotations for a given experiment.
int	deleteAllDataFiles(ExpressionExperiment ee, QuantitationType qt)	Delete all data files for a given QT.
int	deleteAllFiles(ExpressionExperiment ee)	Delete any existing design, coexpression, data, or differential expression data files.
int	deleteAllProcessedDataFiles(ExpressionExperiment ee)	Delete all the processed files for a given experiment.
boolean	deleteCoexpressionDataFile(ExpressionExperiment ee)	Delete a coexpression data file if it exists.
boolean	deleteDataFile(ExpressionExperiment ee, boolean filtered, ExpressionExperimentDataFileType type)	Delete a processed data file if it exists.
boolean	deleteDataFile(ExpressionExperiment ee, QuantitationType qt, ExpressionExperimentDataFileType type)	Delete a raw or single-cell data file if it exists.
boolean	deleteDesignFile(ExpressionExperiment ee)	Delete the experimental design file for a given experiment.
boolean	deleteDifferentialExpressionResultSetTsvFile(Long resultSetId)	Delete the cached TSV (if any) for a single differential-expression result set.
boolean	deleteDiffExArchiveFile(DifferentialExpressionAnalysis analysis)	Delete a diff.
boolean	deleteMetadataFile(ExpressionExperiment ee, ExpressionExperimentMetaFileType type)	Delete a metadata file.
boolean	deleteProcessedDataDesignFile(ExpressionExperiment ee)
LockedPath	getDataFile(String filename, boolean exclusive)	Locate any data file in the data directory.
LockedPath	getDataFile(String filename, boolean exclusive, long timeout, TimeUnit timeUnit)
LockedPath	getDataFile(ExpressionExperiment ee, QuantitationType qt, ExpressionExperimentDataFileType type, boolean exclusive, long timeout, TimeUnit timeUnit)
LockedPath	getMetadataFile(ExpressionExperiment ee, String filename, boolean exclusive)
LockedPath	getMetadataFile(ExpressionExperiment ee, String filename, boolean exclusive, long timeout, TimeUnit timeUnit)
Optional<LockedPath>	getMetadataFile(ExpressionExperiment ee, ExpressionExperimentMetaFileType type, boolean exclusive)	Locate a metadata file.
void	streamAndWriteProcessedExpressionData(ExpressionExperiment ee, boolean filtered, boolean forceWrite, Writer writer, boolean autoFlush)	Stream the processed expression data to the writer while populating the cache file in the same pass.
void	streamAndWriteRawExpressionData(ExpressionExperiment ee, QuantitationType qt, boolean forceWrite, Writer writer, boolean autoFlush)	Raw sibling of streamAndWriteProcessedExpressionData(ExpressionExperiment, boolean, boolean, Writer, boolean).
void	streamAndWriteTabularSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite, Writer writer, boolean autoFlush)	Tabular single-cell sibling of streamAndWriteProcessedExpressionData(ExpressionExperiment, boolean, boolean, Writer, boolean) — the payloads here are the largest in the system, so the duplicated cold build this replaces was at its most expensive on this path: two concurrent full scans of the single-cell vectors per cold request.
int	writeCellBrowserSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, boolean useBioAssayIds, boolean useRawColumnNames, int fetchSize, boolean useCursorFetchIfSupported, Writer writer, boolean autoFlush, Console console)
void	writeDesignMatrix(ExpressionExperiment ee, boolean useProcessedData, Writer writer, boolean autoFlush)	Writes out the experimental design for the given experiment.
void	writeDesignMatrix(ExpressionExperiment ee, QuantitationType qt, Class<? extends DataVector> vectorType, Writer writer, boolean autoFlush)	Writes out the experimental design for the given experiment and quantitation type.
void	writeDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis analysis, OutputStream outputStream)	Write a differential expression analysis archive file for a given analysis to an output stream.
void	writeDiffExAnalysisArchiveFileById(Long id, OutputStream outputStream)
Collection<Path>	writeDiffExAnalysisArchiveFiles(Collection<DifferentialExpressionAnalysis> analyses, Path outputDir, boolean forceWrite)
Collection<Path>	writeDiffExAnalysisArchiveFiles(ExpressionExperiment ee, Path outputDir, boolean forceWrite)	Write all the differential expression data files for a given experiment to a particular directory.
int	writeMexSingleCellExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, ScaleType scaleType, boolean useEnsemblIds, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite, Path destDir, boolean autoFlush, Console console)
int	writeMexSingleCellExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, ScaleType scaleType, boolean useEnsemblIds, OutputStream stream)
int	writeMexSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, boolean useEnsemblIds, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite, Path destDir, boolean autoFlush, Console console)	Write single-cell expression data to a given output stream for a given quantitation type.
int	writeMexSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, boolean useEnsemblIds, OutputStream stream)	Write single-cell expression data to a given output stream for a given quantitation type.
Optional<LockedPath>	writeOrLocateDesignFile(ExpressionExperiment ee, boolean useProcessedQuantitationType, boolean forceWrite)	Locate or create an experimental design file for a given experiment.
Optional<LockedPath>	writeOrLocateDesignFile(ExpressionExperiment ee, boolean useProcessedQuantitationType, boolean forceWrite, long timeout, TimeUnit timeUnit)
LockedPath	writeOrLocateDifferentialExpressionResultSetTsvFile(Long resultSetId, boolean forceWrite)	Locate or create a cached TSV for a single differential-expression result set.
Future<Path>	writeOrLocateDifferentialExpressionResultSetTsvFileAsync(Long resultSetId, boolean forceWrite)
LockedPath	writeOrLocateDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis analysis, boolean forceWrite)	Locate or create the differential expression archive file for a given analysis.
Future<Path>	writeOrLocateDiffExAnalysisArchiveFileAsync(DifferentialExpressionAnalysis analysis, boolean forceWrite)	Build the differential expression analysis archive file off the request thread via the expressionDataFileTaskExecutor.
LockedPath	writeOrLocateDiffExAnalysisArchiveFileById(Long analysisId, boolean forceWrite)	Locate or create the differential expression archive file for a given analysis ID.
Collection<Path>	writeOrLocateDiffExAnalysisArchiveFiles(ExpressionExperiment ee, boolean forceWrite)	Locate or create the differential expression archive file(s) for a given experiment.
Optional<LockedPath>	writeOrLocateJSONProcessedExpressionDataFile(ExpressionExperiment ee, boolean filtered, boolean forceWrite)
LockedPath	writeOrLocateJSONRawExpressionDataFile(ExpressionExperiment ee, QuantitationType type, boolean forceWrite)
LockedPath	writeOrLocateMexSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite, Console console)	Write single-cell expression data to a standard location for a given quantitation type.
Future<Path>	writeOrLocateMexSingleCellExpressionDataAsync(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite)
Optional<LockedPath>	writeOrLocateProcessedDataFile(ExpressionExperiment ee, boolean filtered, boolean forceWrite)	Locate or create a data file containing the 'preferred and masked' expression data matrix, with filtering for low expression applied (currently supports default settings only).
Optional<LockedPath>	writeOrLocateProcessedDataFile(ExpressionExperiment ee, boolean filtered, boolean forceWrite, long timeout, TimeUnit timeUnit)
Future<Path>	writeOrLocateProcessedDataFileAsync(ExpressionExperiment ee, boolean filtered, boolean forceWrite)	Build the processed-expression data file off the request thread via the expressionDataFileTaskExecutor.
LockedPath	writeOrLocateRawExpressionDataFile(ExpressionExperiment ee, QuantitationType type, boolean forceWrite)	Locate or create a new data file for the given quantitation type.
LockedPath	writeOrLocateRawExpressionDataFile(ExpressionExperiment ee, QuantitationType qt, boolean forceWrite, long timeout, TimeUnit timeUnit)
Future<Path>	writeOrLocateRawExpressionDataFileAsync(ExpressionExperiment ee, QuantitationType qt, boolean forceWrite)	Build the raw-expression data file off the request thread via the expressionDataFileTaskExecutor.
LockedPath	writeOrLocateTabularSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite, Console console)	Write single-cell expression data to a standard location for a given quantitation type in tabular format.
Future<Path>	writeOrLocateTabularSingleCellExpressionDataAsync(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean useCursorFetchIfSupported, boolean forceWrite)
int	writeProcessedExpressionData(ExpressionExperiment ee, boolean filtered, ScaleType scaleType, boolean excludeSampleIdentifiers, boolean useBioAssayIds, boolean useRawColumnNames, Writer writer, boolean autoFlush)	Write processed expression data to a given writer for a given quantitation type.
int	writeProcessedExpressionData(ExpressionExperiment ee, List<BioAssay> samples, boolean filtered, ScaleType scaleType, boolean excludeSampleIdentifiers, boolean useBioAssayIds, boolean useRawColumnNames, Writer writer, boolean autoFlush)
int	writeRawExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, ScaleType scaleType, boolean excludeSampleIdentifiers, boolean useBioAssayIds, boolean useRawColumnNames, Writer writer, boolean autoFlush)
int	writeRawExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, boolean excludeSampleIdentifiers, boolean useBioAssayIds, boolean useRawColumnNames, Writer writer, boolean autoFlush)	Write raw expression data to a given writer for a given quantitation type.
int	writeTabularSingleCellExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, ScaleType scaleType, boolean useBioAssayIds, boolean useRawColumnNames, int fetchSize, boolean useCursorFetchIfSupported, Writer writer, boolean autoFlush, Console console)
int	writeTabularSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, boolean useBioAssayIds, boolean useRawColumnNames, int fetchSize, boolean useCursorFetchIfSupported, Writer writer, boolean autoFlush, Console console)	Write single-cell expression data to a given writer for a given quantitation type in tabular format.

Method Details

deleteAllFiles

int deleteAllFiles(ExpressionExperiment ee)

Delete any existing design, coexpression, data, or differential expression data files.

Experiment metadata are not deleted, use deleteMetadataFile(ExpressionExperiment, ExpressionExperimentMetaFileType) for that purpose.

See Also:

• deleteDesignFile(ExpressionExperiment)
• deleteAllDataFiles(ExpressionExperiment, QuantitationType)
• deleteAllProcessedDataFiles(ExpressionExperiment)
• deleteAllAnnotatedFiles(ExpressionExperiment)

deleteDesignFile

boolean deleteDesignFile(ExpressionExperiment ee)

Delete the experimental design file for a given experiment.

deleteProcessedDataDesignFile

boolean deleteProcessedDataDesignFile(ExpressionExperiment ee)

deleteAllAnnotatedFiles

int deleteAllAnnotatedFiles(ExpressionExperiment ee)

Delete all files that contain platform annotations for a given experiment.

This includes all the data and analysis files.

See Also:

• deleteAllDataFiles(ExpressionExperiment, QuantitationType)
• deleteAllProcessedDataFiles(ExpressionExperiment)
• deleteDiffExArchiveFile(DifferentialExpressionAnalysis)

deleteAllDataFiles

int deleteAllDataFiles(ExpressionExperiment ee,
 QuantitationType qt)

Delete all data files for a given QT.

This includes all the possible file types enumerated in ExpressionExperimentDataFileType.

deleteAllProcessedDataFiles

int deleteAllProcessedDataFiles(ExpressionExperiment ee)

Delete all the processed files for a given experiment.

This includes all the possible file types enumerated in ExpressionExperimentDataFileType.

deleteAllAnalysisFiles

int deleteAllAnalysisFiles(ExpressionExperiment ee)

Delete all analyses files for a given experiment.

See Also:

• deleteDiffExArchiveFile(DifferentialExpressionAnalysis)
• deleteCoexpressionDataFile(ExpressionExperiment)

deleteDiffExArchiveFile

boolean deleteDiffExArchiveFile(DifferentialExpressionAnalysis analysis)

Delete a diff. ex. analysis archive file for a given analysis.

Returns:

true if any file was deleted

deleteCoexpressionDataFile

boolean deleteCoexpressionDataFile(ExpressionExperiment ee)

Delete a coexpression data file if it exists.

getMetadataFile

Optional<LockedPath> getMetadataFile(ExpressionExperiment ee,
 ExpressionExperimentMetaFileType type,
 boolean exclusive)
                              throws IOException

Locate a metadata file. If the metadata file is represented by a directory (i.e. ExpressionExperimentMetaFileType.isDirectory() is true), then the latest file in the directory is returned. If no such file exists. Optional.empty() is returned.

If the data file is being written (i.e. with copyMetadataFile(ExpressionExperiment, Path, ExpressionExperimentMetaFileType, boolean), this method will block until it is completed.

Returns:

the metadata file or empty if none is present in the case of a directory-structured metadata

Throws:

IOException

getMetadataFile

LockedPath getMetadataFile(ExpressionExperiment ee,
 String filename,
 boolean exclusive)
                    throws IOException

Throws:

IOException

getMetadataFile

LockedPath getMetadataFile(ExpressionExperiment ee,
 String filename,
 boolean exclusive,
 long timeout,
 TimeUnit timeUnit)
                    throws InterruptedException,
TimeoutException,
IOException

Throws:

InterruptedException

TimeoutException

IOException

copyMetadataFile

Path copyMetadataFile(ExpressionExperiment ee,
 Path existingFile,
 ExpressionExperimentMetaFileType type,
 boolean forceWrite)
               throws IOException

Copy a metadata file to the location of a given metadata type.

Writing to a directory (i.e. ExpressionExperimentMetaFileType.isDirectory() is true) is not supported.

If the metadata file is in use, this method will block until it is released.

Parameters:

existingFile - file to copy, must exist

forceWrite - override any existing metadata file

Returns:

the resulting metadata file, which can also be retrieved with getMetadataFile(ExpressionExperiment, ExpressionExperimentMetaFileType, boolean)

Throws:

IOException

copyMetadataFile

Path copyMetadataFile(ExpressionExperiment ee,
 Path existingFile,
 String filename,
 boolean forceWrite)
               throws IOException

Copy a generic metadata file.

This can only be used for metadata files that are not listed in ExpressionExperimentMetaFileType; using any of these reserved filenames will result in an exception. In addition, CHANGELOG.md is also reserved, use ExpressionMetadataChangelogFileService to manipulate it.

Throws:

IOException

copyMultiQCReport

Path copyMultiQCReport(ExpressionExperiment ee,
 Path existingFile,
 ExpressionExperimentMetaFileType type,
 boolean forceWrite)
                throws IOException

Copy a MultiQC report.

This is a helper based on copyMetadataFile(ExpressionExperiment, Path, String, boolean) that will automatically copy the main HTML report file as well as the log and JSON data files if they exist.:w

Parameters:

type - must be a MultiQC report (i.e. ExpressionExperimentMetaFileType.RNASEQ_PIPELINE_REPORT or ExpressionExperimentMetaFileType.CELL_TYPE_ANNOTATION_PIPELINE_REPORT)

Throws:

IOException

deleteMetadataFile

boolean deleteMetadataFile(ExpressionExperiment ee,
 ExpressionExperimentMetaFileType type)
                    throws IOException

Delete a metadata file.

If the metadata file is organized as a directory, it is deleted recursively.

If the metadata file is in use, this method will block until it is released.

Parameters:

type - the type of metadata file to delete

Returns:

true if a metadata file was deleted

Throws:

IOException

getDataFile

LockedPath getDataFile(String filename,
 boolean exclusive)
                throws IOException

Locate any data file in the data directory.

Parameters:

exclusive - if true, acquire an exclusive lock on the file

Throws:

IOException

getDataFile

LockedPath getDataFile(String filename,
 boolean exclusive,
 long timeout,
 TimeUnit timeUnit)
                throws IOException,
InterruptedException,
TimeoutException

Throws:

IOException

InterruptedException

TimeoutException

getDataFile

LockedPath getDataFile(ExpressionExperiment ee,
 QuantitationType qt,
 ExpressionExperimentDataFileType type,
 boolean exclusive,
 long timeout,
 TimeUnit timeUnit)
                throws IOException,
InterruptedException,
TimeoutException

Throws:

IOException

InterruptedException

TimeoutException

deleteDataFile

boolean deleteDataFile(ExpressionExperiment ee,
 QuantitationType qt,
 ExpressionExperimentDataFileType type)
                throws IOException

Delete a raw or single-cell data file if it exists.

If the data file is in use, this method will block until it is released.

Returns:

true if the file was deleted, false if it did not exist

Throws:

IOException

deleteDataFile

boolean deleteDataFile(ExpressionExperiment ee,
 boolean filtered,
 ExpressionExperimentDataFileType type)
                throws IOException

Delete a processed data file if it exists.

If the data file is in use, this method will block until it is released.

Returns:

true if the file was deleted, false if it did not exist

Throws:

IOException

writeTabularSingleCellExpressionData

int writeTabularSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 Writer writer,
 boolean autoFlush,
 @Nullable
 Console console)
                                  throws IOException

Write single-cell expression data to a given writer for a given quantitation type in tabular format.

Parameters:

ee - the experiment to use

qt - the quantitation type to retrieve

scaleType - a scale type to use or null to leave the data untransformed

useBioAssayIds - whether to use bioassay and biomaterial IDs instead of names or short names

useRawColumnNames - whether to use raw column names instead of R-friendly ones

fetchSize - retrieve data in a streaming fashion

useCursorFetchIfSupported - use cursor fetching if supported by the database. It is not recommended to use this for public-facing operations because it may require a lot of memory

Throws:

IOException

See Also:

• TabularMatrixWriter

writeTabularSingleCellExpressionData

int writeTabularSingleCellExpressionData(ExpressionExperiment ee,
 List<BioAssay> samples,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 Writer writer,
 boolean autoFlush,
 @Nullable
 Console console)
                                  throws IOException

Throws:

IOException

writeOrLocateTabularSingleCellExpressionData

LockedPath writeOrLocateTabularSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite,
 @Nullable
 Console console)
                                                 throws IOException

Write single-cell expression data to a standard location for a given quantitation type in tabular format.

Returns:

a path where the vectors were written

Throws:

IOException

See Also:

• writeTabularSingleCellExpressionData(ExpressionExperiment, QuantitationType, ScaleType, boolean, boolean, int, boolean, Writer, boolean, Console)
• TabularMatrixWriter

writeOrLocateTabularSingleCellExpressionDataAsync

Future<Path> writeOrLocateTabularSingleCellExpressionDataAsync(ExpressionExperiment ee,
 QuantitationType qt,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite)
                                                        throws RejectedExecutionException

Throws:

RejectedExecutionException - if the queue for creating data files is full

See Also:

• writeOrLocateTabularSingleCellExpressionData(ExpressionExperiment, QuantitationType, int, boolean, boolean, Console)

writeCellBrowserSingleCellExpressionData

int writeCellBrowserSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 Writer writer,
 boolean autoFlush,
 @Nullable
 Console console)
                                      throws IOException

Throws:

IOException

writeMexSingleCellExpressionData

int writeMexSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useEnsemblIds,
 OutputStream stream)
                              throws IOException

Write single-cell expression data to a given output stream for a given quantitation type.

Note: this method is memory intensive because the whole matrix needs to be in-memory to write each individual sample matrices. Thus, no streaming is possible.

Parameters:

ee - the experiment to use

qt - the quantitation type to retrieve

scaleType - a scale type to use or null to leave the data untransformed

useEnsemblIds - use Ensembl IDs instead of official gene symbols

Throws:

IOException

See Also:

• MexMatrixWriter

writeMexSingleCellExpressionData

int writeMexSingleCellExpressionData(ExpressionExperiment ee,
 List<BioAssay> samples,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useEnsemblIds,
 OutputStream stream)
                              throws IOException

Throws:

IOException

writeMexSingleCellExpressionData

int writeMexSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useEnsemblIds,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite,
 Path destDir,
 boolean autoFlush,
 @Nullable
 Console console)
                              throws IOException

Write single-cell expression data to a given output stream for a given quantitation type.

Parameters:

ee - the experiment to use

qt - the quantitation type to retrieve

scaleType - a scale type to use or null to leave the data untransformed

useEnsemblIds - use Ensembl IDs instead of official gene symbols

fetchSize - fetch size to use for streaming, or load everything in memory of zero or less

useCursorFetchIfSupported - use cursor fetching if supported by the database. It is not recommended to use this for public-facing operations because it may require a lot of memory

forceWrite - whether to force write and ignore any pre-existing directory

destDir - the destination directory to write the data to. It is now allowed to write under the ${gemma.appdata.home}/dateFiles directory using this method, use writeOrLocateMexSingleCellExpressionData(ExpressionExperiment, QuantitationType, int, boolean, boolean, Console) instead.

console - console to write progress to, ignored if null

Throws:

IOException

See Also:

• MexMatrixWriter

writeMexSingleCellExpressionData

int writeMexSingleCellExpressionData(ExpressionExperiment ee,
 List<BioAssay> samples,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean useEnsemblIds,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite,
 Path destDir,
 boolean autoFlush,
 @Nullable
 Console console)
                              throws IOException

Throws:

IOException

writeOrLocateMexSingleCellExpressionData

LockedPath writeOrLocateMexSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite,
 @Nullable
 Console console)
                                             throws IOException

Write single-cell expression data to a standard location for a given quantitation type.

Returns:

a path where the vectors were written

Throws:

IOException

See Also:

• writeMexSingleCellExpressionData(ExpressionExperiment, QuantitationType, ScaleType, boolean, int, boolean, boolean, Path, boolean, Console)
• MexMatrixWriter

writeOrLocateMexSingleCellExpressionDataAsync

Future<Path> writeOrLocateMexSingleCellExpressionDataAsync(ExpressionExperiment ee,
 QuantitationType qt,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite)
                                                    throws RejectedExecutionException

Throws:

RejectedExecutionException - if the queue for creating data files is full

See Also:

• writeOrLocateMexSingleCellExpressionData(ExpressionExperiment, QuantitationType, int, boolean, boolean, Console)

writeRawExpressionData

int writeRawExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean excludeSampleIdentifiers,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 Writer writer,
 boolean autoFlush)
                    throws IOException

Write raw expression data to a given writer for a given quantitation type.

Note: For compression, wrap a GZIPOutputStream with a OutputStreamWriter. To write to a string, consider using StringWriter.

Parameters:

ee - the expression experiment

qt - a quantitation type to use

scaleType - a scale type to use or null to leave the data untransformed

excludeSampleIdentifiers - whether to exclude sample identifiers for the output file and only use assay identifiers

useBioAssayIds - whether to use bioassay and biomaterial IDs instead of names or short names

useRawColumnNames - whether to use raw column names instead of R-friendly ones

writer - the destination for the raw expression data

Throws:

IOException - if operations with the writer fails

writeRawExpressionData

int writeRawExpressionData(ExpressionExperiment ee,
 List<BioAssay> samples,
 QuantitationType qt,
 @Nullable
 ScaleType scaleType,
 boolean excludeSampleIdentifiers,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 Writer writer,
 boolean autoFlush)
                    throws IOException

Throws:

IOException

writeProcessedExpressionData

int writeProcessedExpressionData(ExpressionExperiment ee,
 boolean filtered,
 @Nullable
 ScaleType scaleType,
 boolean excludeSampleIdentifiers,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 Writer writer,
 boolean autoFlush)
                          throws FilteringException,
IOException

Write processed expression data to a given writer for a given quantitation type.

Note: For compression, wrap a GZIPOutputStream with a OutputStreamWriter. To write to a string, consider using StringWriter.

Parameters:

ee - the expression experiment

excludeSampleIdentifiers - whether to exclude sample identifiers for the output file and only use assay identifiers

useBioAssayIds - whether to use bioassay and biomaterial IDs instead of names or short names

useRawColumnNames - whether to use raw column names instead of R-friendly ones

writer - the destination for the raw expression data

Throws:

IOException - if operations with the writer fails

FilteringException

writeProcessedExpressionData

int writeProcessedExpressionData(ExpressionExperiment ee,
 List<BioAssay> samples,
 boolean filtered,
 @Nullable
 ScaleType scaleType,
 boolean excludeSampleIdentifiers,
 boolean useBioAssayIds,
 boolean useRawColumnNames,
 Writer writer,
 boolean autoFlush)
                          throws FilteringException,
IOException

Throws:

FilteringException

IOException

streamAndWriteProcessedExpressionData

void streamAndWriteProcessedExpressionData(ExpressionExperiment ee,
 boolean filtered,
 boolean forceWrite,
 Writer writer,
 boolean autoFlush)
                                    throws FilteringException,
IOException

Stream the processed expression data to the writer while populating the cache file in the same pass.

One matrix build feeds both consumers. This replaces the cold-path pattern of racing a fire-and-forget cache build against an in-band stream of the same data, which fetched the vectors, thawed the platforms and read the annotations twice per cold request. The two sides fail independently: a caller that goes away mid-stream does not abort the cache build, and a cache-file write error does not abort the stream. When another writer already holds the cache file, this degrades to a plain stream, exactly like writeProcessedExpressionData(ExpressionExperiment, boolean, ScaleType, boolean, boolean, boolean, Writer, boolean); when the cache file turns out to be fresh by the time the lock is acquired, its content is streamed instead of being rebuilt.

Parameters:

forceWrite - rebuild the cache file even if it exists and is up to date

Throws:

FilteringException

IOException

streamAndWriteRawExpressionData

void streamAndWriteRawExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 boolean forceWrite,
 Writer writer,
 boolean autoFlush)
                              throws IOException

Raw sibling of streamAndWriteProcessedExpressionData(ExpressionExperiment, boolean, boolean, Writer, boolean).

Throws:

IOException

streamAndWriteTabularSingleCellExpressionData

void streamAndWriteTabularSingleCellExpressionData(ExpressionExperiment ee,
 QuantitationType qt,
 int fetchSize,
 boolean useCursorFetchIfSupported,
 boolean forceWrite,
 Writer writer,
 boolean autoFlush)
                                            throws IOException

Tabular single-cell sibling of streamAndWriteProcessedExpressionData(ExpressionExperiment, boolean, boolean, Writer, boolean) — the payloads here are the largest in the system, so the duplicated cold build this replaces was at its most expensive on this path: two concurrent full scans of the single-cell vectors per cold request.

Throws:

IOException

writeDesignMatrix

void writeDesignMatrix(ExpressionExperiment ee,
 boolean useProcessedData,
 Writer writer,
 boolean autoFlush)
                throws IOException

Writes out the experimental design for the given experiment.

The bioassays (col 0) matches the header row of the data matrix printed out by the MatrixWriter.

Throws:

IOException

See Also:

• ExperimentalDesignWriter

writeDesignMatrix

void writeDesignMatrix(ExpressionExperiment ee,
 QuantitationType qt,
 Class<? extends DataVector> vectorType,
 Writer writer,
 boolean autoFlush)
                throws IOException

Writes out the experimental design for the given experiment and quantitation type.

Throws:

IOException

writeOrLocateProcessedDataFile

Optional<LockedPath> writeOrLocateProcessedDataFile(ExpressionExperiment ee,
 boolean filtered,
 boolean forceWrite)
                                             throws FilteringException,
IOException

Locate or create a data file containing the 'preferred and masked' expression data matrix, with filtering for low expression applied (currently supports default settings only). It will be gzip-compressed. The file will be regenerated even if one already exists if the forceWrite parameter is true, or if there was a recent change (more recent than the last modified date of the existing file) to any of the experiments platforms.

Parameters:

ee - the experiment

filtered - filtered

forceWrite - force re-write even if file already exists and is up to date.

Returns:

file, or empty if the experiment has no processed vectors

Throws:

FilteringException

IOException

writeOrLocateProcessedDataFile

Optional<LockedPath> writeOrLocateProcessedDataFile(ExpressionExperiment ee,
 boolean filtered,
 boolean forceWrite,
 long timeout,
 TimeUnit timeUnit)
                                             throws TimeoutException,
IOException,
InterruptedException,
FilteringException

Throws:

TimeoutException

IOException

InterruptedException

FilteringException

writeOrLocateProcessedDataFileAsync

Future<Path> writeOrLocateProcessedDataFileAsync(ExpressionExperiment ee,
 boolean filtered,
 boolean forceWrite)
                                          throws RejectedExecutionException

Build the processed-expression data file off the request thread via the expressionDataFileTaskExecutor. Wraps writeOrLocateProcessedDataFile(ExpressionExperiment, boolean, boolean) so cold-cache HTTP callers can return immediately and stream the data while the cached file materializes for the next caller.

Throws:

RejectedExecutionException - if the queue for creating data files is full

writeOrLocateRawExpressionDataFile

LockedPath writeOrLocateRawExpressionDataFile(ExpressionExperiment ee,
 QuantitationType type,
 boolean forceWrite)
                                       throws IOException

Locate or create a new data file for the given quantitation type. The output will include gene information if it can be located from its own file.

Parameters:

type - the quantitation type

forceWrite - To not return the existing file, but create it anew.

Returns:

file

Throws:

IOException

writeOrLocateRawExpressionDataFile

LockedPath writeOrLocateRawExpressionDataFile(ExpressionExperiment ee,
 QuantitationType qt,
 boolean forceWrite,
 long timeout,
 TimeUnit timeUnit)
                                       throws TimeoutException,
IOException,
InterruptedException

Throws:

TimeoutException

IOException

InterruptedException

writeOrLocateRawExpressionDataFileAsync

Future<Path> writeOrLocateRawExpressionDataFileAsync(ExpressionExperiment ee,
 QuantitationType qt,
 boolean forceWrite)
                                              throws RejectedExecutionException

Build the raw-expression data file off the request thread via the expressionDataFileTaskExecutor. Wraps writeOrLocateRawExpressionDataFile(ExpressionExperiment, QuantitationType, boolean) so cold-cache HTTP callers can return immediately and stream the data while the cached file materializes for the next caller.

Throws:

RejectedExecutionException - if the queue for creating data files is full

writeOrLocateDesignFile

Optional<LockedPath> writeOrLocateDesignFile(ExpressionExperiment ee,
 boolean useProcessedQuantitationType,
 boolean forceWrite)
                                      throws IOException

Locate or create an experimental design file for a given experiment. The file will be regenerated even if one already exists if the forceWrite parameter is true, or if there was a recent change (more recent than the last modified date of the existing file) to any of the experiments platforms.

Parameters:

ee - the experiment

useProcessedQuantitationType - if true, produce the design for the assays of the processed data instead of the assays of the experiment. These two are usually the same, but they will differ if there is a sub-sample structure (i.e. single-cell data).

forceWrite - force re-write even if file already exists and is up to date

Returns:

a file or empty if the experiment does not have a design

Throws:

IOException

See Also:

• writeDesignMatrix(ExpressionExperiment, boolean, Writer, boolean)

writeOrLocateDesignFile

Optional<LockedPath> writeOrLocateDesignFile(ExpressionExperiment ee,
 boolean useProcessedQuantitationType,
 boolean forceWrite,
 long timeout,
 TimeUnit timeUnit)
                                      throws TimeoutException,
IOException,
InterruptedException

Throws:

TimeoutException

IOException

InterruptedException

writeOrLocateJSONProcessedExpressionDataFile

Optional<LockedPath> writeOrLocateJSONProcessedExpressionDataFile(ExpressionExperiment ee,
 boolean filtered,
 boolean forceWrite)
                                                           throws FilteringException,
IOException

Throws:

FilteringException

IOException

See Also:

• writeOrLocateProcessedDataFile(ExpressionExperiment, boolean, boolean)

writeOrLocateJSONRawExpressionDataFile

LockedPath writeOrLocateJSONRawExpressionDataFile(ExpressionExperiment ee,
 QuantitationType type,
 boolean forceWrite)
                                           throws IOException

Throws:

IOException

See Also:

• writeOrLocateRawExpressionDataFile(ExpressionExperiment, QuantitationType, boolean)

writeOrLocateDifferentialExpressionResultSetTsvFile

LockedPath writeOrLocateDifferentialExpressionResultSetTsvFile(Long resultSetId,
 boolean forceWrite)
                                                        throws IOException

Locate or create a cached TSV for a single differential-expression result set.

Result sets are immutable post-creation, so the on-disk cache is effectively permanent.

On a cache hit, returns the existing path with a shared lock (no DB hit). On a cache miss, materializes the result set + contrasts + factor values + result-to-genes map via ExpressionAnalysisResultSetService, writes the TSV via ExpressionAnalysisResultSetFileService, stores it gzipped under <dataDir>/resultSets/resultSet_<id>.tsv.gz, and returns the locked path. The /resultSets/{id} REST endpoint sends those bytes verbatim via sendfile under @GZIP(alreadyCompressed = true) — it cannot compress on the fly, because sendfile bypasses the encoder's stream entirely.

Parameters:

resultSetId - the result-set ID to materialize

forceWrite - ignore any existing cached file

Returns:

a locked path to the TSV file, which must be released after use

Throws:

NoSuchElementException - if the result set cannot be found

IOException

writeOrLocateDifferentialExpressionResultSetTsvFileAsync

Future<Path> writeOrLocateDifferentialExpressionResultSetTsvFileAsync(Long resultSetId,
 boolean forceWrite)
                                                               throws RejectedExecutionException

Throws:

RejectedExecutionException - if the queue for creating data files is full

See Also:

• writeOrLocateDifferentialExpressionResultSetTsvFile(Long, boolean)

deleteDifferentialExpressionResultSetTsvFile

boolean deleteDifferentialExpressionResultSetTsvFile(Long resultSetId)

Delete the cached TSV (if any) for a single differential-expression result set.

Intended to be called by the DEA-deletion path so the per-result-set TSV cache does not outlive the underlying entity. Idempotent; returns true only when a cached file was actually removed.

Parameters:

resultSetId - the result-set ID whose cached TSV should be deleted

Returns:

true if a cached file was deleted, false if none existed

writeOrLocateDiffExAnalysisArchiveFiles

Collection<Path> writeOrLocateDiffExAnalysisArchiveFiles(ExpressionExperiment ee,
 boolean forceWrite)
                                                  throws IOException

Locate or create the differential expression archive file(s) for a given experiment.

Parameters:

ee - the experiment

forceWrite - whether to force write

Returns:

collection of files, one per analysis.

Throws:

IOException

See Also:

• writeOrLocateDiffExAnalysisArchiveFiles(ExpressionExperiment, boolean)

writeOrLocateDiffExAnalysisArchiveFileById

LockedPath writeOrLocateDiffExAnalysisArchiveFileById(Long analysisId,
 boolean forceWrite)
                                               throws IOException

Locate or create the differential expression archive file for a given analysis ID.

Throws:

IOException

See Also:

• writeOrLocateDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis, boolean)

writeOrLocateDiffExAnalysisArchiveFile

LockedPath writeOrLocateDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis analysis,
 boolean forceWrite)
                                           throws IOException

Locate or create the differential expression archive file for a given analysis.

We generate an archive that contains following files: - differential expression analysis file (q-values per factor) - file for each result set with contrasts info (such as fold change for each factor value).

Parameters:

analysis - analysis

forceWrite - whether to force creation

Returns:

a locked path to the archive file, which must be released after use

Throws:

IOException

writeOrLocateDiffExAnalysisArchiveFileAsync

Future<Path> writeOrLocateDiffExAnalysisArchiveFileAsync(DifferentialExpressionAnalysis analysis,
 boolean forceWrite)
                                                  throws RejectedExecutionException

Build the differential expression analysis archive file off the request thread via the expressionDataFileTaskExecutor. The returned Future resolves to the archive path once the write completes; callers that only need fire-and-forget semantics may ignore the result. The shared lock acquired by the underlying sync write is released before the future resolves.

Throws:

RejectedExecutionException - if the queue for creating data files is full

See Also:

• writeOrLocateDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis, boolean)

writeDiffExAnalysisArchiveFiles

Collection<Path> writeDiffExAnalysisArchiveFiles(ExpressionExperiment ee,
 Path outputDir,
 boolean forceWrite)
                                          throws IOException

Write all the differential expression data files for a given experiment to a particular directory.

Throws:

IOException

writeDiffExAnalysisArchiveFiles

Collection<Path> writeDiffExAnalysisArchiveFiles(Collection<DifferentialExpressionAnalysis> analyses,
 Path outputDir,
 boolean forceWrite)
                                          throws IOException

Throws:

IOException

writeDiffExAnalysisArchiveFileById

void writeDiffExAnalysisArchiveFileById(Long id,
 OutputStream outputStream)
                                 throws IOException

Throws:

IOException

See Also:

• writeDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis, OutputStream)

writeDiffExAnalysisArchiveFile

void writeDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis analysis,
 OutputStream outputStream)
                             throws IOException

Write a differential expression analysis archive file for a given analysis to an output stream.

Throws:

IOException