Interface ExpressionDataFileService
-
- All Known Implementing Classes:
ExpressionDataFileServiceImpl
public interface ExpressionDataFileService
- Author:
- paul
-
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description Path
copyMetadataFile(ExpressionExperiment ee, Path existingFile, String filename, boolean forceWrite)
Copy a generic metadata file.Path
copyMetadataFile(ExpressionExperiment ee, Path existingFile, ExpressionExperimentMetaFileType type, boolean forceWrite)
Copy a metadata file to the location of a given metadata type.Path
copyMultiQCReport(ExpressionExperiment ee, Path existingFile, boolean forceWrite)
Copy a MultiQC report.int
deleteAllAnalysisFiles(ExpressionExperiment ee)
Delete all analyses files for a given experiment.int
deleteAllAnnotatedFiles(ExpressionExperiment ee)
Delete all files that contain platform annotations for a given experiment.int
deleteAllDataFiles(ExpressionExperiment ee, QuantitationType qt)
Delete all data files for a given QT.int
deleteAllFiles(ExpressionExperiment ee)
Delete any existing design, coexpression, data, or differential expression data files.int
deleteAllProcessedDataFiles(ExpressionExperiment ee)
Delete all the processed files for a given experiment.boolean
deleteCoexpressionDataFile(ExpressionExperiment ee)
Delete a coexpression data file if it exists.boolean
deleteDataFile(ExpressionExperiment ee, boolean filtered, ExpressionExperimentDataFileType type)
Delete a processed data file if it exists.boolean
deleteDataFile(ExpressionExperiment ee, QuantitationType qt, ExpressionExperimentDataFileType type)
Delete a raw or single-cell data file if it exists.boolean
deleteDesignFile(ExpressionExperiment ee)
Delete the experimental design file for a given experiment.boolean
deleteDiffExArchiveFile(DifferentialExpressionAnalysis analysis)
Delete a diff. ex. analysis archive file for a given analysis.boolean
deleteMetadataFile(ExpressionExperiment ee, ExpressionExperimentMetaFileType type)
Delete a metadata file.LockedPath
getDataFile(String filename, boolean exclusive)
Locate any data file in the data directory.LockedPath
getDataFile(String filename, boolean exclusive, long timeout, TimeUnit timeUnit)
LockedPath
getDataFile(ExpressionExperiment ee, QuantitationType qt, ExpressionExperimentDataFileType type, boolean exclusive, long timeout, TimeUnit timeUnit)
LockedPath
getMetadataFile(ExpressionExperiment ee, String filename, boolean exclusive)
LockedPath
getMetadataFile(ExpressionExperiment ee, String filename, boolean exclusive, long timeout, TimeUnit timeUnit)
Optional<LockedPath>
getMetadataFile(ExpressionExperiment ee, ExpressionExperimentMetaFileType type, boolean exclusive)
Locate a metadata file.void
writeDesignMatrix(ExpressionExperiment ee, Writer writer)
Writes out the experimental design for the given experiment.LockedPath
writeDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis analysis, DifferentialExpressionAnalysisConfig config)
int
writeMexSingleCellExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, ScaleType scaleType, boolean useEnsemblIds, int fetchSize, boolean forceWrite, Path destDir)
int
writeMexSingleCellExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, ScaleType scaleType, boolean useEnsemblIds, OutputStream stream)
int
writeMexSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, boolean useEnsemblIds, int fetchSize, boolean forceWrite, Path destDir)
Write single-cell expression data to a given output stream for a given quantitation type.int
writeMexSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, boolean useEnsemblIds, OutputStream stream)
Write single-cell expression data to a given output stream for a given quantitation type.LockedPath
writeOrLocateCoexpressionDataFile(ExpressionExperiment ee, boolean forceWrite)
Write or located the coexpression data file for a given experimentOptional<LockedPath>
writeOrLocateDesignFile(ExpressionExperiment ee, boolean forceWrite)
Locate or create an experimental design file for a given experiment.Optional<LockedPath>
writeOrLocateDesignFile(ExpressionExperiment ee, boolean forceWrite, long timeout, TimeUnit timeUnit)
LockedPath
writeOrLocateDiffExArchiveFile(Long analysisId, boolean forceCreate)
Locate or create the differential expression data file(s) for a given experiment.Collection<LockedPath>
writeOrLocateDiffExpressionDataFiles(ExpressionExperiment ee, boolean forceWrite)
Locate or create the differential expression data file(s) for a given experiment.Optional<LockedPath>
writeOrLocateJSONProcessedExpressionDataFile(ExpressionExperiment ee, boolean forceWrite, boolean filtered)
LockedPath
writeOrLocateJSONRawExpressionDataFile(ExpressionExperiment ee, QuantitationType type, boolean forceWrite)
LockedPath
writeOrLocateMexSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean forceWrite)
Write single-cell expression data to a standard location for a given quantitation type.Future<Path>
writeOrLocateMexSingleCellExpressionDataAsync(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean forceWrite)
Optional<LockedPath>
writeOrLocateProcessedDataFile(ExpressionExperiment ee, boolean filtered, boolean forceWrite)
Locate or create a data file containing the 'preferred and masked' expression data matrix, with filtering for low expression applied (currently supports default settings only).Optional<LockedPath>
writeOrLocateProcessedDataFile(ExpressionExperiment ee, boolean filtered, boolean forceWrite, long timeout, TimeUnit timeUnit)
LockedPath
writeOrLocateRawExpressionDataFile(ExpressionExperiment ee, QuantitationType type, boolean forceWrite)
Locate or create a new data file for the given quantitation type.LockedPath
writeOrLocateRawExpressionDataFile(ExpressionExperiment ee, QuantitationType qt, boolean forceWrite, long timeout, TimeUnit timeUnit)
LockedPath
writeOrLocateTabularSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean forceWrite)
Write single-cell expression data to a standard location for a given quantitation type in tabular format.Future<Path>
writeOrLocateTabularSingleCellExpressionDataAsync(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean forceWrite)
int
writeProcessedExpressionData(ExpressionExperiment ee, boolean filtered, ScaleType scaleType, Writer writer, boolean autoFlush)
Write processed expression data to a given writer for a given quantitation type.int
writeProcessedExpressionData(ExpressionExperiment ee, List<BioAssay> samples, boolean filtered, ScaleType scaleType, Writer writer, boolean autoFlush)
int
writeRawExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, ScaleType scaleType, Writer writer, boolean autoFlush)
int
writeRawExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, Writer writer, boolean autoFlush)
Write raw expression data to a given writer for a given quantitation type.int
writeTabularSingleCellExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, ScaleType scaleType, int fetchSize, Writer writer, boolean autoFlush)
int
writeTabularSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, ScaleType scaleType, int fetchSize, Writer writer, boolean autoFlush)
Write single-cell expression data to a given writer for a given quantitation type in tabular format.
-
-
-
Method Detail
-
deleteAllFiles
int deleteAllFiles(ExpressionExperiment ee)
Delete any existing design, coexpression, data, or differential expression data files.Experiment metadata are not deleted, use
deleteMetadataFile(ExpressionExperiment, ExpressionExperimentMetaFileType)
for that purpose.
-
deleteDesignFile
boolean deleteDesignFile(ExpressionExperiment ee)
Delete the experimental design file for a given experiment.
-
deleteAllAnnotatedFiles
int deleteAllAnnotatedFiles(ExpressionExperiment ee)
Delete all files that contain platform annotations for a given experiment.This includes all the data and analysis files.
-
deleteAllDataFiles
int deleteAllDataFiles(ExpressionExperiment ee, QuantitationType qt)
Delete all data files for a given QT.This includes all the possible file types enumerated in
ExpressionExperimentDataFileType
.
-
deleteAllProcessedDataFiles
int deleteAllProcessedDataFiles(ExpressionExperiment ee)
Delete all the processed files for a given experiment.This includes all the possible file types enumerated in
ExpressionExperimentDataFileType
.
-
deleteAllAnalysisFiles
int deleteAllAnalysisFiles(ExpressionExperiment ee)
Delete all analyses files for a given experiment.
-
deleteDiffExArchiveFile
boolean deleteDiffExArchiveFile(DifferentialExpressionAnalysis analysis)
Delete a diff. ex. analysis archive file for a given analysis.- Returns:
- true if any file was deleted
-
deleteCoexpressionDataFile
boolean deleteCoexpressionDataFile(ExpressionExperiment ee)
Delete a coexpression data file if it exists.
-
getMetadataFile
Optional<LockedPath> getMetadataFile(ExpressionExperiment ee, ExpressionExperimentMetaFileType type, boolean exclusive) throws IOException
Locate a metadata file. If the metadata file is represented by a directory (i.e.ExpressionExperimentMetaFileType.isDirectory()
is true), then the latest file in the directory is returned. If no such file exists.Optional.empty()
is returned.If the data file is being written (i.e. with
copyMetadataFile(ExpressionExperiment, Path, ExpressionExperimentMetaFileType, boolean)
, this method will block until it is completed.- Returns:
- the metadata file or empty if none is present in the case of a directory-structured metadata
- Throws:
IOException
-
getMetadataFile
LockedPath getMetadataFile(ExpressionExperiment ee, String filename, boolean exclusive) throws IOException
- Throws:
IOException
-
getMetadataFile
LockedPath getMetadataFile(ExpressionExperiment ee, String filename, boolean exclusive, long timeout, TimeUnit timeUnit) throws InterruptedException, TimeoutException, IOException
-
copyMetadataFile
Path copyMetadataFile(ExpressionExperiment ee, Path existingFile, ExpressionExperimentMetaFileType type, boolean forceWrite) throws IOException
Copy a metadata file to the location of a given metadata type.Writing to a directory (i.e.
ExpressionExperimentMetaFileType.isDirectory()
is true) is not supported.If the metadata file is in use, this method will block until it is released.
- Parameters:
existingFile
- file to copy, must existforceWrite
- override any existing metadata file- Returns:
- the resulting metadata file, which can also be retrieved with
getMetadataFile(ExpressionExperiment, ExpressionExperimentMetaFileType, boolean)
- Throws:
IOException
-
copyMetadataFile
Path copyMetadataFile(ExpressionExperiment ee, Path existingFile, String filename, boolean forceWrite) throws IOException
Copy a generic metadata file.This can only be used for metadata files that are not listed in
ExpressionExperimentMetaFileType
; using any of these reserved filenames will result in an exception. In addition,CHANGELOG.md
is also reserved, useExpressionMetadataChangelogFileService
to manipulate it.- Throws:
IOException
-
copyMultiQCReport
Path copyMultiQCReport(ExpressionExperiment ee, Path existingFile, boolean forceWrite) throws IOException
Copy a MultiQC report.This is a helper based on
copyMetadataFile(ExpressionExperiment, Path, String, boolean)
that will automatically copy the main HTML report file as well as the log and JSON data files if they exist.:w- Throws:
IOException
-
deleteMetadataFile
boolean deleteMetadataFile(ExpressionExperiment ee, ExpressionExperimentMetaFileType type) throws IOException
Delete a metadata file.If the metadata file is organized as a directory, it is deleted recursively.
If the metadata file is in use, this method will block until it is released.
- Parameters:
type
- the type of metadata file to delete- Returns:
- true if a metadata file was deleted
- Throws:
IOException
-
getDataFile
LockedPath getDataFile(String filename, boolean exclusive) throws IOException
Locate any data file in the data directory.- Parameters:
exclusive
- if true, acquire an exclusive lock on the file- Throws:
IOException
-
getDataFile
LockedPath getDataFile(String filename, boolean exclusive, long timeout, TimeUnit timeUnit) throws IOException, InterruptedException, TimeoutException
-
getDataFile
LockedPath getDataFile(ExpressionExperiment ee, QuantitationType qt, ExpressionExperimentDataFileType type, boolean exclusive, long timeout, TimeUnit timeUnit) throws IOException, InterruptedException, TimeoutException
-
deleteDataFile
boolean deleteDataFile(ExpressionExperiment ee, QuantitationType qt, ExpressionExperimentDataFileType type) throws IOException
Delete a raw or single-cell data file if it exists.If the data file is in use, this method will block until it is released.
- Returns:
- true if the file was deleted, false if it did not exist
- Throws:
IOException
-
deleteDataFile
boolean deleteDataFile(ExpressionExperiment ee, boolean filtered, ExpressionExperimentDataFileType type) throws IOException
Delete a processed data file if it exists.If the data file is in use, this method will block until it is released.
- Returns:
- true if the file was deleted, false if it did not exist
- Throws:
IOException
-
writeTabularSingleCellExpressionData
int writeTabularSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, @Nullable ScaleType scaleType, int fetchSize, Writer writer, boolean autoFlush) throws IOException
Write single-cell expression data to a given writer for a given quantitation type in tabular format.- Parameters:
ee
- the experiment to useqt
- the quantitation type to retrievescaleType
- a scale type to use or null to leave the data untransformedfetchSize
- retrieve data in a streaming fashion- Throws:
IOException
- See Also:
TabularMatrixWriter
-
writeTabularSingleCellExpressionData
int writeTabularSingleCellExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, @Nullable ScaleType scaleType, int fetchSize, Writer writer, boolean autoFlush) throws IOException
- Throws:
IOException
-
writeOrLocateTabularSingleCellExpressionData
LockedPath writeOrLocateTabularSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean forceWrite) throws IOException
Write single-cell expression data to a standard location for a given quantitation type in tabular format.- Returns:
- a path where the vectors were written
- Throws:
IOException
- See Also:
writeTabularSingleCellExpressionData(ExpressionExperiment, QuantitationType, ScaleType, int, Writer, boolean)
,TabularMatrixWriter
-
writeOrLocateTabularSingleCellExpressionDataAsync
Future<Path> writeOrLocateTabularSingleCellExpressionDataAsync(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean forceWrite) throws RejectedExecutionException
- Throws:
RejectedExecutionException
- if the queue for creating data files is full- See Also:
writeOrLocateTabularSingleCellExpressionData(ExpressionExperiment, QuantitationType, int, boolean)
-
writeMexSingleCellExpressionData
int writeMexSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, @Nullable ScaleType scaleType, boolean useEnsemblIds, OutputStream stream) throws IOException
Write single-cell expression data to a given output stream for a given quantitation type.Note: this method is memory intensive because the whole matrix needs to be in-memory to write each individual sample matrices. Thus, no streaming is possible.
- Parameters:
ee
- the experiment to useqt
- the quantitation type to retrievescaleType
- a scale type to use or null to leave the data untransformeduseEnsemblIds
- use Ensembl IDs instead of official gene symbols- Throws:
IOException
- See Also:
MexMatrixWriter
-
writeMexSingleCellExpressionData
int writeMexSingleCellExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, @Nullable ScaleType scaleType, boolean useEnsemblIds, OutputStream stream) throws IOException
- Throws:
IOException
-
writeMexSingleCellExpressionData
int writeMexSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, @Nullable ScaleType scaleType, boolean useEnsemblIds, int fetchSize, boolean forceWrite, Path destDir) throws IOException
Write single-cell expression data to a given output stream for a given quantitation type.- Parameters:
ee
- the experiment to useqt
- the quantitation type to retrievescaleType
- a scale type to use or null to leave the data untransformeduseEnsemblIds
- use Ensembl IDs instead of official gene symbolsfetchSize
- fetch size to use for streaming, or load everything in memory of zero or lessforceWrite
- whether to force write and ignore any pre-existing directory- Throws:
IOException
- See Also:
MexMatrixWriter
-
writeMexSingleCellExpressionData
int writeMexSingleCellExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, @Nullable ScaleType scaleType, boolean useEnsemblIds, int fetchSize, boolean forceWrite, Path destDir) throws IOException
- Throws:
IOException
-
writeOrLocateMexSingleCellExpressionData
LockedPath writeOrLocateMexSingleCellExpressionData(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean forceWrite) throws IOException
Write single-cell expression data to a standard location for a given quantitation type.- Returns:
- a path where the vectors were written
- Throws:
IOException
- See Also:
writeMexSingleCellExpressionData(ExpressionExperiment, QuantitationType, ScaleType, boolean, int, boolean, Path)
,MexMatrixWriter
-
writeOrLocateMexSingleCellExpressionDataAsync
Future<Path> writeOrLocateMexSingleCellExpressionDataAsync(ExpressionExperiment ee, QuantitationType qt, int fetchSize, boolean forceWrite) throws RejectedExecutionException
- Throws:
RejectedExecutionException
- if the queue for creating data files is full- See Also:
writeOrLocateMexSingleCellExpressionData(ExpressionExperiment, QuantitationType, int, boolean)
-
writeRawExpressionData
int writeRawExpressionData(ExpressionExperiment ee, QuantitationType qt, @Nullable ScaleType scaleType, Writer writer, boolean autoFlush) throws IOException
Write raw expression data to a given writer for a given quantitation type.Note: For compression, wrap a
GZIPOutputStream
with aOutputStreamWriter
. To write to a string, consider usingStringWriter
.- Parameters:
ee
- the expression experimentqt
- a quantitation type to usescaleType
- a scale type to use or null to leave the data untransformedwriter
- the destination for the raw expression data- Throws:
IOException
- if operations with the writer fails
-
writeRawExpressionData
int writeRawExpressionData(ExpressionExperiment ee, List<BioAssay> samples, QuantitationType qt, @Nullable ScaleType scaleType, Writer writer, boolean autoFlush) throws IOException
- Throws:
IOException
-
writeProcessedExpressionData
int writeProcessedExpressionData(ExpressionExperiment ee, boolean filtered, @Nullable ScaleType scaleType, Writer writer, boolean autoFlush) throws FilteringException, IOException
Write processed expression data to a given writer for a given quantitation type.Note: For compression, wrap a
GZIPOutputStream
with aOutputStreamWriter
. To write to a string, consider usingStringWriter
.- Parameters:
ee
- the expression experimentwriter
- the destination for the raw expression data- Throws:
IOException
- if operations with the writer failsFilteringException
-
writeProcessedExpressionData
int writeProcessedExpressionData(ExpressionExperiment ee, List<BioAssay> samples, boolean filtered, @Nullable ScaleType scaleType, Writer writer, boolean autoFlush) throws FilteringException, IOException
- Throws:
FilteringException
IOException
-
writeDesignMatrix
void writeDesignMatrix(ExpressionExperiment ee, Writer writer) throws IOException
Writes out the experimental design for the given experiment.The bioassays (col 0) matches the header row of the data matrix printed out by the
MatrixWriter
.- Throws:
IOException
- See Also:
ExperimentalDesignWriter
-
writeOrLocateCoexpressionDataFile
LockedPath writeOrLocateCoexpressionDataFile(ExpressionExperiment ee, boolean forceWrite) throws IOException
Write or located the coexpression data file for a given experiment- Parameters:
ee
- the experimentforceWrite
- whether to force write- Returns:
- file
- Throws:
IOException
-
writeOrLocateProcessedDataFile
Optional<LockedPath> writeOrLocateProcessedDataFile(ExpressionExperiment ee, boolean filtered, boolean forceWrite) throws FilteringException, IOException
Locate or create a data file containing the 'preferred and masked' expression data matrix, with filtering for low expression applied (currently supports default settings only). It will be gzip-compressed. The file will be regenerated even if one already exists if the forceWrite parameter is true, or if there was a recent change (more recent than the last modified date of the existing file) to any of the experiments platforms.- Parameters:
ee
- the experimentfiltered
- filteredforceWrite
- force re-write even if file already exists and is up to date.- Returns:
- file, or empty if the experiment has no processed vectors
- Throws:
FilteringException
IOException
-
writeOrLocateProcessedDataFile
Optional<LockedPath> writeOrLocateProcessedDataFile(ExpressionExperiment ee, boolean filtered, boolean forceWrite, long timeout, TimeUnit timeUnit) throws TimeoutException, IOException, InterruptedException, FilteringException
-
writeOrLocateRawExpressionDataFile
LockedPath writeOrLocateRawExpressionDataFile(ExpressionExperiment ee, QuantitationType type, boolean forceWrite) throws IOException
Locate or create a new data file for the given quantitation type. The output will include gene information if it can be located from its own file.- Parameters:
forceWrite
- To not return the existing file, but create it anew.type
- the quantitation type- Returns:
- file
- Throws:
IOException
-
writeOrLocateRawExpressionDataFile
LockedPath writeOrLocateRawExpressionDataFile(ExpressionExperiment ee, QuantitationType qt, boolean forceWrite, long timeout, TimeUnit timeUnit) throws TimeoutException, IOException, InterruptedException
-
writeOrLocateDesignFile
Optional<LockedPath> writeOrLocateDesignFile(ExpressionExperiment ee, boolean forceWrite) throws IOException
Locate or create an experimental design file for a given experiment. The file will be regenerated even if one already exists if the forceWrite parameter is true, or if there was a recent change (more recent than the last modified date of the existing file) to any of the experiments platforms.- Parameters:
ee
- the experimentforceWrite
- force re-write even if file already exists and is up to date- Returns:
- a file or empty if the experiment does not have a design
- Throws:
IOException
- See Also:
writeDesignMatrix(ExpressionExperiment, Writer)
-
writeOrLocateDesignFile
Optional<LockedPath> writeOrLocateDesignFile(ExpressionExperiment ee, boolean forceWrite, long timeout, TimeUnit timeUnit) throws TimeoutException, IOException, InterruptedException
-
writeOrLocateJSONProcessedExpressionDataFile
Optional<LockedPath> writeOrLocateJSONProcessedExpressionDataFile(ExpressionExperiment ee, boolean forceWrite, boolean filtered) throws FilteringException, IOException
-
writeOrLocateJSONRawExpressionDataFile
LockedPath writeOrLocateJSONRawExpressionDataFile(ExpressionExperiment ee, QuantitationType type, boolean forceWrite) throws IOException
-
writeOrLocateDiffExpressionDataFiles
Collection<LockedPath> writeOrLocateDiffExpressionDataFiles(ExpressionExperiment ee, boolean forceWrite) throws IOException
Locate or create the differential expression data file(s) for a given experiment.- Parameters:
ee
- the experimentforceWrite
- whether to force write- Returns:
- collection of files, one per analysis.
- Throws:
IOException
-
writeOrLocateDiffExArchiveFile
LockedPath writeOrLocateDiffExArchiveFile(Long analysisId, boolean forceCreate) throws IOException
Locate or create the differential expression data file(s) for a given experiment. We generate an archive that contains following files: - differential expression analysis file (q-values per factor) - file for each result set with contrasts info (such as fold change for each factor value)- Parameters:
analysisId
- analysis IDforceCreate
- whether to force creation- Returns:
- file
- Throws:
IOException
-
writeDiffExAnalysisArchiveFile
LockedPath writeDiffExAnalysisArchiveFile(DifferentialExpressionAnalysis analysis, @Nullable DifferentialExpressionAnalysisConfig config) throws IOException
- Throws:
IOException
-
-