Class ProcessedExpressionDataVectorServiceImpl

java.lang.Object

ubic.gemma.persistence.service.AbstractService<ProcessedExpressionDataVector>

ubic.gemma.persistence.service.expression.bioAssayData.AbstractBulkExpressionDataVectorService<ProcessedExpressionDataVector>

ubic.gemma.persistence.service.expression.bioAssayData.ProcessedExpressionDataVectorServiceImpl

All Implemented Interfaces:

BaseImmutableService<ProcessedExpressionDataVector>, BaseReadOnlyService<ProcessedExpressionDataVector>, BaseService<ProcessedExpressionDataVector>, BulkExpressionDataVectorService<ProcessedExpressionDataVector>, ProcessedExpressionDataVectorService

@Service
public class ProcessedExpressionDataVectorServiceImpl
extends AbstractBulkExpressionDataVectorService<ProcessedExpressionDataVector>
implements ProcessedExpressionDataVectorService

Author:

Paul

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	ProcessedExpressionDataVectorServiceImpl(ProcessedExpressionDataVectorDao mainDao)

Method Summary

Modifier and Type	Method	Description
ExpressionDataDoubleMatrix	computeUnmaskedProcessedDataMatrix(ExpressionExperiment expressionExperiment, boolean ignoreQuantitationMismatch)	🛑 Propagation.NEVER, for the same reason as createProcessedDataVectors(ExpressionExperiment, boolean): the normalization between the read and the assembly is where the 44 minutes go, and nothing in it needs a connection.
QuantitationType	createProcessedDataVectors(ExpressionExperiment expressionExperiment, boolean updateRanks)	Create processed vectors and optionally update ranks.
QuantitationType	createProcessedDataVectors(ExpressionExperiment expressionExperiment, boolean updateRanks, boolean ignoreQuantitationMismatch)	🛑 Propagation.NEVER: the quantile normalization in the middle of this takes 44 minutes on a large experiment (GSE260875, measured 2026-09-17) and the connection handling mode is DELAYED_ACQUISITION_AND_HOLD, so a transaction around the whole thing pins a pooled connection for the duration without issuing a statement — against a pool that recycles at 30 minutes.
void	evictFromCache(ExpressionExperiment ee)
Collection<ProcessedExpressionDataVector>	find(Collection<CompositeSequence> designElements, QuantitationType quantitationType)	Retrieve the processed vectors for a set of design elements within a given (processed) quantitation type.
List<DoubleVectorValueObject>	getDiffExVectors(Long resultSetId, double threshold, int maxNumberOfResults)
List<ExperimentExpressionLevelsValueObject>	getExpressionLevels(Collection<ExpressionExperiment> ees, Collection<Gene> genes, boolean keepGeneNonSpecific, String consolidateMode)
List<ExperimentExpressionLevelsValueObject>	getExpressionLevelsByIds(Collection<Long> eeIds, Collection<Gene> genes, boolean keepGeneNonSpecific, String consolidateMode)	Retrieve expression levels by dataset IDs.
List<ExperimentExpressionLevelsValueObject>	getExpressionLevelsDiffEx(Collection<ExpressionExperiment> ees, Long diffExResultSetId, double threshold, int max, boolean keepGeneNonSpecific, String consolidateMode)
List<ExperimentExpressionLevelsValueObject>	getExpressionLevelsPca(Collection<ExpressionExperiment> ees, int limit, int component, boolean keepGeneNonSpecific, String consolidateMode)
Collection<DoubleVectorValueObject>	getProcessedDataArrays(Collection<ExpressionExperiment> expressionExperiments, Collection<Long> genes)
Collection<DoubleVectorValueObject>	getProcessedDataArrays(BioAssaySet bioAssaySet, Collection<Long> genes)
Collection<DoubleVectorValueObject>	getProcessedDataArrays(ExpressionExperiment expressionExperiment)
Collection<DoubleVectorValueObject>	getProcessedDataArraysByProbe(Collection<ExpressionExperiment> expressionExperiments, Collection<CompositeSequence> compositeSequences)
Collection<DoubleVectorValueObject>	getProcessedDataArraysByProbe(ExpressionExperiment ee, Collection<CompositeSequence> compositeSequences)
Collection<ProcessedExpressionDataVector>	getProcessedDataVectors(ExpressionExperiment expressionExperiment)
Slice<ProcessedExpressionDataVector>	getProcessedDataVectors(ExpressionExperiment expressionExperiment, BioAssayDimension dimension, int offset, int limit)
Collection<ProcessedExpressionDataVector>	getProcessedDataVectorsAndThaw(ExpressionExperiment expressionExperiment)	Retrieve and thaw a collection of vectors for a given experiment.
Slice<CompositeSequence>	getProcessedDataVectorsDesignElements(ExpressionExperiment expressionExperiment, BioAssayDimension dimension, int offset, int limit)
Collection<DoubleVectorValueObject>	getRandomProcessedDataArrays(ExpressionExperiment ee, int limit)
Map<ExpressionExperiment, Map<Gene, Collection<Double>>>	getRanks(Collection<ExpressionExperiment> expressionExperiments, Collection<Gene> genes, ProcessedExpressionDataVectorDao.RankMethod method)
int	removeProcessedDataVectors(ExpressionExperiment ee)	Remove the processed vectors of an EE.
void	reorderByDesign(ExpressionExperiment ee)	Creates new bioAssayDimensions to match the experimental design, reorders the data to match, updates.
int	replaceProcessedDataVectors(ExpressionExperiment ee, Collection<ProcessedExpressionDataVector> vectors, boolean updateRanks)	Replace the processed vectors of a EE with the given vectors.
void	updateRanks(ExpressionExperiment ee)	Update the ranks of the processed vectors for the given experiment.

Methods inherited from class AbstractBulkExpressionDataVectorService

find, find, find, findAndThaw, findAndThaw, findAndThaw, thaw, thaw

Methods inherited from class AbstractService

countAll, create, create, ensureInSession, ensureInSession, find, findOrCreate, findOrFail, getElementClass, load, load, loadAll, loadOrFail, loadOrFail, loadOrFail, loadOrFail, loadOrFail, loadOrFail, remove, remove, save, save, streamAll, streamAll, update, update

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface BaseReadOnlyService

countAll, find, findOrFail, getElementClass, load, load, loadAll, loadOrFail, loadOrFail, loadOrFail, loadOrFail, loadOrFail, loadOrFail, streamAll, streamAll

Methods inherited from interface BulkExpressionDataVectorService

find, find, find, findAndThaw, findAndThaw, findAndThaw, thaw, thaw

Constructor Details

ProcessedExpressionDataVectorServiceImpl

@Autowired
protected ProcessedExpressionDataVectorServiceImpl(ProcessedExpressionDataVectorDao mainDao)

Method Details

createProcessedDataVectors

@Transactional(propagation=NEVER)
public QuantitationType createProcessedDataVectors(ExpressionExperiment expressionExperiment,
 boolean updateRanks)
                                            throws QuantitationTypeConversionException

Description copied from interface: ProcessedExpressionDataVectorService

Create processed vectors and optionally update ranks.

Mismatch between quantitation type and data is ignored.

This also adds an audit event and evict the vectors from the cache.

Specified by:

createProcessedDataVectors in interface ProcessedExpressionDataVectorService

Parameters:

updateRanks - whether to update the rnaks of the vectors or not

Returns:

the created processed QT

Throws:

QuantitationTypeConversionException - if the data cannot be converted, generally to log2 scale

See Also:

• ProcessedExpressionDataVectorService.updateRanks(ExpressionExperiment)
• ExpressionExperimentService.createProcessedDataVectors(ExpressionExperiment, Collection)

createProcessedDataVectors

@Transactional(propagation=NEVER)
@AuditedOnError(value=FailedProcessedVectorComputationEvent.class,
                message="Failed to create processed expression data vectors.")
public QuantitationType createProcessedDataVectors(ExpressionExperiment expressionExperiment,
 boolean updateRanks,
 boolean ignoreQuantitationMismatch)
                                            throws QuantitationTypeDetectionException,
QuantitationTypeConversionException

🛑 Propagation.NEVER: the quantile normalization in the middle of this takes 44 minutes on a large experiment (GSE260875, measured 2026-09-17) and the connection handling mode is DELAYED_ACQUISITION_AND_HOLD, so a transaction around the whole thing pins a pooled connection for the duration without issuing a statement — against a pool that recycles at 30 minutes. Read, compute and write are three steps on the helper bean, each with its own transaction or none.

updateRanks goes through self rather than this because a self-invocation does not pass the proxy, and with no transaction inherited from here it would run without one at all.

Specified by:

createProcessedDataVectors in interface ProcessedExpressionDataVectorService

Returns:

the created processed QT

Throws:

QuantitationTypeDetectionException - if the QT caanot be detected from data, never raised if ignoreQuantitationMismatch is set to true

QuantitationTypeConversionException - if the data cannot be converted, generally to log2 scale

See Also:

• ProcessedExpressionDataVectorService.createProcessedDataVectors(ExpressionExperiment, boolean)
• ProcessedExpressionDataVectorService.updateRanks(ExpressionExperiment)

computeUnmaskedProcessedDataMatrix

@Transactional(propagation=NEVER)
public ExpressionDataDoubleMatrix computeUnmaskedProcessedDataMatrix(ExpressionExperiment expressionExperiment,
 boolean ignoreQuantitationMismatch)
                                                              throws QuantitationTypeDetectionException,
QuantitationTypeConversionException

🛑 Propagation.NEVER, for the same reason as createProcessedDataVectors(ExpressionExperiment, boolean): the normalization between the read and the assembly is where the 44 minutes go, and nothing in it needs a connection.

Specified by:

computeUnmaskedProcessedDataMatrix in interface ProcessedExpressionDataVectorService

Throws:

QuantitationTypeDetectionException

QuantitationTypeConversionException

replaceProcessedDataVectors

@Transactional
@Audited(value=ProcessedVectorComputationEvent.class,
         message="Replaced processed expression data.")
@AuditedOnError(value=FailedProcessedVectorComputationEvent.class,
                message="Failed to replace processed expression data vectors.")
public int replaceProcessedDataVectors(ExpressionExperiment ee,
 Collection<ProcessedExpressionDataVector> vectors,
 boolean updateRanks)

Description copied from interface: ProcessedExpressionDataVectorService

Replace the processed vectors of a EE with the given vectors.

Ranks are recomputed, no conversion of QT is done.

This also adds an audit event and evict the vectors from the cache.

Specified by:

replaceProcessedDataVectors in interface ProcessedExpressionDataVectorService

Parameters:

ee - ee

vectors - non-persistent, all of the same QuantitationType

updateRanks - whether to update ranks or not

See Also:

• ExpressionExperimentService.createProcessedDataVectors(ExpressionExperiment, Collection)

removeProcessedDataVectors

@Transactional
@Audited(value=ProcessedVectorComputationEvent.class,
         message="Removed processed expression data.")
@AuditedOnError(value=FailedProcessedVectorComputationEvent.class,
                message="Failed to remove processed expression data vectors.")
public int removeProcessedDataVectors(ExpressionExperiment ee)

Description copied from interface: ProcessedExpressionDataVectorService

Remove the processed vectors of an EE.

This also adds an audit event and evict the vectors from the cache.

Specified by:

removeProcessedDataVectors in interface ProcessedExpressionDataVectorService

See Also:

• ExpressionExperimentService.removeProcessedDataVectors(ExpressionExperiment)

reorderByDesign

@Transactional
@Audited(value=VectorsReorderedEvent.class,
         message="Reordered the data vectors by experimental design")
public void reorderByDesign(ExpressionExperiment ee)

Description copied from interface: ProcessedExpressionDataVectorService

Creates new bioAssayDimensions to match the experimental design, reorders the data to match, updates.

This also adds an audit event and evict the vectors from the cache.

Specified by:

reorderByDesign in interface ProcessedExpressionDataVectorService

updateRanks

@Transactional
@AuditedOnError(value=FailedProcessedVectorComputationEvent.class,
                message="Failed to update ranks for expression data vectors.")
public void updateRanks(ExpressionExperiment ee)

Description copied from interface: ProcessedExpressionDataVectorService

Update the ranks of the processed vectors for the given experiment.

This also adds an audit event and evict the vectors from the cache.

Specified by:

updateRanks in interface ProcessedExpressionDataVectorService

getExpressionLevels

@Transactional(readOnly=true)
public List<ExperimentExpressionLevelsValueObject> getExpressionLevels(Collection<ExpressionExperiment> ees,
 Collection<Gene> genes,
 boolean keepGeneNonSpecific,
 @Nullable
 String consolidateMode)

Specified by:

getExpressionLevels in interface ProcessedExpressionDataVectorService

Parameters:

ees - expressionExperiments

genes - genes

keepGeneNonSpecific - whether to keep vectors that are not specific to the gene

consolidateMode - how to consolidate the vectors when there is more than one

Returns:

value objects containing structured information about the expression levels of given genes

getExpressionLevelsByIds

@Transactional(readOnly=true)
public List<ExperimentExpressionLevelsValueObject> getExpressionLevelsByIds(Collection<Long> eeIds,
 Collection<Gene> genes,
 boolean keepGeneNonSpecific,
 @Nullable
 String consolidateMode)

Description copied from interface: ProcessedExpressionDataVectorService

Retrieve expression levels by dataset IDs.

Specified by:

getExpressionLevelsByIds in interface ProcessedExpressionDataVectorService

See Also:

• ProcessedExpressionDataVectorService.getExpressionLevels(Collection, Collection, boolean, String)

getExpressionLevelsPca

@Transactional(readOnly=true)
public List<ExperimentExpressionLevelsValueObject> getExpressionLevelsPca(Collection<ExpressionExperiment> ees,
 int limit,
 int component,
 boolean keepGeneNonSpecific,
 @Nullable
 String consolidateMode)

Specified by:

getExpressionLevelsPca in interface ProcessedExpressionDataVectorService

Parameters:

ees - expressionExperiments

limit - threshold

component - the principal component

keepGeneNonSpecific - whether to keep vectors that are not specific to the gene

consolidateMode - how to consolidate the vectors when there is more than one

Returns:

value objects containing structured information about the expression levels of genes highly loaded in the given principal component.

getExpressionLevelsDiffEx

@Transactional(readOnly=true)
public List<ExperimentExpressionLevelsValueObject> getExpressionLevelsDiffEx(Collection<ExpressionExperiment> ees,
 Long diffExResultSetId,
 double threshold,
 int max,
 boolean keepGeneNonSpecific,
 @Nullable
 String consolidateMode)

Specified by:

getExpressionLevelsDiffEx in interface ProcessedExpressionDataVectorService

Parameters:

ees - ees

diffExResultSetId - the differential expression result set to access

threshold - threshold

max - max level

keepGeneNonSpecific - whether to keep vectors that are not specific to the gene

consolidateMode - how to consolidate the vectors when there is more than one

Returns:

value objects containing structured information about the expression levels of genes highly loaded in the given principal component.

getProcessedDataArrays

@Transactional(readOnly=true)
public Collection<DoubleVectorValueObject> getProcessedDataArrays(Collection<ExpressionExperiment> expressionExperiments,
 Collection<Long> genes)

Specified by:

getProcessedDataArrays in interface ProcessedExpressionDataVectorService

See Also:

• CachedProcessedExpressionDataVectorService.getProcessedDataArrays(Collection, Collection)

getProcessedDataArrays

@Transactional(readOnly=true)
public Collection<DoubleVectorValueObject> getProcessedDataArrays(BioAssaySet bioAssaySet,
 Collection<Long> genes)

Specified by:

getProcessedDataArrays in interface ProcessedExpressionDataVectorService

See Also:

• CachedProcessedExpressionDataVectorService.getProcessedDataArrays(BioAssaySet, Collection)

getProcessedDataArrays

@Transactional(readOnly=true)
public Collection<DoubleVectorValueObject> getProcessedDataArrays(ExpressionExperiment expressionExperiment)

Specified by:

getProcessedDataArrays in interface ProcessedExpressionDataVectorService

See Also:

• CachedProcessedExpressionDataVectorService.getProcessedDataArrays(BioAssaySet)

getRandomProcessedDataArrays

@Transactional(readOnly=true)
public Collection<DoubleVectorValueObject> getRandomProcessedDataArrays(ExpressionExperiment ee,
 int limit)

Specified by:

getRandomProcessedDataArrays in interface ProcessedExpressionDataVectorService

See Also:

• CachedProcessedExpressionDataVectorService.getRandomProcessedDataArrays(BioAssaySet, int)

getProcessedDataArraysByProbe

@Transactional(readOnly=true)
public Collection<DoubleVectorValueObject> getProcessedDataArraysByProbe(ExpressionExperiment ee,
 Collection<CompositeSequence> compositeSequences)

Specified by:

getProcessedDataArraysByProbe in interface ProcessedExpressionDataVectorService

getProcessedDataArraysByProbe

@Transactional(readOnly=true)
public Collection<DoubleVectorValueObject> getProcessedDataArraysByProbe(Collection<ExpressionExperiment> expressionExperiments,
 Collection<CompositeSequence> compositeSequences)

Specified by:

getProcessedDataArraysByProbe in interface ProcessedExpressionDataVectorService

See Also:

• CachedProcessedExpressionDataVectorService.getProcessedDataArraysByProbe(Collection, Collection)

find

@Transactional(readOnly=true)
public Collection<ProcessedExpressionDataVector> find(Collection<CompositeSequence> designElements,
 QuantitationType quantitationType)

Description copied from interface: ProcessedExpressionDataVectorService

Retrieve the processed vectors for a set of design elements within a given (processed) quantitation type. Unlike getProcessedDataArraysByProbe(...) these are raw entities whose data has NOT had read-time outlier masking applied, so the caller can recover un-masked values for those probes.

Specified by:

find in interface ProcessedExpressionDataVectorService

See Also:

• ProcessedExpressionDataVectorDao.find(Collection, QuantitationType)

getProcessedDataVectors

@Transactional(readOnly=true)
public Collection<ProcessedExpressionDataVector> getProcessedDataVectors(ExpressionExperiment expressionExperiment)

Specified by:

getProcessedDataVectors in interface ProcessedExpressionDataVectorService

See Also:

• ProcessedExpressionDataVectorDao.getProcessedVectors(ExpressionExperiment)

getProcessedDataVectors

@Transactional(readOnly=true)
public Slice<ProcessedExpressionDataVector> getProcessedDataVectors(ExpressionExperiment expressionExperiment,
 BioAssayDimension dimension,
 int offset,
 int limit)

Specified by:

getProcessedDataVectors in interface ProcessedExpressionDataVectorService

getProcessedDataVectorsDesignElements

@Transactional(readOnly=true)
public Slice<CompositeSequence> getProcessedDataVectorsDesignElements(ExpressionExperiment expressionExperiment,
 BioAssayDimension dimension,
 int offset,
 int limit)

Specified by:

getProcessedDataVectorsDesignElements in interface ProcessedExpressionDataVectorService

getProcessedDataVectorsAndThaw

@Transactional(readOnly=true)
public Collection<ProcessedExpressionDataVector> getProcessedDataVectorsAndThaw(ExpressionExperiment expressionExperiment)

Description copied from interface: ProcessedExpressionDataVectorService

Retrieve and thaw a collection of vectors for a given experiment.

Specified by:

getProcessedDataVectorsAndThaw in interface ProcessedExpressionDataVectorService

See Also:

• ProcessedExpressionDataVectorDao.getProcessedVectors(ExpressionExperiment)
• DesignElementDataVectorDao.thaw(Collection)

getRanks

@Transactional(readOnly=true)
public Map<ExpressionExperiment, Map<Gene, Collection<Double>>> getRanks(Collection<ExpressionExperiment> expressionExperiments,
 Collection<Gene> genes,
 ProcessedExpressionDataVectorDao.RankMethod method)

Specified by:

getRanks in interface ProcessedExpressionDataVectorService

See Also:

• ProcessedExpressionDataVectorDao.getRanks(ExpressionExperiment, Collection, ProcessedExpressionDataVectorDao.RankMethod)

getDiffExVectors

@Transactional(readOnly=true)
public List<DoubleVectorValueObject> getDiffExVectors(Long resultSetId,
 double threshold,
 int maxNumberOfResults)

Specified by:

getDiffExVectors in interface ProcessedExpressionDataVectorService

evictFromCache

public void evictFromCache(ExpressionExperiment ee)

Specified by:

evictFromCache in interface ProcessedExpressionDataVectorService