Package ubic.gemma.apps

Class ExpressionExperimentManipulatingCLI

java.lang.Object

ubic.gemma.cli.util.AbstractCLI

ubic.gemma.cli.util.AbstractAuthenticatedCLI

ubic.gemma.cli.util.AbstractAutoSeekingCLI<ExpressionExperiment>

ubic.gemma.apps.ExpressionExperimentManipulatingCLI

All Implemented Interfaces:

org.springframework.beans.factory.Aware, org.springframework.beans.factory.InitializingBean, org.springframework.context.ApplicationContextAware, org.springframework.context.EnvironmentAware, CLI

Direct Known Subclasses:

AffyDataFromCelCli, BatchEffectPopulationCli, DeleteDiffExCli, DeleteExperimentsCli, DifferentialExpressionAnalysisCli, DifferentialExpressionAnalysisWriterCli, ExperimentalDesignImportCli, ExperimentalDesignWriterCLI, ExpressionDataCorrMatCli, ExpressionDataMatrixWriterCLI, ExpressionExperimentDataFileGeneratorCli, ExpressionExperimentDataUpdaterCli, ExpressionExperimentMetadataChangelogEntryAdderCli, ExpressionExperimentMetadataChangelogViewerCli, ExpressionExperimentMetadataFileAdderCli, ExpressionExperimentPlatformSwitchCli, ExpressionExperimentPrimaryPubCli, ExpressionExperimentVectorsManipulatingCli, GeeqCli, LinkAnalysisCli, LockExpressionDataFileCli, MakeExperimentPrivateCli, MakeExperimentsPublicCli, OrderVectorsByDesignCli, ProcessedDataComputeCLI, RefreshExperimentCli, ReplaceDataCli, RNASeqBatchInfoCli, RNASeqDataAddCli, SingleCellCellTypeFactorCreatorCli, SingleCellDataLoaderCli, SingleCellSparsityMetricsUpdaterCli, SplitExperimentCli, SVDCli, UpdateEe2AdCli, UpdateEE2CCli, VectorMergingCli

public abstract class ExpressionExperimentManipulatingCLI
extends AbstractAutoSeekingCLI<ExpressionExperiment>

Base class for CLIs that needs one or more expression experiment as an input. It offers the following ways of reading them in:

• All EEs (if -all is supplied)
• All EEs for a particular taxon.
• A specific ExpressionExperimentSet, identified by name
• A comma-delimited list of one or more EEs identified by short name given on the command line
• From a file, with one short name per line.
• EEs matching a query string (e.g., 'brain')
• (Optional) 'Auto' mode, in which experiments to analyze are selected automatically based on their workflow state. This can be enabled and modified by subclasses who override the "needToRun" method.
• All EEs that were last processed after a given date, similar to 'auto' otherwise.

Some of these options can be (or should be) combined, and modified by a (optional) "force" option, and will have customized behavior.

In addition, EEs can be excluded based on a list given in a separate file.

Author:

Paul

Nested Class Summary

Nested classes/interfaces inherited from interface ubic.gemma.cli.util.CLI

CLI.CommandGroup

Field Summary

Fields

Modifier and Type	Field	Description
protected AuditEventService	auditEventService
protected AuditTrailService	auditTrailService
protected ExpressionExperimentService	eeService
protected EntityLocator	entityLocator
protected EntityUrlBuilder	entityUrlBuilder

Fields inherited from class ubic.gemma.cli.util.AbstractAutoSeekingCLI

FORCE_OPTION

Fields inherited from class ubic.gemma.cli.util.AbstractCLI

ABORTED, FAILURE, FAILURE_FROM_ERROR_OBJECTS, log, SUCCESS

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	ExpressionExperimentManipulatingCLI()

Method Summary

Modifier and Type	Method	Description
protected void	addCellMetadataFileOptions(org.apache.commons.cli.Options options, String what, boolean allowStandardLocation)
protected void	addDataFileOptions(org.apache.commons.cli.Options options, String what, boolean allowStandardLocation)
protected void	addDataFileOptions(org.apache.commons.cli.Options options, String what, boolean allowStandardLocation, boolean allowFile, boolean allowDirectory, boolean allowCurrentDirectory, boolean allowStdout)
protected void	addDesignFileOptions(org.apache.commons.cli.Options options, String what, boolean allowStandardLocation)
protected void	addExpressionDataFileOptions(org.apache.commons.cli.Options options, String what, boolean allowStandardLocation)
protected void	addSingleCellExpressionDataFileOptions(org.apache.commons.cli.Options options, String what, boolean allowStandardLocation)
protected void	addSingleExperimentAutoOption(org.apache.commons.cli.Options options, String opt, String longOpt, String description, String noOpt, String longNoOpt, String noDescription)	Add an auto option that is only available in single-experiment mode.
protected <T extends Enum<T>> void	addSingleExperimentEnumOption(org.apache.commons.cli.Options options, String opt, String longOpt, String description, Class<T> enumClass, EnumMap<T,org.springframework.context.MessageSourceResolvable> descriptions)	Add an enumerated option that is only available in single-experiment mode.
protected void	addSingleExperimentOption(org.apache.commons.cli.Options options, String opt, String longOpt, boolean hasArg, String description)	Add an option that is only available in single-experiment mode.
protected void	addSingleExperimentOption(org.apache.commons.cli.Options options, org.apache.commons.cli.Option option)	Add an option that is only available in single-experiment mode.
protected void	buildExperimentOptions(org.apache.commons.cli.Options options)
protected final void	buildOptions(org.apache.commons.cli.Options options)	Build option implementation.
protected void	doAuthenticatedWork()
protected String	formatExperiment(ExpressionExperiment bas)	Render an experiment to string, with special handling in case of an uninitialized proxy.
protected CellMetadataFileOptionValue	getCellMetadataFileOptionValue(org.apache.commons.cli.CommandLine commandLine, boolean allowStandardLocation)
CLI.CommandGroup	getCommandGroup()	Obtain the command group for this CLI.
protected DataFileOptionValue	getDataFileOptionValue(org.apache.commons.cli.CommandLine commandLine, boolean allowStandardLocation)
protected DataFileOptionValue	getDataFileOptionValue(org.apache.commons.cli.CommandLine commandLine, boolean allowStandardLocation, boolean allowStdout, boolean allowCurrentDirectory)
protected DesignFileOptionValue	getDesignFileOptionValue(org.apache.commons.cli.CommandLine commandLine, boolean allowStandardLocation)
protected ExpressionDataFileOptionValue	getExpressionDataFileOptionValue(org.apache.commons.cli.CommandLine commandLine, boolean allowStandardLocation)
protected SingleCellExpressionDataFileOptionValue	getSingleCellExpressionDataFileOptionValue(org.apache.commons.cli.CommandLine commandLine, boolean allowStandardLocation)
boolean	isAbortOnError()	Indicate if this CLI should abort on error or move on to the next experiment.
protected boolean	noNeedToRun(ExpressionExperiment auditable, Class<? extends AuditEventType> eventClass)	Check if the given auditable can be skipped.
protected void	postprocessExpressionExperiments(Collection<ExpressionExperiment> expressionExperiments)	Post-process ExpressionExperiments that were processed by the CLI.
protected Collection<ExpressionExperiment>	preprocessExpressionExperiments(Collection<ExpressionExperiment> expressionExperiments)	Preprocess the set of ExpressionExperiment before invoking processExpressionExperiments(Collection) or processExpressionExperiment(ExpressionExperiment).
protected void	processAllExpressionExperiments()	Process all ExpressionExperiment.
protected void	processExperimentOptions(org.apache.commons.cli.CommandLine commandLine)
protected void	processExpressionExperiment(ExpressionExperiment expressionExperiment)	Process an ExpressionExperiment.
protected void	processExpressionExperiments(Collection<ExpressionExperiment> expressionExperiments)	Process multiple ExpressionExperiment.
protected final void	processOptions(org.apache.commons.cli.CommandLine commandLine)	Process command line options.
protected String	readChangelogEntryFromConsole(ExpressionExperiment expressionExperiment, String defaultText)	Read a changelog entry from the console.
protected void	refreshExpressionExperimentFromGemmaWeb(ExpressionExperiment ee, boolean refreshVectors, boolean refreshReports)	Refresh a dataset for Gemma Web.
protected void	refreshExpressionExperimentFromGemmaWebSilently(ExpressionExperiment ee, boolean refreshVectors, boolean refreshReports)	A silent version of refreshExpressionExperimentFromGemmaWeb(ExpressionExperiment, boolean, boolean) that produces a warning via AbstractCLI.addWarningObject(Serializable, String, Throwable) instead of throwing an exception.
protected void	setAbortOnError()	Set this to stop processing experiments if an error occurs.
void	setAllIsLazy()	Enable the "all is lazy" mode.
void	setDefaultToAll()	Default to all datasets if no options are provided.
protected void	setSingleExperimentMode()	Enable the single-experiment mode.
protected void	setUseReferencesIfPossible()	Set this to allow reference to be retrieved instead of actual entities.
protected final Serializable	toBatchObject(ExpressionExperiment object)	Convert the given object to a serializable object for batch processing.

Methods inherited from class ubic.gemma.cli.util.AbstractAutoSeekingCLI

addAutoOption, addAutoOption, addErrorObject, addErrorObject, addErrorObject, addForceOption, addForceOption, addLimitingDateOption, addSuccessObject, addSuccessObject, addWarningObject, addWarningObject, getAutoSeekEventType, getLimitingDate, isAutoSeek, isForce, setForce

Methods inherited from class ubic.gemma.cli.util.AbstractAuthenticatedCLI

afterPropertiesSet, doWork, setAuthenticateAnonymously, setEnvironment, setRequireLogin

Methods inherited from class ubic.gemma.cli.util.AbstractCLI

abort, addBatchOption, addErrorObject, addErrorObject, addErrorObject, addSuccessObject, addSuccessObject, addThreadsOption, addWarningObject, addWarningObject, allowPositionalArguments, awaitBatchExecutorService, executeCommand, getApplicationContext, getBatchTaskExecutor, getBatchTaskProgressReporter, getCliContext, getCommandAliases, getCommandName, getHelpFooter, getNumThreads, getOptions, getShortDesc, getUsage, promptConfirmationOrAbort, setAllowPositionalArguments, setApplicationContext, setEstimatedMaxTasks, setReportFrequencyMillis

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

eeService

@Autowired
protected ExpressionExperimentService eeService

auditTrailService

@Autowired
protected AuditTrailService auditTrailService

auditEventService

@Autowired
protected AuditEventService auditEventService

entityLocator

@Autowired
protected EntityLocator entityLocator

entityUrlBuilder

@Autowired
protected EntityUrlBuilder entityUrlBuilder

Constructor Details

ExpressionExperimentManipulatingCLI

protected ExpressionExperimentManipulatingCLI()

Method Details

getCommandGroup

public CLI.CommandGroup getCommandGroup()

Description copied from interface: CLI

Obtain the command group for this CLI.

Specified by:

getCommandGroup in interface CLI

Overrides:

getCommandGroup in class AbstractCLI

addSingleExperimentOption

protected void addSingleExperimentOption(org.apache.commons.cli.Options options,
 org.apache.commons.cli.Option option)

Add an option that is only available in single-experiment mode.

addSingleExperimentOption

protected void addSingleExperimentOption(org.apache.commons.cli.Options options,
 String opt,
 String longOpt,
 boolean hasArg,
 String description)

Add an option that is only available in single-experiment mode.

addSingleExperimentAutoOption

protected void addSingleExperimentAutoOption(org.apache.commons.cli.Options options,
 String opt,
 String longOpt,
 String description,
 String noOpt,
 String longNoOpt,
 String noDescription)

Add an auto option that is only available in single-experiment mode.

See Also:

• OptionsUtils.addAutoOption(Options, String, String, String, String, String, String)

addSingleExperimentEnumOption

protected <T extends Enum<T>> void addSingleExperimentEnumOption(org.apache.commons.cli.Options options,
 String opt,
 String longOpt,
 String description,
 Class<T> enumClass,
 EnumMap<T,org.springframework.context.MessageSourceResolvable> descriptions)

Add an enumerated option that is only available in single-experiment mode.

See Also:

• OptionsUtils.addEnumOption(Options, String, String, String, Class, EnumMap)

addDataFileOptions

protected void addDataFileOptions(org.apache.commons.cli.Options options,
 String what,
 boolean allowStandardLocation)

See Also:

• DataFileOptionsUtils.addDataFileOptions(Options, String, boolean, BiConsumer)

addDataFileOptions

protected void addDataFileOptions(org.apache.commons.cli.Options options,
 String what,
 boolean allowStandardLocation,
 boolean allowFile,
 boolean allowDirectory,
 boolean allowCurrentDirectory,
 boolean allowStdout)

See Also:

• DataFileOptionsUtils.addDataFileOptions(Options, String, boolean, boolean, boolean, boolean, boolean, BiConsumer)

addExpressionDataFileOptions

protected void addExpressionDataFileOptions(org.apache.commons.cli.Options options,
 String what,
 boolean allowStandardLocation)

See Also:

• DataFileOptionsUtils.addExpressionDataFileOptions(Options, String, boolean, BiConsumer)

addSingleCellExpressionDataFileOptions

protected void addSingleCellExpressionDataFileOptions(org.apache.commons.cli.Options options,
 String what,
 boolean allowStandardLocation)

addDesignFileOptions

protected void addDesignFileOptions(org.apache.commons.cli.Options options,
 String what,
 boolean allowStandardLocation)

See Also:

• DataFileOptionsUtils.addDesignFileOptions(Options, String, boolean, BiConsumer)

addCellMetadataFileOptions

protected void addCellMetadataFileOptions(org.apache.commons.cli.Options options,
 String what,
 boolean allowStandardLocation)

See Also:

• DataFileOptionsUtils.addCellMetadataFileOptions(Options, String, boolean, BiConsumer)

getDataFileOptionValue

protected DataFileOptionValue getDataFileOptionValue(org.apache.commons.cli.CommandLine commandLine,
 boolean allowStandardLocation)
                                              throws org.apache.commons.cli.ParseException

Throws:

org.apache.commons.cli.ParseException

getDataFileOptionValue

protected DataFileOptionValue getDataFileOptionValue(org.apache.commons.cli.CommandLine commandLine,
 boolean allowStandardLocation,
 boolean allowStdout,
 boolean allowCurrentDirectory)
                                              throws org.apache.commons.cli.ParseException

Throws:

org.apache.commons.cli.ParseException

getExpressionDataFileOptionValue

protected ExpressionDataFileOptionValue getExpressionDataFileOptionValue(org.apache.commons.cli.CommandLine commandLine,
 boolean allowStandardLocation)
                                                                  throws org.apache.commons.cli.ParseException

Throws:

org.apache.commons.cli.ParseException

getSingleCellExpressionDataFileOptionValue

protected SingleCellExpressionDataFileOptionValue getSingleCellExpressionDataFileOptionValue(org.apache.commons.cli.CommandLine commandLine,
 boolean allowStandardLocation)
                                                                                      throws org.apache.commons.cli.ParseException

Throws:

org.apache.commons.cli.ParseException

getDesignFileOptionValue

protected DesignFileOptionValue getDesignFileOptionValue(org.apache.commons.cli.CommandLine commandLine,
 boolean allowStandardLocation)
                                                  throws org.apache.commons.cli.ParseException

Throws:

org.apache.commons.cli.ParseException

getCellMetadataFileOptionValue

protected CellMetadataFileOptionValue getCellMetadataFileOptionValue(org.apache.commons.cli.CommandLine commandLine,
 boolean allowStandardLocation)
                                                              throws org.apache.commons.cli.ParseException

Throws:

org.apache.commons.cli.ParseException

buildOptions

protected final void buildOptions(org.apache.commons.cli.Options options)

Description copied from class: AbstractCLI

Build option implementation.

Implement this method to add options to your command line, using the OptionBuilder.

This is called right after AbstractCLI.buildStandardOptions(Options) so the options will be added after standard options.

Overrides:

buildOptions in class AbstractCLI

buildExperimentOptions

protected void buildExperimentOptions(org.apache.commons.cli.Options options)

processOptions

protected final void processOptions(org.apache.commons.cli.CommandLine commandLine)
                             throws org.apache.commons.cli.ParseException

Description copied from class: AbstractCLI

Process command line options.

Implement this to provide processing of options. It is called after AbstractCLI.buildOptions(Options) and right before AbstractCLI.doWork().

Overrides:

processOptions in class AbstractAutoSeekingCLI<ExpressionExperiment>

Throws:

org.apache.commons.cli.ParseException - in case of unrecoverable failure (i.e. missing option or invalid value), an exception can be raised and will result in an exit code of AbstractCLI.FAILURE.

processExperimentOptions

protected void processExperimentOptions(org.apache.commons.cli.CommandLine commandLine)
                                 throws org.apache.commons.cli.ParseException

Throws:

org.apache.commons.cli.ParseException

doAuthenticatedWork

protected void doAuthenticatedWork()
                            throws Exception

Specified by:

doAuthenticatedWork in class AbstractAuthenticatedCLI

Throws:

Exception

preprocessExpressionExperiments

protected Collection<ExpressionExperiment> preprocessExpressionExperiments(Collection<ExpressionExperiment> expressionExperiments)

Preprocess the set of ExpressionExperiment before invoking processExpressionExperiments(Collection) or processExpressionExperiment(ExpressionExperiment).

This can be an opportunity to filter or modify the set of experiments.

processAllExpressionExperiments

protected void processAllExpressionExperiments()

Process all ExpressionExperiment.

This is only called when -all and the CLI is configured to treat it lazily with setAllIsLazy().

processExpressionExperiments

protected void processExpressionExperiments(Collection<ExpressionExperiment> expressionExperiments)

Process multiple ExpressionExperiment.

This only called if more than one experiment was found.

processExpressionExperiment

protected void processExpressionExperiment(ExpressionExperiment expressionExperiment)
                                    throws Exception

Process an ExpressionExperiment.

Throws:

Exception

postprocessExpressionExperiments

protected void postprocessExpressionExperiments(Collection<ExpressionExperiment> expressionExperiments)

Post-process ExpressionExperiments that were processed by the CLI.

toBatchObject

protected final Serializable toBatchObject(@Nullable
 ExpressionExperiment object)

Description copied from class: AbstractAutoSeekingCLI

Convert the given object to a serializable object for batch processing.

Specified by:

toBatchObject in class AbstractAutoSeekingCLI<ExpressionExperiment>

refreshExpressionExperimentFromGemmaWebSilently

protected void refreshExpressionExperimentFromGemmaWebSilently(ExpressionExperiment ee,
 boolean refreshVectors,
 boolean refreshReports)

A silent version of refreshExpressionExperimentFromGemmaWeb(ExpressionExperiment, boolean, boolean) that produces a warning via AbstractCLI.addWarningObject(Serializable, String, Throwable) instead of throwing an exception.

See Also:

• refreshExpressionExperimentFromGemmaWeb(ExpressionExperiment, boolean, boolean)

refreshExpressionExperimentFromGemmaWeb

protected void refreshExpressionExperimentFromGemmaWeb(ExpressionExperiment ee,
 boolean refreshVectors,
 boolean refreshReports)
                                                throws Exception

Refresh a dataset for Gemma Web.

Parameters:

refreshVectors - if true, refresh vectors from the caches, that include raw, single-cell, processed and also cached processed vectors in the form of DataVectorValueObject.

Throws:

Exception

setSingleExperimentMode

protected void setSingleExperimentMode()

Enable the single-experiment mode.

setDefaultToAll

public void setDefaultToAll()

Default to all datasets if no options are provided.

This is a very dangerous setting that should be combined with useReferencesIfPossible.

setAllIsLazy

public void setAllIsLazy()

Enable the "all is lazy" mode.

When -all is set, no experiments are actually loaded and processAllExpressionExperiments() is called instead.

setUseReferencesIfPossible

protected void setUseReferencesIfPossible()

Set this to allow reference to be retrieved instead of actual entities.

This only works for entities retrieved by ID.

When this is enabled, do not access anything but ExpressionExperiment.getId(), or else proxy-initialization will be triggered, and you will have to deal with a LazyInitializationException.

The default is false.

isAbortOnError

public boolean isAbortOnError()

Indicate if this CLI should abort on error or move on to the next experiment.

setAbortOnError

protected void setAbortOnError()

Set this to stop processing experiments if an error occurs.

formatExperiment

protected String formatExperiment(ExpressionExperiment bas)

Render an experiment to string, with special handling in case of an uninitialized proxy.

Use this for printing datasets if useReferencesIfPossible is set to prevent LazyInitializationException.

noNeedToRun

protected boolean noNeedToRun(ExpressionExperiment auditable,
 @Nullable
 Class<? extends AuditEventType> eventClass)

Description copied from class: AbstractAutoSeekingCLI

Check if the given auditable can be skipped.

Overrides:

noNeedToRun in class AbstractAutoSeekingCLI<ExpressionExperiment>

Parameters:

auditable - auditable

eventClass - can be null

Returns:

boolean

readChangelogEntryFromConsole

protected String readChangelogEntryFromConsole(ExpressionExperiment expressionExperiment,
 @Nullable
 String defaultText)
                                        throws IOException,
InterruptedException

Read a changelog entry from the console.

Parameters:

defaultText - a default text to be shown in the editor, or null to keep the file empty

Throws:

IOException

InterruptedException