Skip to main content

Compute environment pre-flight checks

Pre-release documentation

Pre-flight checks have not yet been released. This page is preparatory documentation ahead of the release. Existing customers will be notified by Seqera Support when pre-flight checks are due to be released.

Pre-flight checks validate that a compute environment is usable before and at the point of pipeline launch. They run in the background on a recurring schedule and synchronously at launch time. Problems appear before pipeline submission rather than mid-run. Pre-flight checks only flag conditions that would block a pipeline launch.

Pre-flight checks are enabled by default for all Cloud customers and cannot be disabled.

What to verify before creating a compute environment

Before creating or deploying a compute environment, confirm the following:

Credentials

  • The access keys, service account key, or managed identity are valid and have not been rotated or revoked.
  • The IAM role or service account has the permissions required by the target platform. See the relevant compute environment page for the minimum required policy.

Work directory

  • The bucket or storage container exists in the same region as the compute environment (required for AWS Batch and AWS Cloud compute environments only).
  • The credential attached to the compute environment has read and write access to the work directory path.

Wave (if enabled)

  • The Wave service is running and reachable from the Platform instance.

Tower Agent (HPC/grid compute environments only)

  • Tower Agent is reachable from Platform. See Tower Agent for installation and startup instructions.

Validation process

Platform runs three tiers of validation:

1. Credential validation

Runs on a recurring schedule. For each cloud credential (AWS, GCP, Azure) in scope, Platform calls the provider API to verify that the credential is still accepted. For AWS role-based credentials and GCP Workload Identity Federation, this check confirms the credential is well-formed but cannot fully verify the underlying role or identity provider trust configuration.

When a credential fails this check, Platform marks it INVALID and records the provider error on the credential record. This error appears in the launch-time error when a pipeline is blocked, but not in the compute environment banner. To see the specific provider error, check the credential record directly.

2. Compute environment validation

Runs approximately every hour across all AVAILABLE compute environments. Platform validates:

  1. Credential status: If the credential is INVALID, the compute environment is marked INVALID immediately.
  2. Work directory: Platform calls the cloud provider to verify that the configured work directory is accessible. If this check fails, the compute environment is marked INVALID and the provider error appears in the card view and at the top of the form page.

A compute environment marked INVALID displays a banner with the error message. An AVAILABLE compute environment has its lastValidated timestamp refreshed.

note

These checks cover AWS Batch, AWS Cloud, Azure Batch, Azure Cloud, Google Cloud Batch, and Google Cloud compute environments.

3. Pipeline launch-time checks

Runs immediately when a user submits a pipeline launch. If any check fails, the launch is blocked and a specific error is returned. Multiple failures are reported together.

CheckWhat it does
Compute environment statusBlocks launch if the compute environment is marked INVALID
Credential statusBlocks launch if the credential associated with the compute environment is marked INVALID
Work directory overrideIf the user provided a different work directory at launch, validates that path with the cloud provider
Wave connectivityFor compute environments with Wave enabled, verifies the Wave service connection is active
Tower AgentFor HPC compute environments, verifies a Tower Agent is online for the environment

Manual credential validation

When a credential is marked INVALID and you have rotated the keys or fixed the underlying issue, you can trigger an immediate re-validation:

  1. Navigate to Credentials in your workspace.
  2. Find the credential and select Validate.

Platform makes a live call to the cloud provider and updates the credential status immediately. If the check passes, the credential returns to AVAILABLE.

Compute environments marked INVALID because of this credential do not recover automatically. Use Validate on each affected compute environment after restoring the credential.

Manual compute environment validation

When a compute environment is marked INVALID and you have fixed the underlying issue, you can trigger an immediate re-validation without waiting for the next background sweep:

  1. Navigate to Compute environments in your workspace.
  2. Find the compute environment and open its (three-dot) drop-down.
  3. Select Validate.

Platform runs pre-flight checks and updates the compute environment status immediately. If all checks pass, the compute environment returns to AVAILABLE.

Error reference

Compute environment error messages

These banners appear on the compute environment detail page when the compute environment is INVALID.

BannerMeaningAction
Associated credentials are invalid or expired. Update the credentials and validate this compute environment, or contact your workspace maintainer to resolve this.The background sweep found the attached credential is no longer validGo to Credentials, update or rotate the credential, then use Validate on the compute environment
Work directory is invalid. {reason}. Fix it and validate this compute environment, or contact your workspace maintainer to resolve this.The background sweep could not access the configured work directoryFix the bucket/path permissions or location, then use Validate on the compute environment

Launch-time errors

These are returned immediately to the user when a launch is blocked.

ErrorCauseResolution
The selected compute environment '...' is in an invalid stateCompute environment is marked INVALID (see banner for the specific reason)Fix the root cause, then use Validate on the compute environment
The credentials '...' used by this compute environment are invalidCredential is marked INVALIDGo to Credentials, update or rotate the credential, then use Validate on the compute environment
Wave is required by the selected compute environment but the Wave service connection is not active. Verify that Wave is running and check for connectivity issuesPlatform cannot reach the Wave serviceContact your platform administrator. Once Wave is restored, retry the launch
No Tower Agent is online for the selected compute environment. Check that Tower Agent is running at your cluster.No Tower Agent is connected for this compute environment (HPC/grid only)Start or restart Tower Agent on the cluster. See Tower Agent

Credential error messages

When the credential sweep marks a credential INVALID, the provider-specific reason is stored on the credential record. It appears in the launch-time error when a pipeline is blocked, but not in the compute environment banner. To see the specific provider error, check the credential record directly.

ProviderExample message
AWSAWS credentials are invalid or expired. Update or rotate the access keys.
GCPGoogle credentials are invalid or expired. Update the service account key.
GCP Workload Identity FederationGoogle WIF credential validation failed. Verify the provider and service account configuration.
Azure BatchAzure Batch credentials are invalid. Verify the Batch account name and key.
Azure StorageAzure Storage credentials are invalid. Verify the storage account name and key.