To keep the pools up to date, a nightly scheduled job can be utilised which can delete all the scratch orgs in each pool. In the case of developer pools (pools with source tracking and used by developers as opposed to CI pools), care must be taken to delete only the unassigned
pools by omitting the --allscratchorgs
flag.
Be careful when running delete command against developer pools with -allscratchorg
flag. It will delete the scratch orgs that are used by developer and can result in potential loss of work
sfpowerscripts during prepare attempts to create multiple scratch orgs in parallel, while respecting the timeout as provided in Pool configuration. At times when the Salesforce infrastructure is stretched such as during release windows or during maintenance windows, it could be often seen some of the scratch orgs requested results in being timed out and are discarded by the rest of the pool activities. However, some of these scratch orgs are in fact created by Salesforce without respecting the timeout parameter, and counted against the active scratch org limits.
These scratch orgs titled orphaned scratch orgs in sfpowerscripts lingo can be reclaimed by issuing the pool: delete command with --orphans flag. This will delete these scratch orgs and hence allow to recover your active scratch org limits
The following section details on how to operate a pool of scratch orgs
List all the active scratch orgs with the given pool tag, only 'available' and 'in progress' scratch orgs are displayed
sfp's prepare command helps you to build a pool of pre-built scratch orgs which can include managed packages as well as packages in your repository. This process allows you to cut down time in re-creating a scratch org during validation process when a scratch org is used as just-in-time environment or when being used as a developer environment. As you integrate more automated processes into Salesforce, incorporating third-party managed packages into your repository's configuration metadata and code becomes necessary. This integration increases the setup time for CI scratch orgs or developer environments for various tasks, such as running data loading scripts or assigning permission sets.
The prepare command does the following sequence of activities:
Calculate the number of scratch orgs to be allocated (Based on your requested number of scratch orgs and your org limits, we calculate what is the number of scratch orgs to be allocated at this point in time)
Fetch the artifacts from registry if "fetchArtifacts" is defined in config, otherwise build all artifacts
Create the scratch orgs in parallel (respecting the timeout requested in the config), and update Allocation_status_c of each these orgs to "In Progress"
On each scratch org, in parallel, do the following activities:
Install SFPOWERSCRIPTS_ARTIFACT_PACKAGE (04t1P000000ka9mQAA) for keeping track of all the packages which will be installed in the org. You can set an environment variable SFPOWERSCRIPTS_ARTIFACT_PACKAGE to override the installation with your own package id (the source code is available here)
Install all the dependencies of your packages, such as managed packages that are marked as dependencies in your sfdx-project.json
Install all the artifacts that is either built/fetched
If enableSourceTracking
is specified in the configuration, the command will create and deploy "sourceTrackingFiles" using sfpowerscripts artifacts to the scratch org. To store local source tracking files, we re-create it when fetching a scratch org from a pool, using the @salesforce/source-tracking library. Checkout the commit from which each sfpowerscripts artifact was created, and update the local source tracking using the package directory. The files are retrieved to the local ".sf" directory, when using sfpowerscripts:pool:fetch
to fetch a scratch org, and allows users to deploy their changes only, through source tracking. Refer to the decision log for more details.
Mark each completed scratch org as "Available", depending on the pool config `succeedOnDeploymentErrors` is true, else scratch orgs are deleted
Ensure that your DevHub is authenticated using SFDX Auth URL and the auth URL is stored in a secure place (Key Management System or Secure Storage).
A developer can fetch a scratch org with all the dependencies installed and the latest code base has been pushed from a pool for their feature development. the developer can start developing new features without spending a few hours preparing the development environment.
prepare command has inbuilt capability to orchestrate installation of external package dependencies. Package dependencies are defined in the sfdx-project.json. More information on defining package dependencies can be found in the Salesforce docs.
Let's unpack the concepts utilizing the above example:
There are two unlocked packages
Expense Manager - Util is an unlocked package in your DevHub, identifiable by 0H in the packageAlias
Expense Manager - another unlocked package which is dependent on ' Expense Manager - Util', 'TriggerFramework' and 'External Apex Library - 1.0.0.4'
External Apex Library is an external dependency, It could be a managed package or any unlocked package released on a different Dev Hub. All external package dependencies have to be defined with a 04t ID, which can be determined from the installation URL from AppExchange or by contacting your vendor.
sfpowerscripts parses sfdx-project.json and does the following in order
Skips Expense manager - Util as it doesn't have any dependencies
For Expense manager
Checks whether any of the package is part of the same repo, in this example 'Expense Manager-Util' is part of the same repository and will not be installed as a dependency
Installs the latest version of TriggerFramework ( with major, minor and patch versions matching 1.7.0) to the scratch org
Install the 'External Apex Library - 1.0.0.4' by utilizing the 04t id provided in the packageAliases
If any of the managed package has keys, it can be provided as an argument to the prepare command. Check the command's flags for more information.
The format for the 'keys' parameter is a string of key-value pairs separated by spaces - where the key is the name of the package, the value is the protection key of the package, and the key-value pair itself is delimited by a colon .
e.g. --keys "packageA:12345 packageB:pw356 packageC:pw777"
The time taken by this command depends on how many managed packages and your packages that need to be installed. Please note, if you are triggering this command in a CI server, ensure proper time outs are provided for this task, as most cloud-based CI providers have time limits on how long a single task can be run. Explore multi-stage prepare jobs in case this is a blocker for the team.