• Document Up to Date

Deployer Processors Guide

Crafter Deployer includes an extensive list of deployment processors that can be easily added to any target to meet specific requirements. Some examples of the use cases that can be addressed with deployment processors are:

  • Pushing content created/edited in Crafter Studio to an external service

  • Pulling content created/edited from an external service

  • Execute actions every time a deployment succeeds or fails

Note

When adding processors or changing the deployment pipeline for a target keep in mind that the processors will be executed following the order defined in the configuration file and some processors require a specific position in the pipeline

Main Deployment Processors

The main deployment processors can do any task related to detect changed files or process changed files that were detected by other processors. To process changed files a processor may interact with any external service as needed.

All deployment processors support the following properties:

Name

Required

Default Value

Description

processorLabel

None

Label that other processors can use to jump to this one

jumpTo

None

The label of the processor to jump to after a successful execution

includeFiles

None

List of regular expressions to check the files that should be included

excludeFiles

None

List of regular expressions to check the files that should be excluded

alwaysRun

false

Indicates if the processor should run even if there are no changes in the current deployment

failDeploymentOnFailure

false

Enables failing a deployment when there’s a processor failure

Git Pull Processor

Processor that clones/pulls a remote Git repository into a local path in the filesystem.

Note

This needs to be the first processor in the pipeline

Properties

Name

Required

Default Value

Description

remoteRepo.url

The URL of the remote Git repo to pull

remoteRepo.name

origin

The name to use for the remote repo when pulling from it

remoteRepo.branch

The default branch in the repo

The branch of the remote Git repo to pull

remoteRepo.username

The username for authentication with the remote Git repo. Not needed when SSH with RSA key pair authentication is used

remoteRepo.password

The password for authentication with the remote Git repo. Not needed when SSH with RSA key pair authentication is used

remoteRepo.ssh.privateKey.path

The SSH private key path, used only with SSH with RSA key pair authentication

remoteRepo.ssh.privateKey.passphrase

The SSH private key passphrase, used only with SSH withRSA key pair authentication

failDeploymentOnFailure

true

Enables failing a deployment when there’s a processor failure

Example

Git Pull Processor using basic auth
1- processorName: gitPullProcessor
2  remoteRepo:
3    url: https://github.com/myuser/mysite.git
4    branch: live
5    username: myuser
6    password: mypassword
Git Pull Processor using SSH with RSA key pair
1- processorName: gitPullProcessor
2  remoteRepo:
3    url: https://github.com/myuser/mysite.git
4    branch: live
5    ssh:
6      privateKey:
7        path: /home/myuser/myprivatekey
8        passphrase: mypassphrase

Git Diff Processor

Processor that, based on a previous processed commit that’s stored, does a diff with the current commit of the deployment, to find out the change set. If there is no previous processed commit, then the entire repository becomes the change set.

Note

This processor needs to be placed after the gitPullProcessor and before any other processor like the searchIndexingProcessor

Properties

Name

Required

Default Value

Description

includeGitLog

false

Indicates if the git log details should be included in the change set

updateCommitStore

true

Indicates if the processed commit value should be modified

failDeploymentOnFailure

true

Enables failing a deployment when there’s a processor failure

Example

Git Diff Processor
1- processorName: gitDiffProcessor
2  includeGitLog: true

Git Push Processor

Processor that pushes a local repo to a remote Git repository.

Properties

Name

Required

Default Value

Description

localRepoBranch

The branch of the local repo to push

remoteRepo.url

The URL of the remote Git repo to push to

remoteRepo.branch

The default branch in the repo

The branch of the remote Git repo to push to

remoteRepo.username

The username for authentication with the remote Git repo. Not needed when SSH with RSA key pair authentication is used

remoteRepo.password

The password for authentication with the remote Git repo. Not needed when SSH with RSA key pair authentication is used

remoteRepo.ssh.privateKey.path

The SSH private key path, used only with SSH with RSA key pair authentication

remoteRepo.ssh.privateKey.passphrase

The SSH private key passphrase, used only with SSH withRSA key pair authentication

force

false

Sets the force preference for the push

pushAll

false

If all local branches should be pushed to the remote

Example

Git Push Processor using basic auth
1- processorName: gitPushProcessor
2  remoteRepo:
3    url: https://github.com/myuser/mysite.git
4    branch: deployed
5    username: myuser
6    password: mypassword
Git Push Processor using SSH with RSA key pair
1- processorName: gitPushProcessor
2  remoteRepo:
3    url: https://github.com/myuser/mysite.git
4    branch: deployed
5    ssh:
6      privateKey:
7        path: /home/myuser/myprivatekey
8        passphrase: mypassphrase

Git Update Commit Id Processor

Processor that updates the processed commits value with the current commit

Example

Git Update Commit Id Processor
1- processorName: gitUpdateCommitIdProcessor

Groovy Script Processor

A custom Groovy processor that can process published content.

Properties

Name

Required

Default Value

Description

scriptPath

The relative path of the script to execute

Note

The default path scripts are loaded from is $CRAFTER_HOME/bin/crafter-deployer/processors/scripts

Example

Groovy Script Processor
1- processorName: scriptProcessor
2  scriptPath: 'myscripts/mychanges.groovy'

The following variables are available for use in your scripts:

Variable Name

Description

logger

The processor’s logger, http://www.slf4j.org/api/org/slf4j/Logger.html

applicationContext

The application context of the current target, https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/context/ApplicationContext.html

deployment

The current deployment, http://javadoc.craftercms.org/3.1.31/deployer/org/craftercms/deployer/api/Deployment.html

execution

The execution for this processor, http://javadoc.craftercms.org/3.1.31/deployer/org/craftercms/deployer/api/ProcessorExecution.html

filteredChangeSet

A subset of ‘originalChangeSet’ that matches the includeFiles pattern and not the excludeFiles pattern for this processor, http://javadoc.craftercms.org/3.1.31/deployer/org/craftercms/deployer/api/ChangeSet.html

originalChangeSet

The original change set returned by the previous processors in the pipeline, http://javadoc.craftercms.org/3.1.31/deployer/org/craftercms/deployer/api/ChangeSet.html



Let’s take a look at an example script that you can use for the Groovy script processor. Below is a script that only includes a file from the change set if a parameter is present in the deployment:

Example Groovy script to be run by a script processor
 1import org.craftercms.deployer.api.ChangeSet
 2
 3logger.info("starting script execution")
 4
 5def specialFile = "/site/website/expensive-page-to-index.xml"
 6
 7// if the file has been changed but the param was not sent then remove it from the change set
 8if (originalChangeSet.getUpdatedFiles().contains(specialFile) && !deployment.getParam("index_expensive_page")) {
 9    originalChangeSet.removeUpdatedFile(specialFile)
10}
11
12// return the new change set
13return originalChangeSet

File Based Deployment Event Processor

Processor that triggers a deployment event that consumers of the repository (Crafter Engines) can subscribe to by reading a file from the repository.

Properties

Name

Required

Default Value

Description

deploymentEventsFileUrl

deployment-events.properties

Relative path of the deployment events file

eventName

Name of the event to trigger

Example

File Based Deployment Event Processor
1- processorName: fileBasedDeploymentEventProcessor
2  eventName: 'events.deployment.rebuildContext'

Command Line Processor

Processor that runs a command line process.

Properties

Name

Required

Default Value

Description

workingDir

Deployer’s directory

The directory from which the process will run

command

The full command that the process will run

processTimeoutSecs

30

The amount of seconds to wait for the process to finish

includeChanges

false

Additional parameters will be added to the command

Example: script.sh SITE_NAME OPERATION (CREATE | UPDATE | DELETE) FILE (relative path of the file)

Example

Command Line Processor
1- processorName: commandLineProcessor
2  workingDir: '/home/myuser/myapp/bin'
3  command: 'myapp -f --param1=value1'

Search Indexing Processor

Processor that indexes the files on the change set, using one or several BatchIndexer. After the files have been indexed it submits a commit.

Note

This processor uses the Crafter Search API to index in Apache Solr, it should be used only for sites from 3.0.x that will not be migrated to Elasticsearch.

Properties

Name

Required

Default Value

Description

ignoreIndexId

false

If the index ID should be ignored

indexId

Value of siteName

The specific index ID to use

reindexItemsOnComponentUpdates

true

Flag that indicates that if a component is updated, all other pages and components that include it should be updated too

Example

Search Indexing Processor
1- processorName: searchIndexingProcessor

Elasticsearch Search Indexing Processor

Processor that indexes the files on the change set, using one or several BatchIndexer. After the files have been indexed it submits a commit.

Properties

Name

Required

Default Value

Description

ignoreIndexId

false

If the index ID should be ignored

indexId

Value of siteName

The specific index ID to use

reindexItemsOnComponentUpdates

true

Flag that indicates that if a component is updated, all other pages and components that include it should be updated too

Example

Elasticsearch Indexing Processor
1- processorName: elasticsearchIndexingProcessor

HTTP Method Call Processor

Processor that does a HTTP method call.

Properties

Name

Required

Default Value

Description

url

The URL to call

method

The HTTP method

Example

HTTP Method Call Processor
1- processorName: httpMethodCallProcessor
2  method: GET
3  url: 'http://localhost:8080/api/1/site/cache/clear.json?crafterSite=mysite'

Delay Processor

Processor that stops the pipeline execution for a given number of seconds.

Properties

Name

Required

Default Value

Description

seconds

5

Amount of seconds to wait

Example

Delay Processor
1- processorName: delayProcessor
2  seconds: 10

Find And Replace Processor

Processor that replaces a pattern on the content of the created or updated files.

Note

The files changed by this processor will not be committed to the git repository and will be discarded when the next deployment starts

Properties

Name

Required

Default Value

Description

textPattern

Regular expression to search in files

replacement

Expression to replace the matches

failDeploymentOnFailure

true

Enables failing a deployment when there’s a processor failure

Example

Find And Replace Processor
1- processorName: findAndReplaceProcessor
2  textPattern: (/static-assets/[^"<]+)
3  replacement: 'http://mycdn.com$1'

AWS Processors

All deployment processors related to AWS services support the following properties:

Name

Required

Default Value

Description

region

If not provided the AWS SDK

default providers will be

used

The AWS Region

accessKey

The AWS Access Key

secretKey

The AWS Secret Key

url

AWS S3 bucket URL to upload files

failDeploymentOnFailure

true

Enables failing a deployment when there’s a processor failure


S3 Sync Processor

Processor that syncs files to an AWS S3 Bucket.

Example

S3 Sync Processor
1- processorName: s3SyncProcessor
2  url: s3://serverless-sites/site/mysite

S3 Deployment Events Processor

Processor that uploads the deployment events to an AWS S3 Bucket

Properties

Name

Required

Default Value

Description

deploymentEventsFileUrl

deployment-events.properties

URL of the deployment events file, relative to the local git repo

Example

S3 Deployment Events Processor
1- processorName: s3DeploymentEventsProcessor
2  region: ${aws.region}
3  accessKey: ${aws.accessKey}
4  secretKey: ${aws.secretKey}
5  url: {{aws.s3.url}}

Cloudfront Invalidation Processor

Processor that invalidates the changed files in the given AWS Cloudfront distributions.

Properties

Name

Required

Default Value

Description

distributions

List of distributions ids

Example

Cloud Front Invalidation Processor
1- processorName: cloudfrontInvalidationProcessor
2  distributions:
3    - E15UHQPTKROC8Z

Post Deployment Processors

The post deployment processors assume that all changed files have been handled and the result of the deployment is already known (either successful or failed) and take actions based on those results, because of that they need to be placed after all main deployment processors to work properly.

File Output Processor

Post processor that writes the deployment result to an output CSV file under CRAFTER_HOME/logs/deployer for later access, whenever a deployment fails or files were processed.

Example

File Output Processor
1- processorName: fileOutputProcessor

Mail Notification Processor

Post processor that sends an email notification with the result of a deployment, whenever a deployment fails or files were processed. The output file generated by the fileOutputProcessor is attached if it’s available.

Properties

Name

Required

Default Value

Description

templateName

default

The name of the Freemarker template used for email creation

from

noreply@example.com

The value of the From field in the emails

to

The value of the To field in the emails

subject

Deployment Report

The value of the Subject field in the emails

html

true

Whether the emails are HTML

serverName

Current local host name

The hostname of the email server

dateTimePattern

MM/dd/yyyy hh:mm:ss.SSS a z

The date time pattern to use when specifying a date in the email

status

ON_ANY_STATUS

Indicates for which deployment status emails should be sent.

Possible values:

  • ON_ANY_STATUS Notifications sent for all deployments

  • ON_ANY_FAILURE Notifications sent for deployments where at least one processor has failed

  • ON_TOTAL_FAILURE Notifications will be sent for deployments in which the general status indicates failure

Example

Mail Notification Processor for any failure
1- processorName: mailNotificationProcessor
2  to:
3    - admin@example.com
4    - author@example.com
5  status: ON_ANY_FAILURE

Full Pipeline Example

The following example shows how the deployment processors work together to deliver a serverless site using AWS services.

Serverless Delivery Pipeline
 1pipeline:
 2  # -------------------- START OF MAIN PIPELINE --------------------
 3
 4  # First clone or update the local repository from github
 5  - processorName: gitPullProcessor
 6    remoteRepo:
 7      url: https://github.com/myuser/mysite.git
 8      branch: live
 9      username: myuser
10      password: my_secret_password
11
12  # Then find the added/changed/deleted files since the previous pull (if any)
13
14  - processorName: gitDiffProcessor
15
16  # Change all references to static-assets to use a CDN URL instead of the local URL
17  - processorName: findAndReplaceProcessor
18    includeFiles: ['^/site/.*$', '^/templates/.*$', '^/static-assets/.*(js|css|html)$']
19    textPattern: (/static-assets/[^"<]+)
20    replacement: 'http://d111111abcdef8.cloudfront.net$1'
21
22  # Index the changes in Elasticsearch
23  - processorName: elasticsearchIndexingProcessor
24
25  # Sync the changes in a S3 bucket
26  - processorName: s3SyncProcessor
27    url: s3://serverless-sites/site/mysite
28
29  # Add a small delay to allow the S3 changes to propagate
30  - processorName: delayProcessor
31
32  # Invalidate the changed files in the CDN
33  - processorName: cloudfrontInvalidationProcessor
34    includeFiles: ['^/static-assets/.*$']
35    distributions:
36      - E15UHQPTKROC8Z
37
38  # Trigger deployment events so any Crafter Engine listening can update accordingly:
39  # Rebuild the site context if any config or script has changed
40  - processorName: fileBasedDeploymentEventProcessor
41    includeFiles: ["^/?config/.*$", "^/?scripts/.*$"]
42    excludeFiles: ['^/config/studio/content-types/.*$']
43    eventName: 'events.deployment.rebuildContext'
44
45  # Clear the cache if any static-asset has changed
46  - processorName: fileBasedDeploymentEventProcessor
47    excludeFiles: ['^/static-assets/.*$']
48    eventName: 'events.deployment.clearCache'
49
50  # Rebuild the GraphQL schema if any content-type has changed
51  - processorName: fileBasedDeploymentEventProcessor
52    includeFiles: ['^/config/studio/content-types/.*$']
53    eventName: 'events.deployment.rebuildGraphQL'
54
55  # Push the updated events to the S3 bucket
56  - processorName: s3SyncProcessor
57    includeFiles: ['^/?deployment-events\.properties$']
58    url: s3://serverless-sites/site/mysite
59
60  # -------------------- END OF MAIN PIPELINE --------------------
61  # Only Post Processors can be in this section
62
63  # Record the result of the deployment to a CSV file
64  - processorName: fileOutputProcessor
65
66  # Notify the site admin & an author if there were any failures during the deployment
67  - processorName: mailNotificationProcessor
68    to:
69      - admin@example.com
70      - author@example.com
71    status: ON_ANY_FAILURE