• Document Under Review
  • Updated On 4.1.2

Engine Configuration

Configuration Files

Crafter Engine can be configured at the project/site level or at the instance level.

Project-level/Site-level Configuration Files

Crafter Engine provides a flexible configuration system that allows site administrators to change the behavior of the project without the need to modify any code. Some properties are used by Crafter Engine itself, but developers can also add any custom property they need for their code.

The main configuration files for a project/site can be edited within Crafter Studio’s Project Tools > Configuration UI or via Git. These files are:

Engine Project Configuration Files

Configuration File

Description

Engine Project Configuration (config/engine/site-config.xml)

Contains project properties used by Crafter Engine

Engine Project Application Context (config/engine/application-context.xml)

Contains bean definitions for the site context associated with the Webapp

URL Rewrite Configuration (XML Style) (config/engine/urlrewrite.xml)

Contains URL rewrite rules

Proxy Config (config/engine/proxy-config.xml)

Configures the proxy servers for the Preview server (Crafter Engine in Preview Mode)

Note

All configuration files can be overridden by environment. Learn more about multi-environment support in Engine Multi-Environment Support.

The configuration file site-config.xml has some additional considerations. This file can be defined in:
  • /config/engine/env/{envName}/site-config.xml: This is the environment override, and is loaded first if present.

  • /config/engine/site-config.xml: This is the main configuration file for the project/site. This file is loaded if the environment override is not present.

Note

All properties will be available for developers in the Freemarker templates and Groovy scripts using the siteConfig variable. The siteConfig variable is an instance of the XMLConfiguration class.

Instance-level Configuration

The main files for configuring Crafter Engine at the instance level are:

Engine Instance Level Configuration Files

Configuration File

Description

server-config.properties

Contains server configurable parameters such as URLs, paths, etc.

services-context.xml

Contains the bean definition for services layer

rendering-context.xml

Contains the bean definition for rendering

logging.xml

Contains loggers, appenders, etc.

These configuration files for Crafter Engine is located under CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension, where CRAFTER_HOME is the install directory of your CrafterCMS authoring or delivery environment.

The files can be accessed by opening the files using a text editor. Any changes made to any of the files listed above will require a restart of Crafter Engine.


Engine Configuration Properties

In this section we will highlight some of the more commonly used properties in the configuration of Crafter Engine. For most properties, please see the server-config.properties file, and for additional configuration files and properties, see Configuration Files.

Common Configuration Properties

Property

Purpose

Engine Root Folder

Allows you to set the content root folder

Turn Off Show Error

Allows you to turn off showing errors in line with content

Groovy Sandbox Configuration

Allows you to configure the Groovy sandbox to tweak the Groovy security layer

Request Filtering Configuration

Allows you to configure request filtering

Forwarded Headers

Allows you to configure forwarded headers

Policy Headers

Allows you to configure policy headers

Search Timeouts

Allows you to configure the search client connection timeout, socket timeout and number of threads

Content-Length Headers

Allows you to configure the content-length header

Static Methods in Freemarker Templates

Allows you to configure static methods in Freemarker templates

Spring Expression Language

Allows you to configure SpEL expressions for custom app contexts

Engine URL Rewrite Configuration

Allows you to configure URL rewriting

Proxy Configuration

Allows you to configure the proxy for the Preview server (Crafter Engine in Preview Mode)



Proxy Configuration

CrafterCMS supports a proxy system to proxy GraphQL, Engine, NodeJS or other application delivery systems. Whenever Crafter Engine receives a request, it is matched against the patterns of each server and the first match would then get the request sent to the server with the matching pattern. In some systems, multiple servers are used for search, Studio, etc. Using the proxy helps simplify the system.

One of the benefits of using the proxy in CrafterCMS is that it can connect to any remote server as the preview server, which allows for easier authoring of projects built with other programming languages and technology, React, Angular, or Vue for example.

The proxy configuration file contains configuration for the preview proxy servers. To modify the proxy configuration, click on projectTools from the bottom of the Sidebar, then click on Configuration and select Proxy Config from the dropdown list.

Configurations - Open Proxy Configuration

Here’s a sample Proxy Configuration file (click on the triangle on the left to expand/collapse):

Sample "proxy-config.xml"
CRAFTER_HOME/data/repos/sites/SITENAME/sandbox/config/engine/asset-processing/proxy-config.xml
 1<?xml version="1.0" encoding="utf-8"?>
 2
 3<!--
 4  ~ Copyright (C) 2007-2022 Crafter Software Corporation. All Rights Reserved.
 5  ~
 6  ~ This program is free software: you can redistribute it and/or modify
 7  ~ it under the terms of the GNU General Public License version 3 as published by
 8  ~ the Free Software Foundation.
 9  ~
10  ~ This program is distributed in the hope that it will be useful,
11  ~ but WITHOUT ANY WARRANTY; without even the implied warranty of
12  ~ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
13  ~ GNU General Public License for more details.
14  ~
15  ~ You should have received a copy of the GNU General Public License
16  ~ along with this program.  If not, see <http://www.gnu.org/licenses/>.
17  -->
18
19<!--
20    This file configures the proxy servers for preview.
21
22    Every request received by Engine will be matched against the patterns of each server
23    and the first one that matches will be used as proxy.
24
25    <server>
26        <id/> (id of the server, can have any value)
27        <url/> (url of the server, if missing or empty the request will be executed locally)
28        <patterns>
29            <pattern/> (regex to match requests)
30        </patterns>
31    </server>
32-->
33
34<proxy-config>
35    <version>1</version>
36    <servers>
37        <!-- Proxy all GraphQL requests to this server (can be any HTTP compatible GraphQL server) -->
38        <server>
39            <id>graphql</id>
40            <url>http://my-graphql-server</url>
41            <patterns>
42                <pattern>/api/1/site/graphql.*</pattern>
43            </patterns>
44        </server>
45
46        <!-- Proxy all Crafter Engine API requests to this server -->
47        <server>
48            <id>engine</id>
49            <url>http://my-crafter-egine-server</url>
50            <patterns>
51                <pattern>/api/.*</pattern>
52            </patterns>
53        </server>
54
55        <!-- Proxy all Crafter Engine static-assets requests to this server -->
56        <server>
57            <id>static-assets</id>
58            <url>http://my-crafter-engine-server</url>
59            <patterns>
60                <pattern>/static-assets/.*</pattern>
61            </patterns>
62            <headersToServer>
63                <header>
64                    <name>authorization</name>
65                    <value>Bearer eyJhbGciOiJQQkVTMi1IUzUxMit...</value>
66                </header>
67            </headersToServer>
68            <headersToClient>
69                <header>
70                    <name>Cache-Control</name>
71                    <value>no-cache\, no-store\, max-age=0\, must-revalidate</value>
72                </header>
73                <header>
74                    <name>Access-Control-Allow-Origin</name>
75                    <value>http://my-crafter-engine-server</value>
76                </header>
77                <header>
78                    <name>Access-Control-Allow-Methods</name>
79                    <value>GET\, PUT</value>
80                </header>
81            </headersToClient>
82        </server>
83
84        <!-- Proxy any other request to this server (can be any web or application server) -->
85        <server>
86            <id>preview</id>
87            <url>http://my-web-server</url>
88            <patterns>
89                <pattern>.*</pattern>
90            </patterns>
91        </server>
92    </servers>
93</proxy-config>


Note

Deleting the config file (proxy-config.xml) from the repo completely disables the proxy feature.

Proxy Example: React

For example, you would like to work on a React application within Studio. What is normally included inside Studio is the build output of the React application, so that a user making edits to the React code would need to build the React code then copy it into Studio in order to preview the changes. This becomes cumbersome when developing, as many edits are normally done before reaching the final version of the React app. Using the proxy, the user can preview the React app in Studio and is able to work on both the React app and CrafterCMS.

Let’s take a look at an example of setting up the proxy for a React application.

We’ll look at the Video Center Blueprint, a React application available from the public marketplace, that runs on localhost:3000, then setup the Studio proxy so we can preview the React application inside Studio. Finally, we’ll make some changes in the React application and view the changes made inside Studio.

Let’s begin:

  1. Setup the React application.

    • Clone the video center blueprint by running git clone https://github.com/craftercms/video-center-blueprint.git

       git clone https://github.com/craftercms/video-center-blueprint.git
      Cloning into 'video-center-blueprint'...
      remote: Enumerating objects: 6433, done.
      remote: Total 6433 (delta 0), reused 0 (delta 0), pack-reused 6433
      Receiving objects: 100% (6433/6433), 77.12 MiB | 4.92 MiB/s, done.
      Resolving deltas: 100% (4041/4041), done.
      
    • Run the React application

      Inside the video center blueprint folder that we just cloned above, navigate to video-center-blueprint/sources/app. We need yarn installed in your system. Running yarn with no command will run yarn install. In the example below, yarn is already installed in the system

        app git:(master) yarn
      yarn install v1.22.4
      [1/4] 🔍  Resolving packages...
      success Already up-to-date.
      ✨  Done in 0.68s.
      

      Build the React application by running yarn start

        app git:(master) yarn start
      yarn run v1.22.4
      $react-scripts start
      
      Compiled successfully!
      
      You can now view video-center-blueprint in the browser.
      
      Local:            http://localhost:3000/
      On Your Network:  http://192.168.1.135:3000/
      
      Note that the development build is not optimized.
      To create a production build, use yarn build.
      

      The command above will open a browser window where we can view the app

      Video Center Blueprint preview on "localhost:3000"

    • Edit the base url in your React app to the server we are using for development, which in our case, is the Studio server, on localhost:8080, and the websocket port for hot module reloading (Crafter does not proxy websocket so you will need to point to the origin server of the websocket), then save the changes and restart the React server.

      To edit the base url, navigate to video-center-blueprint/sources/app then open the .env.development file using your favorite editor and set the following variables: REACT_APP_BASE_URL and WDS_SOCKET_PORT

      video-center-blueprint/sources/app/.env.development
      # A blank REACT_APP_SITE_NAME variable will make the app
      # try to find the crafterSite cookie which, provided you're
      # running both your local crafter and node dev web server for
      # the app are the same, it should be set for you when you created
      # the project. Manually set the here otherwise.
      REACT_APP_BASE_URL=http://localhost:8080
      
      # If you're using the CrafterCMS's Preview Proxy to view the dev mode app inside Preview,
      # configuring the port makes live reload work inside the CrafterCMS Preview frame.
      # If you're using any other port to run your webpack dev server, you should adjust this to
      # that port too.
      WDS_SOCKET_PORT=3000
      

      For Windows users on Docker, add DANGEROUSLY_DISABLE_HOST_CHECK=true to the .env.development file.
      Setting the environment variable DANGEROUSLY_DISABLE_HOST_CHECK to true disables the host check, which allows us to pretend that the the host header of the request and the listening address of the host are running on the same host and port which is important since browsers block cross-origin requests. If not set, the user will see an Invalid Host header message in Studio. Remember that disabling the host check is insecure and should only be used in a development environment.

      Remember to restart the React server for the settings to take effect.

      Note

      If you’re using the create-react-app, please note that react-scripts versions earlier than 3.4.0 does not support custom sockjs pathname for hot reloading the server. Make sure that your react-scripts version used is 3.4.0 or above for the live reload work inside CrafterCMS to work.

  2. Setup Studio

    • Create a project using the video center blueprint from the Public Marketplace.

      From the Main Menu, click on Project, then click on the Create Project button. This will open the Create Site dialog. Look for Video Center, then click on the Use button, fill in the required information then click on the Review button, then finally the Create Site button. This Video Center blueprint we selected from the Marketplace is the same react application

      Select Video Center blueprint from the Public Marketplace

    • Setup the proxy for the video center React application we started above

      Open the Sidebar, click on projectTools, then click on Configuration. Select Proxy Config from the dropdown menu.

      Configurations - Open Proxy Configuration

      Scroll down to the preview server and edit the url to point to the url used for the React application (localhost:3000) we setup in the beginning. Save your changes.

      CRAFTER_HOME/data/repos/sites/sandbox/SITENAME/sandbox/config/engine/proxy-config.xml
      <server>
        <id>preview</id>
        <url>http://localhost:3000</url>
        <patterns>
          <pattern>.*</pattern>
        </patterns>
      </server>
      

      For users running Studio on Docker, use http://host.docker.internal:3000 for the url of the React application. Docker containers can access local services running on the host by connecting to host.docker.internal. See https://docs.docker.com/docker-for-windows/networking/#use-cases-and-workarounds for more information on connecting from a container to a service on the host.

      At this point, the preview we are seeing in Studio should be the one from our React application.

    • Modify the React application then verify that we can preview the changes made inside Studio.

      For this part, we’ll change the text Featured Channels in the home page to My Featured Channels. Using your favorite editor, in your React app, navigate to video-center-blueprint/sources/app/src/containers/Home and open the Home.js file. Scroll down to the line with key: 'featured-channels' and edit the value:

      {
        key: 'featured-channels',
        value: 'My Featured Channels',
        type: 'channel-card-alt',
        ...
      

      Save your changes. Notice that in the React app preview (localhost:3000), the page is reloaded with our changes now visible. Now let’s take a look at Studio. Notice that Studio preview has reloaded and the changes we made in the React app is now visible.

      Changes made in the React app now visible in the Studio preview


Engine URL Rewrite Configuration

URL rewriting turns hard to remember, long and complicated URLs into easier to remember URLs.

CrafterCMS comes with the Tuckey URLRewrite filter, a Java Web Filter with functionality like Apache’s mod_rewrite, that lets you setup rewrite rules for your site.

To add a URL rewrite rule, in Studio, open the Sidebar then click on projectTools. Click on Configuration then select Engine URL Rewrite Configuration (XML Style).

Configurations - Open URL Rewrite Configuration

Sample

Here’s a sample urlrewrite.xml file (click on the triangle on the left to expand/collapse):

Sample "urlrewrite.xml"
CRAFTER_HOME/data/repos/sites/PROJECTNAME/sandbox/config/engine/urlrewrite.xml
 1<?xml version="1.0" encoding="utf-8"?>
 2
 3<!--
 4  ~ Copyright (C) 2007-2022 Crafter Software Corporation. All Rights Reserved.
 5  ~
 6  ~ This program is free software: you can redistribute it and/or modify
 7  ~ it under the terms of the GNU General Public License version 3 as published by
 8  ~ the Free Software Foundation.
 9  ~
10  ~ This program is distributed in the hope that it will be useful,
11  ~ but WITHOUT ANY WARRANTY; without even the implied warranty of
12  ~ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
13  ~ GNU General Public License for more details.
14  ~
15  ~ You should have received a copy of the GNU General Public License
16  ~ along with this program.  If not, see <http://www.gnu.org/licenses/>.
17  -->
18
19<urlrewrite>
20
21    <rule>
22        <from>^/some/olddir/(.*)$</from>
23        <to type="redirect">/very/newdir/$1</to>
24    </rule>
25
26    <rule match-type="wildcard">
27        <from>/blog/archive/**</from>
28        <to type="redirect">/roller/history/$1</to>
29    </rule>
30
31</urlrewrite>

After making your changes and saving the configuration, remember to publish the configuration file just saved (urlrewrite.xml file). To publish the configuration file, from the Sidebar, click on Dashboard. In the Unpublished Work dashlet, check the box next to the urlrewrite.xml file, and click Publish from the context nav to publish.

Configurations - Publish URL Rewrite Config File from Dashboard

For more information on the UrlRewriteFilter, see http://tuckey.org/urlrewrite/


Engine Root Folder

Crafter Engine requires a root folder path to be configured if the defaults are not used.

The default root folder path has the pattern: crafter.engine.site.default.rootFolder.path=file:${CRAFTER_DATA_DIR}/repos/sites/{siteName}/ This relies on the CRAFTER_DATA_DIR environment variable being set. Crafter Engine will then resolve the {siteName} variable to the name of the site being requested.

To change the root folder path, you can either set the CRAFTER_DATA_DIR environment variable or change the default root folder path in the server-config.properties file (see more about that file in server-config.properties. The variable to modify is:

{delivery-env-directory}/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
crafter.engine.site.default.rootFolder.path=file:${CRAFTER_DATA_DIR}/repos/sites/{siteName}/

Turn Off Show Error

Templates in CrafterCMS will display the errors in line with content as they encounter them to help the template developer during the coding process. On production environments, you do not want the errors to show up because it will highlight site issues and expose information that may be a security concern. To turn off showing errors in line with content, do the following:

  1. Place the following property and value in the server-config.properties file

    CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
        crafter.engine.template.error.displayInView=false
    
  2. Restart the Crafter Engine application or the Tomcat service.

  3. Test by deploying an FTL file with an error in it. Note that the error will not show up but is printed out in the server’s log file.



Request Filtering Configuration

Since 4.1.0

The following allows you to setup a filter to deny access to any request matching the value/s defined in the property.

CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
crafter.security.forbidden.urls=/templates/**


Forwarded Headers

The following section allows you to configure forwarded headers to resolve the actual hostname and protocol when it is behind a load balancer or reverse proxy. Forwarded headers are disabled by default.

CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
1# Indicates if Forwarded or X-Forwarded headers should be used when resolving the client-originated protocol and
2# address. Enable when Engine is behind a reverse proxy or load balancer that sends these
3crafter.engine.forwarded.headers.enabled=false


Search Timeouts

The following allows you to configure the search client connection timeout, socket timeout and number of threads.

1# The connection timeout in milliseconds, if set to -1 the default will be used
2crafter.engine.search.timeout.connect=-1
3# The socket timeout in milliseconds, if set to -1 the default will be used
4crafter.engine.search.timeout.socket=-1
5# The number of threads to use, if set to -1 the default will be used
6crafter.engine.search.threads=-1


Content-Length Headers

The following allows you to configure the content-length header sent for responses. The content-length header is sent for all responses by default.

CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
1# Indicates if the 'etag' header should be added
2crafter.engine.header.etag.enable=false
3# Indicates the urls that will have the 'etag' header (comma separated ant matchers)
4crafter.engine.header.etag.include.urls=/**


Policy Headers

Since 4.1.2

Referer Policy

The following allows you to configure what information is made available in the Referer header in a request. This can be set to a different value as needed.

CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
1# The value of the Referer-Policy header that should be set in all requests. Supported
2# values are: no-referrer, no-referrer-when-downgrade, same-origin, origin, strict-origin,
3# origin-when-cross-origin, strict-origin-when-cross-origin, unsafe-url
4crafter.security.headers.referrerPolicy.value=no-referrer

Content Security Policy

The following allows you to configure which resources can be loaded (e.g. JavaScript, CSS, Images, etc.) and the URLs that they can be loaded from. This should be tuned to the specific requirements of each project.

CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
1# The value of the Content-Security-Policy header that should be set in all requests.
2crafter.security.headers.contentSecurityPolicy.value=default-src 'self' 'unsafe-inline'
3# Set to true to enable the Content-Security-Policy-Report-Only header (this will report in the user agent console instead of actually blocking the requests)
4crafter.security.headers.contentSecurityPolicy.reportOnly=true

To block offending requests, set crafter.security.headers.contentSecurityPolicy.reportOnly to false. This property is set to true by default.

X-Permitted-Cross-Domain-Policies

The following allows you to configure what other domains you want to allow access to your domain. The X-PERMITTED-CROSS-DOMAIN-POLICIES header is set to none (do not allow any embedding) by default.

CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
1# The value of the X-PERMITTED-CROSS-DOMAIN-POLICIES header that should be set in all requests
2crafter.security.headers.permittedCrossDomainPolicies.value=none


Spring Expression Language

The following allows you to configure SpEL expressions for custom app contexts. SpEL expressions support is disabled by default.

CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
1# Indicates if the custom site application contexts should support SpEL expressions
2crafter.engine.context.expressions.enable=false
3# Indicates if the whole servlet & spring context should be available for templates & scripts
4crafter.engine.disableVariableRestrictions=false
5# Patterns for beans that should always be accessible from the site application context
6crafter.engine.defaultPublicBeans=crafter\\.(targetIdManager|targetedUrlStrategy)


Static Methods in Freemarker Templates

The following allows you to configure access to static methods in Freemarker templates. Access to static methods in Freemarker templates is disabled by default.

CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
1# Indicates if access for static methods should be allowed in Freemarker templates
2crafter.engine.freemarker.statics.enable=false


Cache

Max Items

The following allows you to configure the maximum number of objects in Engine’s cache:

# The max number of items that each site cache can have
crafter.engine.site.default.cache.maxAllowedItems=250000

URL Transformations

The following allows you to configure whether the URL transformation performed by the view resolver will be cached:

# Flag that indicates if the URL transformations performed by the view resolver should be cached
crafter.engine.page.view.resolver.url.transformation.cache=false

Preloaded Folders

The following allows you to configure folders to be preloaded in the cache:

#################
# Cache Warm Up #
#################
# Indicates if cache warming should be enabled. This means the site cache will be warmed up (according to a list of
# cache warmers) on context init and instead of cache clear, a new cache will be warmed up and switched with the
# current one
crafter.engine.site.cache.warmUp.enabled=false
# The descriptor folders that need to be preloaded in cache, separated by comma. Specify the preload depth with
# :{depth} after the path. If no depth is specified, the folders will be fully preloaded.
crafter.engine.site.cache.warmUp.descriptor.folders=/site:4
# The content folders that need to be preloaded in cache, separated by comma. Specify the preload depth with
# :{depth} after the path. If no depth is specified, the folders will be fully preloaded.
crafter.engine.site.cache.warmUp.content.folders=/scripts,/templates

where:

  • The descriptor folders are paths that contain XML that needs to be parsed, loaded and merged e.g. for inheritance. Most of the time this would be folders under /site

  • The content folders are mostly static, non-processed content, e.g. scripts, templates, static-assets

For all projects, the cache is preloaded using the above configuration. CrafterCMS warms up the cache on every publish and startup. Note also that what’s cache warmed will be warmed on every publish and startup and will live as long as nothing kicks it out of the cache due to least recently used (LRU) cache.

S3 Object

Since 4.1.0

The following allows you to configure a white list of paths for caching in memory when using S3 store and also the maximum content length for S3 objects allowed to be cached in memory

# Maximum content length (in bytes) for S3 objects to be cached in memory. Larger files will be retrieved
# directly from S3 every time they are requested.
# Default set to 10M = 10 * 1024 * 1024
crafter.engine.store.s3.cache.contentMaxLength=10485760
# White list of paths to be cached in memory when using S3 store.
crafter.engine.store.s3.cache.allowedPaths=\
  /config/.*,\
  /site/.*,\
  /scripts/.*,\
  /templates/.*,\
  /static-assets/css/.*,\
  /static-assets/js/.*,\
  /static-assets/fonts/.*


Groovy Sandbox Configuration

When a Groovy script is executed all code is validated against a blacklist of insecure expressions to prevent code that could compromise the system. When you try to execute a script that contains insecure expressions you will see an error similar to this:

UnsupportedOperationException: Insecure call staticMethod java.lang.Runtime getRuntime ...

It is recommended to keep the default configuration if possible. However, if access to one or more of the blacklisted expressions is required, it is possible to override the blacklist configuration. Configuration is global and affects all scripts on the server.

Warning

When you allow a script to make an insecure call you should make sure it can only be executed with known arguments and never with unverified user input.


Groovy Sandbox Properties

The following allows you to configure the Groovy sandbox. The Groovy sandbox is enabled by default and can be disabled by changing the property crafter.engine.groovy.sandbox.enable to false.

CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
1# Indicates if the sandbox should be enabled for all sites
2crafter.engine.groovy.sandbox.enable=true
3# Indicates if the blacklist should be enabled for all sites (this will have no effect if the sandbox is disabled)
4crafter.engine.groovy.sandbox.blacklist.enable=true
5# The location of the default blacklist to use for all sites (this will have no effect if the sandbox is disabled)
6crafter.engine.groovy.sandbox.blacklist.path=classpath:crafter/engine/groovy/blacklist

Using a Custom Blacklist

Crafter Engine includes a default blacklist that you can find here. Make sure you review the branch/tag you’re using.

To use a custom blacklist follow these steps:

  1. Copy the default blacklist file to your classpath, for example:

    CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/groovy/blacklist

  2. Remove or comment (adding a # at the beginning of the line) the expressions that your scripts require

  3. Update the server-config.properties configuration file to load the custom blacklist:

    CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
    # The location of the blacklist to use for all sites (this will have no effect if the sandbox is disabled)
    crafter.engine.groovy.sandbox.blacklist.path=classpath:crafter/engine/extension/groovy/blacklist
    

    Note

    In CrafterCMS v3.1.14 and prior, the name of the property is crafter.engine.groovy.sandbox.blacklist

  4. Restart CrafterCMS

Now you can execute the same script without any issues.

Disabling the Sandbox Blacklist

It is possible to disable the blacklist to allow the execution of most expressions, in case you need to use a considerable number of the expression included in the blacklist while keeping some basic restrictions. To disable the blacklist for all projects/sites update the server configuration file server-config.properties:

CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
# Indicates if the blacklist should be enabled for all sites (this will have no effect if the sandbox is disabled)
crafter.engine.groovy.sandbox.blacklist.enable=false

Important Notes

There are some limitations that should be noted when working with the Groovy Sandbox.

One limitation is that an exception is thrown during execution when a Groovy class has a property and a getter method for the property. Here’s an example code that throws an exception during execution:
class Test {
  private String message

  public String getMessage() {
     return this.message
  }
}

def t = new Test()
t.message = "this is a test"

return t.getMessage()

Here’s the error thrown in the logs by the code above:

Caused by: java.lang.StackOverflowError
     at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:693)
     at groovy.lang.GroovyClassLoader$InnerLoader.loadClass(GroovyClassLoader.java:450)
     at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:812)
     at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:800)
     at sun.reflect.GeneratedMethodAccessor340.invoke(Unknown Source)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:498)
     at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
     at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
     at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:1845)
     at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:3773)
     at Test.getProperty(test.get.groovy)
     at org.codehaus.groovy.runtime.InvokerHelper.getProperty(InvokerHelper.java:190)
     at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.getProperty(ScriptBytecodeAdapter.java:469)
     at org.kohsuke.groovy.sandbox.impl.Checker$7.call(Checker.java:392)
     at org.kohsuke.groovy.sandbox.GroovyInterceptor.onGetProperty(GroovyInterceptor.java:68)
     at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:297)
     at org.kohsuke.groovy.sandbox.impl.Checker$7.call(Checker.java:390)
     at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:394)
     at org.kohsuke.groovy.sandbox.impl.Checker$checkedGetProperty$1.callStatic(Unknown Source)
     at Test.getMessage(test.get.groovy:5)

Workarounds

There are a couple of things you can do to get around the exception being thrown:

  • Do not use getter methods and instead access the property directly
    Using the example above, we’ll access the property directly:

    class Test {
      private String message
    }
    
    def t = new Test()
    t.message = "this is a test"
    
    return t.message
    

  • Use a different name for the property and the getter method
    Again, using the example above, we’ll use a different name from the property for the getter method:

    class Test {
      private String theMessage
    
      public String getMessage() {
         return this.theMessage
      }
    }
    
    def t = new Test()
    t.theMessage = "this is a test"
    
    return t.getMessage()
    


Configuration Properties

This example file contains the properties used by Crafter Engine (click on the triangle on the left to expand/collapse):

Sample file containing the properties used by Crafter Engine
  1<?xml version="1.0" encoding="UTF-8"?>
  2<!--
  3  ~ Copyright (C) 2007-2022 Crafter Software Corporation. All Rights Reserved.
  4  ~
  5  ~ This program is free software: you can redistribute it and/or modify
  6  ~ it under the terms of the GNU General Public License version 3 as published by
  7  ~ the Free Software Foundation.
  8  ~
  9  ~ This program is distributed in the hope that it will be useful,
 10  ~ but WITHOUT ANY WARRANTY; without even the implied warranty of
 11  ~ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 12  ~ GNU General Public License for more details.
 13  ~
 14  ~ You should have received a copy of the GNU General Public License
 15  ~ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 16  -->
 17
 18<!--
 19    This file configures site properties used by Crafter Engine
 20
 21    Below are the properties used by Crafter Engine:
 22
 23    (General Properties)
 24    <indexFileName />  (The name of a page's index file)
 25    <defaultLocale />  (Default locale for the site)
 26
 27    (Navigation Properties)
 28    <navigation>
 29        <additionalFields /> (List of additional fields to include for dynamic navigation items)
 30    </navigation>
 31
 32    (Single Page Application Properties (React JS, Angular, Vue.js, etc.))
 33    <spa>
 34        <enabled /> (Enable/disable SPA mode, default is false)
 35        <viewName /> (The view name for the SPA (Single Page Application). Current view names can be a page URL (like /) or a template name (like /template/web/app.ftl). Default is /)
 36    </spa>
 37
 38    (Compatibility properties)
 39    <compatibility>
 40        <disableFullModelTypeConversion /> Disables full content model type conversion for backwards compatibility mode (false by default)
 41    </compatibility>
 42
 43
 44    (Filter Properties)
 45    <filters> (Defines the filter mappings)
 46        <filter>
 47            <script> (Specifies the complete path to the filter script)
 48                <mapping>
 49                    <include /> (Contains Ant patterns (separated by comma) that request URLs should match for the filter to be executed)
 50                    or
 51                    <exclude /> (Contains patterns that the requests shouldn't match)
 52                </mapping>
 53            </script>
 54        </filter>
 55    </filters>
 56
 57    (CORS Properties)
 58    <cors>
 59        <enable>true</enable> (Enable/disable CORS headers, default is false)
 60        (Values for each of the headers that will be added to responses)
 61        <accessControlMaxAge>3600</accessControlMaxAge>
 62        <accessControlAllowOrigin>*</accessControlAllowOrigin>
 63        <accessControlAllowMethods>GET\, OPTIONS</accessControlAllowMethods>
 64        <accessControlAllowHeaders>Content-Type</accessControlAllowHeaders>
 65        <accessControlAllowCredentials>true</accessControlAllowCredentials>
 66    </cors>
 67
 68    (Content Targeting Properties)
 69    <targeting>
 70        <enabled /> (Enable/disable content targeting, default is false)
 71        <rootFolders /> (Root folders handled for content targeting)
 72        <excludePatterns /> (Regex patterns used to exclude certain paths from content targeting)
 73        <availableTargetIds /> (Valid target IDs)
 74        <fallbackTargetId /> (Target ID used as a last resort when resolving targeted content)
 75        <mergeFolders /> (Sets whether the content of folders that have the same "family" of target IDs should be merged)
 76        <redirectToTargetedUrl /> (Sets whether the request should be redirected when the targeted URL is different from the current URL)
 77    </targeting>
 78
 79    (Profile Properties)
 80    <profile>
 81        <api>
 82            <accessTokenId /> (The access token to use for the Crafter Profile REST calls.  This should always be specified on multi-tenant configurations)
 83        </api>
 84    </profile>
 85
 86    (Security Properties)
 87    <security>
 88        <saml>
 89            <token/> (The expected value for the secure key request header)
 90            <groups>
 91                <group>
 92                    <name/> (The name of the group from the request header)
 93                    <role/> (The value to use for the role in the profile)
 94                </group>
 95            </groups>
 96            <attributes>
 97                <attribute>
 98                    <name/> (The name of the request header for the attribute)
 99                    <field/> (The name of the field to use in the profile)
100                </attribute>
101            </attributes>
102        </saml/>
103        <login>
104            <formUrl /> (The URL of the login form page)
105            <defaultSuccessUrl /> (The URL to redirect to if the login was successful and the user could not be redirected to the previous page)
106            <alwaysUseDefaultSuccessUrl /> (Sets whether to always redirect to the default success URL after a successful login)
107            <failureUrl /> (The URL to redirect to if the login fails)
108        </login>
109        <logout>
110            <successUrl /> (The URL to redirect after a successful logout)
111        </logout>
112        <accessDenied>
113            <errorPageUrl /> (The URL of the page to show when access has been denied to a user to a certain resource)
114        </accessDenied>
115        <urlRestrictions> (Contains any number of restriction elements)
116            <restriction> (Restriction element, access is denied if a request matches the URL, and the expression evaluates to false)
117                <url /> (URL pattern)
118                <expression /> (Spring EL expression)
119            </restriction>
120        </urlRestrictions>
121    </security>
122
123    (Social Properties)
124    <socialConnections>
125        <facebookConnectionFactory>
126            <appId /> (The Facebook app ID required for establishing connections with Facebook)
127            <appSecret /> (The Facebook app secret required for establishing connections with Facebook)
128        </facebookConnectionFactory>
129    </socialConnections>
130
131    (Job Properties)
132    <jobs>
133        <jobFolder> (Specifies a folder which will be looked up for scripts to be scheduled using a certain cron expression)
134            <path /> (Path absolute to the site root)
135            <cronExpression /> (Cron expression)
136        </jobFolder>
137        <job> (Specifies a single script job to be scheduled)
138            <path />
139            <cronExpression />
140        </job>
141    </jobs>
142
143    (Cache Warm Up properties)
144    <cache>
145        <warmUp>
146            <descriptorFolders /> The descriptor folders that need to be pre-loaded in cache, separated by comma.
147            <contentFolders /> The content folders that need to be preloaded in cache, separated by comma.
148        </warmUp>
149    </cache>
150
151    (Header Mappings properties)
152    <headerMappings>
153        <mapping>
154            <urlPattern>/products/**</urlPattern> (Ant path pattern)
155            <headers>
156                <header>
157                    <name>Cache-Control</name>
158                    <value>max-age=60\, s-maxage=300</value>
159                </header>
160            </headers>
161        </mapping>
162        <mapping>
163            <urlPattern>/**/*.pdf</urlPattern> (Ant path pattern)
164            <headers>
165                <header>
166                    <name>X-Crafter-Document</name>
167                    <value>true</value>
168                </header>
169            </headers>
170        </mapping>
171    </headerMappings>
172
173    You can learn more about Crafter Engine site configuration in the docs at
174    http://docs.craftercms.org
175
176-->
177
178<site>
179    <!-- General properties -->
180    <indexFileName>index.xml</indexFileName>
181    <defaultLocale>en</defaultLocale>
182
183    <!-- Navigation properties -->
184    <!--
185    <navigation>
186        <additionalFields>navIcon,componentType</additionalFields>
187    </navigation>
188    -->
189
190    <!-- Compatibility properties -->
191    <compatibility>
192        <disableFullModelTypeConversion>false</disableFullModelTypeConversion>
193    </compatibility>
194
195
196    <!-- Filter properties -->
197    <filters>
198        <filter>
199            <script>/scripts/filters/testFilter1.groovy</script>
200            <mapping>
201                <include>/**</include>
202            </mapping>
203        </filter>
204        <filter>
205            <script>/scripts/filters/testFilter2.groovy</script>
206            <mapping>
207                <include>/**</include>
208            </mapping>
209        </filter>
210        <filter>
211            <script>/scripts/filters/testFilter3.groovy</script>
212            <mapping>
213                <include>/**</include>
214                <exclude>/static-assets/**</exclude>
215            </mapping>
216        </filter>
217    </filters>
218
219    <!-- CORS Properties -->
220    <cors>
221        <enable>true</enable>
222        <accessControlMaxAge>3600</accessControlMaxAge>
223        <accessControlAllowOrigin>*</accessControlAllowOrigin>
224        <accessControlAllowMethods>GET\, OPTIONS</accessControlAllowMethods>
225        <accessControlAllowHeaders>Content-Type</accessControlAllowHeaders>
226        <accessControlAllowCredentials>true</accessControlAllowCredentials>
227    </cors>
228
229    <!-- Content targeting properties -->
230    <targeting>
231        <enabled>true</enabled>
232        <rootFolders>/site/website</rootFolders>
233        <excludePatterns>/site/website/index\.xml</excludePatterns>
234        <availableTargetIds>en,ja,ja_JP,ja_JP_JP</availableTargetIds>
235        <fallbackTargetId>en</fallbackTargetId>
236        <mergeFolders>true</mergeFolders>
237        <redirectToTargetedUrl>false</redirectToTargetedUrl>
238    </targeting>
239
240    <!-- Profile properties -->
241    <profile>
242        <api>
243            <accessTokenId>${enc:q3l5YNoKH38RldAkg6EAGjxlI7+K7Cl4iEmMJNlemNOjcuhaaQNPLwAB824QcJKCbEeLfsg+QSfHCYNcNP/yMw==}</accessTokenId>
244        </api>
245    </profile>
246
247    <!-- Security properties -->
248    <security>
249        <saml>
250            <token>SOME_RANDOM_TOKEN</token>
251            <groups>
252                <group>
253                    <name>MEMBER</name>
254                    <role>memberUser</role>
255                </group>
256            </groups>
257            <attributes>
258                <attribute>
259                    <name>givenName</name>
260                    <field>firstName</field>
261                </attribute>
262            </attributes>
263        </saml>
264        <login>
265            <formUrl>/signin</formUrl>
266            <defaultSuccessUrl>/home</defaultSuccessUrl>
267            <alwaysUseDefaultSuccessUrl>true</alwaysUseDefaultSuccessUrl>
268            <failureUrl>/signin?error=loginFailure</failureUrl>
269        </login>
270        <logout>
271            <successUrl>/home</successUrl>
272        </logout>
273        <accessDenied>
274            <errorPageUrl>/signin?error=accessDenied</errorPageUrl>
275        </accessDenied>
276        <urlRestrictions>
277            <restriction>
278                <url>/*</url>
279                <expression>hasRole('USER')</expression>
280            </restriction>
281        </urlRestrictions>
282    </security>
283
284    <!-- Social properties -->
285    <socialConnections>
286        <facebookConnectionFactory>
287            <appId>${enc:Nk4ZJWGGNIf9tt0X8BudixQhHekkBbG1AJE6myeqxp8=}</appId>
288            <appSecret>${enc:JOqVSAHHPYmIO8dC5VCz4KDBbKK466zKeAEowuDRqDammJ+07XmRbB+2ob5T8mg6gAEjDs5WxMuMiMPaDr4wOg==}</appSecret>
289        </facebookConnectionFactory>
290    </socialConnections>
291
292    <!-- Job properties -->
293    <jobs>
294        <jobFolder>
295            <path>/scripts/jobs/morejobs</path>
296            <cronExpression>0 0/15 * * * ?</cronExpression>
297        </jobFolder>
298        <job>
299            <path>/scripts/jobs/testJob.groovy</path>
300            <cronExpression>0 0/15 * * * ?</cronExpression>
301        </job>
302    </jobs>
303
304    <!-- Cache Warm Up properties -->
305    <cache>
306        <warmUp>
307            <descriptorFolders>/site:3</descriptorFolders>
308            <contentFolders>/scripts,/templates</contentFolders>
309        </warmUp>
310    </cache>
311
312    <!-- Header Mappings properties-->
313    <headerMappings>
314        <mapping>
315            <urlPattern>/products/**</urlPattern>
316            <headers>
317                <header>
318                    <name>Cache-Control</name>
319                    <value>max-age=60\, s-maxage=300</value>
320                </header>
321            </headers>
322        </mapping>
323        <mapping>
324            <urlPattern>/**/*.pdf</urlPattern>
325            <headers>
326                <header>
327                    <name>X-Crafter-Document</name>
328                    <value>true</value>
329                </header>
330            </headers>
331        </mapping>
332    </headerMappings>
333</site>


Crafter Engine Properties
  • indexFileName: The name of a page’s index file (default is index.xml).

  • defaultLocale: The default locale for the project. Used with content targeting through localization.

  • navigation.additionalFields: List of additional fields to include for dynamic navigation items (Example: <additionalFields>myTitle_s,myAuthor_s,…</additionalFields>)

  • spa: Used for Single Page Application (SPA) Properties (React JS, Angular, Vue.js, etc.). Contains <enabled> element which enables/disables SPA mode (default is false) and <viewName> element, the view name for the SPA (Single Page Application. Current view names can be a page URL (like /) or a template name (like /template/web/app.ftl). Default is /)

  • compatibility.disableFullModelTypeConversion: Disables full content model type conversion for backwards compatibility mode (false by default)

    Up to and including version 2: Crafter Engine, in the FreeMarker host only, converts model elements based on a suffix type hint, but only for the first level in the model, and not for _dt. For example, for contentModel.myvalue_i Integer is returned, but for contentModel.repeater.myvalue_i and contentModel.date_dt a String is returned. In the Groovy host no type of conversion was performed.

    In version 3 onwards: Crafter Engine converts elements with any suffix type hints (including _dt) at at any level in the content model and for both Freemarker and Groovy hosts.

  • filters: Used to define the filter mappings. Each <filter> element must contain a <script> element that specifies the complete path to the filter script, and a <mapping> element. In the <mapping> element, the <include> element contains the Ant patterns (separated by comma) that request URLs should match for the filter to be executed, while the <exclude> element contains the patterns that requests shouldn’t match.

  • cors.enable:true if CORS headers should be added to REST API responses when not in preview mode. Defaults to false.
    . The elements <accessControlMaxAge>, <accessControlAllowOrigin>, <accessControlAllowMethods>, <accessControlAllowHeaders> and <accessControlAllowCredentials> have the values that will be copied to each response.

    <accessControlAllowOrigin> values are split using ,. Remember that commas inside patterns need to be escaped with a \, like this: <accessControlAllowOrigin>http://localhost:[8000\,3000],http://*.other.domain</accessControlAllowOrigin>

    <accessControlAllowMethods> and <accessControlAllowHeaders> values are split using ,. Remember to escape the commas , separating the values like this: <accessControlAllowHeaders>X-Custom-Header\, Content-Type</accessControlAllowHeaders> or <accessControlAllowMethods>GET\, OPTIONS</accessControlAllowMethods>

    Note

    When engine is in preview mode, it is a proxy and therefore will not add CORS headers to REST API responses even if CORS is enabled.

  • targeting.enabled:true if content targeting should be enabled. Defaults to false.

  • targeting.rootFolders: The root folders that should be handled for content targeting.

  • targeting.excludePatterns: Regex patterns that are used to exclude certain paths from content targeting.

  • targeting.availableTargetIds: The valid target IDs for content targeting (see Content Targeting Guide).

  • targeting.fallbackTargetId: The target ID that should be used as last resort when resolving targeted content. (see Content Targeting Guide).

  • targeting.mergeFolders: true if the content of folders that have the same “family” of target IDs should be merged. (see Content Targeting Guide).

  • targeting.redirectToTargetedUrl: true if the request should be redirected when the targeted URL is different from the current URL. (see Content Targeting Guide).

  • profile.api.accessToken: The access token to use for the Profile REST calls. This parameter should be always specified on multi-tenant configurations.

  • security.saml.token: The expected value for the secure key request header

  • security.saml.groups: Contains any number of <group> elements. Each <group> element contains a <name> element (The name of the group from the request header) and a <role> element (The value to use for the role in the profile).

  • security.saml.attributes: Contains any number of <attribute> elements. Each <attribute> element contains a <name> element (The name of the request header for the attribute) and a <field> element (The name of the field to use in the profile).

  • security.login.formUrl: The URL of the login form page. The default is /login.

  • security.login.defaultSuccessUrl: The URL to redirect to if the login was successful and the user couldn’t be redirected to the previous page. The default is /.

  • security.login.alwaysUseDefaultSuccessUrl: true if after successful login always redirect to the default success URL. The default is false.

  • security.login.failureUrl: The URL to redirect to if the login fails. The default is /login?login_error=true.

  • security.logout.successUrl: The URL to redirect after a successful logout. The default is /.

  • security.accessDenied.errorPageUrl: The URL of the page to show when access has been denied to a user to a certain resource. The default is /access-denied.

  • security.urlRestrictions: Contains any number of restriction elements. Each restriction is formed by an Ant-style path pattern (<url>) and a Spring EL expression (<expression>) executed against the current profile. If a request matches the URL, and the expression evaluates to false, access is denied. For more information, check UrlAccessRestrictionCheckingProcessor.java and AccessRestrictionExpressionRoot.java

    Note

    For the <url> Ant-style path pattern, <url>/*</url> indicates just one level of the URL and <url>/**</url> indicates all urls. For more information on Ant-style path pattern matching, see https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/util/AntPathMatcher.html

  • socialConnections.facebookConnectionFactory.appId: The Facebook app ID required for establishing connections with Facebook.

  • socialConnections.facebookConnectionFactory.appSecret: The Facebook app secret required for establishing connections with Facebook.

  • jobs.jobFolder: Specifies a folder which will be looked up for scripts to be scheduled using a certain cron expression. The folder path should be specified with <path>, and should be absolute to the project root. The cron expressions is specified in <cronExpression>.

  • jobs.job: Specifies a single script job to be scheduled. The job path should be specified in <path>, and the cron expression in <cronExpression>.

  • cache.warmUp.descriptorFolders: The descriptor folders (paths that contain XML that needs to be parsed, loaded and merged e.g. for inheritance. Most of the time this would be folders under /site) that need to be pre-loaded in cache, separated by comma, when not in preview mode. Specify the preload depth with :{depth} after the path. If no depth is specified, the folders and all their sub-folders will be fully preloaded. Example: <descriptorFolders>/site:3</descriptorFolders>

  • cache.warmUp.contentFolders: The content folders (mostly static, non-processed content, e.g. scripts, templates, static-assets) that need to be pre-loaded in cache, separated by comma, when not in preview mode. Specify the preload depth with :{depth} after the path. If no depth is specified, the folders and all their sub-folders will be fully pre-loaded. Example: <contentFolders>/scripts,/templates</contentFolders>

    Note

    Cache and ActiveCache do not function the same way as specified above when engine is in preview because the preview server does not cache to ensure the latest updates are seen immediately.

  • headerMappings.mapping.urlPattern Ant path pattern to match for adding headers to response

  • headerMappings.mapping.headers The headers that will be added to responses. Each <header> element must contain a <name> element that specifies the name of the header e.g. Cache-Control, and a <value> element containing directives, etc. (separated by an escaped comma) e.g. max-age=60\, s-maxage=300.

Note

Crafter Engine will not be able to load your Project Context if your configuration contains invalid XML or incorrect configuration.

CrafterCMS supports adding headers to responses when there are matched configuration patterns in the Engine Project Configuration file

To setup HTTP response headers, do the following: - Configure the Ant path pattern to match for adding headers to response in headerMappings.mapping.urlPattern - Configure the <header> element and the <value>` element ` with your desired values under headerMappings.mapping.headers.

<headerMappings>
  <mapping>
    <urlPattern>/**/*.pdf</urlPattern>
    <headers>
      <header>
        <name>X-Crafter-Document</name>
        <value>true</value>
      </header>
    </headers>
  </mapping>
</headerMappings>

Cache headers allows specifying caching policies such as how an item is cached, maximum age before expiring, etc. These headers are extremely useful for indicating cache TTLs to CDNs and browsers on certain requests.

To setup cache headers, do the following:

  • Configure the Ant path pattern to match for adding headers to response in headerMappings.mapping.urlPattern

  • Configure the <header> element with the value Cache-Control and the element <value> with your desired Cache-Control directive under headerMappings.mapping.headers.

    See here for a list of available directives to use with Cache-Control.

Your configuration should look something like below:

<headerMappings>
  <mapping>
    <urlPattern>/articles/**</urlPattern>
    <headers>
      <header>
        <name>Cache-Control</name>
        <value>max-age=60\, s-maxage=300</value>
      </header>
    <headers>
  </mapping>
</headerMappings>

Please note that the Cache-Control header inserted to responses by default is set to No-Cache.

Spring Configuration

Each project can also have it’s own Spring application context. Just as with site-config.xml, beans can be overwritten using the following locations:

Spring Configuration Files
  • /config/engine/application-context.xml (This file can be accessed easily from any project created through the out-of-the-box blueprints, by navigating from the Studio sidebar to Project Tools > Configuration, and finally picking up the Engine Project Application Context option from the dropdown).

    Engine Project Application Context
  • /config/engine/{crafterEnv}-application-context.xml

  • $TOMCAT/shared/classes/crafter/engine/extension/sites/{siteName}/application-context.xml

The application context inherits from Engine’s own service-context.xml, and any class in Engine’s classpath can be used, including Groovy classes declared under /scripts/classes/*.

As an example, assuming you have defined a Groovy class under /scripts/classes/mypackage/MyClass.groovy, you can define a bean like this:

application-context.xml
 1      <?xml version="1.0" encoding="UTF-8"?>
 2      <beans xmlns="http://www.springframework.org/schema/beans"
 3             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 4             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
 5
 6  <bean class="org.springframework.context.support.PropertySourcesPlaceholderConfigurer" parent="crafter.properties"/>
 7
 8  <bean id="greeting" class="mypackage.MyClass">
 9    <property name="myproperty" value="${myvalue}"/>
10  </bean>
11
12</beans>

A org.springframework.context.support.PropertySourcesPlaceholderConfigurer (like above) can be specified in the context so that the properties of site-config.xml can be used as placeholders, like ${myvalue}. By making the placeholder configurer inherit from crafter.properties, you’ll also have access to Engine’s global properties (like crafter.engine.preview).

Note

Crafter Engine will not be able to load your Project Context if your context file contains invalid XML, incorrect configuration or if your beans do not properly handle their own errors on initialization.

Engine Multi-Environment Support

The following engine configuration files can be setup for different environments:

  • site-config.xml

  • application-context.xml

  • urlrewrite.xml

To setup an environment for engine configuration files, do the following:

  1. Create a folder under data/repos/sites/${site}/sandbox/config/engine called env

  2. Inside the folder, create a directory called myenv (or whatever you want to call the environment)

  3. Copy the configuration file you want to override in the new environment you are setting up, inside your myenv folder

  4. Remember to commit the files copied so Studio will pick it up.

  5. In the crafter-setenv.sh file in TOMCAT/bin set the following property to desired environment:

    bin/crafter-setenv.sh
    # -------------------- Configuration variables --------------------
    export CRAFTER_ENVIRONMENT=${CRAFTER_ENVIRONMENT:=myenv}
    

  6. Restart Crafter

Examples

Creating a Custom Environment Example

Let’s take a look at an example of creating a new environment, called mycustomenv with the urlrewrite.xml file overridden in the new environment for a project created using the Website Editorial blueprint. This example is very similar to the example shown above for Studio except for the location of the custom configuration file:

  1. We’ll create a folder called env under data/repos/sites/my-editorial/sandbox/config/engine

    1data/
    2  repos/
    3    sites/
    4      my-editorial/
    5        sandbox/
    6          config/
    7            engine/
    8              env/
    

  2. Inside the env folder, create a directory called mycustomenv

  3. We will now create the configuration file for the urlrewrite.xml that we want to override in the new environment we are setting up, inside our mycustomenv folder:

        env/
          mycustomenv/
            urlrewrite.xml
    

    We will redirect the page to /articles/2021/12/Top Books For Young Women when the page /articles/2020/12/Top Books For Young Women is previewed. Copy the following inside the urlrewrite.xml file.

    Urlrewrite.xml file for environment mycustomenv
    1<?xml version="1.0" encoding="utf-8"?>
    2<urlrewrite>
    3  <rule>
    4    <from>/articles/2020/12/(.*)$</from>
    5    <to type="redirect">/articles/2021/12/$1</to>
    6  </rule>
    7</urlrewrite>
    

    For our example, the folder articles/2020/12 was copied to articles/2021 with the page under articles/2021/12, modified to display the title as a dupe. This was done so when we click on the page under articles/2020/12, we can easily tell that it’s being redirected to the page under articles/2021/12. Of course, you can also just look at the url of the page previewed to verify that it was redirected to the right page.

    Folder with page copied from 2020 to 2021

    Here’s the original page:

    Original page before being redirected

    Here’s the page we want to be redirected to when previewing the page above:

    Page we want to be redirected to

  4. Remember to commit the files copied so Studio will pick it up.

      sandbox git:(master)  git add .
    ➜  sandbox git:(master)  git commit -m "Add urlrewrite.xml file for mycustomenv"
    

  5. Open the crafter-setenv.sh file in TOMCAT/bin and set the value of CRAFTER_ENVIRONMENT to the environment we setup above (myenv) to make it the active environment:

    bin/crafter-setenv.sh
    # -------------------- Configuration variables --------------------
    export CRAFTER_ENVIRONMENT=${CRAFTER_ENVIRONMENT:=mycustomenv}
    

  6. Restart Crafter. To verify our newly setup environment, open the Sidebar and click on projectTools, then select Configuration. Notice that the active environment mycustomenv will be displayed on top of the configurations drop-down box and when you select the Engine URL Rewrite Configuration (XML Style), it should display the file we created in one of the previous step:

    Active Environment Displayed in Project Tools Configuration

    Let’s verify that our urlrewrite.xml is in effect. From the Sidebar, click on Home -> Entertainment -> Top Books For Young Women or, navigate to /articles/2020/12/ and click on Top Books For Young Women.

    Preview the page mentioned in the urlrewrite.xml that will be redirected

    The preview page should take you to /articles/2021/12/Top Books For Young Women

Environment Specific Configurations Example

Environments are useful for managing values such as paths or database connections without the need to change any code directly in the servers.

In this example, we show how to manage a database connection that will change depending on the server where the project is deployed. We will have three environments dev, auth and delivery

  1. First create the environments by following the example above for creating the environments. We’ll then have the following folders called dev, auth and delivery under CRAFTER_HOME/data/repos/sites/SITENAME/sandbox/config/engine/env

  2. Next, include the appropriate connection string for each environment in the site-config.xml file:

    Local Development Configuration: /config/engine/env/dev/site-config.xml
    1<?xml version="1.0" encoding="UTF-8"?>
    2<site>
    3  <db>
    4    <uri>mongodb://localhost:27017/mydb?maxPoolSize=1&amp;minPoolSize=0&amp;maxIdleTimeMS=10000</uri>
    5  </db>
    6</site>
    
    Authoring Configuration: /config/engine/env/auth/site-config.xml
    1<?xml version="1.0" encoding="UTF-8"?>
    2<site>
    3  <db>
    4    <uri>mongodb://localhost:27020/mydb?maxPoolSize=5&amp;minPoolSize=2&amp;maxIdleTimeMS=10000</uri>
    5  </db>
    6</site>
    
    Delivery Configuration: /config/engine/env/delivery/site-config.xml
    1<?xml version="1.0" encoding="UTF-8"?>
    2<site>
    3  <db>
    4    <uri>mongodb://delivery-db-server:27020/delivery-db?maxPoolSize=10&amp;minPoolSize=5&amp;maxIdleTimeMS=1000</uri>
    5  </db>
    6</site>
    

    Remember to commit the files copied so Studio will pick it up.

  3. Finally, notice when using this approach the code is completely independent of the environment so we only need one bean that will always connect to the right database:

    Default Application Context: /config/engine/application-context.xml (shared by all environments)
     1<?xml version="1.0" encoding="UTF-8"?>
     2<beans xmlns="http://www.springframework.org/schema/beans"
     3        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     4        xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
     5
     6  <bean class="org.springframework.context.support.PropertySourcesPlaceholderConfigurer" parent="crafter.properties"/>
     7
     8  <bean id="mongoUri" class="com.mongodb.MongoClientURI">
     9     <constructor-arg value="${db.uri}"/>
    10  </bean>
    11
    12  <bean id="mongoClient" class="com.gmongo.GMongoClient">
    13    <constructor-arg ref="mongoUri"/>
    14  </bean>
    15
    16</beans>
    


SAML2 Multi-Environment Support Enterprise only feature

When configuring SAML2 in an environment-specific project configuration file (site-config.xml), since the SAML2 configuration folder sits outside the environment folder, you can point to environment-specific SAML2 files in the SAML2 folder for the following path/file configuration of SAML2:

Property
Description
Default Value

keystore.path

The path of the keystore file in the repo

/config/engine/saml2/keystore.jks

identityProviderDescriptor

The path of the identity provider metadata XML descriptor in the repo

/config/engine/saml2/idp.xml

serviceProviderDescriptor

The path of the service provider metadata XML descriptor in the repo

/config/engine/saml2/sp.xml

Use the format /config/engine/saml2/saml2-path-file-config-{myCustomEnv}.*** for naming your SAML2 environment specific configuration files where {myCustomEnv} is the name of your environment.

Example

Say we’re setting up SAML2 files for an environment named dev. Using the format mentioned above, our environment specific SAML2 files will be the following:

  • /config/engine/saml2/keystore-dev.jks

  • /config/engine/saml2/idp-dev.xml

  • /config/engine/saml2/sp-dev.xml

Below is the SAML2 configuration using the above files in the project configuration file:

Example SAML2 configuration for a custom environment
<saml2>
  ...
  <keystore>
    <defaultCredential>abc-crafter-saml</defaultCredential>
    <path>/config/engine/saml2/keystore-dev.jks</path>
    <password encrypted="true">${enc:value}</password>
    <credentials>
      <credential>
        <name>abc-crafter-saml</name>
        <password encrypted="true">${enc:value}</password>
      </credential>
    </credentials>
  </keystore>
  <identityProviderName>http://www.okta.com/abc</identityProviderName>
  <identityProviderDescriptor>/config/engine/saml2/idp-dev.xml</identityProviderDescriptor>
  <serviceProviderName>https://intranet.abc.org/saml/SSO</serviceProviderName>
  <serviceProviderDescription>/config/engine/saml2/sp-dev.xml</serviceProviderDescription>
</saml2>

See Engine SAML2 Configuration for more information on configuring SAML2.

Engine Multi-target Support

There are some cases where the Engine configuration files need to have different values per publishing target. Say for a production environment where you have staging to test out your project and live , the project to be used by end users, you may need different SAML authentication mechanics or different URL rewrites.

The Engine Multi-Environment Support section detailed how to setup Engine configuration files per environment. CrafterCMS supports overriding Engine configuration files, not just per environment, but also per publishing target. It supports a base configuration per environment with the ability to override per publishing target.

The following engine configuration files can be setup for different publishing targets:

  • site-config.xml

  • application-context.xml

  • urlrewrite.xml

Here are the available publishing targets for the configuration files listed above:

  • preview

  • staging

  • live

Overriding Engine Configuration Files per Publishing Target

To override a configuration file in any of the publishing targets

  1. Add the new configuration file/s for overriding to Configurations under projectTools -> Configuration

    Multi-target Configuration - Open Configurations

    The overriding configuration file should be named configuration-to-be-overridden.publishing-target.xml. Depending on the publishing target you wish the configuration file to override, the files should look like one of the following:

    • configuration-to-be-overridden.preview.xml

    • configuration-to-be-overridden.staging.xml

    • configuration-to-be-overridden.live.xml


    Say, to add a urlrewrite.xml file override for staging, add the following in the Configurations

    Configurations - SITENAME/config/studio/administration/config-list.xml
    <file>
      <module>engine</module>
      <path>urlrewrite.staging.xml</path>
      <title>Engine URL Rewrite (XML Style) Staging</title>
      <description>Engine URL Rewrite (XML Style) Staging</description>
      <samplePath>sample-urlrewrite.xml</samplePath>
    </file>
    

    For more information on Configurations config file, see Configurations

  2. Fill in your desired additions/modifications to the override configuration file. Refresh your browser. The configuration file you added from above should now be available from projectTools -> Configuration. Open the new configuration file and make the necessary additions/modifications for the override file then save your changes.

    Multi-target Configuration - New configuration files added to dropdown list

  3. If the configuration file to be overridden is not for preview, publish the configuration file to the intended publishing target, staging or live

Example

Let’s take a look at an example of overriding the Project Configuration used by Engine site-config.xml for the staging and live publishing targets so that each target has a different SAML authentication mechanics (different identity provider in staging and live). In our example, we will use a project created using the Website Editorial blueprint named mysite

  1. Add the new configuration file/s for overriding to Configurations under projectTools -> Configuration. We will be overriding the site-config.xml file in the staging and live publishing targets, so we will add to the configuration a site-config.staging.xml and site-config.live.xml files.

    Configurations - SITENAME/sandbox/config/studio/administration/config-list.xml
     1<file>
     2  <module>engine</module>
     3  <path>site-config.staging.xml</path>
     4  <title>Engine Project Configuration Staging</title>
     5  <description>Project Configuration used by Engine for the Staging publishing target</description>
     6  <samplePath>sample-engine-site-config.xml</samplePath>
     7</file>
     8<file>
     9  <module>engine</module>
    10  <path>site-config.live.xml</path>
    11  <title>Engine Project Configuration Live</title>
    12  <description>Project Configuration used by Engine for the Live publishing target</description>
    13  <samplePath>sample-engine-site-config.xml</samplePath>
    14</file>
    

  2. The configurations we added above will now be available from projectTools -> Configuration.

    Multi-target Configuration - Project Tools override configuration files now listed in "Project Tools" -> "Configuration"

    Enable SAML2 in the configuration with identity provider My IDP1 for the site-config.staging.xml and use identity provider My IDP2 for the site-config.live.xml.

    SITENAME/sandbox/config/engine/site-config.staging.xml
     1<site>
     2  <version>4.0.1</version>
     3
     4  <security>
     5    <saml2>
     6      <enable>true</enable>
     7      <attributes>
     8        <mappings>
     9          <mapping>
    10            <name>DisplayName</name>
    11            <attribute>fullName</attribute>
    12          </mapping>
    13        </mappings>
    14      </attributes>
    15      <role>
    16         <mappings>
    17            <mapping>
    18               <name>editor</name>
    19               <role>ROLE_EDITOR</role>
    20            </mapping>
    21         </mappings>
    22      </role>
    23      <keystore>
    24         <defaultCredential>my-site</defaultCredential>
    25         <password>superSecretPassword</password>
    26         <credentials>
    27            <credential>
    28               <name>my-site</name>
    29               <password>anotherSecretPassword</password>
    30            </credential>
    31         </credentials>
    32      </keystore>
    33      <identityProviderName>My IDP1</identityProviderName>
    34      <serviceProviderName>Crafter Engine</serviceProviderName>
    35   </saml2>
    36  </security>
    37
    38</site>
    

    For more information on SAML2 configuration, see Engine SAML2 Configuration

  3. Publish site-config.live.xml to live and site-config.staging.xml to staging.

    To publish the override configuration files setup above, open the Dashboard via the Navigation Menu on the top right or via the Sidebar. Scroll to the Unpublished Work dashlet.

    Multi-target Configuration - New configuration files listed in the "Unpublished Work" dashlet in the Dashboard

    To publish the site-config.live.xml configuration file to publishing target live, put a check mark next to the file in the dashlet, then click on Publish from the context nav. Remember to set the Publishing Target to live in the Publish dialog

    Multi-target Configuration - Set "Publishing Target" to "live" in dialog for site-config.live.xml

    To publish the site-config.staging.xml file to publishing target staging put a check mark next to the file in the dashlet, then click on Publish from the context nav. Remember to set the Publishing Target to staging in the Publish dialog.

    The Engine site-config.live.xml configuration will now be loaded when viewing your project in live and the Engine site-config.staging.xml configuration will now be loaded when viewing your project in staging instead of the default Engine site-config.xml files

Setup Engine to Deliver a Project

Server-based Delivery

In this section, we will be working in the delivery environment of CrafterCMS and describing how to setup your project for a delivery environment.

Setup Crafter Deployer Target

CrafterCMS out of the box has a script to help you create your deployer target for the delivery environment.

In the bin folder in your CrafterCMS delivery environment, we will use the script init-site.sh to help us create the deployer target.

From your command line, navigate to your {Crafter-CMS-delivery-environment-directory}/bin/ , and execute the init-site script. The following output of init-site.sh -h explains how to use the script:

usage: init-site [options] [site] [repo-path]
 -a,--notification-addresses <addresses>   A comma-separated list of email
                                           addresses that should receive
                                           deployment notifications
 -b,--branch <branch>                      The name of the branch to clone
                                           (live by default)
 -f,--passphrase <passphrase>              The passphrase of the private
                                           key (when the key is passphrase
                                           protected)
 -h,--help                                 Show usage information
 -k,--private-key <path>                   The path to the private key, when
                                           using private-key authentication
                                           through SSH to the remote Git repo
 -p,--password <password>                  The password for the remote Git
                                           repo, when using basic
                                           authentication
 -u,--username <username>                  The username for the remote Git
                                           repo, when using basic
                                           authentication
 --addresses <>                            A comma-separated list of email
                                           addresses that should receive deployment notifications

EXAMPLES:
 Init a site from the default repo path (../../crafter-authoring/data/repos/sites/{sitename}/published)
     init-site mysite
 Init a site from a specific local repo path
     init-site mysite /opt/crafter/authoring/data/repos/sites/mysite/published
 Init a site from a specific local repo path, cloning a specific branch of the repo
     init-site -b master mysite /opt/crafter/authoring/data/repos/sites/mysite/published
 Init a site that is in a remote HTTPS repo with username/password authentication
     init-site -u jdoe -p jdoe1234 mysite https://github.com/jdoe/mysite.git
 Init a site that is in a remote SSH repo with public/private key authentication (specific private key path
 with no passphrase)
     init-site -k ~/.ssh/jdoe_key mysite ssh://myserver/opt/crater/sites/mysite
 Init a site that is in a remote SSH repo with public/private key authentication (specific private key path
 with passphrase)
     init-site -k ~/.ssh/jdoe_key -f jdoe123 mysite ssh://myserver/opt/crater/sites/mysite

Note

Remember that when using private key SSH authentication, the private key path must be set explicitly using the -k option. Here’s an example:

init-site -k ~/.ssh/jdoe_key myeditorial ssh://myserver/opt/crater/sites/myeditorial

We recommend using Secure Shell (SSH) with your project’s published repo Git URL and for authentication, to use either username/password authentication or public/private key authentication.

The SSH Git URL format is: ssh://[user@]host.xz[:port]/path/to/repo/ where sections between [] are optional.

Example #1: ssh://server1.example.com/path/to/repo

Example #2: ssh://jdoe@server2.example.com:63022/path/to/repo

Note

When using ssh keys for authentication, the keys need to be generated using RSA as the algorithm and with no passphrase.

Crafter requires the key to be RSA and does not support keys generated using an algorithm other than RSA. The Jsch library that Jgit uses only supports RSA and does not support other keys such as OpenSSH. Crafter also currently doesn’t support using a passphrase with SSH keys.

To generate your Secure Shell (SSH) keys for authentication, run the following command ssh-keygen -m PEM -b 4096 -t rsa. Your output should look something like this:

 ssh-keygen -m PEM -b 4096 -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/myuser/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/myuser/.ssh/id_rsa.
Your public key has been saved in /Users/myuser/.ssh/id_rsa.pub.
.
.

Check that the file starts with the following header: -----BEGIN RSA PRIVATE KEY----- to verify that the key is using RSA.

After generating your private and public keys, you will need to add your new public key to where your remote git repository is located. If you are using GitHub, you will need to add your public key (e.g., id_rsa.pub) into your GitHub account. If your remote Git repository is hosted on a server, you will need to copy your public key (e.g., id_rsa.pub) to the host server.

If you are just working on another directory on disk for your delivery, you can just use the filesystem. When your repository is local, make sure to use the absolute path. Here is an example project’s published repo Git url when using a local repository:

/opt/crafter/authoring/data/repos/sites/my-project/published

Note

  • When using ssh, you might see in the logs com.jcraft.jsch.JSchException: UnknownHostKey errors. These errors are common in Ubuntu, and are caused by known host keys being stored in non-RSA format. Please follow the instructions in Troubleshooting Deployer Issues under SSH Unknown Host to resolve them.

  • Git needs to be installed in authoring when using SSH to connect the delivery to the authoring.

    If you see the following error in the delivery Deployer: Caused by: java.io.IOException: bash: git-upload-pack: command not found you’ll need to add the location of git (usually /usr/bin) to your non-login shell startup file (e.g. ~/.bashrc).

    To get the location of Git, run the following command: which git-upload-pack

  • You can limit SSH access by using Git Shell, see https://git-scm.com/docs/git-shell for more information.

Viewing your Site for Testing

To test viewing your project, open a browser and type in the URL of your project.

If you have multiple projects setup, to view a certain project, in your browser, enter the following:

{Server URL}?crafterSite={siteName}

Here we have an example of a delivery setup in another directory on disk (local), where there are two projects, my-awesome-editorial and hello-world

Setup Project for Delivery - Project List

To set crafterSite to the hello-world project, in your browser, type in

http://localhost:9080?crafterSite=helloworld
Setup Project for Delivery - Hello World Project

To set the site to the myawesomesite, in your browser, type in

http://localhost:9080?crafterSite=myawesomesite
Setup Site for Delivery - My Awesome Site

Serverless Delivery

CrafterCMS can be configured to serve sites directly from AWS services, following this guide you will:

  • Create a AWS Elasticsearch domain (optional)

  • Configure a Crafter Studio in an authoring environment to call the Crafter Deployer to create an AWS CloudFormation with a CloudFront and S3 bucket for each site

  • Configure a Crafter Engine in a delivery environment to read files from the S3 bucket and query to AWS Elasticsearch (optional)

Prerequisites

  • An AWS account

  • A CrafterCMS authoring environment

  • A CrafterCMS delivery environment

Step 1: Create an Elasticsearch Domain for Delivery (optional)

Since serverless delivery requires a single Elasticsearch endpoint readable by all Engine instances, we recommend you create an AWS Elasticsearch domain for delivery. If you don’t want to use an AWS Elasticsearch domain then you should create and maintain your own Elasticsearch cluster.

Important

Authoring can also use an Elasticsearch domain, but be aware that in a clustered authoring environment each authoring instance requires a separate Elasticsearch instance. If you try to use the same ES domain then you will have multiple preview deployers writing to the same index.

To create an AWS Elasticsearch domain please do the following:

  1. In the top navigation bar of your AWS console, click the Services dropdown menu, and search for Elasticsearch Service.

  2. Click on Create a new domain.

  3. Select Deployment Type and on the Elasticsearch version, pick 7.2.

    Serverless Site - Elasticsearch Deployment Type

  4. On the next screen, enter the domain name. Leave the defaults on the rest of the settings or change as needed per your environment requirements, then click on Next.

  5. On Network Configuration, we recommend you pick the VPC where your delivery nodes reside. If they’re not running on an Amazon VPC, then pick Public Access.

    Serverless Site - Elasticsearch Network Access

  6. Select the Access Policy that fits your Crafter environment, and click on Next (if on the same VPC as delivery, we recommend Do not require signing request with IAM credential).

    Serverless Site - Elasticsearch Access Policy

  7. Review the settings and click on Confirm.

  8. Wait for a few minutes until the domain is ready. Copy the Endpoint. You’ll need this URL later to configure the Deployer and Delivery Engine which will need access to the Elasticsearch.

    Serverless Site - Elasticsearch Endpoint

Step 2: Configure the Delivery for Serverless Mode

  1. Edit the services override file to enable the Serverless S3 mode (DELIVERY_INSTALL_DIR/bin/apache-tomcat/shared/classes/crafter/engine/extension/services-context.xml):

    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
    
      <import resource="classpath*:crafter/engine/mode/multi-tenant/simple/services-context.xml" />
      <!-- S3 Serverless Mode -->
      <import resource="classpath*:crafter/engine/mode/serverless/s3/services-context.xml" />
    
    </beans>
    

  2. Edit the properties override file to point Engine to consume the site content from S3 (DELIVERY_INSTALL_DIR/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties). The properties you need to update are the following:

    • crafter.engine.site.default.rootFolder.path

    • crafter.engine.s3.region

    • crafter.engine.s3.accessKey

    • crafter.engine.s3.secretKey

    An example of how the server-config.properties would look with configuration to read from an S3 bucket per site (which is the most common use case), is the following (values in X are not displayed since they’re sensitive):

    DELIVERY_INSTALL_DIR/bin/apache-tomcat/shared/classes/crafter/engine/extension/server-config.properties
    # Content root folder when using S3 store. Format is s3://<BUCKET_NAME>/<SITES_ROOT>/{siteName}
    crafter.engine.site.default.rootFolder.path=s3://serverless-test-site-{siteName}/{siteName}
    ...
    
    # S3 Serverless properties
    # S3 region
    crafter.engine.s3.region=us-east-1
    # AWS access key
    crafter.engine.s3.accessKey=XXXXXXXXXX
    # AWS secret key
    crafter.engine.s3.secretKey=XXXXXXXXXXXXXXXXXXXX
    

    As you can see, the bucket name portion of the root folder S3 URL contains a prefix and then the site name. This prefix is mentioned also as a “namespace” later on in the Studio serverless configuration.

    Important

    You can also provide the AWS region, access key and secret key without having to edit the config file properties. Please see Set up AWS Credentials and Region for Development.

  3. We recommend that the AWS credentials configured belong to a user with just the following permission policy (all strings like $VAR are placeholders and need to be replaced):

    aws-serverless-engine-policy.json
     1{
     2    "Version": "2012-10-17",
     3    "Statement": [
     4        {
     5            "Effect": "Allow",
     6            "Action": "s3:ListAllMyBuckets",
     7            "Resource": "*"
     8        },
     9        {
    10            "Effect": "Allow",
    11            "Action": [
    12                "s3:ListBucket",
    13                "s3:GetBucketLocation",
    14                "s3:GetObject"
    15            ],
    16            "Resource": "arn:aws:s3:::$BUCKET_NAME_PREFIX-*"
    17        }
    18    ]
    19}
    

  4. Edit the ES_URL in DELIVERY_INSTALL_DIR/bin/crafter-setenv.sh to point to the Elasticsearch endpoint you created in the previous step:

    export ES_URL=https://vpc-serverless-test-jpwyav2k43bb4xebdrzldjncbq.us-east-1.es.amazonaws.com
    

  5. Make sure that the you have an application load balancer (ALB) fronting the Delivery Engine instances and that it’s accessible by AWS CloudFront.

Step 3: Configure Authoring for Serverless Deployment

Instead of having one Crafter Deployer per node in delivery, for serverless you just need a single Deployer uploading files to S3. The authoring preview deployer can also be used for serverless deployment, when there’s only one authoring node. When there’s multiple authoring nodes (a cluster), then you’ll need to have a separate deployer pulling from a load balanced SSH/HTTPS URL fronting the Studio Git repos.

In both cases you still need to configure Studio to call the Deployer to create the serverless targets on site creation. You can find this configuration under CRAFTER_HOME/bin/apache-tomcat/shared/classes/crafter/studio/extension/studio-config-override.yaml. The properties are well documented in the file so they won’t be explained here, but there are still some important things to notice:

  • You need to add the URL of the Elasticsearch domain created in a previous step under studio.serverless.delivery.deployer.target.template.params.elasticsearch_url:

    studio.serverless.delivery.deployer.target.template.params:
      # The delivery Elasticsearch endpoint (optional is authoring is using the same one, specified in the ES_URL env variable)
      elastic_search_url: https://vpc-serverless-test-jpwyav2k43bb4xebdrzldjncbq.us-east-1.es.amazonaws.com
    

  • When using the aws-cloudformed-s3 target template (the default one), the Deployer creates first an AWS CloudFormation stack with an S3 bucket where the site content will be uploaded and a CloudFront that will serve /static-assets directly and will redirect any other requests to the Delivery Engine LB (which you specify in studio.serverless.delivery.deployer.target.template.params.aws.cloudformation.deliveryLBDomainName).

  • The aws.cloudformation.namespace is basically the prefix of the S3 bucket mentioned in the previous step. This prefix will be part of the name of most of the AWS resources created by the serverless deployer.

  • You need to specify proper AWS credentials for creating the CloudFormation stack and uploading files to S3, which can be done in the following ways:

    • As environment variables or under the default AWS credentials path, like explained in Set up AWS Credentials and Region for Development.

    • In the aws.default_access_key and aws.default_secret_key properties under studio.serverless.delivery.deployer.target.template.params:

      studio.serverless.delivery.deployer.target.template.params:
        aws:
          # AWS access key (optional if specified through default AWS chain)
          default_access_key: XXXXXXXXXX
          # AWS secret key (optional if specified through default AWS chain)
          default_secret_key: XXXXXXXXXXXXXXXXXXXX
      

    • In aws.cloudformation.access_key and aws.cloudformation.secret_key properties under studio.serverless.delivery.deployer.target.template.params, when specific CloudFormation credentials are needed:

      studio.serverless.delivery.deployer.target.template.params:
        aws:
          ...
          cloudformation:
            # AWS access key (optional if aws.accessKey is specified)
            access_key: XXXXXXXXXX
            # AWS secret key (optional if aws.secretKey is specified)
            secret_key: XXXXXXXXXXXXXXXXXXXX
      

  • We recommend that the AWS credentials configured belong to a user with just the following permission policy (all strings like $VAR are placeholders and need to be replaced):

    aws-serverless-deployer-policy.json
     1{
     2    "Version": "2012-10-17",
     3    "Statement": [
     4        {
     5            "Effect": "Allow",
     6            "Action": [
     7                "cloudformation:CreateStack",
     8                "cloudformation:DescribeStacks",
     9                "cloudformation:DeleteStack"
    10            ],
    11            "Resource": "arn:aws:cloudformation:$REGION:$ACCOUNT_ID:stack/$CLOUDFORMATION_NAMESPACE-*/*"
    12        },
    13        {
    14            "Effect": "Allow",
    15            "Action": [
    16                "cloudfront:CreateDistribution",
    17                "cloudfront:GetDistribution",
    18                "cloudfront:GetDistributionConfig",
    19                "cloudfront:UpdateDistribution",
    20                "cloudfront:DeleteDistribution",
    21                "cloudfront:CreateInvalidation",
    22                "cloudfront:TagResource",
    23                "cloudfront:UntagResource"
    24            ],
    25            "Resource": "arn:aws:cloudfront::$ACCOUNT_ID:distribution/*"
    26        },
    27        {
    28            "Effect": "Allow",
    29            "Action": [
    30                "cloudfront:CreateCloudFrontOriginAccessIdentity",
    31                "cloudfront:GetCloudFrontOriginAccessIdentityConfig",
    32                "cloudfront:GetCloudFrontOriginAccessIdentity",
    33                "cloudfront:DeleteCloudFrontOriginAccessIdentity"
    34            ],
    35            "Resource": "*"
    36        },
    37        {
    38            "Effect": "Allow",
    39            "Action": [
    40                "s3:CreateBucket",
    41                "s3:ListBucket",
    42                "s3:DeleteBucket",
    43                "s3:GetBucketLocation",
    44                "s3:GetBucketPolicy",
    45                "s3:PutBucketPolicy",
    46                "s3:DeleteBucketPolicy",
    47                "s3:PutBucketCORS",
    48                "s3:GetObject",
    49                "s3:PutObject",
    50                "s3:DeleteObject"
    51            ],
    52            "Resource": "arn:aws:s3:::$CLOUDFORMATION_NAMESPACE-*"
    53        }
    54    ]
    55}
    

  • By default, the CloudFront created by Deployer will have a *.cloudfront.net domain name. To have CloudFront use additional domain name(s) please specify the AWS ARN of the domain SSL certificate (cloudfrontCertificateArn) and the alternate domain name(s) (alternateCloudFrontDomainNames):

    studio.serverless.delivery.deployer.target.template.params:
      aws:
        cloudformation:
          ...
          # The SSL certificate ARN the CloudFront CDN should use (optional when target template is aws-cloudformed-s3)
          cloudfrontCertificateArn: arn:aws:acm:...
          # The alternate domains names (besides *.cloudfront.net) for the CloudFront CDN (optional when target template is aws-cloudformed-s3)
          alternateCloudFrontDomainNames: myawesomesite.com,www.myawesomesite.com
    

An example of serverless deployment configuration where there’s a single authoring instance and no specific domain name requirements is the following:

##########################################################
##                 Serverless Delivery                  ##
##########################################################
# Indicates if serverless delivery is enabled
studio.serverless.delivery.enabled: true
# The URL for the serverless delivery deployer create URL
studio.serverless.delivery.deployer.target.createUrl: ${studio.preview.createTargetUrl}
# The URL for the serverless delivery deployer delete URL
studio.serverless.delivery.deployer.target.deleteUrl: ${studio.preview.deleteTargetUrl}
# The template name for serverless deployer targets
studio.serverless.delivery.deployer.target.template: aws-cloudformed-s3
# Replace existing target configuration if one exists?
studio.serverless.delivery.deployer.target.replace: false
# The URL the deployer will use to clone/pull the site's published repo. When the deployer is in a separate node
# (because of clustering), this URL should be an SSH/HTTP URL to the load balancer in front of the Studios
studio.serverless.delivery.deployer.target.remoteRepoUrl: ${env:CRAFTER_DATA_DIR}/repos/sites/{siteName}/published
# The deployer's local path where it will store the clone of the published site. This property is not needed if
# the deployer is not the preview deployer, so you can leave an empty string ('') instead
studio.serverless.delivery.deployer.target.localRepoPath: ${env:CRAFTER_DATA_DIR}/repos/aws/{siteName}
# Parameters for the target template. Please check the deployer template documentation for the possible parameters.
# The following parameters will be sent automatically, and you don't need to specify them: env, site_name, replace,
# disable_deploy_cron, local_repo_path, repo_url, use_crafter_search
studio.serverless.delivery.deployer.target.template.params:
   # The delivery search endpoint (optional if authoring is using the same one, specified in the SEARCH_URL env variable)
   search_url: https://vpc-serverless-test-jpwyav2k43bb4xebdrzldjncbq.us-east-1.es.amazonaws.com
   aws:
     # AWS access key (optional if specified through default AWS chain)
     default_access_key: XXXXXXXXXX
     # AWS secret key (optional if specified through default AWS chain)
     default_secret_key: XXXXXXXXXXXXXXXXXXXX
     cloudformation:
       # Namespace to use for CloudFormation resources (required when target template is aws-cloudformed-s3)
       namespace: serverless-test
       # The domain name of the serverless delivery LB (required when target template is aws-cloudformed-s3)
       deliveryLBDomainName: serverless-test-lb-1780491458.us-east-1.elb.amazonaws.com

Step 4: Create the Site in the Authoring Environment

  1. Login to the Crafter Studio in the authoring environment from your browser.

  2. Click the Create Site button

  3. Choose the Editorial blueprint, enter the Site Id (e.g. editorial), and then review and create.

  4. Go to your AWS console in your browser and on the Services dropdown search for CloudFormation. You should then see the CloudFormation for the site you just created with the status CREATE_IN_PROGRESS. After several minutes, the status should change to CREATE_COMPLETE, which tells the Crafter Deployer that it is able to start uploading files to S3.

    Serverless Site - CloudFormation

  5. Wait at least 2 minutes for the Crafter Deployer to finish uploading the files and for the delivery Crafter Engine to warm up the new site in cache.

    deployer.log
     12019-12-20 20:48:58.780  INFO 18846 --- [deployment-3] llCloudFormationStackUsableLifecycleHook : CloudFormation stack 'serverless-test-site-editorial' is usable (status 'CREATE_COMPLETE')
     22019-12-20 20:48:58.781  INFO 18846 --- [deployment-3] org.craftercms.deployer.impl.TargetImpl  : Creating deployment pipeline for target 'editorial-serverless-delivery'
     32019-12-20 20:48:58.854  INFO 18846 --- [deployment-3] org.craftercms.deployer.impl.TargetImpl  : Checking if deployments need to be scheduled for target 'editorial-serverless-delivery'
     42019-12-20 20:48:58.855  INFO 18846 --- [deployment-3] org.craftercms.deployer.impl.TargetImpl  : Deployments for target 'editorial-serverless-delivery' scheduled with cron 0 * * * * *
     52019-12-20 20:49:00.001  INFO 18846 --- [deployment-8] org.craftercms.deployer.impl.TargetImpl  : ============================================================
     62019-12-20 20:49:00.001  INFO 18846 --- [deployment-8] org.craftercms.deployer.impl.TargetImpl  : Deployment for editorial-serverless-delivery started
     72019-12-20 20:49:00.001  INFO 18846 --- [deployment-8] org.craftercms.deployer.impl.TargetImpl  : ============================================================
     8...
     9...
    10...
    112019-12-20 20:49:15.882  INFO 18846 --- [deployment-8] org.craftercms.deployer.impl.TargetImpl  : ============================================================
    122019-12-20 20:49:15.882  INFO 18846 --- [deployment-8] org.craftercms.deployer.impl.TargetImpl  : Deployment for editorial-serverless-delivery finished in 15.878 secs
    132019-12-20 20:49:15.882  INFO 18846 --- [deployment-8] org.craftercms.deployer.impl.TargetImpl  : ============================================================
    

    engine.log
    1[INFO] 2019-12-20T20:50:00,061 [pool-3-thread-10] [] [context.SiteContextManager] | ==================================================
    2[INFO] 2019-12-20T20:50:00,061 [pool-3-thread-10] [] [context.SiteContextManager] | <Creating site context: editorial>
    3[INFO] 2019-12-20T20:50:00,061 [pool-3-thread-10] [] [context.SiteContextManager] | ==================================================
    4...
    5...
    6...
    7[INFO] 2019-12-20T20:50:04,393 [pool-3-thread-10] [] [context.SiteContextManager] | ==================================================
    8[INFO] 2019-12-20T20:50:04,393 [pool-3-thread-10] [] [context.SiteContextManager] | </Creating site context: editorial>
    9[INFO] 2019-12-20T20:50:04,393 [pool-3-thread-10] [] [context.SiteContextManager] | ==================================================
    

Step 5: Test the Delivery Site

Open a browser and go to https://DOMAIN_OF_YOUR_CLOUDFRONT. You should be able to see your Editorial site!

Serverless Site - Editorial Screenshot

Note

The following error appears in the deployer logs (CRAFTER_HOME/logs/deployer/crafter-deployer.out) when a site hasn’t been published:

2020-07-07 15:33:00.004 ERROR 22576 --- [deployment-9] l.processors.AbstractDeploymentProcessor : Processor 'gitDiffProcessor' for target 'ed-serverless-delivery' failed
org.craftercms.deployer.api.exceptions.DeployerException: Failed to open Git repository at /home/ubuntu/craftercms/crafter-authoring/data/repos/sites/ed/published;

Once the site has been published, the error above will go away.