Reindexing Content Without Disrupting Service in Production¶
In some scenarios it’s not possible/appropriate to delete a live index and wait for the index to rebuild in production. Perhaps the index is driving dynamic features on the site that will break while the index is empty or being rebuilt. In these scenarios you need a process for building the index off line and swapping it in.
Step 1: Prepare a re-index core¶
The first step is to prepare an additional empty core on Solr where you can index the content:
- Go to
SOLR_PORTin Authoring is normally 8694, while in Delivery it’s 8695).
- Click on
Core Adminon the left menu.
- Click on
Add Core. A popup will appear with the core properties you need to fill. Name the new core however you want, making sure it’s not the same name as the current core (e.g.
instanceDirshould be the path to the
crafter_configsconfigset in Solr , which should be under
dataDirshould be the path of the core’s data directory under Crafter’s
schemawith their default values, and click on
Step 2: Content freeze¶
Once you are about to start a re-index you need to freeze your authoring/editing activity. If content is being updated in the live environment while you are rebuilding your indexes, you may miss updates. Ask the authors not to publish during your re-index process.
Step 2: Set up a new temporary target¶
The next step is to create a temporary deployment target that is basically a copy of the production target, but with a different ID. The easiest way to do this is to:
- Go to the
- Copy and paste the target’s YAML to somewhere temporary outside the
targetsfolder (to avoid the Deployer from picking the new target while you’re modifying it).
- Replace the original site name from the YAML file name with the name of the Solr core you just created (e.g.
- Change the
siteNameproperty value inside the YAML to the name of the Solr core (e.g.
- Copy the the YAML file back to the
Step 4: Re-index¶
On a live environment, the Deployer will execute the deployment of a target on schedule every minute by default, so after creating the new temporary target
the Deployer should pick it up automatically and start re-indexing. If the Deployer is not working on a schedule, you can follow the process in
Reindexing Content for Search and Queries, starting in
Step 2: Invoke the reprocessing and using the
siteName (or Solr core name) you set in the temporary target
Step 5: Wait¶
You will see indexing activity in the deployment log located in
INSTALL_DIRECTORY/logs/deployer/crafter-deployer.out. Indexing activity time is dependent
on the amount of content which must be re-processed. When the deployment/indexing finishes you should see something like the following in the log:
2017-07-25 16:52:03.762 INFO 21896 --- [pool-2-thread-1] org.craftercms.deployer.impl.TargetImpl : ------------------------------------------------------------ 2017-07-25 16:52:03.763 INFO 21896 --- [pool-2-thread-1] org.craftercms.deployer.impl.TargetImpl : Deployment for editorial-tmp-prod finished in 2.359 secs 2017-07-25 16:52:03.763 INFO 21896 --- [pool-2-thread-1] org.craftercms.deployer.impl.TargetImpl : ------------------------------------------------------------
Step 6: Swap indexes¶
Now that indexing is complete you need to load the re-indexed content. Follow these steps:
- In the Solr console (from Step 1), under the
Core Admin, click
Swap Coresto swap from the production core to the temporary core.
- Backup the original core folder under
CRAFTER/data/indexes(should have the same name as the site, e.g.
- Consider creating a copy of the re-indexed core with the original name and swapping again to preserve file/folder names:
- Go to the
CRAFTER/data/indexesand delete the original core folder.
- Rename the swapped core folder (
editorial-tmp) to the original core folder name (
- Swap the cores again.
- Go to the
- Unload the temporary core.
Step 7: Unfreeze Content¶
Now that you are certain everything is working as it should, notify your authors that they may start editing and publishing activity.
Step 8: Clean up¶
Now that your process is complete you can clean up some of the artifacts created by the process.
- The re-index core if swapped out
- The temporary target YAML file