Error Troubleshooting – SearchStax Backup Failure

user

2 years ago

The Sitecore Experience Platform provides different Search provider options to connect with the Sitecore platform and one of the options is SearchStax.

From Sitecore :
SearchStax offers an easy, reliable, performant, and scalable Solr service for Sitecore to automate, manage and scale Solr infrastructure. With Sitecore 10.x, Sitecore has deprecated Azure Search and recommends Solr as the search engine to prevent future upgrade obstacles.

From SearchStax :
• Spend Less Time Managing And Supporting Solr Infrastructure For Your Sitecore Cms
• Implement Faster With Our Sitecore-solr Plugin
• Focus On Building Better Experiences
• Let Searchstax Focus On Uptime, Patching & More
• Improve Security And Compliance – Hipaa & Gdpr

SearchStax also provides the following Disaster Recovery (DR) Options for Solr Deployments:
• Hot Disaster Recovery
• Warm Disaster Recovery
• Cold Disaster Recovery

Please find below an example of a Fully Scaled Cross Region Failover and Disaster Recovery system:

The Disaster Recovery option has a service level agreement (SLA) to restore your site and for this backup is required for your website’s Solr-managed instance, sometimes if you are writing content more frequently to the Solr then it will tend towards the backup failure because we are trying to write very frequently in the Solr document for indexing.

For example, if you are having a website which is heavy in terms of Sitecore xConnect Search API utilization point of view, then more frequent commits are required for Sitecore xConnect related search indexes, and this can be one of the causes that SearchStax Solr can’t be able to complete durable backups (Why do Backups Fail? – SearchStax Docs) of the xDB index.

To manage the commit (or “hard commit“) rate to write on the Solr indexes, we have to modify the solrconfig.xml to lower commit rates, and need to adjust the following Solr configurations as per our need:
• autoCommit
• autoSoftCommit
• MaximumCommitMilliseconds

You can check more details at Optimize Solr for Sitecore | SearchStax Blog.

In this article, I will share the information related to the update of the MaximumCommitMilliseconds setting on a Docker based solution to resolve the SearchStax backup issue.

The Sitecore Docker Example Visual Studio Solution has the following structure:

The MaximumCommitMilliseconds setting is present at sc.Xdb.Collection.IndexWriter.SOLR.xml file of one of the xConnect window services.

The updated setting of the MaximumCommitMilliseconds attribute as per suggestions present on Known Issues – Performance optimizations when using Solr (sitecore.com) should be:

<?xml version="1.0" encoding="utf-8" ?>
<Settings>
  <Sitecore>
    <XConnect>
      <SearchIndexer>
        <Services>
          <Solr.SolrWriterSettings>
            <Options>
              <MaximumCommitMilliseconds>600000</MaximumCommitMilliseconds>
            </Options>
            </Solr.SolrWriterSettings>
        </Services>
      </SearchIndexer>
    </XConnect>
  </Sitecore>
</Settings>

In Sitecore, we generally use the Sitecore Patch Configuration Files instead of directly changing Sitecore OOTB configuration files.

If you have to patch the config of xConnect, Cortex or Marketing automation windows services then need to be aware of the following information:

• The patch files applied to Sitecore in alphabetical order
• The patch file name should be the same as the original file name, e.g., sc.Xdb.Collection.IndexWriter.SOLR.xml
• Either you will modify the original file and update the details, or create the new folder and place the patch file, but the folder should come after the original folder, e.g., zz.SearchIndexer folder

We will add sc.Xdb.Collection.IndexWriter.SOLR.xml at \src\DockerExamples.XConnect\App_Data\Config\Sitecore\zz.SearchIndexer in Visual Studio Solution.

The following table shows Sitecore images for both XP0 (Single) and XP1 (Scaled) topologies on which sc.Xdb.Collection.IndexWriter.SOLR.xml file needs to be deployed:

Sitecore image / role	XP0	XP1	sc.Xdb.Collection.IndexWriter.SOLR.xml
xconnect	x		root\App_Data\Config\sitecore\zz.SearchIndexer
xdbsearchworker	x	x	root/App_Data/Config/sitecore/zz.SearchIndexer
xdbautomationworker
cortexprocessingworker
prc
xdbcollection		x	root\App_Data\Config\sitecore\zz.SearchIndexer
xdbsearch		x	root\App_Data\Config\sitecore\zz.SearchIndexer
xdbautomation

Note the root path varies for each image: C:\service for the worker roles, C:\inetpub\wwwroot for all others.

The Sitecore Docker-based Visual Studio Solution will deploy the patch config sc.Xdb.Collection.IndexWriter.SOLR.xml to all the roles wherever you are deploying xConnect artifacts from the solution:

# Copy solution XConnect files
COPY --from=solution \artifacts\xconnect\ .\

xConnect Artifacts Deployment to different Sitecore roles

From the above, we have to explicitly copy sc.Xdb.Collection.IndexWriter.SOLR.xml file to the service for xDBSearchWorker role and there may be a chance depending upon your Docker deployment (Dockerfile) you may need to delete this file from some roles where you are deploying xConnect artifacts.

Please find below details about how to copy/remove deployment artifacts (files) from Docker Containers:

Copy file to Docker Container

# xDBSerchWorker Dockerfile

# escape=`

ARG BASE_IMAGE
ARG SOLUTION_IMAGE

FROM ${SOLUTION_IMAGE} as solution
FROM ${BASE_IMAGE}

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

WORKDIR C:\service

# Copy solution sc.Xdb.Collection.IndexWriter.SOLR.xml
COPY --from=solution \artifacts\xconnect\App_Data\Config\Sitecore\zz.SearchIndexer\sc.Xdb.Collection.IndexWriter.SOLR.xml .\App_Data\Config\Sitecore\zz.SearchIndexer\sc.Xdb.Collection.IndexWriter.SOLR.xml

Remove file from Docker Container

Suppose you have deployed the xConnect artifacts to the xDBRefData role container and sc.Xdb.Collection.IndexWriter.SOLR.xml file is not required on this Sitecore role then you have to remove/delete files from the docker container

# xDBRefData Dockerfile

# escape=`

ARG BASE_IMAGE
ARG SOLUTION_IMAGE

FROM ${SOLUTION_IMAGE} as solution
FROM ${BASE_IMAGE}

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

WORKDIR C:\inetpub\wwwroot

# Copy solution XConnect 
COPY --from=solution /artifacts/xconnect/ ./

# Remove file from docker cotainer
RUN RM C:\inetpub\wwwroot\App_Data\Config\Sitecore\zz.SearchIndexer\sc.Xdb.Collection.IndexWriter.SOLR.xml

I hope this helps to resolve the SearchStax backup failure issue and performance of your Sitecore instance if you are using Solr as the search provider.

Credit/References

Amit is an IT Solution Architect with Assurex. Reach out to Amit on

Did you find this useful? If so, please share with your connections.