Sitecore XP collects data on contact visits and interactions as standard, for both anonymous and identified contacts. This data is saved to the Sitecore xDB and the database size grows over time.
If the xDB grows large with data that is no longer providing business value, this has many downstream impacts on the performance and cost of the system, including:
- Slowdown of TTFB on pages due to slower contact lookup
- Increased cost due to capacity and processing requirements on the SQL server and xDB Processing server
- Sitecore upgrade impediment due to large amount of data to be migrated
Service Intelligence - xDB Maintenance
To address this issue, Dataweavers has developed the MaintainXdb application within our Service Intelligence function app.
This application can monitor xDB size and data issues such as that described in Sitecore's guide "Troubleshooting issues caused by contacts with an excessive number of interactions in SQL collection database". It also cleans up old data that is considered not to be providing business value, meaning contacts that have not made any interactions with the Sitecore platform for an agreed period of time (either anonymous or known).
The application operates outside of the codebase and at the Azure platform level. It will only run outside of business hours, and there will be no public-facing impact on your business.
Once the initial cleanup is completed (which may take several nights depending on the amount of data to be removed) it will run automatically every night to monitor and cleanup the xDB.
How to take advantage of this solution
We recommend our customers take advantage of this solution, provided business requirements are met.
Please submit your request to support@dataweavers.com advising us of the answers to these questions:
- For what period of time do details of an anonymous contact's last interaction provide business value?
- For what period of time do details of a known contact's last interaction provide business value?
As a general guide, we recommend the following as a good balance between data retention vs. size of xDB:
- Anonymous contacts - 12 months
- Known contacts - 24 months
We will then configure these times in the application, and once contacts pass these thresholds they will be permanently deleted from the xDB.
Feel free to get in touch if you'd like to discuss the solution in further detail, we're here to help.