Sitecore XP collects data on contact visits and interactions as standard, for both anonymous and identified contacts. This data is saved to the Sitecore xDB and the database size grows over time.
If the xDB grows large with data that is no longer providing business value, this has many downstream impacts on the performance and cost of the system, including:
- Slowdown of TTFB on pages due to slower contact lookup
- Increased cost due to capacity and processing requirements on the SQL server and xDB Processing server
- Sitecore upgrade impediment due to large amount of data to be migrated
xDB Data Intelligent Maintenance application
To address this issue, Dataweavers has developed the xDB Data Intelligent Maintenance application (set up as an Azure Function App).
This application can monitor xDB size and data issues such as this potential issue. It also cleans up old data that is considered not to be providing business value, meaning contacts that have not made any interactions with the Sitecore platform for an agreed period of time (either anonymous or known).
The application operates outside of the codebase and at the Azure platform level. It will only run outside of business hours, and there will be no public-facing impact on your business.
Once the initial cleanup is completed (which may take several nights depending on the amount of data to be removed) it will run automatically every night to monitor and cleanup the xDB.
GDPR compliance
For our customers that require GDPR compliance, this system ensures that personal data is kept in a form that permits the identification of data subjects for no longer than is necessary for the purposes for which the personal data is processed.
How to take advantage of this solution
We recommend our customers take advantage of this solution, provided business requirements are met.
Please submit your request to support@dataweavers.com advising us of the answers to these questions:
- For what period of time do details of an anonymous contact's last interaction provide business value?
- For what period of time do details of a known contact's last interaction provide business value?
As a general guide, we recommend the following as a good balance between data retention vs. size of xDB:
- Anonymous contacts - 12 months
- Known contacts - 24 months
We will then configure these times in the application, and once contacts pass these thresholds they will be permanently deleted from the xDB.
Feel free to get in touch if you'd like to discuss the solution in further detail, we're here to help.