Understanding Data Integrity in Apricot
Overview
Data Integrity is a Que-powered Apricot skill that scans your forms for likely duplicate records. Instead of hunting for duplicates manually before each reporting cycle, the tool runs every night and surfaces flagged pairs in a dedicated view, grouped by form and ready for administrator review.
Data Integrity is part of Que Intelligence. It appears in Que Manager as the Data Integrity Review skill.
Here’s how it works. You choose which forms to scan. Each night, Que compares the records on those forms and flags pairs that are likely duplicates. Each flagged pair has a confidence level (High, Medium, or Low) and a short Que rationale that explains why the records were flagged. You decide what happens next: merge the pair, or mark it as not duplicates. Que never changes your records on its own.
To get started, in the Apricot Administrator tab, open Record Manager from the left navigation menu, then select Data Integrity. In the Duplicate Detection table, select the forms you want scanned. The first scan runs overnight, and results are ready the next morning. After that, results update every night with no extra steps.
Once you have results, open a form to see its flagged pairs in three tabs: For Review, Not Duplicates, and Merged. If you merge by mistake, you can undo a merge for up to 30 days.
Review the articles below to set up detection and resolve duplicates.
What can Data Integrity do for you?
Tomas is the Apricot Administrator at a workforce development nonprofit. Every quarter, he spends days checking for duplicate participant records before his team runs funder reports. This quarter, he enrolls his Participant Profile form the evening before he wants results.
The next morning, he opens Data Integrity and finds flagged pairs ranked by confidence. He reads the Que rationale for each, compares the values, merges the clear duplicates, and marks the rest as not duplicates. By the time his reporting cycle starts, the cleanup is done. And since the tool runs every night, new duplicates show up as they appear.
Key Concepts
| Term | What it means |
| Eligible form | Any form with at least one field where Duplicate Check is enabled in Form Designer. |
| Duplicate detection | The Que skill that scans enrolled forms, identifies likely duplicate record pairs, and ranks them by confidence level for administrator review. |
| Nightly scan | The tool runs once per night across all enrolled forms. New results are available each morning. The first scan for a newly enrolled form builds embeddings for all active records. |
| Enrollment threshold | Forms with up to 50,000 active records are eligible for enrollment. |
| Embeddings | Vector representations of record data built during the first scan. They power the similarity comparisons behind duplicate detection and are updated incrementally with each nightly run. |
Before You Begin
-
Confirm which forms have at least one field with Duplicate Check enabled. Only those forms appear as eligible in the Duplicate Detection table.
-
Identify which eligible forms to prioritize first, especially those with known manual cleanup burden ahead of reporting cycles.
-
Plan to enable detection the evening before you want results. The first scan runs overnight.
Safe to explore: Enrolling a form doesn’t modify records or expose results to standard users. You control enrollment and can unenroll at any time.
Feature Availability
-
Data Integrity is part of Que Intelligence.
-
It’s used by Administrators from the Apricot Administrator tab.
-
Forms above 50,000 active records aren’t eligible during the current pilot.
-
Duplicate detection is the first Data Integrity skill. See the FAQs for what we’re planning next.
What do you need help with?
How do I find duplicate records in Apricot? | Where is Data Integrity in Apricot? | What is Data Integrity in Apricot Que? | Who can use Data Integrity in Apricot? | Is Data Integrity available to all Apricot users? | Does Data Integrity change my records automatically?