Managing duplicate contacts

General merge information

Some of the different types of record information that might be merged include the following information:

• Identity
• Missing data in the retained record is populated when the data exists for the duplicate record.
• Related data (for example, Physical Address, Phone, Email, or Fax for a specific address) in the retained record is updated only when corresponding related data exists in the duplicate record, and the data fields have not been populated in the retained record prior to the merge.
• Contact ID field values in all related items (for example, Orders, Invoices, Event Registrations, and Group Memberships) for the duplicate record are replaced with the contact ID of the retained record.
• Where both contacts have an engagement category assignment, the retained contact is set to the higher of the two category assignments.
• Engagement scores from a discarded contact are retained if both contacts have engagement scores. This results in two separate engagement scores, neither of which accurately reflects the merged contact. To fix this, you must run the Recalculate Historical Scores process.

Note: If you run Recalculate Historical Scores, engagement scores will be recalculated for your entire database. This could affect performance.

• Financial
• All financial information, including order history, orders, billing parties, and shipping parties, are merged so that the retained record contains the combined history from both records.
• Gift history is recalculated after a merge so that the donor history reflects the combined giving from both records.

Note: Records that contain AutoPay data cannot be merged. You must unenroll the contact from AutoPay before you can merge the contact.

Note: Donor club records are neither created nor deleted, and can be duplicated if both records contain qualifying gift history.

• Activity History
• Any activity that was associated with the discarded contact record that was not retained is merged with the retained contact.
• If custom activities exist, they are updated after a merge.
• User credentials (individuals only)
  • When merged, any permissions that were attached to the Username of the discarded contact are given to the retained contact. The Username of the discarded contact is deleted, and that login will no longer work.
  • If the retained contact does not have a user account, but the discarded contact does have a user account, then the discarded contact's Username will become the retained contact's Username after a merge.

• CalculatedEngagementScore table
  • When both sets of scores exist, engagement scores are recalculated for the retained contact.
  • If an engagement score exists only for the retained contact, no action is taken.
  • If an engagement score exists only for the discarded contact, all CalculatedEngagementScore entries are updated to reflect the retained contact's ContactKey.
• EngagementCategoryContact table
• For data points where both contacts have a category assignment, the retained contact is set to the higher of the two.
• If a category assignment exists only for the discarded contact, that category assignment is copied to the retained contact.
• If a category assignment exists only for the retained contact, no action is taken. All EngagementCategoryContact entries for the discarded contact are then purged.

When merging contact records, you are provided with detailed information about the potential duplicates to help you determine whether the records belong to the same person or organization. When reviewing potential duplicates, the following items are important to consider:

• In the Other Memberships field, groups to which contacts belong are displayed. This includes Membership and Subscription groups as well as Chapters, Committees, Event registration rosters, and Relationships, and all event functions for which a contact is a member.
• In the Custom Data field:
  • For single instance data sources, you can expand and compare data line-by-line, and any missing data in the retained record is populated when the data exists for the duplicate record.

  • For multi-instance sources, you only see the count of how many instances exist for each contact, and the entire record for each instance from the duplicate record is preserved in the retained record after the merge.

To ensure important data is not overwritten or otherwise lost, merges are prevented in the following scenarios:

• Events — If two people are registered for the same event, and neither person is cancelled, then their records will not be merged. Their records can be merged if one of the individuals cancels their registration.
• Groups — If two contacts are active members in the same Membership or Subscription group, for example, Dues, Chapter, Section, or Subscription:
  • If subscription records have overlapping dates, then:
    • If the subscription requires payment for membership, these contacts will not be merged.
    • If no payment is required for membership, then these contacts will be merged, and the membership is extended to the furthest date.
  • If subscription records do not have overlapping dates, then these contacts will be merged and the membership is extended to the furthest date.
  • If the number of subscription copies specified does not match, then these contacts are prevented from being merged.
• Certification — If two contacts are enrolled in Certification programs, and both contacts are or have ever been enrolled in the same Certification program, then the contacts cannot be merged, regardless of status.
• AutoPay enrollments - For the duplicate record to be merged, no AutoPay enrollment for recurring donations or recurring memberships can exist. To check if an AutoPay enrollment is still active, go to the contact’s Giving and Preferences tabs. To cancel an enrollment, see Cancelling AutoPay enrollments.
• Contact types - Merges are not permitted for records with the following credentials:
  • Individuals cannot be merged with organizations.
  • Combo records (organization records that also contain first and / or last names) cannot be merged.
• User credentials - Merges are not permitted for records with the following credentials:
  • Staff users cannot be merged. To remove staff credentials, see Assigning security credentials and staff access.
  • Organizations associated with user accounts (usernames and passwords) cannot be merged. Remove the organization’s user credentials to complete the merge.
  • Organizations associated with invalid iMIS IDs cannot be merged.
• On behalf of – Merging is not permitted if the discarded organization is currently selected for On behalf of. Clear the selected On behalf of contact to complete the merge.

New content added to the out-of-the-box account pages is not automatically added to custom account pages. If your custom account pages do not include the Resolve duplicates action, you may need to update the Actions content item to include the out-of-the-box configuration.

To add the Resolve duplicates option, open the custom account page for edit and Configure the Actions (Big Button Panel) content item to include the following action:

• Link Text – Resolve duplicates
• Link Target – @/iCore/Tools/Duplicate_Merge
• URL Parameter – ID=@SelectedID
• Use On Behalf Of – Disabled

Alternatively, if you do not have an Actions button on your account pages, do the following to copy the out-of-the-box Actions content item:

1. Go to RiSE > Page Builder > Manage content.
2. Open to the out-of-the-box account page content records:
   • Individuals: @/iCore/Contacts/ContactLayouts/ Account_Page_Staff
   • Organizations: @/iCore/Contacts/OrganizationLayouts/Account_Staff
3. Find the Actions content item, then click Copy To.



4. From the Document Browser, navigate to the custom account page you want to move the content to.
5. Click OK, then close the content record.
6. Navigate to the individual or organization account page you moved the content to, then open it for edit.
7. Move the copied Actions content item to the desired location.
8. Click Save & Publish.

For more information, see Adding out-of-the-box content to custom account pages.

If you are experiencing a problem with existing members or users (who already have usernames or accounts) creating new accounts instead of signing in with their existing accounts, the following tips might help cut down on the number of duplicate accounts being created:

• Consider adding user instructions on your Event Showcase regarding signing in versus creating a new account. Add a Content Html content item just above or just below the existing Contact Sign In content item. Emphasize that existing members should sign in with their account or use the Forgot my password or Forgot my username links, rather than creating a new account. You can even provide readily-accessible contact information or links so that users can receive assistance from staff if they are still having trouble signing in with their existing account.

Note: If you are using the default Event page, you will need to make a copy of it in order to add the Content Html content item, and then edit your EventDetail shortcut to point to your new copy of the page.

• You could add notes inside your event description area. For example, add a note explaining that users must sign in first, and to use their existing account.
• You could configure the existing Contact Sign In content item on that page, and change the text for the Create account link text, for example, you could enter Create a new account only if you do not have an existing account.

Note: If you are using the default Event page, you will need to make a copy of it in order to add the Content Html content item, and then edit your EventDetail shortcut to point to your new copy of the page.

• You could add another warning or set of instructions at the top of the Create Account page with the Content Html content item. Emphasize that it is very important that users log in with their existing account, rather than creating a new account. Using the Content Html content item, you can put your note in bold, use a color, and so forth.

Note: If you are using the default Event page, you will need to make a copy of it in order to add the Content Html content item, and then edit your EventDetail shortcut to point to your new copy of the page.

Adding one or more of these things to your site should help to mitigate the risk of duplicate account creation.

To merge contacts directly from a profile page, do the following:

1. Go to the account page of an individual contact.
2. Under the profile picture, click Actions, then select Resolve duplicates.



The Resolve Duplicates window is displayed. The Master contact column displays pre-populated information for the contact.

3. Click Select a contact to merge to open a Merge contact window that allows you to select a record which you suspect might be a duplicate.

The queries you can access through this window are stored in the $/ContactManagement/DefaultSystem/Queries/Contacts folder. These queries can be modified by an Administrator if necessary. However, these queries are the same queries used by the On behalf of tool. Any modifications made to the queries for a merge will also be reflected in On behalf of.

4. (optional) If you are licensed for Contact Merge - Plus, select a contact from the Potential duplicates drop-down. The drop-down will not appear if you are not licensed, or if the Duplicate Search task did not find any potential duplicates:
   • From the drop-down, you can select a new contact to load into the second column.
   • The contacts in the drop-down will always be potential duplicates of the contact in the first column.



Note: To customize the criteria for identifying potential duplicate organizations, see Searching for duplicate contacts.

5. Search for someone with the same name. If there is a duplicate contact, select that contact. The column for the duplicate contact will be populated with the duplicate contact’s information.
6. Select the Keep this contact radio button for the contact you want to retain. The information from the contact you are not keeping will be merged. If there is an empty field in the kept record, information from the other contact is used to fill the field.

Note: After a contact’s record has been merged, a user might see errors when attempting to log in with the merged user credentials. This problem is due to temporary caching. These errors will pass after a short timeframe, typically between five and ten minutes.

7. After you have reviewed the contacts' detailed information, click Merge. The ID number of the contact that was deleted is recorded. The change log also records data changes for any field for which changes are normally tracked.
8. While reviewing the contact information, if you decide that the contacts should not be merged and are not duplicates of each other, click Not Duplicates.
9. A confirmation message displays once the merge is initiated. Select Merge logs to review the status of the merge and the properties that changed. For more information, see Reviewing the duplicate merge logs.

To merge duplicate organizations, do the following:

1. Navigate to the profile page of an organization.
2. From the Actions drop-down, select Resolve duplicates.

Note: If you are using a custom account page and do not see the Resolve duplicates option, you may need to update the custom account page to include the latest out-of-the-box content. For more information, see Adding the resolve duplicates action to custom account pages.

3. The organization details appear in the Master organization column. To locate the duplicate, Select an organization to merge.



4. (optional) If you are licensed for Contact Merge - Plus, select an organization from the Potential duplicates drop-down. The drop-down will not appear if you are not licensed, or if the Duplicate Search task did not find any potential duplicates:
   • From the drop-down, you can select a new organization to load into the second column.
   • The organizations in the drop-down will always be potential duplicates of the contact in the first column.
Click to view image



Note: To customize the criteria for identifying potential duplicate organizations, see Searching for duplicate contacts.

5. Search for the duplicate organization, then select its Company name.

Note: The search query available from the Merge organizations window is located at $/ContactManagement/DefaultSystem/Queries/Contacts/Companies Only.

6. Information for the selected duplicate populates the right-most column. Review the detailed information about the pair to determine which organization to retain in the merge.
7. Enable Keep this organization for the organization you want to retain.



8. Click Merge. The information from the organization you are not keeping will be merged. If there is an empty field in the kept record, information from the discarded organization is used to fill the field.



9. A confirmation message appears once the merge is initiated. Select Merge log to review the results. For more information, see Reviewing the duplicate merge logs.

Note: This is a licensed feature. For more information, contact your AiSP or ASI Technical Support.

iMIS Contact Merge - Plus locates all potential duplicate records and organizes them into duplicate pairs. You can manage these pairs to determine which records to retain, which records to merge into the retained records, and which records to mark as Not Duplicates.

System Administrators, and users with a Customers: 8 module authorization level can manage all potential duplicate contacts from a single interface. System Administrators can also enable a task that searches the database for potential duplicates on a specific date and time and customize the criteria for identifying duplicates.

All records identified as potential duplicates are paired together and displayed in the Manage Duplicates tab. An alert appears on the profile pages of the potential duplicate contacts. This alert is only visible to staff users. To learn more about automated alerts, please see Creating and using alerts.

Searching for duplicate contacts

From the Manage duplicates settings, schedule or run a task that searches for potential duplicates. The task identifies contacts as potential duplicates based on the criteria defined in the Potential duplicate formulas. Adjust the criteria for potential duplicates by selecting from the available formulas.

Note: Names differing only in accents or special characters are not identified as potential duplicates during searches.

Do the following to search for duplicate contacts:

1. Go to Community > Manage duplicates.
2. Click the Settings tab.
3. (optional) The task identifies individuals and organizations as potential duplicates based on the criteria in the Potential duplicate formulas. To update the criteria for potential duplicates, choose from one of the available formulas:

For individual contacts:

• None – Does not search for duplicate individuals.
• Standard – Matches contacts if they have the same first four characters of their last name, first character of their first name, and first five digits of their postal code.
• Email – Matches solely on the email address.
• Mobile Phone – Matches based on the mobile phone number (Name.MOBILE_PHONE).
• Strict – Matches contacts if they have the same first five characters of their last name, first three characters of their first name, first five digits of their postal code, and portions of their email address.

For organizations:

• None – Does not search for duplicate organizations.
• Standard – Matches based on the first four characters of the company name and the first five digits of the postal code.
• Email – Matches solely on the email address.
• Strict – Matches contacts if they have the same first five characters of their last name, first three characters of their first name, first five digits of their postal code, and portions of their email address.
• Name and Domain – Matches based on the first five characters of the company name and the domain portion of the email address.
• Company Administrator – Identifies duplicates by matching companies that have the same Company Administrator.
• Company Name – Matches based on the full company name.

Note: All organization formulas except Company Name ignore prefixes listed in the Company sort excluded prefixes setting (Settings > Organization) when searching for potential duplicates.

4. Click Save to update the formulas.



5. (optional) Click Find Duplicates Now to immediately search for potential duplicate contacts. Depending on how large your database is, this could take several minutes. You can navigate around your site while the task runs.
6. To automatically search for potential duplicates at a later time, select Run task on a scheduled basis.
7.  From the Frequency drop-down, select how often you want your database to be searched for potential duplicate contacts.

Note: Dates and times are based on the system Time zone setting.

One time

Select a Start date and time.

Daily

• Schedule - Select how often and at what time you want the task to run.
• Start - Select the date and time you want to begin running daily searches.
• End - Select the date you would like the daily searches to stop.

Weekly

• Schedule - Select how often and at what time you want the task to run.
• Day of week - Select the days of the week you want the task to run.
• Start - Select the date and time you want to begin running weekly searches.
• End - Select the date and time you would like the weekly searches to stop.

Monthly

• Schedule - Select how often you want the task to run.
• Day of month - Select the days of the month you want the task to run.
• Start - Select the date and time you want the task to start running.
• End - Select the date and time you want the task to stop running.

Any time you create and save a new scheduled task, the previous task is overridden and will no longer run.

8. Click Save.

Note: Depending on how large of a database you have, the task could take several minutes to run. It is suggested you run the scheduled task at a time your system is not being heavily used. For example, on weekends or overnight.

9. Once the task is completed, all records identified as potential duplicates are paired together and displayed in the Manage Duplicates tab.

Managing duplicates

From the Manage duplicates tab, review contacts who have been identified as possible duplicates, and resolve by either merging the contacts or indicating that they are not duplicates.

1. Go to Community > Manage duplicates.
2. From the Manage Duplicates tab, search for potential duplicates or scroll through the list.

Note: To ensure a quicker response, when the results return more than 100 potential duplicate contacts, not all of the pairs will display. In that case, if you search for a specific potential duplicate contact by ID or full name, it will always be found.

The following columns of information are available for each potential duplicate pair:

• Contact 1 - The potential duplicate of Contact 2. Select the contacts name to view more profile information.
• Contact 2 - The potential duplicate of Contact 1. Select the contacts name to view more profile information.
• Date Found - The date the potential duplicate was found.



3. Select the arrow to expand the potential duplicate pair.



Both potential duplicate contacts in the pair contain information specific to that account:

• Name
• Primary Email
• Primary Organization
• Status
• Date Created
• Last Updated
• Username

The contact information displayed is currently not customizable.

4. If the two contacts are determined not to be duplicates, select Not Duplicates. This permanently marks them as not duplicates of each other.



5. Select Resolve to open the Merge Contacts window. From here you can view detailed information about the pair to help you determine if you should merge the contacts, or mark them as Not Duplicates.



6. After reviewing the contacts' detailed information, if you decide the contacts are not duplicates of each other, click Not Duplicates.
7. If the contacts are duplicates of each other, select a contact to keep, then click Merge. The information from the contact you are not keeping will be merged. If there is an empty field in the kept record, information from the discarded contact is used to fill the field.



Note: Once the contacts have been merged, they will no longer be shown in the table.

8. A confirmation message displays once the merge is initiated. Select Merge logs to review the status of the merge and the properties that changed. For more information, see Reviewing the duplicate merge logs.

If you would like to export the Manage duplicates table, select the desired export icon. Select Expand All to expand all of the nodes in each potential duplicate pair.

Clearing potential duplicates

System administrators have the option to Clear Potential Duplicates for either individuals or organizations, removing all potential duplicates from the Manage duplicates tab and profile pages of the contacts and eliminating clutter from outdated searches. This is useful to do before changing the formula used to identify potential duplicates.

To clear potential duplicates, do the following:

1. Go to Community > Manage duplicates.
2. Click the Settings tab.
3. From the Clear Potential Duplicates drop-down, choose to clear all potential duplicates for Individuals or Organizations.



4. Click OK from the confirmation window. The potential duplicates for all individuals or organizations are removed from the Manage duplicates tab and the profile pages of the contacts.

Note: Clearing potential duplicates does not mark the contacts as Not duplicates. Cleared contacts may reappear if they meet the criteria for new searches.

The Duplicate merge logs detail the progress of recent contact merges and the properties affected by the merges. To access the logs, go to Community > Duplicate merge logs.

Note: Successful merge logs are retained for 30 days, and failed merge logs are retained for 90 days. This retention policy is not configurable.

The Duplicate merge logs display the following information for each recent merge request:

• Task Run ID – Select the Task Run Id associated with a merge to view the Duplicate merge log detail screen. This screen provides a breakdown of the tasks performed during the merge and the change history of properties affected by the merge.
• Created By – The user who initiated the merge.
• Start Time – The date and time the user initiated the merge.
• Status – The success of the merge. The status can be one of the following:
  • In progress - The task has been initiated and is currently running. The task has not yet ended.
  • Failed - The task was initiated and ended, but the task was not completed successfully.
  • Succeeded - The task was initiated and ended, and the task completed successfully.
• Information – The results of the merge, including the merge status and the contact IDs merged.

Select the Task Run Id to review the following merge details:

• Duplicate merge log detail – Displays the following columns of information for each task performed during the merge:
  • Task Run Detail ID – The ID of the task.
  • Created On – The date and time the task was initiated.
  • Status – The success of the task. The status can be one of the following:
    • In progress - The task has been initiated and is currently running. The task has not yet ended.
    • Failed - The task was initiated and ended, but the task was not completed successfully.
    • Succeeded - The task was initiated and ended, and the task completed successfully.
  • Information – The activities involved in the task.
  • ID – The iMIS ID affected by the task.
• Change logs – The change history of properties affected by the merge. The Change logs display the following columns of information for each property that changed:
  • Date – The date the property changed.
  • Action – The type of change (Add, Delete, Update).
  • User – The contact who changed the property.
  • Property – The property that has changed. For more information, see Properties logged in the Change history.
  • Previous Value – The original value of the property.
  • New Value – The current value of the property.

  Example: Contact A does not have an address defined, while Contact B does have an address defined. When Contact A is merged with Contact B, the address of Contact B is retained, updating the Name.FULL_ADDRESS property. The change log records Contact B’s address as the New Value for Name.FULL_ADDRESS.

The data from the Change logs is also available in the Change history of the merged contact.

Although there is no report that details all duplicate contacts, you can export the duplicate contact list that iMIS generates at Community > Manage duplicates.

Do the following:

1. Go to Community > Manage duplicates.
2. Click the Export button, then choose the export type.



The list downloads locally in the selected file format.