Data transfer utilities
The Import Utility can be used by customers who are new to iMIS. By making use of templates and SQL scripts to make their existing data meet the mapping requirements of the iMIS database tables, the new customers can use the Import Utility to load their existing data into the iMIS database.
Existing customers can also use the Import Utility to import data from a third-party source that does not match iMIS database formats.
iMIS customers can use the Export Utility to create customized layouts of exported information that is easier for third parties to consume and use.
This document provides an example designed to walk users through the process of importing records into the Name table in the iMIS database.
There are several steps in the record import process:
- Create an import template.
- Load an import template to the Import Utility.
- Import a records file.
This document will also identify NRDS-specific steps, which are performed only when conducting NRDS-specific imports. NRDS is an acronym for National Realtors Database System. NRDS was designed as a single database residing on the Internet to store the member and office records for the Realtor organization.
A template file is a comma-delimited text file that defines the format of the data imported into the Name table in the iMIS database. The template file acts as an interpreter, defining for the database the data that is imported, and enables the iMIS database to consume the data accurately.
The import template is a guide only and must be created based on the data being imported.
Create the template file in a flat editor. The rows in the template file correspond to the columns in your .csv (comma-separated values) file that contains the data you are importing.
Note: The template file and the corresponding .csv file must be in agreement on the order of the columns to ensure the data is imported accurately.
There will be six items that will be imported for each record:
- First name
- Last name
- Company
- Home phone
- Work phone
Do the following to import a file with five records:
- Create a template file. The template file for this import will look like the following:
- Give the template file a name, for example, template.txt. The corresponding rows in the .csv file must match the order of the items in the template file.
- Create a .csv file to import. Give it a name, for example, Contacts.csv. Include the following information in the Contacts.csv file:
Name.FIRST_NAME,0,0,C
Name.LAST_NAME,0,0,C
Name.COMPANY,0,0,C
Name.EMAIL,0,0,
Name.HOME_PHONE,0,0,
Name.WORK_PHONE,0,0,
Note: In the template file, the values 0,0,C correspond to the Position, Length, and Processing Code in the Import Utility. These are explained later in this document.
Each row in the template file represents an entry in the header row of a .csv file, and corresponds to a column name in the iMIS database. For example, data entered through the Import Utility as Name.LAST_NAME,0,0,C
and Name.FIRST_NAME,0,0,C
is placed under the Name (Last, First) column in the iMIS database.
ALAN,ADAMS,ADANAC ELECTRIC,AADAMS@YAHOO.COM,512-334-7834,512-222-5389
MICHAEL,BAKER,ADANAC ELECTRIC,MICHAEL99@GMAIL.COM,512-672-8302,512-332-7378
JENNIFER,CARSON,ADANAC ELECTRIC,CARS212@HOTMAIL.COM,512-373-2383,512-564-9765 GENE,DARRENS,ADANAC ELECTRIC,GD_555@DARRENS.COM,512-254-9492,512-759-8545
LISA,EBERSOL,ADANAC ELECTRIC,EBERS73@GMAIL.COM,512-998-7259,512-213-2596
Do the following to load import template files in the Import Utility:
- In the Advanced Accounting Console, go to Utilities > Data Transfer Utilities > Import Utility to open the Import Utility window.
- Click New.
- Enter the Import Name (description of the import task). For this example, use CONTACT.
- Select an Import Type:
- Select (C)omma for comma-delimited (.csv) files.
- (NRDS only) Select (F)ixed Field when working with NRDS imports.
- Select Name for the File to Modify.
- (NRDS only) Select the NRDS Import (Skip Header Records) checkbox when performing a NRDS import. This instructs the import utility to skip the header records.
- (NRDS only) Select the Member Import or the Office Import checkbox to specify whether you are importing the NRDS Member file or the NRDS Office file.
- Click Save.
- Click Edit.
- Click Load and select the template.txt file.
- Edit the template as needed, if necessary:
- In the template area, click Add to append a record row to the end of the template, or click Insert to add a row within the template. You can highlight a record row and click Delete to remove the record row.
- Click Ctrl+Z to open an edit window.
- Specify the fields to be edited:
- Enter the Field Name in the conversion table.
- (NRDS only) Enter the record Position from the NRDS file for that field.
- (NRDS only) Enter the Length of the field.
- Enter the Processing Code option, if used. This code identifies which record to update. The Processing Codes table on the window lists the options and their descriptions.
If the field is a Name_Address field, for example, ADDRESS_1, CITY, STATE_PROVINCE, and so forth, you will need to specify which address purpose to use. The #1, #2, #3 processing codes specify the first, second or third address purpose. When importing address information, you need only specify the ADDRESS_1, CITY, STATE_PROVINCE, COUNTRY, and ZIP fields in Name_Address. iMIS will build the FULL_ADDRESS field and populate the corresponding fields in the Name table. The same is true for Name information. If you populate the LAST_NAME and FIRST_NAME, iMIS will automatically build the FULL_NAME field.
(NRDS only) The following table defines some NRDS-related processing codes used in the import templates:
- #1 – Updates the member’s current main address
- #2 – Updates the member’s second address
- #3 – Updates the member’s third address
- H = 1 – Main Address in the Name Maintenance window
- M = 2 – Second Address in the Address window
- O = 3 – Third Address in the Address window
- Click Save to save the changes to the template.
Code |
Definition |
---|---|
#1, #2, #3 |
Address format. Used with one of the three PURPOSEs used in iMIS. Note: This must match the address exported to the National Association of REALTORS® (NAR). |
N |
Numeric format, removes leading zeroes. Do not use N for importing international phone numbers. |
P |
Phone format. |
L |
Instructs the import utility to insert a new Name.ID and Name.MAJOR_KEY if no match is found for that field (must have both). |
D |
Date format. Note: The NRDS import file contains two fields for zip code. To convert these fields to the iMIS zip code format, enter two zip code fields in the table. Specify Length = 5 for the first field. For the second code, specify Length = 4 and specify A- processing code (Append Data to Field, followed by a hyphen). Note: Use the A- processing code (Append Data to Field, followed by a hyphen) for importing Social Security numbers in the standard format (999-99-9999). |
The Member import template uses the following additional Processing Codes:
Code |
Definition |
---|---|
@ |
Takes the NRDS ID number in the import file and finds the Name record in iMIS where this number is populated in the MAJOR_KEY field to get the iMIS ID. The utility then populates the Name.CO_ID field with this iMIS ID for the member record to be imported. Any necessary company information flowdown is then processed. |
% |
Used to populate the preferred mail checkbox. The import file contains a field consisting of H, M, or O. The Import Utility maps each letter to a specific address. In the Name_Address.PREFERRED_MAIL field, enter a % processing code with a 3-digit code representing each of your three iMIS addresses. For example, using %312 means the third address in iMIS corresponds to the Home Address, the Main Address corresponds to the mailing address, and the Second Address corresponds to the office address. |
Before importing the Contacts.csv file, you must ensure that the data in the file is accurate and correct. Review all the fields in the Contacts.csv file, as iMIS will import exactly what is in each field.
Do the following to import the Contacts.csv file:
- In the Advanced Accounting Console tool bar, go to Utilities > Data Transfer Utilities > Import Utility to open the Import Utility window.
- Under Available Imports, highlight the CONTACT import.
- Click Go!
- Browse to find the Contacts.csv file.
- Select the Contacts.csv file.
- Click Open.
- On the Import Utility window, the number of records imported will display, along with 100% Complete once the import is complete.
- Ensure the number of records imported matches the number of records in your Contacts.csv file.
Do the following to verify the import was successful:
- In the Staff site, go to Community > Find Contacts.
- Perform a By all common criteria search .
- Enter adanac in the Company starts with field.
- Verify that you can see the records you imported into the Name table in the iMIS database.
The Export Utility lets you export information from the iMIS system to another location or to a third-party software application, such as a word processor, spreadsheet program, or compatible database application. The Export Utility lets you:
- Include information from multiple records of the same file on one export record.
- Update fields in the iMIS database when the export is run.
- Specify if/then processing of the output.
- Create an activity record that records the date the export record was created and includes a copy of each member’s export record.
- Schedule the export execution time.
To use the iMIS Export Utility, you must have the EXPORT license code in your system control file. Contact your authorized iMIS reseller to obtain an updated system control file.
The primary table (file) in iMIS is the Name table, which is the internal system label for the member/contact file. It contains name, address, categorization, and demographic information for each member/contact that you track through iMIS. All iMIS modules interact with and retrieve information from this table.
The other main table that supports membership processing is the general purpose Activity table. It contains records that track historical and participation details, such as meetings attendance, committee service, dues payments, product purchases, correspondence sent, and calls logged. The information in this table is commonly accessed for analysis or marketing reasons.
Another table you may want to use is the Subscription table, which tracks all dues and subscription line items billed and collected.
When you want to access or reference a field in iMIS, you can often select from a field list. To export iMIS fields, format the field names exactly, including precise capitalization and punctuation. Table (file) names appear with initial capitals (with underscores as needed), field names appear in all capitals (with underscores as needed), and the two parts are connected with a period.
The following are the recommended iMIS tables to use to export data:
- Name
- Activity
- Subscription
- Orders
- Order_Lines
- Trans
- Invoice
- Invoice_Lines
Avoid using other iMIS tables for export, because data in the other tables might not be easily linked to the primary tables.
Do the following to export data using the iMIS Export Utility:
- Create the export definition.
- Select the iMIS tables and fields to be exported.
- Specify the search criteria to be used.
- Specify the export field formats.
- Run the export.
The first step to export data with the Export Utility is to create the basic export definition. The export definition specifies and formats the data to be exported.
Do the following to create the export definition:
- In the Advanced Accounting Console, go to Utilities > Data Transfer Utilities > Export Utility.
- Click New.
- Type a name for the export definition in the Name field.
- Select a Delimiter (character to be placed between fields):
- Fixed – no delimiter.
- Tab
- Comma
- Other – specifies other delimiter in a field that is displayed.
- Quotes – encloses text in double quotes. This is helpful if a comma delimiter is required and some text fields contain commas.
- (Optional) Select the Create Activity Record option to record each export record as a member activity in the iMIS database.
- Define the Path for the export file.
- (Optional) Select the Allow OverWrites option to allow the file to be overwritten each time the export is run.
- Click Save.
Note: The Quotes field displays only when a delimiter other than Fixed is selected.
Note: You can include an incremental sequence number in the name of the file by including a question mark in the name of the file. For example, if you enter c:\Temp\national?.txt, the file c:\Temp\national1.txt is created the first time you run the export, c:\Temp\national2.txt is created the second time you run the export, and so forth.
After you create the export definition, select the iMIS tables and fields to be included in the export definition. These Select Files are the basis for the export. You can sort the export records based on a file or field value in the Select Files and you can define search criteria to designate data in the Select Files to be exported.
Do the following to select the iMIS tables and fields to be exported:
- In the Advanced Accounting Console, go to Utilities > Data Transfer Utilities > Export Utility.
- Select an export definition from the Available Exports.
- Click Edit.
- Click Setup ‘Select’ Files to open the File Export Utility – Select Records window.
- Select up to three iMIS tables as Select Search Files.
- (Optional) Select the Sort Fields for these files. This determines the order the records are exported.
- (Optional) Select the Break option if you want to create an export record each time the value in Sort Fields changes.
- Define the joins for the Select Search Files in the File Connections fields. For more information, see Defining joins for select files.
- Click OK to return to the Export Utility window.
- Click Save.
Note: To eliminate duplicate member export records, specify the Name.ID field as a Break field. The Export Utility will create only one export record for each unique Name.ID.
Specify the general search criteria for the export. The export will contain only the information in the Select Search Files that meets the search criteria.
Do the following to specify search criteria:
- In the Advanced Accounting Console, go to Utilities > Data Transfer Utilities > Export Utility.
- Click Edit Search Criteria to open the Select Search Instructions window.
- Click New to open the Search Instructions window.
- Enter the Search Name.
- (Optional) Select the Mode. This option allows you to export only the files or fields that meet defined criteria such as Comparison, Calculation, AND, and OR.
- Click Save.
- Click OK to return to the Export Utility window.
Note: If the Comparison Value is a field value, enter an asterisk (*) before the field name.
Note: You can specify search criteria from a record in a file that is not one of the three Select Search Files, or from another record within a Select File. If you are pulling information from a file other than a Select File, you must relate the file to one of the Select Search Files in the search criteria.
You can use SQL statements to specify the Comparison Value. For example, if Name is a Select File and you want to bring in the Home Address record, enter Name_Address.ID to search for a specific address. Enter *[ASISql(Name.ID)] for the Comparison Value, as shown in the following example.
Export field formatting is entered in the Comparison Value section of the Export Utility window. You can specify formatting for constants, iMIS field values, and Omnis calculations. You can also specify if/then processing of the output by formatting the export output with condition statements. This step narrows the general search criteria for specific fields.
Do the following to format export fields:
- In the Advanced Accounting Console, go to Utilities > Data Transfer Utilities > Export Utility.
- Select an Available Export.
- Click Edit. There are three types of values that can be exported: a constant, a value contained in an iMIS field, and an Omnis calculation. Enter the formatting in the Value field.
- To specify the format for a constant (literal) value, type a double-quotation mark followed by the value. For example, if the first four characters of the export record must contain the value 0001, enter "0001 in the Comparison Value field.
- To specify the format for an iMIS field value, type the field name, for example, Name.MEMBER_TYPE.
- To specify the format for an Omnis calculation, type an equal sign followed by the calculation.
For example, if you want to export the area code of the home telephone number field, type =mid(Name.HOME_PHONE,2,3). This statement exports the second, third, and fourth characters from the Name.HOME_PHONE field. - (Optional) Specify the desired Length of the field. This allows you to truncate or extend the field length.
- (Optional) Enter a code in the Format field.
- Click Add to save the format.
- After all formatting instructions are added, click Save to save the formats.
The following table lists the available Format codes.
Code |
Definition |
---|---|
1 |
Exports the value on the first record only (useful for header records). Must be in the first position of the Format field. |
C |
Formats the output to initial capital letters. |
D6 |
Exports the value as a date in the MMDDYY format. |
D6/ |
Exports the value as a date in the MM/DD/YY format. |
D8 |
Exports the value as a date in the MMDDYYYY format. |
D8/ |
Exports the value as a date in the MM/DD/YYYY format. |
F |
Appends an LF (line feed) return to the end of the field. |
L |
Formats the output to all lowercase letters. |
P |
Pads the field to the length specified in the Length field with the character immediately following the P. |
P0 |
Pads the field without zeros. |
N |
Specifies all blank or zero field values to be output as Null. |
N2 |
Same as N except that the characters . , - $ ( ) are saved. |
R |
Appends a CR (carriage return) feed at the end of the field. Note: This code should be used with the last field of each format. |
U |
Converts the output to all uppercase letters. |
S |
Skips the delimiter and concatenates with the next export column. |
X |
Updates data in the iMIS database. Note: You must include search criteria and a calculation with this format. The search criterion identifies the specific record to be updated. The calculation determines the value that is output in the export record. |
6X |
Indicates which record will be selected when searching extra records. For example, the code 62 will take the field value from the second record that matches the search criteria. |
Y |
Outputs a YES or NO. If the field value or calculation in the Value field evaluates True, a YES is exported. Otherwise a NO is exported. |
y |
Same as Y, except outputs a Y or N. |
You can format the export output with condition statements by creating a series of IF statements with corresponding output values.
- In the Advanced Accounting Console, go to Utilities > Data Transfer Utilities > Export Utility.
- Select an Available Export.
- Click Search. Highlight an export Value.
- Click Edit Calculations to display the Export Calculations Utility window.
- Click New to create a new calculation.
- Click Insert at the bottom of the window and click in the Name field to add a new name for the calculation.
- Click Add to add a calculation or condition statement. In the Calculation field, type an equals sign followed by the calculation, for example, =Name.MEMBER_TYPE=‘CM’.
- In the Output field, type a double quotation mark followed by the desired output value, for example "CO. In this example, every CM Member Type is exported as a CO value.
- Click Save to save the calculation. The calculation is now listed as an Available Calculation on the Export Calculations Utility window.
You can add as many calculations as you need. The utility exports the result of the last True calculation. The result can be an iMIS field name, a constant, or an Omnis calculation.
To load records from compatible third-party software applications, the records must be comma- delimited or tab-delimited. Each record to be loaded must contain the following fields in this exact sequence: Value, Length, Format, Calculation, and Search.
Do the following to load records from compatible third-party software applications:
- In the Advanced Accounting Console, go to Utilities > Data Transfer Utilities > Export Utility.
- Select an Available Export.
- Click Edit.
- Click Load to select the file to load into the export definition.
- Click Open to start the load process.
- Click Save.
You can run an export on demand. Do the following to run an export on demand:
- In the Advanced Accounting Console, go to Utilities > Data Transfer Utilities > Export Utility.
- Select the Available Export to run.
- Click Go.
- When prompted to confirm running the export, select Yes.
You can also schedule an export. The export should be scheduled after you have finished work for the day.
Do the following to schedule an export:
- In the Advanced Accounting Console, go to Utilities > Data Transfer Utilities > Export Utility.
- Click Schedule to open the Schedule Program window.
- Select the time to start the export by using the up and down arrows.
- Click OK. The <remaining time> until Program Start message is displayed.
You can clone an existing export definition and make changes to create a new export definition.
Do the following to clone an export definition:
- In the Advanced Accounting Console, go to Utilities > Data Transfer Utilities > Export Utility.
- Select an Available Export.
- Click Save As on the Export Utility window to display the Export Definition Save As window.
- Type the New Name for the export definition.
- Click Save to save the file with a new name.
- Make changes in the new file and save.
If the Create Activity Record option is selected in the Export Utility window, iMIS records the date the export record was created and saves a copy of each member’s export record.
Do the following to view the activity record:
- Make sure the Name file is one of the Select Files in the export definition.
- In the Advanced Accounting Console, go to Configure > Membership > Set up activity types.
- Create an activity type named EXPORT.
- Click Save.
- On the Staff site, go to RiSE > Page Builder > Manage content.
- Select a Content Folder.
- Go to New > Website Content.
- Enter a Title and a Publish File Name.
- Click add content.
- In the Content Gallery, go to Content Types > Content > Panel Editor.
- Click OK.
- In the Panel list, select (New Panel).
- Name the panel.
- Select the Show data sources that allow multiple entries per object instance radio button.
- Double-click Activity-EXPORT. The columns available for the EXPORT activity record are displayed.
- Click Save & Close and then OK.
- Click Save & Publish.
Note: When creating an EXPORT activity type, be sure to define some columns. For example:
UF1 - UF3 – Use these fields to enter prompts for the values entered in the Char 1 - Char 3 fields on the Event System Preferences window.
UF4 - UF5 – Use these fields to enter prompts for the values entered in the Number 1 - Number 2 fields on the Event System Preferences window.
UF6 - UF7 – Use these fields to enter prompts for the values entered in the Date 1 - Date 3 fields on the Event System Preferences window.
These columns will generate data that will be visible in the Panel Editor content item that you will create in order to view the activity records.
You might require data from more than one iMIS table for export. For example, you might need FULL_NAME and COMPANY from the Name table and TYPE=‘DUES’ and BALANCE from the Subscription table. Since there is no membership data in the Subscription table, you must access the Name table. To use data from two or more tables, you must join them.
To join files (tables) is to specify which fields are equivalent so the files can relate to each other correctly, record-by-record. In effect, joining glues the files together for the export process. The most common joins in iMIS center on member ID numbers. A join across three tables might look like the following:
Name.ID = Activity.ID
Name.ID = Subscription.ID
The order of your table joins can impact performance. Although the optimal order depends on the report, the key iMIS joins are:
Name.ID = Activity.ID
Name.ID = Subscription.ID
Name.ID = Invoice.BT_ID
Orders.ORDER_NUMBER = Order_Lines.ORDER_NUMBER
Product.PRODUCT_CODE = Order_Lines.PRODUCT_CODE
Name.ID = Orders.BT_ID
If you experience problems such as too much data being returned, multiple records being returned, or endless data retrieval, you may not have joined the files correctly.
If you are experiencing performance problems with a report that uses a join, try switching the order of the files.
Make sure that the files included in the join are Select Files in the export definition.