Delta Hashing

Input Output
  • Any (JSON)
  • Required: Array of Objects (Dataset)
  • Any (JSON) (Passthrough)
  • Filtered Dataset of Changes (array)

Summary

Delta Hashing takes a “tabular” dataset (an array of columns where each object has the same columns), and for each row, determines if the data has changed since the previous workflow run. Only changed rows are included in the output.

How It Works

An MD5 hash of the record is taken and cached. It is identified by the “Key Column”, which must be a unique value.

On the first run, all records will be included in the output since there are no stored hashes for those records.

On subsequent runs, if the MD5 hash does not match the previous run, that record is included in the output.

What Is Stored

Only the key column is stored in plaintext in our datacenter. The other columns/values in the dataset are not stored.

Input Object Delta Hash Configuration Stored Data 
{
  "id": "12345",
  "firstName": "Joe",
  "lastName": "Smith",
  "dob": "2001-03-07"}
 Key Column Name: id 
{
  "id": "12345",
  "hash": "DC2F3BC7459B8B3B37FD2F64C16FCFA6"}

Properties

Name Type Templatable Notes
Key Column Property Name

Text

No

Enter the name of the column (the property name) that should be used as the “primary key” for hashing.

The data in this field must be unique.

Tip! Use iMIS ID, sequence values, or GUIDs for this field.
Input Array

Array

Yes

Enter an array expression that points to the dataset that should be hashed for changes.

For example, if you previously ran an IQA action and stored the results in the property myIqaData, then in this field, you would enter:

{{ input.myIqaData | array }}
Output Property

Text

No

Specify the name of the property to store the CSV data in.