Entity matching

How it works
Entity fields match information on a source document to user-provided reference data. This lets you establish a link between documents and records from an existing business database or directory.
A common use-case for entity matching is to link an invoice issuer to a supplier database. Entity fields automatically learn a fuzzy-match between information on the document (e.g. a Supplier Name, Address or Business number) and reference data fields you've uploaded. The reference data is then returned as a standard prediction result, allowing you to build on these matches to power complex automation rules, derived field values and validation checks.
Getting started
To configure and use an entity match field, the following basic process applies:
1.Upload entity data to Sypht via the entity storage API
2.Configure and train an entity match field
3.Extract the entity field from documents to establish a match
As entities are pushed from your database to Sypht there is no need to open up network or API access into secure internal data stores. You have complete control over what data is pushed and when.
Overview of the basic entity matching process.
Keeping reference data in sync
When an entity field is extracted on a document, data is matched against all available entities in the Sypht data store at that time. As reference data changes over time you can push smaller differential updates via the entity storage API to replicate addition, removal or modification of entities.
We recommend establishing a regular automated ETL process to keep reference data in sync.
Using the entities API
This section explains the available API endpoints for storage, retrieval and search of entity data. Entities are stored in isolated collections for a given company_id and within that company there may be multiple distinct entity types (e.g. there may be distinct entity types for supplier and the receiving office ; each having distinct attributes and match logic).
Checkout the open-source Sypht Python Client on GitHub for a reference implementation using the entities API.
Pushing entities
PUT storage/{company_id}/entity/{entity_type}/{entity_id}
Path Parameters:
company_id your Sypht Company Id
entity_type type of entity e.g. supplier, vehicle or employee
entity_id a unique identifier for the entity
Request Body:
JSON-encoded entity data
An object with keys and values representing attributes of the entity
Complex json data structures (e.g. nested objects or lists) may be stored but are currently not supported as reference fields for search and match
Empty values should be represented as null
Example Supplier Entity data
1
{
2
   "name": "Joe's Plumbing",
3
   "address": "123 Water Street, Chippendale, 2008",
4
   "expense_code": "GL1234",
5
   "project_id": "PRJ-9876",
6
   "last_sync": "2021-01-01"
7
}
Copied!
POST storage/{company_id}/bulkentity/{entity_type}/
Path Parameters:
company_id your Sypht Company Id
entity_type type of entity e.g. supplier, vehicle or employee
Request Body:
JSON-encoded list of objects with an entity_id and data to be store
1
[
2
  {
3
    "entity_id": "id0",
4
    "data": {
5
      "rego_no": "qwer",
6
      "Contract Start": "2020-11-04"
7
    }
8
  },
9
  {
10
    "entity_id": "id1",
11
    "data": {
12
      "rego_no": "wert",
13
      "Contract Start": "2020-11-05"
14
    }
15
  },
16
  {
17
    "entity_id": "id2",
18
    "data": {
19
      "rego_no": "erty",
20
      "Contract Start": "2020-11-06"
21
    }
22
  }
23
]
Copied!
Removing entities
DELETE storage/{company_id}/entity/{entity_type}/{entity_id}
Path Parameters:
company_id your Sypht Company Id
entity_type type of entity e.g. supplier, vehicle or employee
entity_id a unique identifier for the entity
Retrieving entities
GET storage/{company_id}/entity/{entity_type}/{entity_id}
Path Parameters:
company_id your Sypht Company Id
entity_type type of entity e.g. supplier, vehicle or employee
entity_id a unique identifier for the entity
Response Body:
1
{
2
   "name": "Joe's Plumbing",
3
   "address": "123 Water Street, Chippendale, 2008",
4
   "expense_code": "GL1234",
5
   "project_id": "PRJ-9876",
6
   "last_sync": "2021-01-01"
7
}
Copied!
Searching entities
POST storage/{company_id}/entitysearch/{entity_type}/
Path Parameters:
company_id your Sypht Company Id
entity_type type of entity e.g. supplier, vehicle or employee
Request Body:
JSON-encoded string containing exact and fuzzy match search constraints
Each of these should be an object with keys denoting an attribute to search against and value denoting the query string to search for
e.g. To search for exact match for "rego_no" == "qwer" 
1
{
2
    "exact": {},
3
    "fuzzy": {"name":"Plumbing"}
4
}
Copied!
Response Body:
1
[
2
    {
3
        "item": {
4
            "name": "Joe's Plumbing",
5
            "address": "123 Water Street, Chippendale, 2008",
6
            "expense_code": "GL1234",
7
            "project_id": "PRJ-9876",
8
            "last_sync": "2021-01-01"
9
        },
10
        "score": 0.5753642
11
    },
12
    ...
13
]
Copied!
Searching entities by id
POST storage/{company_id}/entitysearch/{entity_type}/by_id
Path Parameters:
company_id your Sypht Company Id
entity_type type of entity e.g. supplier, vehicle or employee
Request Body:
JSON-encoded list of objects with entity_id
1
[
2
  {
3
    "entity_id": "id_0"
4
  },
5
  {
6
    "entity_id": "id_1"
7
  }
8
]
Copied!
Response Body:
1
{
2
    "entities": [
3
        {
4
            "entity_id": "id_0",
5
            "data": {
6
                "name": "Joe's Plumbing",
7
                "address": "123 Water Street, Chippendale, 2008",
8
                "expense_code": "GL1234"
9
            },
10
            "error": false
11
        },
12
        {
13
            "entity_id": "id_1",
14
            "data": null,
15
            "error": true
16
        }
17
    ]
18
}
Copied!
Retrieving list of entity_id
GET storage/{company_id}/entitysearch/{entity_type}
Path Parameters:
company_id your Sypht Company Id
entity_type type of entity e.g. supplier, vehicle or employee
Query Parameters:
page  page token, if None (not provided) will return first page by default, otherwise request for specified page which would be grabbed from next_page of previous response
limit maximum count for responded entity_ids
Response Body:
1
{
2
    "next_page": "page token",
3
    "entities": [
4
        "102013",
5
        "102015",
6
        "102019",
7
        "102034",
8
        "102051",
9
        "102056",
10
        "102057",
11
        "102068",
12
        "102072",
13
        "102074"
14
    ]
15
}
Copied!
Using Sypht Client
GitHub - sypht-team/sypht-python-client: A python client for the Sypht API
GitHub
pip install sypht
Retrieving all entity_ids
This client method is a wrapper to loop over pagination endpoint to get all entity_ids for specified entity_type
Returns list of objects if verbose (by default)
[{"entity_id": "id_0"}, {"entity_id": "id_1"}, ...]
Returns list of entity_id if not verbose
["id_0", "id_1", ...]
1
from sypht.client import SyphtClient
2
​
3
sc = SyphtClient('<client_id>', '<client_secret>')
4
sc.get_all_entity_ids(entity_type='test_type')
Copied!
Workflows - Previous
Smart document split
Next - Field types
Line Items
Last modified 8mo ago
Copy link
Contents
How it works
Getting started
Using the entities API
Searching entities by id
Retrieving list of entity_id
Using Sypht Client
Retrieving all entity_ids