Search
gridsz
GRIDSZ
Contact us
Request demo
GRIDSZ

Equitri protocol

Equitri is the protocol the GRIDSZ Workflow framework uses to keep parties working on the same task in sync. It powers the Fulfillment (TASK) and Assurance (SERVICE) APIs and assumes a relatively low volume of updates on long-lived tasks (typically running longer than a day). Behind a deliberately simple API, Equitri hides the complexity of backend systems, middleware, overnight batch processing, and realtime systems.

Every connected party must implement both a client and a server: the client initiates updates and fetches task information from another party’s server, while the server exposes the task data to another party’s client (typically Gridsz). The creator (InTask) and receiver (OutTask) use different APIs, but the underlying protocol is identical.

After describing some terminology and the protocol itself, this page details the steps for both directions:

  • Gridsz generating a new task or updating an existing task
  • An integrated system generating a new task or updating an existing task

The term ‘integrated system’ and ‘you’ refers to the party implementing Equitri.

This page does not focus on the underlying connectivity on http(s) level , VPNs, proxies et cetera and it does not expand on what the party should do with the received task information. The bare minimum is for the integrated system to be able to handle the Equitri API (INDICATION / FETCH / SYNC) both ways with valid json and authorization, taking into account sunny and rainy day cases.

Note: it is possible is to handle the FETCH separately from the INDICATION and SYNC process – to collect information without updating the task.

Glossary

A short description of terms used.

Equitri

The name provided to the API-mechanism as used on the GRIDSZ workflow APIs. This mechanism applies to both Fulfilment and Assurance flows, and for both the creator (InTask) and receiver (OutTask) APIs.

INDICATION

An INDICATION is the 1st message in the Equitri protocol. It is a fire-and-forget POST message simply stating that there is a task to be fetched at the senders’ endpoint. The content in the body specifies which task to fetch from which endpoint. This can either be a completely new task, or an update on an existing task.

FETCH

A FETCH is the 2nd message in Equitri and the actual ‘power-stroke’. With a simple HTTP GET to the right URL, the entire task content is returned. The FETCH could also be done as a standalone call, without a previous INDICATION, and will then also not require a SYNC.

SYNC

A SYNC message is the 3rd and last message in the Equitri protocol. It is a PUT message which states that the task was processed, and no further INDICATION should be sent for this specific update (version). The SYNC also contains a field for clarification and reason, which will be filled in case the update does not pass the data validations.

UpdateCount

Also known as the ‘version’. It specifies which version a specific task is currently on. On GRIDSZ side, the version is the master and can only increase or stay the same (if update fails).

Equitri protocol rules

The following general rules apply:

  1. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.
  2. HTTP headers which are not recognized will be silently ignored
  3. When a party sends a task to another party. And, directly after that fetches that task with the same party, the received task contains exactly the same data as the original.
  4. When you receive an INDICATION for a task, you are not to send any INDICATION for that task yourself, until you first FETCH and SYNC for the incoming INDICATION
  5. When receiving multiple updates for the same task, only the last update (meaning highest updateCount) is relevant. Old updates will be skipped.
  6. A task always concerns 1 TASK on 1 connection. The connection (DHid) and taskType of an existing task is not allowed change

Retry schema

Things can, and will at some point, go wrong. Whenever the update of an existing task goes wrong, the party must rely on the retry schema:

  • Unless a SYNC is sent back, INDICATION messages will be send again until eternity by the other party.
  • Since the receival of an INDICATION is often used as the trigger to do a FETCH and a SYNC, the minimum time between two indication requests for the same task is 10 seconds, and typically starts at 1 minute on production.
  • Implementations of Equitri could implement a progressive backup where the retry time between two indication requests increases after each request up to a maximum of 1 INDICATION per 4 hours.

All INDICATON requests for the same combination of task and updatecount belong to the same conversation.

Relative task versioning

It is possible, and even likely, that there will outages of 1 or more of the systems of parties involved in one task at some time. Also, parties will have a different cadence in responding to updates in a task. To help prevent versioning issues the Equitri task always has an updatecount. The updatecount acts a form of version number. It must always start with the integer value 1 for a new task. For every update a party communicates with other parties, the updateCount must be increased by (at least) 1 compared to the current value.
When the updateCount is 500 or more, the task has spiralled into a (potentially endless) loop. The integrated system should not send any more INDICATION for this task and make sure manual action is taken to correct the situation.

UpdateCount lower or equal than the current updateCount

This happens when another party send an update on a previous version of the task. The other party was probably offline and did not receive last updates on the task. GRIDSZ’s version of the task will be the master.
For instance, comments that were not present in the version of the integrated system will be inserted in the correct order. Attachments can be added. Even the status of the task can be updated, but only if it does not collide with the status in the current task version.
After the merge, the system will SYNC with the other party to signal the task update was received and processed. For syncing it is important that the integrated system will have the same updateCount compared to the version currently on GRIDSZ. If a lower updateCount is synced, then the system that initiated the INDICATION should still send INDICATIONS for the un-synced latest updateCount.
Afterwards, the integrated system can start a new conversion, starting with an INDICATION, to bring the other party up to date with your version of the task.
Note: it is NOT possible for GRIDSZ to return to a previous version. The current version is and will remain the truth, also if the other system cannot accept this.

UpdateCount higher than the current UpdateCount

This is the sunny-day scenario. Another party made an update and is now further in updates than the other side is. The version of the task with the lower updateCount will be the deprecated. And the received version of the task (with the higher updateCount) will serve as the new ‘truth’.
It is now the job to merge the received task and stored task together into a new consistent version. Most of the time this can be as simple as overwriting the old version of the task with the fetched version. Things get more complex when the received update is not acceptable by the other side. For instance, if GRIDSZ has the task in an end-status and then receives an update from integrated system to set it back to an open-status.
In this case, GRIDSZ will send a SYNC back with a clarification + reason filled. This automatically means that the version is not accepted, and the previous version is still the current truth. As GRIDSZ cannot return to a previous version, any SYNC with the updateCount equal to the current updateCount will be considered as a SYNC-ok from integrated system, even if the clarification + reason is filled.
By not sending a SYNC at all, the other system will initiate the retry schema and keep sending INDICATIONS. Recommend in these scenarios is to find the root cause why the update is not accepted. In case it concerns an unexpected JSON response or value, it is best to contact GRIDSZ to examine it further together.
Note: it is NOT possible for GRIDSZ to return to a previous version. The current version is and will remain the truth, also if the other system cannot accept this.

Support for smoke tests

The protocol supports an optional HTTP header called ‘x-gridsz-test’. When the HTTP header is present in a request and its value is ‘TRUE’ it is a signal that the given conversation is part of a test. Typically, this header is used for smoke tests on production. It is to be processed as normal, but no actual physical action should be taken. Parties implementing the Equitri protocol are encouraged to implement debug level logging which is triggered by this x-gridsz-test=TRUE header.

A normal task (created without the x-gridsz-test=TRUE header) which receives an update with a x-gridsz-test=TRUE header is cause for alarm. Possibly test-tasks and normal tasks are getting mixed up. A test task (created with the x-gridsz-test=TRUE header) which receives an update without a x-gridsz-test=TRUE header is not cause for alarm. Possibly the other party cannot support the HTTP header.
Caution is advised with using test tasks on the Production environment. Be sure that all involved parties are aligned, to prevent actual work from being completed physically.

GRIDSZ updating an existing task or creating a new task

This breaks down in 3 steps: the incoming indication request (Gridsz towards the integrated system), the fetch request (the integrated system providing information to Gridsz), and the sync to confirm.

1. Incoming Indication

The indication initiated by Gridsz:

  1. Authorize the request, as agreed during implementation, based on the authorization credentials stored in Gridsz (may require OAuth 2.0 token collection)
  2. (If X-gridsz-test header has value ‘TRUE’ turn on debug logging)
  3. Read updateCount from the INDICATION
  4. (Could: log “received indication for update updatecount on taskId from orgId”)
  5. Store the fact that an update is available for the task with the following data:
    • orgId (must)
    • systemId (must)
    • taskId (must)
    • updateCount (must)
    • x-request-id (could),
    • x-correlation-id (could)
    • requesttime (could)
    • value of x-gridsz-test (must if present)
  6. Erase previous update entries for same orgId and taskId with lower updatecount

2. Outgoing Fetch

The fetch from Gridsz:

  1. Determine endpoint to call based on environment, orgId and taskId in the INDICATION
  2. (Read value for x-gridz-test and, if found, set x-gridsz-test http-header to its value)
  3. do FETCH request on endpoint; retrieve new task information
  4. Check to see if task data is well formed:
    • Valid json which complies to the open API definition for the latest API version
    • Valid json which passes the business validations
  5. If the task is not well-formed: investigate the rootcause and contact Gridsz if needed
    Note: GRIDSZ will keep sending INDICATION until a SYNC has been sent
    (END OF FLOW)
  6. If the data is well formed:
    • Lock the task in the task-store
    • Read old task info from the task store
    • Do task merge between old and new task info
    • If the result of the merge is different than the old task info write merged task info to the task store
    • Unlock the task in the task-store
  7. Send SYNC to endpoint.

3. Outgoing Sync

The sync to confirm:

  1. Send SYNC request to endpoint of the FETCH
  2. There is no retry for a SYNC message. If the request/response fails, you could log the exception and any error codes & clarifications as provided within the sync message. Rely on the retry schema of the other party to try again
  3. Delete the update information written during indication

Integrated System updating an existing task or creating a new task

This again breaks down in 3 steps: the indication request (the integrated system towards Gridsz), the fetch request (Gridsz collecting information from the integrated system), and the sync to confirm.

1. Outgoing Indication

The integrated system detects a change (not caused by Gridsz in the previous scenario) and wants to inform Gridsz about this by sending an indication to Gridsz:

  1. Authorize the request, as agreed during implementation (may require OAuth 2.0 token collection)
  2. (If X-gridsz-test header has value ‘TRUE’ turn on debug logging)
  3. Add 1 to the updateCount stored on the existing task record within the integrated system (use 1 for new tasks)
  4. (Could: log “sending indication for update updateCount on taskId from orgId”)
  5. Send indication to Gridsz with the following data:
    • orgId (must)
    • systemId (must)
    • taskId (must)
    • updateCount (must)
    • x-request-id (could),
    • x-correlation-id (could)
    • requesttime (could)
    • value of x-gridsz-test (must if present)

Don’t wait for or try to read any response. Don’t try to recover from any exception, but rely on retry schema to try again.

2. Incoming Fetch

Gridsz will respond to the indication from the previous step by fetching the task from the agreed endpoint:

  1. Accept the call to the endpoint based on the agreed authentication
  2. (Read value for x-gridz-test and, if found, set x-gridsz-test http-header to its value)
  3. Provide 404 if task is not found in the integrated system
  4. Provide FETCH information by loading and formatting the task information from the integrated system

3. Incoming Sync

Gridsz will process the information from the previous step and will respond with a PUT to the endpoint to confirm the sync:

  1. Process the authenticated SYNC request to endpoint of the FETCH
  2. When the SYNC contains a filled reason and clarification, it means GRIDSZ did not accept the update and the previous version is still the master

The Indication-Fetch-Sync might not always run perfectly, but the protocol should gracefully retry as described above. However, reason codes and clarifications from the sync step should be processed properly (and reported to the users) to prevent endless loops.