Polling Events
About Polling Events
Polling Events are one of the two types of Events that your connector can contain. Polling Events are designed to automatically run on a recurring basis based on the user’s plan (typically 1-5 minutes). This means that these Events will need to grab a collection of records, then process collections of records and determine which records have happened since its last iteration.
To be able to keep track of the last record that was collected, Polling Events use a tool called the “since” value. This value allows your API to understand when the last query sent from your Connector was, and can take many forms, although it often is a timestamp sent in your HTTP request. When the user turns on a Flow, the Flow will call the API to get the current since value. Then, at a specified interval, the Event will call the API again and use the since value saved during the previous run to fetch new records. The engine will automatically execute the Flow once for every new record.
The Since Value
The “since” value is a single piece of data that persists between executions of an Event. This means that the data that was set on a previous run of an Event can be compared against a current run’s dataset. Usually the “since” value is a timestamp or the ID of the last record collected of a previous iteration, that you can use to compare against the records collected on this iteration.
For example, if a service returns a list of all records, and each record looks like so:
{ “recordName” : “A”, “createdAt” : “2017-02-15 8:58:00” }
then, so long as I can keep track of the time the last iteration occurred, I can filter out my list of records to only include those whose “createdAt” value is greater than that timestamp. The “since” value would keep track of that timestamp to filter by, and then would be reset to the current time after the list was filtered.
The “since” value can be defined in the Control.Since module, and once defined may be referred to with Mustache as “{{since}}” throughout that Event method.
Building a Polling Event
Polling Events have a fairly well-defined structure, that can be used in most Connectors. Generally speaking, they’ll always include these steps, in approximately this order:
- Get a list of items from an API
- Filter that list based off of the since value that was set on a previous execution. As a side note, sometimes services will allow you to provide a timestamp as a query parameter in the initial request. In those cases, this step is unnecessary, and instead you would pass the since value as that query parameter in the initial API interaction.
- Update your “since” value with the Control.Since module
- (Optional, but often needed) Map over each item in the filtered list to render that item into the right format
- End the method with the new list of formatted and filtered items.
Note: A Polling Event must return a list, even if that list contains just one item in a given iteration
Testing a Polling Event
To test a Polling Event in the connector builder, you can artificially set the “since” value in the “Run” window, inside the “Since” textarea. Inside of this area, put in any value that you want to simulate as the since value from the previous iteration of a method execution. You can also leave it as “null” to simulate the first run of your Polling Event.