ibmcloudant.features.changes_follower
A helper for using the changes feed.
Module Contents
Classes
ChangesFollower is a helper for using the changes feed. |
|
The ChangesFollowerIterator implements iterator interface. |
|
Enums for changes follower’s operation mode. |
|
Enums for changes follower’s transient errors suppression mode. |
Data
API
- ibmcloudant.features.changes_follower.BASE_DELAY = 100
- ibmcloudant.features.changes_follower.BATCH_SIZE = 10000
- class ibmcloudant.features.changes_follower.ChangesFollower(service: ibmcloudant.cloudant_v1.CloudantV1, *, error_tolerance: int = FOREVER, **kwargs)
ChangesFollower is a helper for using the changes feed.
There are two modes of operation: start_one_off() to fetch the changes from the supplied since sequence until there are no further pending changes. start() to fetch the changes from the supplied since sequence and then continuing to listen indefinitely for further new changes.
The starting sequence ID can be changed for either mode by using “since”.
By default when using: start_one_off() the feed will start from the beginning. start() the feed will start from “now”.
In either mode the iterator can be terminated early by calling stop().
By default ChangesFollower will suppress transient errors indefinitely and endeavour to run to completion or listen forever. For applications where that behaviour is not desirable an alternate options is available where a duration may be specified to limit the time since the last successful response that transient errors will be suppressed.
It should be noted that errors considered terminal, for example, the database not existing or invalid credentials are never suppressed and will throw an exception immediately.
The named arguments for “post_changes” are used to configure the behaviour of the ChangesFollower. However, a subset of the options are invalid as they are configured internally by the implementation and will cause an ValueError exception to be thrown if supplied.
These invalid options are: - descending - feed - heartbeat - last_event_id - timeout
Only the value of “_selector” is permitted for the post_changes’s “filter” option. Selector based filters perform better than JS based filters and using one of the alternative JS based filter types will cause ChangesFollower to throw a ValueError exception.
It should also be noted that the “limit” parameter will truncate the iterator at the given number of changes in either operating mode.
The ChangesFollower requires the Cloudant client to have HTTP call and read timeouts of at least 1 minute. The default client configuration has sufficiently long timeouts.
- Parameters:
service (CloudantV1) – A client for the Cloudant service.
error_tolerance (int) – A duration to suppress transient errors for set in milliseconds.
- Returns:
None
Initialization
- property error_tolerance: int
- property options: Dict
- start() Iterator[ibmcloudant.cloudant_v1.ChangesResultItem]
Return all available changes and keep listening for new changes until reaching an end condition.
The end conditions are: - a terminal error (e.g. unauthorized client). - transient errors occur for longer than the error suppression duration. - the number of changes received reaches the limit specified in the “post_changes” args used to instantiate this ChangesFollower. - ChangesFollower’s stop() is called.
The same change may be received more than once.
Returns an iterator of ChangesResultItem per change.
Throws ValueError if ChangesFollower’s start() or start_one_off() was already called or ApiException if a terminal error or unsupressed transient error is recevied from the service when fetching changes
- start_one_off() Iterator[ibmcloudant.cloudant_v1.ChangesResultItem]
Return all available changes until there are no further changes pending or reaching an end condition.
The end conditions are: - a terminal error (e.g. unauthorized client). - transient errors occur for longer than the error suppression duration. - the number of changes received reaches the limit specified in the “post_changes” args used to instantiate this ChangesFollower. - ChangesFollower’s stop() is called.
The same change may be received more than once.
Returns an iterator of ChangesResultItem per change.
Throws ValueError if ChangesFollower’s start() or start_one_off() was already called or ApiException if a terminal error or unsupressed transient error is recevied from the service when fetching changes
- stop() None
Stop this ChangesFollower.
Note that synchronous iterator blocks so this stop method needs to be called from a different thread to have any effect.
- class ibmcloudant.features.changes_follower.ChangesFollowerIterator(changes_caller, mode: ibmcloudant.features.changes_follower.Mode, error_tolerance: int)
The ChangesFollowerIterator implements iterator interface.
This class is for internal use by ChangesFollower, which provides the user facing API.
Args: changes_caller: A partial function that pulls changes feed from CouchDB for a given “since” parameter. mode: Enum representing either one-off consumption of changes feed (FINITE) or constant following the changes feed (LISTEN) error_tolerance: The duration to suppress errors, measured from the previous successful request.
Initialization
- property limit: int
- retry_delay()
Method retry_delay implements full jitter delay algorithm. This is an exponential capped backoff with added jitter to spread retry calls in case of multiple followers started simultaneously for different feeds on the same account.
The base delay is set to 100 ms and cap is set to the changes feed pull’s timeout.
Algorithm reference: https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
- property since: str
- stop() None
- ibmcloudant.features.changes_follower.EXP_RETRY_GATE = None
- ibmcloudant.features.changes_follower.FOREVER = None
- ibmcloudant.features.changes_follower.LONGPOLL_TIMEOUT = None
- ibmcloudant.features.changes_follower.MIN_CLIENT_TIMEOUT = 60000