The new bulk v3 framework provided by SAP Cloud Platform Open Connectors platform includes several APIs to help streamline the creation and management of your bulk jobs. This article describes the available bulk APIs and the job criteria parameters. In addition to the information in this article, you can also find the bulk APIs in a Postman collection here.
Bulk v3 APIs
The bulk v3 framework APIs are described in the following table:
|POST /bulk/download||Initiates a bulk job to download all the records as per the given job criteria, which is specified as part of the request body. For further information on job-criteria, see the Job Criteria section.|
Provides the status of the job. The response will include the following information in a JSON format:
|GET /bulk/:id||Produces the results or data of the download job in the form of a streaming file. The default format of the file will match the specified format in the Job-Criteria; if no format is defined, the default value is JSONL format.|
|Paginates the results in the bulk file that we can download through the |
|PUT /bulk/:id/cancel||Cancels the job at any time, provided the job is not in COMPLETED or ABORTED state.|
|GET /bulk/:id/errors||Yields the errors that occurred during the job runtime.|
Continues a job where it left off, this scenario may occur if there is an outage in the downstream service, API limits are hit on the vendor side, or the apiLimit set on the bulk request was reached for that day.
The following table lists and describes the configuration parameters that can be sent as a payload to the /bulk/download API.
|objectName*||The resource name for which you want to download the records. This should match with the API supported in the connector. For example, a Salesforce Sales Cloud user wants all the accounts available in his/her account. In this case, the objectName will be |
|format*||Allows the user to choose the format that a user wishes to download. The supported values for this filed are |
|selectFields||Gives the user the ability to retrieve whichever fields they are interested in. For example, the |
|limit||Restricts the bulk job to download a certain amount of records rather downloading all records. When the job reaches a number of records equivalent to the specified limit, the job automatically stops and changes its state to COMPLETED.|
|apiLimit||Throttles the bulk job when the API call count reaches a specified value. This field is helpful when service providers offer only a certain amount of calls per job.|
|filterNulls||Removes null values from the response payload when specified true. Otherwise, no filtering will happen on the response payload for nulls.|
|from||Accepts a date as a value in ISO 8601 Date format, which is helpful to filter the records from the given date against the date field in the response payload, which can be identified through the filterDateField. In other words, the bulk job will skip the records which are identified to be modified or inserted before the date given with from. Note that if from date is given, filterDateField should also be present in the request body.|
|to||Works the other way around from field, meaning the bulk job will skip all the records which are identified to be modified or inserted after the date given with from. Note that if to date is given, filterDateField should be present in the request body.|
|filterDateField||Required when from or to dates are given, a conditionally required field with from and to. This filed helps the bulk job identify which field to compare with when from or to date is given.|
|notificationUrl||Accepts the notification URL at which the user will be notified when the bulk job is complete.|
|where||A well-known typical OCNQL query for any SEARCH API. This can help improve the performance of the job and directly apply the filtering to the service provider.|
|query||Any other API related and supported filters (query params) will be accepted through this field, which is of type JSONObject (Map).|
|pageSize||The default value is 200 records.|
If a user wants to upload their file to a documentation hub once the job completed, we use this field to automatically collect some information about the documentation hub and streamline the process.
This feature is supported through our documentation hub connectors, eg Google Drive, Dropbox, Box, etc. This field accepts a JSON map as its value with the following properties:
This flag helps to initiate a bulk download job with respect to another job that was completed successfully. This property accepts another bulk job id as the value and when submitted, the new job considers the context of the old job (that will be identified with the given id) and starts downlaod data from then.