Introduction: This document describes how to fetch data in chunks using pagination from S/4 Hana cloud’s OData API.
Let’s understand pagination first,
In the context of OData (Open Data Protocol), pagination refers to the practice of dividing a large set of data into smaller, more manageable chunks or pages. This is done to improve the performance of data retrieval and to reduce the amount of data transferred over the network. Pagination is a common technique in APIs and web services to handle large result sets efficiently.
In OData, pagination is typically achieved through the use of query parameters, specifically the $skip and $top parameters. Here’s a brief explanation of these parameters:
- $skip: This parameter is used to specify the number of items that should be skipped from the beginning of the result set. It is often used in conjunction with $top to implement paging. For example, if you want to retrieve results 11 to 20, you would set $skip=10.
- $top: This parameter is used to specify the maximum number of items to be returned in the result set. It works in conjunction with $skip to define the size of each page. For example, if you want to retrieve the first 10 items, you would set $top=0.
- $inlinecount: This parameter is used to get total count of records by passing value “allpages” to this parameter.
By using $skip,$top and $inlinecount together, you can navigate through the result set in chunks, effectively implementing pagination.
For instance, to retrieve the second page of results (items 501-600), you might use $top=100 and $skip=500.
Now, you have a scenario where you need to fetch the full load from the Business Partner API or any other API of S/4 Hana public cloud with 500 records per page.
Here, $inlinecount parameter will come in the picture.
First get call to the API with https://<S4hanaAPIHostName>:<Port>/API_BUSINESS_PARTNER/A_BusinessPartner?$top=500&$skip=0$inlinecount=allpages
in response, you will get 500 record if records are equal to 500 or more else full load will come in first page it self.
Along with these records, you will also get count of total records in “count” element
if (count>500)
then calculate the number of API call you need to make based on total count
After every API call, Increase the value of skip by 500.
To do all this, you need to write a program in language supported by you application or middleware system.
Let’s call this API in CPI with Pagination
In CPI, OData adapter can manage all parameters, we need to just manage the number of times API should be called by “looping process call” artifact.
As you can see in the below picture, fetching data from S/4 Hana, transform the data in Salesforce Soap API format and transferring to Salesforce system is happing in “Local Integration Process 1”
And Integration Process is only use to call “Local Integration Process 1” in loop.
In OData adapter, just tick “Process in Pages” and enter the total number or records should be called in a single call in “Page size”.
Every time when API call fetches the data, a property “${property.<RECEIVER>.<Channel>.hasMoreRecords} ” is set with true if there are still records to fetch from S/4 Hana or set with false if no records are left to fetch
This property we need to set in Lopping Process Call to give the condition for stopping loop
Receiver: S4Hana
Channel: ODataS4
Condition in Lopping Process Call: ${property.S4Hana.ODataS4.hasMoreRecords} contains ‘true’
Conclusion: After reading this document, you can easily use OData API with pagination and how to use it in CPI.
