Achieving Seamlessness: Master Data Upload in S/4HANA Cloud

Introduction

Resource utilization is an important aspect of SAP IPAAS (Integration Suite). Since we need to work with a finite amount of processing speed and memory, we need to take this into consideration when developing new interface.

SAP S/4HANA Cloud Sales Certification Preparation Guide

Here I will present an alternate way of accomplishing a simple task of master data upload to S/4HANA Cloud using the combination of both CI and OC. So instead of building the entire solution in CI, we are spreading the load to different products of integration suite – for optimal load distribution. Also since we are combining the capacities of different products, it may take less effort to build solution to complex problem statement.

Problem Statement

We need to upload Business Partner from a flat file to S/4HANA cloud.
We need to correlate the result of the post to every record of the file.
We need to send the correlation result in email.

Solution

Cloud Integration
- Poll the file from SFTP server.
- Convert CSV to JSON format.
- Push the content to Open Connector bulk activated endpoint.
Open Connector
- Load the file from the staging area and push BUPA to S/4HANA cloud.
- Get the status of the processing and send email.

As we see above – we distributed the task between CI and OC – and are using suitable capability of the products to achieve the solution.

Before development – assume following pre-condition task are completed in SAP S/4HANA Cloud:

Creation of communication system.
Creation of communication user.
Creation of communication arrangement SAP_COM_0008.

Create new Cloud Element for SAP S/4HANA Cloud in Open Connector

Create a new element for the SAP S/4HANA Cloud Business Partner – API. Download the EDMX from api.sap.com and import the same. Import A_BusinessPartner GET/POST operations, also provide some cool logo.

Provide the tenant details and the authentication credentials.

Now authenticate this element and create a connector instance.

Create resources for the newly created cloud element

All the POST operation on API’s of SAP S/4HANA Cloud is protected by X-CSRF-Token. We need to create a new resource to fetch the token. Create a new /token endpoint.

The idea is to get the token and the cookie in the response body. For that we need to create a post hook to replace the $metadata response with a custom body.

let token = response_headers['x-csrf-token'];
let cookie = response_headers['set-cookie'];
done({
  response_body: {
    'token': token,
    'cookie': cookie[1]
  },
  continue: true
});

Now if we test this /token API, we get the following response.

Update the POST A_BusinessPartner resource to include /token API for fetching the CSRF token before BUPA POST

Now since we have created the endpoint to get the CSRF token, we merely need to call it to obtain the tokens. These token will be then passed over to the actual A_BusinessPartner API for successful post. We can write this logic in the pre-hook.

const https = require('https');
let u = '?;
let o = '?';
let e = '?';

var options = {
    hostname: 'api.openconnectors.eu10.ext.hana.ondemand.com',
    port: 443,
    path: '/elements/api-v2/token',
    method: 'GET',
    headers: {
        "Authorization": `User ${u}, Organization ${o}, Element ${e}`
    }
};

//Get SFDC connector from CE and return the results
https.get(options, (res) => {
    let rawData = '';
    res.on('data', (chunk) => rawData += chunk);
    res.on('end', () => {
        console.log(rawData);
        try {
            let parsedData = JSON.parse(rawData);
            let v = request_vendor_headers;
            v['x-csrf-Token'] = parsedData.token;
            v['cookie'] = parsedData.cookie;
            done({ "request_vendor_headers": v });
        } catch (e) {
            done({ "response_error": e.message });
        }
    });
    res.on('error', (e) => {
        done({ "response_body": e.message });
    });
});

replace ? with actual credentials.

We have now accomplished the task to POST new BUPA to SAP S/4HANA Cloud. Now we need to activate bulk on this API.

Create Formula which will get the status of the BUPA POST and inform the results over email

Next we need to create the formula which will be triggered once all the BUPA are posted in SAP S/4HANA Cloud. Within this formula, we will check if there are any errors in the posting, if yes, then an email will be sent to designated person.

Following code to check if there are any errors.

let d = steps.GetStatus.response.body.length;
if(d > 0){
  return done(true);
}
else{
  return done(false);
}

Following code to format the email body and send email.

let m = '<table border="1" style="border-collapse: collapse;" cellpadding="5">';
let d = steps.GetStatus.response.body;
d.forEach((row) => {
  m = m + '<tr><td>Line</td><td>' + row.rowNum + '</td></tr>';
  m = m + '<tr><td>Error</td><td>' + row.status + '</td></tr>';
  m = m + '<tr><td>File Row</td><td>' + row.response + '</td></tr>';
})
m = m + '</table>';
notify.email('test@gmail.com',
  'BUPA Replication',
  m);
done(true);

Create new IFLOW in Cloud Integration

The IFLOW is now going to be very simple. For the sake of simplicity, we have not included the SFTP adapter. As of the day of writing this blog – open connector receiver adapter is unable to convert to multipart/form-data – which is the expected format of the /bulk API, so HTTP adapter is being used.

Important thing to note is that – while we call the /bulk endpoint – we need to also specify the URL that will be called upon completion of the bulk task. This URL is the API endpoint of the formula that we created above.

Message header:

Content-Type

multipart/form-data; boundary=—-abc

Authorization

User ?, Organization ?, Element ?

(credential of S/4HANA Cloud element)

Elements-Async-Callback-Url

https://api.openconnectors.eu10.ext.hana.ondemand.com/elements/api-v2/formulas/instances/41574/executions

Sample Payload:

[
  { "LastName": "Bhowmick", //will succeed
    "CorrespondenceLanguage": "EN", 
    "FirstName": "Deepankar",
    "BusinessPartnerGrouping": "BPEE",
    "BusinessPartnerCategory": "1"
  },
  {
    "LastName": "LastThree", //will error
    "CorrespondenceLanguage": "EN",
    "FirstName": "FirstThree",
    "BusinessPartnerGrouping": "BPEE",
    "BusinessPartnerCategory": 9
  },
  {
    "LastName": "LastThree", //will error
    "CorrespondenceLanguage": "EN",
    "FirstName": "FirstThree",
    "BusinessPartnerGrouping": "BPEE",
    "BusinessPartnerCategory": 9
  }
]

Script to manually prepare the multipart/form-data format.

import com.sap.gateway.ip.core.customdev.util.Message
import java.util.HashMap
import com.sap.gateway.ip.core.customdev.util.Message
import org.apache.camel.impl.DefaultAttachment
import javax.mail.util.ByteArrayDataSource
def Message processData(Message message) {
    String msg = """------abc\r\n""" + 
                """Content-Disposition: form-data; name="file"; filename="hello.json"\r\n""" +
                """Content-Type: application/json\r\n\r\n""" +
                """${message.getBody(java.lang.String).replace('\n','\r\n')}\r\n""" +
                """------abc--"""
    message.setBody(msg)
    return message
}

HTTP receiver adapter.

Testing

After executing the IFLOW, we should have a BUPA created for “Deepankar Bhowmick”. The rest 2 should appear in the email with correlation.

BUPA created successfully.

Error email notification with correlation (which file record has what error).

Key Takeaways

Simple implementation of a relatively complex requirement. No fancy $batch mapping required.

Execution time in CI is ~100ms for even large payloads, if we implement the entire logic in CI alone, then execution time could be in minutes. Performance improvement in CI is > 99%.

We created SWAGGER complaint API on top of A_BusinessPartner in OC, this is a scalable solution. We can only improve from here.

Optimal resource utilization of Integration Suite. Load is distributed to several Integration Suite products.