Connectors > AWS® EventBridge, Kinesis, SNS, S3
Overview
ASAPIO enables SAP systems for event-driven connectivity, with many platforms and application.
Unlike many other solutions, ASAPIO uses SAP application layer integration instead of a database-only/CDC integration.
ASAPIO Add-on detects when and which data exactly is being created or changed by SAP applications, and allows to send out that data as single messages in real-time, or even in batch-mode.
Benefits of using ASAPIO for your data and process integration with AWS and other platforms or applications:
- Native connectivity with SAP application layer
- Low-code and no-code interface creation
- Lowest performance impact
- Full control of data and workload on the SAP server
- No separate server required
Architecture
ASAPIO is an SAP Add-on, which is installed directly into the supported SAP systems and enables them for a direct connectivity with the AWS services. No middleware or iPaaS systems are required.
Getting started
The following sections explain how to connect SAP systems with ASAPIO Integration Add-on to Amazon® Web Services (AWS).
Supported AWS services
Required skills to perform the settings: familiarity with the AWS services in scope, e.g. in terms of initial configuration and use.
An Amazon® Web Services (AWS) account is required, with a subscription to at least one or more of the following services:
Service | Billable? | Mandatory? |
Amazon EventBridge | Yes | Optional |
Amazon SNS (Simple Notification Service) | Yes | Optional |
Amazon Kinesis | Yes | Optional |
Amazon S3 | Yes | Optional |
Further info on AWS services
There is extensive documentation out there to get you started on how to set-up the required Amazon services for ASAPIO. We collected some of them for your convience.
Please note that the following sections contains links to external websites of Amazon and other third parties, on whose contents we have no influence. Therefore, we cannot assume any liability for the external contents.
EventBridge
The following tutorials help you explore the features of EventBridge and how to use them.
Amazon SNS
These links help you become more familiar with Amazon SNS by showing you how to manage topics, subscriptions, and messages using the Amazon SNS console.
- Prerequisites
- Step 1: Create a topic
- Step 2: Create a subscription to the topic
- Step 3: Publish a message to the topic
- Step 4: Delete the subscription and topic
- Next steps
Amazon S3
Find help and tutorials on how to set-up S3 buckets to store data received from ASAPIO at Get Started with S3
Amazon Kinesis
ASAPIO supports connecting to Amazon Kinesis Data streams.
Please follow the Developer Guide on info how to set-up the services.
Getting-started tutorials:
- Step 1: Set Up an Account and Create an Administrator User
- Step 2: Set Up the AWS Command Line Interface (AWS CLI)
- Step 3: Create Your Starter Amazon Kinesis Data Analytics Application
- Step 4 (Optional) Edit the Schema and SQL Code Using the Console
Pre-requisites
Pre-requisites for the SAP system
Please see Installation/ for infos on how to download and install the components into your SAP systems.
Please also make sure you have the required roles and authorizations for the SAP system to configure the ASAPIO Add-on (please see Roles and authorizations)
The following ASAPIO components are required:
Add-on/component name | Type |
ASAPIO Integration Add-on – Framework | Base component (required) |
ASAPIO Integration Add-on – Connector for AWS | Additional package |
Pre-requisites for the AWS services in scope
IAM users and roles/access privileges
Warning: do not use root user, this is not required – use a dedicated and restricted user, as explained below.
Please create a dedicated IAM user for each service that is used and also create a limited policy to restrict access of that user to that single service and for publishing events.
Necessary actions to publish events are:
Service | Action |
Amazon EventBridge | events:PutEvents |
Amazon SNS (Simple Notification Service) | Publish |
Amazon Kinesis | PutRecord and PutRecords |
Amazon S3 | PutObject |
When creating users and policies please adhere to the principle of least privilege, as described at https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege.
You can find further info on access control at:
https://docs.aws.amazon.com/sns/latest/dg/sns-authentication-and-access-control.html
https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-iam.html
https://docs.aws.amazon.com/streams/latest/dev/security-best-practices.html
https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-access-control.html
Key generation
Recommendation: Create specific users in IAM without access to the AWS management console to publish events into the services that is used. This user should only be used by the SAP system to publish the configured events.
The access key id is then configured in the Default Values of the connection (also refer to Set-up authentication) and the access key secret is stored in the SAP Secure Store. Check out this link to find a guide how to use SAP Secure Store: Save Secret in Secure Store.
Data storage, sensitive data
ASAPIO does not control the kind of data which is sent as event message payload or metadata, therefore per default no sensitive data (as per definition of EU GDPR and applicable German laws and regulations) is required to be stored or transmitted.
Please make sure that topics, queues, buckets and data streams are only accessible to the users/systems of the use case.
Customer data does not have to be stored anywhere in the configuration.
Set-up connectivity
Please make sure you have the required roles and authorizations for the SAP system to configure the ASAPIO Add-on (please see Roles and_authorizations)
Typical time required to complete this configuration: 1 hour
To establish connectivity with services on AWS platform, please proceed with the following activities and refer to the specific documentation articles please.
- Create RFC destinations to Amazon platform in SAP system settings
- Set-up connection instance to Amazon platform in ASAPIO Integration Add-on
- Endpoint configuration to set additional connection parameters
- Configure example outbound message for a simple example to test the connectivity
Create RFC destinations
Create RFC destination for messaging endpoints
Create a new RFC destination of type “G” (HTTP Connection to External Server).
- Transaction: SM59
- Create new destination of type “G”
- Specify Target Host: endpoint of the service to connect to
Add the certificates for the created destinations to the certificate list selected in tab Logon & Security:
Add Certificates to Trust Store
- Transaction: STRUST
- Select Certificate List as used in RFC destination created above
- Click button Import certificate (1)
- Click button Add to Certificate List (2)
Activate BC-Set
Business Configuration sets (BC-Set) contain customizing and configuration-related table entries that are not imported with the add-on.
- Transaction: SCPR20
- BC-Set includes:
- Configuration for cloud adapter
- Configuration for cloud codepages
- Definition of IDoc segments
- Activate the BC-Set with default values: /ASADEV/ACI_BCSET_FRAMEWORK_AWS
Set-up connection instance
Add an entry for the connector to the list of cloud adapters:
- Transaction: SPRO
- Goto ASAPIO Cloud Integrator – Maintain Cloud Adapter.
- Add New Entry and specify:
- Cloud Type: name with which to reference this type of connector
- ACI Handler Class: /ASADEV/CL_ACI_AWS_HANDLER
Set-up connection instance
Create the connection instance customizing that ties together the RFC destination created earlier and the cloud connector type:
- Transaction: SPRO
- Goto ASAPIO Cloud Integrator – Connection and Replication Object Customizing
- Or go directly to transaction: /ASADEV/68000202
- Add New Entry and specify:
- Field Instance: a name for this connection in
- Field RFC Dest. (Upload): the RFC destination created for the messaging endpoint
- Field ISO Code: the code page to use
- Field Cloud Type: AWS (or the name you chose when adding the connector)
Set-up Authentication
The AWS connector uses the Signature Version 4 (SigV4) signing process for authentication.
For this process you have to specify the following values:
Default Attribute | Default Attribute Value |
---|---|
AWS_ACCESS_KEY | part of the credentials to access the service |
AWS_REGION | AWS region the service runs in |
AWS_SERVICE | which service you connect to (possible values are: sns, kinesis, s3 and events) |
Save Secret in SAP Secure Store
Enter the secret access key in the SAP Secure Store:
- Transaction: SPRO
- Goto ASAPIO Cloud Integrator – Set the cloud connection password
- Or go directly to transaction: /ASADEV/SCI_TPW
- Select the create Cloud Instance
- Enter the secret access key in the Cloud Shared Secret field and execute
Set-up outbound messaging
For outbound messaging, you can use and even combine the following methods:
- Simple Notifications
- Message Builder (Generic View Generator)
- IDoc capturing
- Custom-built triggers and extractors
Pre-requisite for all methods is to create a message type, which will be used throughout the configuration process.
The following articles explain the individual possibilities.
Create a new Message Type
Create Message Type
Note:
In the example below, we use the Sales Order (BUS2032) event.
Please use suitable data for your use-case as required.
For each object to be sent via ACI you have to create a message type:
- Transaction: WE81
- Add New Entry and specify:
- Message Type: unique name for the integration
- Description: description of the purpose
Activate Message Type
- Transaction: BD50
- Add New Entry and specify:
- Message Type: the created message type
- Active: tick the checkbox
Simple Notifications
Create Outbound Object configuration
- Transaction: SPRO
- Goto ASAPIO Cloud Integrator – Connection and Replication Object Customizing
- Or go directly to transaction: /ASADEV/68000202
- Select the created Connection
- Go to section Outbound Objects
- Add New Entry and specify:
- Object: name of the outbound configuration
- Extraction Func. Module: /ASADEV/ACI_SIMPLE_NOTIFY
- Message Type: the created message type
- Load Type: Incremental Load
- Trace: activate for testing purposes
- Response Function: /ASADEV/ACI_AWS_RESP_HANDLER
This example uses the built-in notification event.
[ps2id id=’endpoint_config’ target=”/]Set-up target endpoint in ‘Header Attributes’
Configure the endpoint to send the events to. This is different depending on the AWS service you connect to:
Amazon EventBridge endpoint
The values you can specify for the Amazon EventBridge are used for routing in the EventBridge service. They are all optional depending on your specific use case. Please refer to the EventBridge documentation.
- Go to section Header Attributes
- Add New Entry and specify the following attributes:
Header Attribute |
Header Attribute Value |
AWS_EVENTBRIDGE_DETAIL_TYPE | Optional. Describes the event |
AWS_EVENTBRIDGE_ENDPOINT_ID | Optional. Specifies an endpoint in AWS |
AWS_EVENTBRIDGE_EVENT_BUS_NAME | Optional. If not given the default event bus is used. |
AWS_EVENTBRIDGE_SOURCE | Optional. The source of the event |
Amazon SNS endpoint
- Go to section Header Attributes
- Add New Entry and specify the following attributes:
Header Attribute |
Header Attribute Value |
AWS_TOPIC | Topic to send data to |
AWS_TOPIC_OWNER | Account ID of the topic owner |
Amazon Kinesis endpoint
Note:
Only data streams are supported
- Go to section Header Attributes
- Add New Entry and specify the following attributes:
Header Attribute |
Header Attribute Value |
AWS_KINESIS_STREAM_NAME | Name of the data stream |
Amazon S3 endpoint
- Go to section Header Attributes
- Add New Entry and specify the following attributes:
Header Attribute |
Header Attribute Value |
AWS_S3_BUCKET |
Name of the S3 bucket |
Set up ‘Business Object Event Linkage’
Link the configuration of the outbound object to a Business Object event:
- Transaction: SWE2
- Add New Entry and specify:
- Object Category: BO BOR Object Type
- Object Type: the Business Object Type sending the event
- Event: the event to react to
- Receiver Type: the message type of the outbound object (this is the link to the Addon configuration)
- Receiver Call: Function Module
- Receiver Function Module: /ASADEV/ACI_EVENTS_TRIGGER
- Linkage Activated: tick the checkbox
Test the outbound event creation
In the example above, please pick any test sales order in transaction /nVA02 and force a change event, e.g. by changing the requested delivery date on header level.
Message Builder (Generic View Extractor)
The message builder is based on database views. It can extract and format data based on the configured database view. The formatting tries to nest the data based on the used database tables.
Create database view
For the data events also configure the DB view that is used to define the extraction:
- Transaction: SE11 (for SAP ERP or S/4HANA on-prem deployments with SAP GUI access)
- Alternatively, you can use Eclipse with ABAP Development Tools, or the SAP Fiori App “Create Custom CDS Views” to create a database view if you have this app available in SAP S/4HANA.
Example: Material master view (e.g. to be used for Material Master (BUS1001) change events)
Create Outbound Object configuration
- Transaction: SPRO
- Goto ASAPIO Cloud Integrator – Connection and Replication Object Customizing
- Or go directly to transaction: /ASADEV/68000202
- Select the created Connection
- Go to section Outbound Objects
- Add New Entry and specify:
- Object: name of the outbound configuration
- Extraction Func. Module: /ASADEV/ACI_GEN_VIEW_EXTRACTOR
- Message Type: the created message type
- Load Type: Incremental Load
- Trace: activate for testing purposes
- Formatting Function: /ASADEV/ACI_GEN_VIEW_FORMATTER
- Extraction View Name: name of the DB view
Set-up ‘Header Attributes’
Header attributes change based on the AWS service, please see section endpoint configuration for the correct values.
Please note the header attributes can be endpoint/platform specific. Here are examples:
Header attribute | Header attribute value | Example | required for Connector |
---|---|---|---|
S3_BUCKET | S3 bucket the data is stored in | jze-test-bucket | AWS S3 |
ACI_ADD_LOGSYS | To add the logical system to the top level of the payload.
Only works for the generic view extractors / formatters! |
X | All connectors – optional |
Set up ‘Business Object Event Linkage’
Link the configuration of the outbound object to a Business Object event:
- Transaction: SWE2
- Add New Entry and specify:
- Object Category: BO BOR Object Type
- Object Type: the Business Object Type sending the event
- Event: the event to react to
- Receiver Type: the message type of the outbound object (this is the link to the Addon configuration)
- Receiver Call: Function Module
- Receiver Function Module: /ASADEV/ACI_EVENTS_TRIGGER
- Linkage Activated: tick the checkbox
Test the outbound event creation
In the example above, please pick any test sales order in transaction /nVA02 and force a change event, e.g. by changing the requested delivery date on header level.
Set-up Packed Load (split large data)
Create Outbound Object configuration
- Transaction: SPRO
- Goto ASAPIO Cloud Integrator – Connection and Replication Object Customizing
- Or go directly to transaction: /ASADEV/68000202
- Select the created Connection
- Go to section Outbound Objects
- Add New Entry and specify:
- Object: name of the outbound configuration
- Extraction Func. Module: /ASADEV/ACI_GEN_VIEW_EXT_PACK
- Message Type: the created message type(optional)
- Load Type: Packed Load
- Trace: activate for testing purposes
- Formatting Function: /ASADEV/ACI_GEN_VIEW_FORMATTER (depending on your use case)
Create database view
Note
Please also refer to chapter 4.6.3.1
For the data events also configure the DB view that is used to define the extraction:
- Transaction: SE11 (for SAP ERP or S/4HANA on-prem deployments with SAP GUI access)
- Alternatively, you can use Eclipse with ABAP Development Tools, or the SAP Fiori App “Create Custom CDS Views” to create a database view if you have this app available in SAP S/4HANA.
Example: Material master view
Set-up ‘Header Attributes’
- Go to section Header Attributes of the outbound object created previously
- Add New Entry and specify the header attributes and values
Header attribute | Header attribute value | Example |
---|---|---|
ACI_PACK_BDCP_COMMIT | Flag for changepointer creation.
If set, changepointers will be generated for every entry. IF this flag is set, a messagetype has to be maintained in the outbound object. Caution: This may heavily impact performance. |
X |
ACI_PACK_TABLE | Name of the table to take the keyfields from. This is typically different then the db view specified in ‘ACI_VIEW‘ as we only want to build packages based on the header object and the db view typically contains sub-objects as well | MARA |
ACI_PACK_RETRY_TIME | Time in seconds. This is the duration in which the framework will attempt to get a new resource from the servergroup | 300 |
ACI_PACK_WHERE_COND | Condition that is applied to the table defined in ‘ACI_PACK_TABLE‘ | |
ACI_PACK_SIZE | Number of entries to send | 500 |
ACI_PACK_KEY_LENGTH | Length of the key to use from the ACI_PACK_TABLE (e.g. MANDT + MATNR) | 13 |
ACI_VIEW | name of a SAP database view that is key compatible with the ACI_PACK_TABLE | Z_MARM_TEST |
Execute the initial load
Warning
depending on the amount of data this can stress the SAP system servers immensely.
Please always consult with your basis team for the correct server group to use!
- Transaction: /ASADEV/ACI
- Select the Connection and hit enter
- Select Upload Type: P
- Select Replication Object
- Select a Servergroup (this is mandatory)
SNS Payload Offloading
With Payload Offloading enables you to manage payloads that is larger than the current SNS limit of 256 KB and up to a maximum of 2 GB. This is made possible by offloading the data to S3. SNS then only receives a reference to where the payload was stored.
The following header attributes must be added to the outbound object for SNS Payload Offloading.
- Go to section Header Attributes
- Add New Entry and specify the following attributes:
Header Attribute | Header Attribute Value | Example |
---|---|---|
AWS_PAYLOAD_OFFLOADING | Flag for activation of offloading | X |
AWS_PAYLOAD_OFFLOADING_SIZE | Threshold value for offloading in bytes | 140 |
AWS_S3_BUCKET | Name of the S3 bucket | |
AWS_S3_DESTINATION | RFC destination for Managed S3 endpoint | |
AWS_TOPIC | Topic to send data to | |
AWS_TOPIC_OWNER | Account ID of the topic owner |
Note: If the threshold value is not exceeded, standard SNS messaging is used.