LinkedIn

Send Data from Salesforce to Data Cloud using Ingestion API and Flow

-

As part of this blog post, we are going to see a Sprint 24 feature - Send Data to Data Cloud using Flows and Ingestion API. The release note is available here.



 Introduction

As we all know, Data Cloud helps us to build a unique view of customers by harmonizing data from multiple source systems and coming up with meaningful data segmentation and insights that can be used in different platforms for additional processing. 

If you are new to Data Cloud refer to Salesforce documentation or these videos that are part of Salesforce Developers youtube channel.

What is Ingestion API

As per Salesforce documentation, Ingestion API is a REST API and offers two interaction patterns: bulk and streaming. The streaming pattern accepts incremental updates to a dataset as those changes are captured, while the bulk pattern accepts CSV files in cases where data syncs occur periodically. The same data stream can accept data from the streaming and the bulk interaction.
This Ingestion API will be helpful to send interaction-based data to Data Cloud in near real-time and
 this is getting processed every 15 minutes.

Use Case

When external users are submitting donations from a public site, the donation details need to be sent to Data Cloud for segmentation and creating insights.

Implementation Details

Let us see the details:

1. Create a Donation custom Object

We can create a custom object in Salesforce with minimum details like: 

  • Name 
  • Category
  • Donation Amount

2. Create a screen flow to input donation details

Add screen elements to input the above 3 variables and create a record once the values are entered.




3. Add this flow component to a Public site/ Salesforce page

Normally donations are executed from the experience cloud site. But for test purposes let us add this to the internal home page.




4. Create Ingestion API Connector and upload the Schema

From Data Cloud Set up screen, create a new Ingestion API Connector

  • Click on New Button from Data Cloud setup->Ingestion API Connector


  • Input API name



  • Upload Schema of Ingestion API. For this, we need to give the file in yaml format following open API format. Sample Schema file:


  • Click on Upload Schema file

  • Select the file

  • Preview schema and save it




5. Create Data Stream from Ingestion API

Data streams in the Data Cloud allow you to ingest data to the Data Cloud from Salesforce and other data sources. To ingest data to the Data Cloud using the Donation_Ingestion API that we just created, a data stream needs to be defined as shown below:

  • Create new Data Stream 
Go to Data Cloud from Appa Launcher and open Data Streams Tab. Click on new button:




  • Select the source for the Data Stream. In our case it is Ingestion API


  • Select the newly created Donation Ingestion API






  • Configure the selected object and field details as shown below:
Here choose the Data Category as "Engagement" and primary key as donation_id which is getting mapped to the standard name field of Donation that is configured as auto number and map the Event Time field to the created date.



  • Deploy the Data Stream

  • The final output of the Data Stream will be a new Data Lake object as shown below:


6. Do Data mapping and create a Data Model Object

For additional processing of these ingested data, they need to be mapped to Data Model objects in Data Cloud. A single Data Lake Object can be mapped to one or more standard/custom Data model objects. After that, all additional processing can be applied to this data mapped to Data Model Objects.
In our case, we are going to create a custom Data Model Object and do the mapping.

  • Start Data Mapping and click on Select Objects

  • Select the Custom Data Model Tab and click on the new Custom Object

  • Select the fields to be created in the new DM Object from Data Lake


  • The mapping looks like the below and save it


  • In the Data Model Object Tab, we can see the newly created Object


7. Modify the flow to send the new Donation record to Data Cloud

As per the Sprint 24 feature, once an Ingestion API connector has been successfully created, it will be available by default as an action in the Screen flow/ record triggered flow.
In record triggered flow this action can be added using the scheduled path only, since this is a callout operation

  • Modify flow to retrieve new Donation record name and created date


  • Add a new Action in the screen flow and select "Send to Data Cloud" from the category


  • Newly created Ingestion API - Donation Ingestion, will be available by default


  • Do all the input variable mapping

  • Screen flow got updated successfully, save and activate it

8. Test the flow Action

  • Open the screen flow and enter the details

  • Record created successfully

  • In Data Cloud App, Go to Data Explorer Tab and check Data Lake Object - Donation. We can see that the new record created in Salesforce got synched with Data Cloud now:

  • Similarly, check the corresponding Data Model Object, and we can see the data is synched there also:

Thus with this Sprint 24 capability Data sync from Salesforce flow to the Data Cloud has been made very easy.

Comments

Post a Comment

Popular posts from this blog

Subscribing to Salesforce Platform Events using External Java Client - CometD

-
Image
Platform Events follow event based Architecture, where multiple subscribers can subscribe to a published event in real time manner.  Below diagram shows the basic Architecture of the event-based communication Model: Event - This is a change in state for a business process Event Message - Data about the event Producer - Publisher of the event Event Bus - A communication and storage service for published events Event Consumer - A subscriber to the channel Platform events can be subscribed internally and also externally Platfrom events  - Internal Subscription Below are some scenarios in which you can consume platform events internally: 1. When we need to execute functionalities asynchronously to save govornor limits 2. To build a proper exception logging framework - You can find more details here  How to subscribe Internally? we can see more details about this in a different blogpost. Platfrom events  - External Consumption Many Organizations are moving to event-based integration appr
Read more

Salesforce Security - Restriction Rules and Scoping Rules

-
Image
1. Overview Salesforce provides a very effective security model where we always start with the most restrictive access and then open up access using additional mechanisms like sharing rules, and manual sharing based on the requirements.   But there might be scenarios where we need to restrict access based on specific criteria. Salesforce recently introduced 2 approaches to support this requirement. Those are restriction rules and scoping rules. Let us see this in detail. 2. Restriction Rules - Overview Using restriction rules we can apply an additional level of filter on top of records to which a specific user is having access. 3. Scoping Rules - Overview Scoping rules help to filter the default records visible for a user based on specific criteria. But it is not preventing access to other records. 4. Compare Restriction Rules and Scoping Rules 5. Restriction Rule - Example Company ABC is having Job Applications getting created through their company portal and synched to Salesforce. T
Read more

How to develop reusable Invocable Apex methods for Flows

-
Image
 Salesforce is adding many new features and capabilities to Flows. But there are still some scenarios where you need to call apex methods from Flows to execute complex operations - when Flow is not capable of doing that or the performance will be low when you implement that using flows. Let us see how we can build reusable apex methods for Flows. For this let us consider 2 sample scenarios: Scenario 1 - Quick Create Account/Contact I want to build a screen, where the user can select the object needs to be created, and then create a record of that object by entering the required details. Also if the record creation is getting failed, I want to create an exception record with details. Solution Let us see how we can utilize Apex action for this. For this first let us create an Apex Invocable method, which can accept a list of wrapper class as shown below: You can see, this invocable method is accepting a List of wrapper class variables and also returning a list of wrapper class variables.
Read more