Salesforce

How to Merge Einstein Analytics Data Into Salesforce Objects

By May 26, 2020 October 12th, 2020 No Comments

How to Merge Einstein
Analytics Data Into
Salesforce Objects

Einstein Analytics

Einstein Analytics has now become a popular business intelligence (BI) tool for many allowing users to analyze, aggregate and visualize them to connect the data from various platforms into a single place and explore the complex data easily and quickly.

Work with Einstein Analytics datasets into Salesforce Objects

Einstein, however, does not enable us to run reports, workflows, process flows based on the information contained in datasets as this information is not contained in salesforce fields.

Recently we have come across a requirement to merge the data from Einstein Analytics datasets into the Salesforce objects. This is to capture certain key information in salesforce custom fields to run campaigns, reports and workflows through it. 

Using SAQL

Integrating data from the datasets back into SF objects is possible but needs to be done programmatically. Following sections explain how this can be achieved using the Analytics Apex SDK:

Three ways to use SAQL in Einstein Analytics

  • SAQL
  • Analytics REST API
  • Apex class (to query the data)

SAQL Overview 

SAQL is a Salesforce Analytics Query Language that is used to access the data from Analytics datasets. You can use SAQL for multiple operations such as to create advanced dashboards, to perform data manipulations and calculations to bring it to the datasets etc. In our case, we used it to access the data from the datasets and integrate with (insert into) the Salesforce objects.

Each SAQL statement has an input stream, an operation, and an output stream. Statements can span multiple lines and must end with a semicolon. 

Example:

Einstein Analytics

You can create, modify and execute the SAQL queries in the SAQL editor in the Analytical Studio:

Einstein Analytics Studio

Analytics REST API 

Analytics features such as datasets, dashboards and lenses can be accessed programmatically using the Analytics REST API. Developers can write SAQL to access the required information/data via these Analytics REST API. 

Apex Class to Query Data from Analytics

The Analytics REST API is based on the Chatter REST API and follows its conventions. We can create a well-formed query in the Apex class with the help of the executeQuery method and query builder classes in Analytics Apex SDK. It is pretty easy to create and send a SAQL query from apex class to Analytics and get the response back in the form of JSON string.

Example:

Einstein Analytics

As we see, the executeQuery method is exposed via ConnectedApi.Wave namespace and is the safest way to create a query string for execution. You can find more information on Analytics Apex SDK here.

In our scenario, a scheduled apex job has been created to run every hour that sends the request to analytics and fetch the data from datasets in JSON format. This response is then deserialized and updated into Salesforce objects. 

Sample JSON response:

Einstein Analytics

Records node contains the actual data in the field-value pair.

Challenges

There are few challenges faced during this integration:

Large data to merge:

When a large amount of data exists in a dataset or if the response contains more than 12MB of data it fails with the heap size limit error. 

To handle this, we implemented the query ‘limit’ and ‘offset’ in SAQ to get the paginated response from the Analytics. 

Example SAQL with ‘offset’ and ‘limit’:

Einstein Analytics

This example loads the dataset, sorts the rows in alphabetical order by field specified in the order by the statement, and returns rows 51–101

Latest Version Id of the dataset:

If you notice, in all of the SAQL we pass the ‘version id’, one of the required input parameters in the query. It’s important to pass the latest version id in order for SAQL to work as expected. We can not hardcode the version id in the code as it changes every time the dataset is modified. Also, it’s cumbersome using the custom settings and manually updating the value every time the version id changes.

To overcome this challenge, we have used an API to fetch the latest version id of the dataset. So overall, two API calls are made to the Analytics, first one to get the latest version id of the dataset and second call to fetch the data from the dataset.

Following are the configurations that are done as a prerequisite to making an API call to Analytics to retrieve the dataset version id:

  1. Configured Connected App to provide secure access to REST API
  2. Created Auth Provider to grant restricted access 
  3. Named credentials (to avoid hardcoding the API key and secret) for authorization

Once the above components are configured, we can make the API call using named credentials as given below:

Einstein Analytics

The response contains the version id for the given dataset.

Einstein Analytics

Test class
:

Getting the code coverage for the apex classes with connectapi.literaljson was a bit of a challenge. We had no option but to use the seeAllData=true as most Chatter in Apex methods require access to real organization data, and fail unless used in test methods marked @IsTest(SeeAllData=true).


The above blog helps you understand Salesforce Einstein Analytics for effective business insights and how to merge the data from Einstein Analytics into Salesforce objects using REST APIs. Stay tuned for more information

 TechForce Services specialise in Salesforce implementation and work with businesses to help them utilise Salesforce and improve their current business processes. Einstein Analytics is the perfect tool for any company that wants to get more insights from its data. Add the value for your Einstein Analytics Investment. Connect with our expert today, We are happy to help!

Check out our other blogs :

How could your business improve with help from Einstein

A primer to Salesforce Einstein

References Links:
Testing ConnectApi Code
Analytics REST API Overview

Leave a Reply