Mapping data sources with analytics data

Businesses generate a lot of data with respect to their customers, especially if they are present online. If you use a variety of digital systems and tools to track this data, you can use analytics to comb, combine and analyse it in one place.

For example — you are an e-commerce company and your analytics tool captures data of the user’s entire journey until the order is placed online. If the user also visited the physical store and made a purchase, that data would only be stored in the CRM database. To view and analyse the user’s purchase journey across offline and online stores, it is necessary to see the unified data, presented as a single report.

Data Import

When a user visits or calls the store and provides more information such as an email id or phone number, the userID and other data attributes get stored in the CRM database. You can then export the CRM data into an excel sheet and import it into the analytics tool with the userID as the unique identifier. This allows you to analyse the combined data in the tool itself.

customer-data-analysis

Importing external data to an analytics tool can be achieved in different ways for different tools. In Google Analytics, it is accomplished via Data Import, whereas in Adobe Analytics, either Data Sources or SAINT classifications would do the job.

Data Import in Google Analytics

Data Import in Google Analytics lets you combine the data generated by your offline business systems with the online data collected by a Website/Mobile Analytics tool.
For instance, consider that you have an online education website, and there is an option for users to upgrade or enrol to a program using a call-in method. The goal now is to analyse these two streams of data at a user level.

data-import-google

The steps to be followed for data import are:

  • Create the required Custom Dimensions on Google Analytics

The data to be uploaded to Google Analytics can be stored in custom dimensions. For this, create the custom dimensions (number of fields you are uploading with those many custom dimensions) before uploading data.

  • Prepare the data set (in CSV format)

A “data set” is a container that holds the data you upload to Analytics. Data Set Schema defines the structure that joins the data you upload with the existing analytics data. You manage your uploaded data files using a data set. Data sets are associated with views, giving you flexibility regarding which data you want to import in your reports.

The schema has two important parts:

  • Key

“Key” is a dimension using which you will join the data that already exists in GA with the data you are uploading.

  • Location for uploaded data

To upload the data collected from the CRM and from users who called the call centre and subsequently enrolled, create a Data Set. The Data Set Schema would define a key, using email ID (for instance) as a dimension, and import dimensions for metadata such as courses, city, revenue, etc. You can then upload the information as needed to the relevant Data Set.

Follow these steps to define the schema and configure options that affect how imported data is connected to your property.

    • Sign in to Google Analytics
    • Click Admin and navigate to the property to which you want to upload data
    • In the PROPERTY column, click Data Import. This displays the Data Sets page
    • Click +New Data Set
    • Select the Data Set Type (for example, “Cost Data”)
    • Provide a name for the data source (for example, “Ad Network Data”)
    • Select one or more Views in which you want to see this data
    • Define the Data Set schema by selecting the Key and Target dimensions
    • Select the Overwrite behaviours
    • Click Save

Once saved, click on “Get Schema” button to get the headers of the CSV or you can download the template here.

Use the downloaded template to fill out the details

    • Upload the data to Google Analytics

There are two ways to upload data – manually, using the Analytics web interface; and programmatically via the Management API’s ‘Uploads’ resource.

    • Analyse the uploaded data in reports

Data uploaded using the custom import feature can be included in Custom Reports.
Please note, uploaded data needs to be processed before it can show up in reports. Once processing is complete, it may take up to 24 hours before the imported data begins to inform the incoming data. Also, real-time reports do not display imported dimensions.

Data Sources in Adobe Analytics

Data Sources in Adobe Analytics lets you manually import additional online or offline data for reporting.

Two methods are available to submit data:

  • FTP

You can create and manage FTP-based data sources through marketing reports, which uses FTP file transfer to import data files into Data Sources in Adobe Analytics. After creating a data source, Adobe provides you with an FTP location that you can use to upload Data Source files. Once uploaded, Data Sources automatically locates and processes them. Once processed, the data is available for marketing reports.

  • API

Adobe offers a Data Sources API that lets you programmatically link your applications to Data Sources. This eliminates the need for an intermediary FTP server, and transfers data via HTTP, SOAP, and REST.

data-sources

Please note:

    • Each FTP account is limited to 50 MB of total data for all files. Processing pauses if the size exceeds 50 MB and does not resume until it is contained below that size.
    • You can upload data for 90 unique dates. If you exceed this limit, the upload will fail with an error message stating that you have exceeded the maximum unique days.
    • Only data with current or past dates can be imported. Do not attempt to use future dates in your Data Sources data.
    • All rows must have a date specified to enable report graphing capabilities. If a row does not include a date, Data Sources generates an error and rejects the file. The date/time format varies by data source type:
      1. Full Processing Data Sources: Use the ISO 8601 date format of YYYY-MM-DDThh:mm:ss±UTC_offset (for example, 2013-09-01T12:00:00-07:00), or Unix Time Format (the number of seconds elapsed since January 1, 1970).
      2. Standard and Integration Data Sources: Use the following date format: MM/DD/YYYY/HH/mm/SS (for example, 01/01/2013/06/00/00)

Data Sources Manager

Create, manage, and view the use of data sources in a report suite (Analytics > Admin > Data Sources).

The Create tab lets you configure a new data source for the currently selected report suite. When you activate a data source, the “Data Sources” wizard guides you through the process of creating a Data Sources template and creates an FTP location for uploading data. The selection you make on the Create tab determines the initial fields in the template.