The Brand
As one of the largest, global information technology services enterprises, the brand provides business consulting, information technology management, and process outsourcing services. Its B2B marketing strategy relies heavily on the lead generation and funnel conversion processes.
The Opportunity
For marketing to be effective during mid-funnel nurturing and opportunity specific campaigns, the brand needed to understand which content is being downloaded and by whom. This requirement had two fronts:
- Understand how and when assets are downloaded from their website on an hourly basis using Adobe Analytics.
- Take log level information from Adobe and process it through Demandbase API for firmographic enrichment. This is how they would learn and act upon the Who part of the download metrics.
The business process that the company tried to deploy required hourly batch files extracted from Adobe to be processed and enriched in real-time using Demandbase API. This was then uploaded back into Adobe for decision-making and sales follow-ups.
The Challenge
The Demandbase API available to meet such a requirement had limit on the number of records it could be enriched per call. This limit was breached almost every time. So, this processing required careful record metering enrichment with successive API calls.
A similar constraint existed in Adobe API for back upload of enriched records, and again, careful metering was needed. Due to metering, the two connectors needed a validation stage to ensure all the required records were enriched and uploaded back into Adobe.
Nabler’s Response
Three Stage Solution
Data extraction was achieved using S3 watcher service that checks for the weblog file drops into S3 bucket with a particular pattern. Once the condition is met, the files are read, and the data is stored in RDS database.
- Data transformation to enrich the firmographic data to the raw data of the web log files primarily involved Demand base API calls. Transformation also involved unification of the data based on the two identifiers. Enriched column for reach record is then updated back to database.
- Load data back into Adobe with Adobe Datasource upload connector. This uploads the enriched data back to the Adobe console with required fields for tracking.
- The limitations from API were handled with batched data pulls and uploads to API. Jobs are scheduled on an hourly basis to ensure the all the requests per day are processed, enriched, and uploaded back to Adobe for further reporting.
A total of three connectors were built: Demandbase, S3 watcher-based file upload, and Adobe.
Architecture Components
- nPrep Platform: nPrep is Nabler’s proprietary data preparation platform. Built in Java, it supports built-in connectors for data extraction, job scheduling capability, and data availability in popular databases like MYSQL and MSSQL with advanced logging for job run traceability and alerts
- S3 Buckets: The weblog files were uploaded by the brand’s IT team. This mechanism was chosen because of the need for secure file drops within the AWS environment, and certain company level policy restrictions. Dropped hourly files were approximately 100 -500MB size in compressed formats.
- S3 Watcher: This nPrep platform service was built to poll — or watch — the S3 buckets for files with pre-defined patterns. Once the pattern files are found, they are downloaded into the nPrep platform for further processing.
- RDS MYSQL Database: RDS MYSQL Server is a centralized database for storing all their marketing data. Based on their current data sources, sizes, and types, RDS was the optimal and most cost-effective managed service available.
- Connectors: The Demandbase connector was built to enrich the weblog file data with company specific information based on unique identifier mapping. Adobe Datasource Upload connector was built to upload the data back to the Adobe reporting API.
Creating Data Pipeline and Scheduling
Connectors were scheduled to read the data from files dropped in S3 and then store the data into MYSQL database. Data enrichment from Demandbase API was handled at the database level. The enriched data was then pushed to the Adobe Reporting API. See the final data set sample in the chart to the right.
Outcome
The solution was successfully deployed and has been running for more than a year. Today, it is integrated into the brand’s key mid-funnel sales and marketing processes. Nabler is the agency of record maintaining the solution.