This was a collaboration between Setfive and Data Driven. The end customer was a client of Data Driven's that we'll refer to as WF. WF is an affiliate marketing company which owns and operates about a dozen different website properties. WF works with advertisers to match users with offers and discounts as users browse WF sites. WF does this by combining behavioral, intent, and contextual data about their users and sites. For historical reasons, WF selected Elasticsearch as their primary datastore to collect and record their user data. Unfortunately, as WF started servicing more advertisers and users, they began struggling to use Elasticsearch for their reporting needs.
WF met with Data Driven and Setfive to discuss the difficulties of using Elasticsearch and the business risks around relying on a third party for revenue reports. After this discussion, Data Driven and Setfive came up with a twofold solution. First, we would create an Extract Transform Load (ETL) pipeline to regularly move the data from Elasticsearch and into Amazon Redshift. The second step was to make the data in Redshift easy to query by members of the WF business team. To accomplish this, we setup and configured Looker to connect and query the WF Redshift cluster.
This collaboration was a success and WF continues to use Looker today.