Overview
The Data Warehouse S3 Connector automatically syncs all the data in your Funnel account to Amazon S3. All you have to do is connect your ad accounts to Funnel, as usual, set up an S3 bucket with write access for Funnel, and enable the upload.
This can be useful as a way to get programmatic access to the cost data from your ad platforms or serve as a way of importing the data to QuickSight or another part of your Data Warehouse pipeline.
Schema
The Data Warehouse connector allows you to pick and choose whichever dimensions and metrics you like. Any changes to your Funnel account or the fields available to the connected sources will not be reflected in your Shares until you manually make a change to the set of selected fields. Once your Share has run you'll find a number of files in your bucket. Most notably you'll see one file per month for the date range you have selected, e.g. funnel_data_2018_01
, funnel_data_2018_02
and so on.
Along with these, you'll find a file named schema.funnel_data_{startYYYY}_{startMM}.sql, containing the CREATE TABLE
syntax for a table matching your configured Share.
โ
The files will contain the fields in the same order they are configured in the Share.
Formatting
The Data Warehouse Share allows for some configurability in terms of file format and formatting of the data. This also includes settings for different types of header rows within each file and the formatting of metric values.ย
Requirements
To get started you will need to prepare an S3 bucket in your AWS account. More information on this process can be found in the article Amazon S3 bucket configuration.
Data Share configuration
In Funnel open your Account, select "Data Warehouse" in the left-hand side navigation and click the "New Share" button to set up a new Share.
Choose Amazon S3 as the destination for your Share and enter a name.
Enter the details of your bucket, the path to where you wish to share your data and a file name template to use when naming the shared files.
Once you have filled in your details it may be useful to verify that everything is correctly set up by clicking the Test access button. If there are any connection errors, please refer to the article Amazon S3 bucket configuration.
The next step is selecting a file format for your shared files. If you are planning on importing the data to Redshift, suitable options for your S3 Share would be:
gzip turned on
file format TSV
metric format Export
headers None
Once you have selected a file format you should click the Choose fields button in order to select which dimensions and metrics to include in your Share. This step will also let you specify if you want monetary values currency converted into a single currency or use the original currency they are reported in, as well as specify a schedule for when to run your Share.
Verify the bucket
Once you have saved your configuration you will need to verify the bucket is correct.
When you click the 'Verify...' button shown below we will create a file containing a token and upload it to your bucket.
Follow the link in the dialog, open the file, and copy the contents into the box:
Enable the Data Share
Once everything is set up you can review your configuration.
When everything looks good you need to toggle the Enable switch in the top right-hand corner.
The Share will run according to the schedule you specified.
In case you wish to verify that everything works as intended you can click the Run now button.