1. Overview
Funnel's BigQuery connector lets you share your Funnel data to a BigQuery Dataset of your choice. This guide will cover what you need to do in your Google Cloud Console in order for Funnel to be able to share data there.
Before you get started you need a Google Cloud Project with billing enabled https://console.cloud.google.com/billing
2. Authentication
When authenticating you can select either a Google account (via OAuth) or a service account. The first option is recommended unless you don't want to tie the authentication to one user.
2.1 Using a Google Account
The Google user that you use to authenticate with needs access to Google Cloud Storage (GCS) and BigQuery and billing needs to be enabled in your Google project. The reason why we need GCS access as well is that we first put the files in a bucket that we create, transfer the files from there to BigQuery and then delete the files.
Required scopes
bigquery.datasets.createbigquery.jobs.createstorage.buckets.createstorage.buckets.getstorage.objects.createstorage.objects.delete
Optional scopes
bigquery.datasets.delete- A test table will be created before every export to validate access, this will allow us to delete it afterwards.
2.2 Using a Service account
When setting up a Share it will list what service account you give access to, it depends on what instance of Funnel you are using.
App URL | Service account |
app.funnel.io | |
app.eu.funnel.io |
Grant our Service account access to your Google Cloud Project
Go to https://console.cloud.google.com/iam-admin/iam/ and select your project. Add the Funnel service account from the table above with the role "BigQuery User".
3. Configure a BigQuery dataset
3.1 Let Funnel create the dataset
Funnel can create a dataset for you, this is the default behaviour.
3.1.1 BigQuery Dataset Location
For the BigQuery dataset location you can either select US (multi-region) or EU (multi-region). If you want your data in another location follow the step 3.2 to create your own dataset and give Funnel permissions to export to it.
3.2 Give service account access to your own dataset
If you choose to manage the dataset yourself and are using the service account authentication you need to grant Funnel access to it.
Go to https://bigquery.cloud.google.com and create a new dataset.
After creating the dataset, select the dataset and press "Share dataset".
Add the Funnel service account found in the table above with the "BigQuery Data Owner" role.
All done!
Having completed the two steps above you can now configure your Funnel Share to send data to this BigQuery Dataset. Do this from within your Funnel account.