1. Overview
Funnel's BigQuery connector lets you share your Funnel data to a BigQuery Dataset of your choice. This guide will cover what you need to do in your Google Cloud Console in order for Funnel to be able to share data there.
Before you get started you need a Google Cloud Project with billing enabled https://console.cloud.google.com/billing
2. Authentication
When authenticating you can select either a Google account (via OAuth) or a service account. The first option is recommended unless you don't want to tie the authentication to one user.
2.1 Using a Google Account
The Google user that you use to authenticate with needs access to Google Cloud Storage (GCS) and BigQuery and billing needs to be enabled in your Google project. The reason why we need GCS access as well is that we first put the files in a bucket that we create, transfer the files from there to BigQuery and then delete the files.
Required scopes
bigquery.datasets.createbigquery.datasets.getbigquery.tables.createbigquery.jobs.createstorage.buckets.createstorage.buckets.getstorage.objects.createstorage.objects.delete
Optional scopes
bigquery.tables.delete
A test table might be created before the export to validate access, this will allow us to delete it afterwards.
Also when doing certain changes like removing date from the Data Share or changing all fields at once, it's required by us to delete the table and create a new one. For these auto-fix capabilities to work we require this scope, but it is also possible to delete the table manually and click "Run job" to fix all of these issues.
2.2 Using our Service account
When setting up a Share it will list what service account you give access to, it depends on what instance of Funnel you are using.
App URL | Service account |
app.funnel.io | |
app.eu.funnel.io |
Grant our Service account access to your Google Cloud Project
Go to https://console.cloud.google.com/iam-admin/iam/ and select your project. Add the Funnel service account from the table above with the role BigQuery Data Editor.
3. Configure a BigQuery dataset
3.1 Let Funnel create the dataset
Funnel can create a dataset for you, this is the default behaviour.
3.1.1 BigQuery Dataset Location
For the BigQuery dataset location you can either select US (multi-region) or EU (multi-region). This can be selected under "Advanced settings" when setting up a Data Share. If you want your data in another location follow the step 3.2 to create your own dataset and give Funnel permissions to export to it.
3.2 Give service account access to your own dataset
If you choose to manage the dataset yourself and are using the service account authentication you need to grant Funnel access to the dataset.
To grant the access required add the Funnel service account found in the table above to the project with the role "BigQuery Job User" and on the dataset add the role "BigQuery Data Owner" to the service account.
3.2.1 How to grant BigQuery Data Owner on the dataset
Go to the dataset you want to export to and press "Sharing" then "Permissions".
Press Add principal then add the service account with the role "BigQuery Data Owner"
3.2.2 Domain Restrictions (optional)
When using service account and if you have domain restrictions enabled in GCP then below is the organization and workspace ID's for the service accounts used in Funnel Shares.
Organization resource ID | 1033907189250 |
Google Workspace customer ID | C03lzhzjd |
All done!
Having completed the two steps above you can now configure your Funnel Share to send data to this BigQuery Dataset. Do this from within your Funnel account.