For some connectors, Funnel offers the ability to pull data from custom defined reports. This lets you customize precisely what fields you want to pull into Funnel. Examples of connectors with this capability are DoubleClick Bid Manager and DoubleClick Campaign Manager.
A common problem we see with our connectors is that of data overload due to too big requests in custom reports. These problems are due to volume quotas enforced by the platform Funnel is integrating with.
In order to avoid this problem it is useful to understand how to avoid requesting huge data sets in custom reports. These tend to be the result of combining multiple dimensions in the report. Let's take a look at how adding dimensions expands the volume of data.
Consider following fields in a custom report:
- Date, e.g. 2019-01-01 or 2019-01-02
- Country, e.g. "Sweden" or "Norway"
- Campaign, e.g. "Black Friday" or "Valentines day"
With three different dimensions where each dimension could alter between two different values it would results in total 8 unique combinations (2*2*2) as per the below table.
If we expand the fields by adding "gender" to the custom report the number of rows would double (2*2*2*2 = 16 rows):
If we increase the interval from two to four days and adding two countries to the Country dimension the number of rows grows from 16 to 64 (4*4*2*2).
The point is that requesting many different dimensions and the values for those results in a substantially bigger data set.
For this particular example the number of rows grows from 8 to 64 by adding two days and four additional breakdowns ("Finland", "Iceland", "Male", "Female") to the data set.
In real world examples the data sets are obviously much larger and the impact from multiple dimensions substantially higher. The main point, however, is that as more dimensions get added to a query, the amount of rows grow exponentially and eventually lead to quota issues. Custom reports make this extra challenging as Funnel has no control over how the dataset gets constructed; it only gets a report ID and attempts to download the corresponding dataset.
We keep investing in better tools for quickly diagnosing these kinds of problems and finding ways of working around platform quotas. Until all problems are engineered away, however, you can get around issues with data sources based on custom reports by eliminating dimensions that have lesser impact on your analysis.