All Collections
Data Connectors
Error messages and troubleshooting
Custom reports in Core Connectors - Troubleshooting
Custom reports in Core Connectors - Troubleshooting

Learn how to avoid problems with data sources based on custom reports

Viktor Tell avatar
Written by Viktor Tell
Updated over a week ago

For some Core Connectors, Funnel offers the ability to pull data from custom defined reports. This lets you customize precisely what fields you want to pull into Funnel. Examples of connectors with this capability are DoubleClick Bid Manager and DoubleClick Campaign Manager.

Changes to the custom report

When connecting a source in Funnel we download up to 2 years of historical data. The historical data will contain the dimensions and metrics that were included in the custom report at the time of connecting the source. Changes that are made to the custom report, in the platform it was created in, will not automatically update the data in Funnel.

Solution

To solve this requires you to write in to support and explaining that your source requires a refresh of its data due to changes made to the custom report

Data size

A common problem we see with our connectors is that of data overload due to too big requests in custom reports. These problems are due to volume quotas enforced by the platform Funnel is integrating with. 

In order to avoid this problem it is useful to understand how to avoid requesting huge data sets in custom reports. These tend to be the result of combining multiple dimensions in the report. Let's take a look at how adding dimensions expands the volume of data.

Consider following fields in a custom report: 

  • Date, e.g.  2019-01-01 or 2019-01-02

  • Country, e.g. "Sweden" or "Norway"

  • Campaign, e.g. "Black Friday" or "Valentines day"

With three different dimensions where each dimension could alter between two different values it would results in total 8 unique combinations (2*2*2) as per the below table.

If we expand the fields by adding "gender" to the custom report the number of rows would double (2*2*2*2 = 16 rows): 

If we increase the interval from two to four days and adding two countries to the Country dimension the number of rows grows from 16 to 64 (4*4*2*2).

The point is that requesting many different dimensions and the values for those results in a substantially bigger data set. 

For this particular example the number of rows grows from 8 to 64 by adding two days and four additional breakdowns ("Finland", "Iceland", "Male", "Female") to the data set.

In real world examples the data sets are obviously much larger and the impact from multiple dimensions substantially higher. The main point, however, is that as more dimensions get added to a query, the amount of rows grow exponentially and eventually lead to quota issues. Custom reports make this extra challenging as Funnel has no control over how the dataset gets constructed; it only gets a report ID and attempts to download the corresponding dataset.

Solution

We keep investing in better tools for quickly diagnosing these kinds of problems and finding ways of working around platform quotas. Until all problems are engineered away, however, you can get around issues with data sources based on custom reports by eliminating dimensions that have lesser impact on your analysis.

Did this answer your question?