There are effectively two parts to Funnel's importing of data. An initial import of historical data and ongoing downloads to keep your data up to date as time progresses.
Initial import of historical data
When you first connect a data source to Funnel, the initial sync of data typically goes back 2 years. Some source platforms limit this, in which case the initial import window will be shorter. The time this takes to complete depends on a few factors. The main ones are:
- How big the dataset is, which you can affect with the configuration of your data source. The more granular the breakdown is, the bigger the dataset.
- The response time of the API. This comes down to the nature of the source platform.
The frequency of imports vary between different platforms, where the majority import several times a day, while in a few instances imports are made once a day. This is mainly affected by two things:
- How often the source platform allows new API requests
- How often the source data is updated in the platform (example of data freshness in Google Ads)
The import frequency for a given connector is subject to change as Funnel adapts to changing quotas and rate limits from the source platforms. Funnel might also temporarily reduce download frequency if a source platform is not responding quickly enough in order to avoid building up download queues that risk choking the system and effectively halt all downloads for the connector.
As a download job is completed, Funnel queues up the next one according to the download frequency set for the connector. This means that if a download job takes 10 minutes to complete, the next download job will effectively start 1 hour and 10 minutes after the previous one was started. You will then have new data 1 hour and 10 minutes after you last received new data, assuming the second download job also took 10 minutes to complete.
Keeping up with historical changes of source data
As part of every ongoing download job, Funnel downloads a certain number of days in the past. For most source platforms, data tends to keep changing for a given period of time. Funnel looks at the typical time period for which data changes for each platform and uses this as the guideline for how many days to cover in each download. You can learn more about the reasons for historical changes to data in the article Advertising platform data mismatches.
To learn more about Funnel's take on downloading and storing data, see the article Retrieval and storage of data from each source.
If you have any specific questions about either the update frequency or amount of historical data for a certain platform, please reach out to support.