Single Partition Data sources

What is single partition and how is it used in custom connectors

Zarah Fager avatar
Written by Zarah Fager
Updated over a week ago

Funnel keys on date

As described in this help article, Funnel use the Date dimension as its primary key when importing new data. This means that in a multi partition source the data is split up by date in the Funnel backend. In some cases the Date primary key might not make sense to use, for example when lookups are needed.

Multi partition

As seen in the table below, the rows of data are partitioned by date. This is typically how data sources work in Funnel.

Single partition

In the below table, all rows of data are in a single partition, meaning that any new data received by the data source will overwrite the existing partition.

NOTE! If you send data to Funnel, as opposed to Funnel pulling data for you, this means you must send the entire dataset each time, as the entire partition is overwritten upon import.

In Funnel, single partition sources are mainly used for lookup tables to blend meta data with data keyed on date. An example use case of this could be to join Campaign Status (a field that isn't necessarily tied to a specific date) with marketing data from a standard connector data source.

You can see if a data source is multi- or single partition by going to the Data sources page in Funnel; here multi partition sources show the imported date range and single partition sources only display the message "Data has been imported".

Did this answer your question?