Background

To get an overview of historical jobs and the latest status of your Exports to Snowflake you can create an export_summary table from your Snowflake Export in Funnel. All exports in the same Funnel Workspace will append rows to the same table. The export summary table is always written after all data has been shared and can therefore be used to trigger other steps in your pipeline or to ensure that the data in Snowflake is up to date.

The use of a summary table is enabled by checking the following option that can be found under Advanced when creating/editing the Export in Funnel.

Table Schema

Field name	Type	Description
account_id	STRING	Funnel account ID
export_id	STRING	Unique Funnel Export configuration
run_id	STRING	Unique ID for a single sharing job
total_count	INTEGER	Files exported
fail_count	INTEGER	Files that failed to be shared
success_count	INTEGER	Files that were exported successfully
status	STRING	Status of the export job, 'Success' or 'Fail'
start_at	TIMESTAMP	UTC timestamp of when the exporting job started
end_at	TIMESTAMP	UTC timestamp of when the exporting job ended
message	STRING	Status message from failed exporting job
files	ARRAY	Files in an export job
files[x].status	VARIABLE <STRING>	Status of file export, 'Success' or 'Fail'
files[x].message	VARIABLE <STRING>	Status message from a failed file export
files[x].updated	VARIABLE <BOOLEAN>	If the file changed, true or false

Example Queries

Query bytes per export run

SELECT
  run_id,
  SUM(value['file_size_bytes']) as bytes
FROM
  EXPORT_SUMMARY,
  LATERAL FLATTEN( input => files )
GROUP BY run_id;

List export run part statuses and error messages

SELECT
  run_id,
  value['status'] as status,
  value['message'] as message
FROM
  EXPORT_SUMMARY,
  LATERAL FLATTEN( input => files );

Query the number of failed/successful export jobs per day

SELECT
  DATE(end_at) as date,
  SUM(total_count) as total_total,
  SUM(success_count) as success_total,
  SUM(fail_count) as fail_total
FROM
  EXPORT_SUMMARY
GROUP BY
  1
ORDER BY
  1 DESC