Background

To be able to get an overview of historical jobs and the latest status of your Data Shares to Snowflake you can create an export_summary table from your Snowflake Data Share in Funnel. All Data Shares in the same Funnel Workspace will append rows to the same table. The export summary table is always written after all data has been shared and can therefore be used to trigger other steps in your pipeline or to make sure that the data in Snowflake is up to date.

The use of a summary table is enabled by checking the following option that can be found under Advanced when creating/editing the Share in Funnel.

Table Schema

Field name	Type	Description
account_id	STRING	Funnel account ID
export_id	STRING	Unique Funnel Share configuration
run_id	STRING	Unique ID for a single sharing job
total_count	INTEGER	Files shared
fail_count	INTEGER	Files that failed to be shared
success_count	INTEGER	Files that were shared successfully
status	STRING	Status of the sharing job, 'Success' or 'Fail'
start_at	TIMESTAMP	UTC timestamp of when the sharing job started
end_at	TIMESTAMP	UTC timestamp of when the sharing job ended
message	STRING	Status message from failed sharing job
files	ARRAY	Files in a sharing job
files[x].status	VARIABLE <STRING>	Status of file sharing, 'Success' or 'Fail'
files[x].message	VARIABLE <STRING>	Status message from failed file sharing
files[x].updated	VARIABLE <BOOLEAN>	If the file changed, true or false

Example Queries

Query bytes per export run

SELECT
  run_id,
  SUM(value['file_size_bytes']) as bytes
FROM
  EXPORT_SUMMARY,
  LATERAL FLATTEN( input => files )
GROUP BY run_id;

List export run part statuses and error messages

SELECT
  run_id,
  value['status'] as status,
  value['message'] as message
FROM
  EXPORT_SUMMARY,
  LATERAL FLATTEN( input => files );

Query the number of failed/successful export jobs per day

SELECT
  DATE(end_at) as date,
  SUM(total_count) as total_total,
  SUM(success_count) as success_total,
  SUM(fail_count) as fail_total
FROM
  EXPORT_SUMMARY
GROUP BY
  1
ORDER BY
  1 DESC