When exporting to file storage solutions Funnel can export a summary file containing information about the export. This can be used to:
- Trigger next stages in your data pipeline
- Determine which files have been updated
- Error handling for failed exports
The Export summary file can be configured for the following export destinations:
- Google Cloud storage
- Amazon S3
- Microsoft Azure Blob storage
The summary file can be configured in the 'advanced' settings, and defaults to JSON for new exports.
This example comes from a Google Cloud storage export:
The summary file formats that are available are:
- JSON file - Contains meta information about the export like status (Success or Failed), error messages, file size, unique IDs, start and end dates, the name of files that were exported and if they were updated or not since the last run.
- Csv file - Contains a list of the files that were exported and if they were updated or not since the last run.
- Text file (legacy) - A plain text file. This file can be used to trigger next steps in a data pipeline but is considered legacy as it contains no information about the export, and is included for backwards compatibility with older exports.
- None - you can choose to receive no summary file at all.