Sharing Funnel data to SFTP

How to setup Funnel for sharing data to SFTP

Niclas Bångman avatar
Written by Niclas Bångman
Updated over a week ago

Overview

The Data Warehouse Share for SFTP automatically syncs the data in your Funnel account to a server using the SSH File Transfer Protocol.

SFTP server permissions

We need access to write and delete (to overwrite) files at the location of the path you have selected, where meta files (summary, schema, test files, etc) will be stored.

Your settings can optionally block all meta files if you want to be strict, except for the file starting with test_ which is sent over and deleted to test access.

The data files are written to the "file name template" which optionally can include a path, eg data/{startDate}.csv which also needs to be writable. This path is only used for the data files and not the meta data files.

Data Share setup

Choose what formats, fields to share, and schedule you want.

You need to have an SFTP server that can be accessed by external services. We have no requirements as to what authentication that your server should use, but we support using password, private/public key and IP whitelisting to enforce security.

We really recommend that the credentials that you use are unique and don't allow access to other resources!

Check with your IT provider that you have these:

  • Hostname and port

  • Username and an optional password and/or private key with permissions to add and overwrite files

  • Path to put files under and what filenames to use.

IP whitelisting

Another level of security you can choose to add is IP whitelisting. Funnel's SFTP traffic always comes from either of these IP Addresses:

  • 54.81.136.8 - if you are using our app.funnel.io service,

  • 18.193.167.135 - if you are using our app.eu.funnel.io service.

Public key authentication

Since passwords by themselves rarely are long and secure enough to withstand brute force attacks they are recommended to be accompanied by public key authentication. Where we have the private key and your SFTP server, that have the public key, can verify that the request comes from us.

You can create a private key and public key on your computer by running this command in the terminal:

ssh-keygen

Use your own private key (recommended)

We recommend that you create your own private key and public key. After creating the keys you can then input the private key when creating or editing the SFTP Data Share in Funnel. By adding the public key to your SFTP server you will then enable public key authentication and start verifying all incoming traffic.

Use the Funnel default private key

By not inputting a private key at all you will use Funnel's default private key and public key. Then you just have to add the public key below to your SFTP server to enable the public key authentication.

Global - If you are using our app.funnel.io service

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDAmszGM2cFkJa61r8pyTkmBqyw1Gm4qfwgIwJQ62+xugARioYIPsQ3RCcHWwlpqpA2ALnATg2knqh2csBtrADSrMhreFRNUbzXX53Sgbe0G9ZgUfFEVORA28hHqCGTbZAb11AkhHviICZBZ85f4+0CnJ3Tw1HYKfKoCvrPE7brEgh63L8jw6OFgsvpQrnDrisKT23vKnrbqIwOWbSeYzji+NIV64LsJ8fB4Ld4gBUVUMFi/Sj0wVmKauCDwNr4TInuIFDH0UgwnJTK8JK092IqC6bSdm2BXtuuhEGCuqBbzP3kodVW7h4Fq5tLzYzK/ykTrFiMJ4794g0NqkOtDr2A3129momqbuKraxUdsNBTWd2M/bMMeAk2ft7JjdKAclHgra161X/F2bNlk1B2sGYNx2Er10u9ni6Fv0GZQ6ZY1qR4vgZ1JC+crvEar7pBsW7+S/TOHJLvXS0GafAmLo7rB74NAkqpjFnZp/oEo254OaBtgrT6xt8VTdc6yoAJzGw2ycg4SUCiauATFZFQZkmitslGywvBgg2e92SudkD3dvt7f3BivD3bjVDp7cYMwyRSNoqR64OGFoHUSFb7K5CPsl4icWpswbBx9xMUGNCJuZMCnchMSnoLuYNkr61jUFeL9P+NQSkPXsuFhfQGddPCnSEup8IA/YxaEqnyMSfcNw== funnel

EU - If you are using our app.eu.funnel.io service

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9r5i0b9N+XZnwdxyYd3BQ/T/EtNjSGOP6eqJ6PnE9H8bNTacZWlZ4HZh3fL7s6MrQ6VsFc7kqsilXFh1ixV0mtKI72kidz4F8kk+qHHjoGeMn46UNIHf3zO3vXyrGU6Y+Uuvwcr5I1g89Sb7nAUvd8yuucyCoGxAwHWDVNyJYyVsK8y6Brp5sSlvLQi2mH26MFnwjQ56GcCp82PBhw7izt+uM/6jlE4ErS2ysexhSHQrINobIOxTNBjnE6DvRAOAGuq7quSv89c94gSH3jT4+ZM3vHYkancnJFEjcLjWWfqQTo+eiy1TP17asKuAsQGuwwx8HeotLefHuElQq4nkLGpDdsTz0u4wBFSl/WvnSF9AWi8DUg/tPl3oogO0pCynLToL/qU15HdGMqbXHj8xZy0pCAXktpglXIRRdZpPV24JBwUY5TRVyVPmlgE0RR1BMyNtmZx62J2dqZTotwG06yjAGyELN1ztvrd3JUw/x1mBH4z7IJve7ZMHEeR6oI/q6Aqnv06YoIZC06zLAcv4fgWlBQF1+4603akWsMMHHipxbeYzwK0qsaqAHc9b6HrYFJwP71f7j03TheugI5iCdYDruuwzzgnf5S7IsysZI4eYbmJzXmEnin/DUwtJZZcRBZIj+DHoQ8Me3Z10bNgJc1t4MlYqlcTBxTSNq83V8OQ== funnel

Troubleshooting

Below are some error messages that can occur:

  • "getaddrinfo ENOTFOUND"  means that the hostname is not known, check the hostname and that it is a name or IP that is accessible from the internet.

  • "no matching client->server cipher" typically means that the server implementation is old. Our services use most currently preferred ciphers. See below for more on this.

  • "Unable to contact server" typically indicates that Funnel could not connect to the SFTP server. If using IP security in a firewall ensure that AWS can access the server. We are on either us-east-1 or eu-central-1 region and ELB service for now and the current details are on https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html

  • "All configured authentication methods failed" happens when Funnel could reach the server but didn't get access. Validate that you've put in the correct username/password combination and that if your server blocks password only authentication, you have also allowed our public-key

How to check if your server supports Funnel's SFTP sharing

For protocol details, see Protocol support below

To find out if your server supports a certain cipher, you can type this into a terminal window:
sftp -v -c aes128-ctr your.sftp.server

The SFTP server does not support the requested cipher if the response looks like this:

Unable to negotiate with xxx.xxx.xxx.xxx. port 22: no matching cipher found. 
Their offer: twofish256-cbc,twofish-cbc,twofish128-cbc,blowfish-cbc,3des-cbc,arcfour,cast128-cbc,aes256-cbc,aes128-cbc

This will also tell you which ciphers the server does support.

Protocol support

These are all the protocols we support and your server needs to support at least one of the protocols listed below for each category.

Ciphers

HMAC (message authentication code)

KEX (key exchange)

  • curve25519-sha256

  • ecdh-sha2-nistp256

  • ecdh-sha2-nistp384

  • ecdh-sha2-nistp521

  • diffie-hellman-group14-sha256

  • diffie-hellman-group15-sha512

  • diffie-hellman-group16-sha512

  • diffie-hellman-group17-sha512

  • diffie-hellman-group-exchange-sha1

  • diffie-hellman-group14-sha1

  • diffie-hellman-group1-sha1

  • diffie-hellman-group18-sha512 (very slow)

  • diffie-hellman-group-exchange-sha256 (very slow)

Server Host Key (algorithms)

  • ssh-ed25519

  • ecdsa-sha2-nistp256

  • ecdsa-sha2-nistp384

  • ecdsa-sha2-nistp521

  • rsa-sha2-512

  • rsa-sha2-256

  • ssh-rsa

  • ssh-dss

Did this answer your question?