This integration will allow you to connect with Sftp and start collecting your data.
Setting up the Sftp Integration
How to create your pipeline
You should log in to your Blendo account. In order to create a new pipeline, click on the Sftp icon.
On the first step of the setup, Blendo is providing you with the public key, that needs to be added in the authorized_keys of the user that wants to connect to Blendo. You have to fill in the name of the host and the user. Host is the IP of the instance that needs to connect to Blendo and contains the data that Blendo will sync. User is the name of this instance.
After you have filled them in you should click on “Next”.
On the second step you should configure the resource of your Sftp pipeline. Every Sftp pipeline has only one resource, so in case you want to sync more resources with different configuration, you should create new Sftp pipelines.
In order to configure the resource of your pipeline, you have to fill in the following fields:
- Resource ID: as every pipeline run syncs only updates, the resource ID is needed for the deduplication of data in your database. In case it is left empty, no deduplication will be performed.
- Resolve Type: you should select either “path” or “pattern”.
- Path/Pattern: the Path indicates the folder that Blendo accesses in order to sync the data that this folder contains. Blendo will not access the subfolders of the path. When selecting a Pattern, the Path is optional, to indicate the subfolders that Blendo will access in order to sync data. The Pattern indicates the specific pattern of files that you want to be synced (e.g. in case you want to select all files that end with .csv you should type: (.*).csv ).
- Parser: the type of file that exists in Sftp and the way that data will be parsed in the destination table.
On the next step you will be asked if you want to use a prefix to your table name that will be created in your database. In case you leave it blank, the table will use the default name (“document”). You are also asked to select your database schema that the table will end up to. After you have filled them in you should click on “Next”.
Your pipeline is just created. If you want to initiate the sync, you should click on the “Sync now” button on the right or wait for the sync to start at the time indicated.
