This integration will allow you to connect with Mongo and start collecting your data.
Setting up the Mongo Integration
Setting up the Mongo pipeline is really simple. Firstly you need to log into your account on Blendo and select the Mongo icon for adding a pipeline.
On the first step you should add your credentials to connect to your Mongo.
- Mongo Host is the host of your Mongo instance.
- Port is the port of your Mongo instance. The default Mongo port (27017) is preselected. Change this if your setup is different.
- Authentication Database is the database inside your instance, which contains the Mongo users, which might be different than the data collection’s databases. By default the authentication database is “admin”, commonly used by cloud providers such as MongoDB Atlas.
- Username and Password are the credentials of a Mongo user that has read access to the collections that need to be synced.
If your MongoDB requires an SSL connection, please make sure that you select “Connect using SSL” underneath.
After you have filled in all the information you should click on “Next”.
On the second step of the setup you should select the database schema that the new tables will end up to. After selecting that, you should click on “Next”.
On the third step you should select your pipeline resources, each resource corresponding to a database table. If it is the first time that you setup a Mongo pipeline there will not be any available resources to select, so you have to create the resources that you want, by clicking on the “+ Create Resources” button.
On the window that opens you are asked to configure the resource. The resource is by default “collection” and under that you should type the name of the resource as you want it to appear at the resource list. This will also be used as the name of the table in your data warehouse. Setting up a Mongo collection resource also requires the collection’s database. The databases are automatically detected based on the databases the authenticated user has access to. Then you should select the collection that you want as a resource.
An update field is a collection field, with date type, that is used by Blendo to detect when a collection document has been updated. The field should be typed exactly as it appears in Mongo, as Blendo cannot verify if the field exists in each document and if it has a date type. Incremental updates will check this value and load only documents with time values after the last time your pipeline synced.
If an update field is selected, your pipeline will detect which documents have been updated and append them to your destination table. This will effectively store in your destination table all latest versions of your source data as they change.
After you have created the resource, you are back on the “Resource Selection” step. There you can:
- create a new resource by clicking on the “Create Resource” button
- select/deselect any of the existing ones by clicking on the check box on the left on the resource name
- configure a created resource by clicking on the Cog icon on the right of the resource name
- delete a created resource by clicking on the Bin icon on the right of the resource name (Please keep in mind that in case you delete a resource, the same resource is deleted from all the pipelines that include it)
- create the pipeline by clicking on the “Create” button
Your pipeline is just created. If you want to initiate the sync, you should click on the “Sync now” button on the right or wait for the sync to start at the time indicated.
That’s it! Your pipeline is ready to receive data!
Configuration of the pipeline
When selecting the pipeline, you can see some information about it. On the top right you can see the pipeline ID, the data warehouse that the data is synced to and the schema that it is using.
Below you can find three tabs:
The first tab that you can see while selecting your pipeline is the Status tab. There you can see if the pipeline is syncing at the moment and for how long or if it has stopped. Additionally, you can manually stop the pipeline. Moreover, you can see the details of the current or previous sync, specifically the status of each of the tables that are syncing, and how many records have synced.
The second tab is the Resources tab. There you can edit the resource selection for your pipeline by selecting or deselecting any of the resources, or you can edit or create a new resource as described above.
The third tab is the Settings tab. There you can see the account information of your Mongo pipeline and you can either pick another account or re-authenticate the pipeline.
Under that, you can see the Destination configuration and you can edit the database schema that your tables end up to.
Blendo collects the data as soon as they arrive and will sync them to your data warehouse according to your pipeline’s schedule. The default behaviour is to do this once per hour, but you can configure it to run as often as once per 15 minutes or once per day, by clicking on “Reschedule” button. Additionally, next to the “Reschedule” button you can find the “Pause” button that pauses the pipeline and keeps it paused until you manually resume it.
On the bottom of the page you can see the button “Remove”. By clicking it you delete your pipeline. This action is irreversible.
Keep in mind that any changes that may be done to the pipeline will be valid starting from the following sync and on.
Expected Mongo Data
Mongo Integration does not have a Schema, since the tables that will be created are custom and depend on the selected resources of the pipeline. The tables that will be created in your database will be named after the Name that is filled in during the resource configuration of the third step of the creation of the Mongo pipeline.