Loading data from Snowflake
On this page, we'll explain you how to add a Snowflake data source and create a loading job using it to synchronize your source with your SlicingDice's databases.
Whitelist SlicingDice's IPs
If your Snowflake database has network policies configured, you'll need to whitelist the SlicingDice's Data Loading IP addresses. To whitelist our IPs, follow the steps below.
How to whitelist our Data Loading IPs in Snowflake
Log in your Snowflake instance and select a role that has permission to create network policies, like
Click on the "Account" icon on the top header, then click on Policies.
Create a new Network Policy by clicking on the "Create" option.
Insert a name and our IP addresses on the "Allowed IP Addresses", separated by commas.- 220.127.116.11
Click on the "Finish" button
Your IP address!
If you don't have any Snowflake policy that allows your IP address to access Snowflake, you'll need to insert your own IP on the Allowed IP Addresses whitelist.
Select your new Network Policy and check if everithing is ok.
You'll see a list of Allowed IP Addresses and Blocked IP Addresses, if any exists
Click on "Activate..." option to enable your Network Policy.
You'll receive a success message and now your Network Policy will have a label called "Active" after activation.
Now SlicingDice's Data Loading Module will be able to access your Snowflake databases as soon as you create a data source on SlicingDice. This is what we'll do next!
Add a Snowflake Data Source
Before adding your Snowflake data source on SlicingDice, you need to be logged in our Control Panel. Then, you need to go to the Data Sources page so we can start our tutorial.
How to add a Snowflake data source on SlicingDice
Before creating your data source you need to have the following information in order to connect to your Snowflake
- The hostname where your Snowflake instance is running;
- Your Snowflake username and password
- Your database name
- Your Snowflake warehouse name
Data Source setup
The first step is the configuration of your data source identification on SlicingDice. The following screen shows step 1.
Three fields will appear. Each field function is described at the table below.
Field Description Data Source Name The name of your data source. Can be edited at any time. Mandatory Data Source Labels/Tags Labels/tags you might want to associate to a source, in order to organize your sources. Can be edited at any time. Optional Data Source Description The description of your data source. Can be edited at any time. Optional
When ready, click on the Save & Continue button to go to Step 2.
Data Source Details
Below you can see an example of the information and credentials that you should provide so SlicingDice can be able to connect to your Snowflake database.
Field Description Data Source Type The type of data source. In this case we're using Snowflake. Host The Snowflake host. The access URL without HTTPS. Username The username to log in to Snowflake Password The username's password to log in to Snowflake Database The database that will be loaded into SlicingDice Warehouse The name of your virtual warehouse.
You can test the connection by clicking on the Test Connection button. If everything goes ok you'll see a success message.
Now you can go to the next step clicking on Save & Continue
Here you'll see a summary of the configurations defined for this data source before you finally create it.
The following image shows an example of a confirmation screen, which the name of the data source is
If everything is ok, click on the Submit button and then you'll receive a success creation message.
Now you'll be able to find your new data source at the data sources list as you can see in the following image.
That's it! The next step is to load your Snowflake data into SlicingDice creating a loading job.
Add a new loading job using Snowflake sources
Now all the connection configuration with your Snowflake source is completed, so the next step is just to create and execute a loading job using this Snowflake Data Source you've configured on SlicingDice.
Here are the creation jobs tutorials for each type of loading job. Choose the most helpful for your use case:
Remember that while setting up your loading job, you should set the Snowflake Data Source you've created as the Data Source to be used on this job.
- One-time loading job: The one-time loading job loads your data once, needing manual intervention to execute it. This loading job type is useful if you don't update your data frequently.
- Manual incremental loading job: The manual
incremental loading job loads your data on-demand when new data needs to be inserted in SlicingDice. You
need to manually start this loading job.
Differently from an one-time loading job, only new rows will be inserted in the database. Your dataset needs to have a timestamp column in order to use this loading job type.
- Automatic loading job: The automatic loading
your data frequently, specified by a predetermined time interval. You don't need to manually start this
loading job, as it executes automatically.
Your dataset needs to have a timestamp column in order to use this loading job type.