Loading data automatically
On this section, you'll learn how to create and execute an automatic loading job to the Data Loading & Transformation module and execute it.
Create a data source!
Before you create a new Loading Job, you'll need to have at least one Data Source configured. If you didn't create a Data Source yet, read the Adding data sources guide to create one.
Creating an automatic loading job
The automatic job loads your data frequently, specified by a predefined time interval. You don't need to manually start this loading job, as it executes automatically.
Column with timestamps!
For this guide, your dataset needs to have a timestamp column (Date, Datetime), or the Automatic Loading Job won't work. This column serves to SlicingDice only insert new data to your database.
In the Control Panel, go to the Loading Jobs page and click on Create New Loading Job to start creating your job.
The first step is to configure your loading job identification as you can see below.
Here you'll define your loading job name, description, the registered data source to connect, the job type (Automatic/Scheduled in this case) and job frequency that this job will be executed. When ready, click on Save & Continue.
Now you'll define in which of your databases and dimensions the data will be loaded.
When everything is set, click on Save & Continue.
After this you'll map your source columns and match them with the SlicingDice columns and their supported data types.
For it, the first thing you should do is to detect your source schema, selecting the data you want to retrieve. You can do it choosing the Access mode - query or table - where you can define a SQL query or just inform the name of the table to retrieve data.
After detecting your schema, the selected source columns will appear at the Attribute and Event Columns mapping section. Map your source's columns to SlicingDice's existing columns or create new ones if needed. Also define the column to use on automatic job that is necessary to guarantee that just new data will be loaded. After that click on Save & Continue.
Mapping an Entity ID is mandatory!
On SlicingDice an Entity ID column behaves similarly to a primary key column in a database with unique values stored on it, so any data you insert will always be associated with a single identifier. This is why you MUST define one of your columns from your sources to be your Entity ID column, otherwise you'll receive an error while creating your loading job.
Finally just check your loading job configuration and click on Submit. Now you'll be able to find your created loading job at the loading jobs list. That's it!, now your data will be loaded automatically considering the frequency you selected.
Executing and checking an automatic loading job status
As your Loading Job is set as Automatic, it'll start automatically.
Time to execute the job!
The Loading Job execution time can vary by the size of your dataset and the chosen Rows per request per second configured at step 3. When the "Status" parameter modifies to "Completed", you can start querying your data.