Loading data incrementally

On this section, you'll learn how to create and execute an incremental manual loading job to the Data Loading & Transformation module.


Creating an incremental manual loading job

The manual incremental loading job loads your data on-demand when new data needs to be inserted in SlicingDice. You need to manually start this job. Differently from one-time loading job, only new rows will be inserted in the database.

In the Control Panel, to go to the Loading Jobs page and click on Create New Loading Job to start creating your job.

The first step is to configure your loading job identification as you can see below.

Screen to define
        the loading job identification

Here you'll define your loading job name, description, the registered data source to connect and the job type (Manual Incremental in this case). When ready, click on Save & Continue.

Now you'll define in which of your databases and dimensions the data will be loaded.

Screen to define the loading
         job data destination

When everything is set, click on Save & Continue.

After this you'll map your source columns and match them with the SlicingDice columns and their supported data types.

For it, the first thing you should do is to detect your source schema, selecting the data you want to retrieve. You can do it choosing the Access mode - query or table - where you can define a SQL query or just inform the name of the table to retrieve data.

Map columns from source to SlicingDice columns

After detecting your schema, the selected source columns will appear at the Attribute and Event Columns mapping section. Map your source's columns to SlicingDice's existing columns or create new ones if needed. Also define the column to use on incremental job that is necessary to guarantee that just new data will be loaded. After that click on Save & Continue.

Finally just check your loading job configuration and click on Submit. Now you'll be able to find your created loading job at the loading jobs list. That's it! The next step is to execute your loading job manually.


Executing and checking an incremental manual loading job status

Once the loading job is created, it will automatically appear on the list of loading jobs. As your Loading Job is set as Manual Incremental Job just start it by pressing on the Play button at the Actions section on your loading jobs list.

You can also see the Status of your Loading Job changing to Running.