Loading JSON data into dashDB from Cloudant

Loading JSON data into dashDB from Cloudant
What this tutorial is about
This tutorial will show you how to create a new dashDB instance and populate it with
data directly from a Cloudant account.
What you should be able to do






Provision a new dashDB instance from Cloudant
Populate the dashDB instance with data from Cloudant
Work with the created tables in dashDB
Run real-time replication from Cloudant to dashDB
Stop real-time replication
Delete the dashDB instance
What you need before you start

An IBM Bluemix Account
If you don't have an account, sign up for a free account at
https://bluemix.net

An IBM Cloudant Account
If you don't have an account, you can sign up for a free account at
https://cloudant.com
At least one small database in your Cloudant account
This tutorial will work with 2 databases from
https://examples.cloudant.com called movies_demo and geo. You can
either replicate those two databases into your own account or work with
any other Cloudant database you may already have.

Page 1
Loading JSON data into dashDB from Cloudant
Step 1: Provision a dashDB instance from Cloudant
1. Log in to your Cloudant dashboard. If you don't have a Cloudant account, jump to
the appendix in this tutorial for detailed steps.
2. Optional: On the Databases tab, view the list of databases in your Cloudant account
to find suitable database names, then open a database.
3. Optional: Open and inspect individual documents in your database to understand
their data and structure.
Page 2
Loading JSON data into dashDB from Cloudant
4. Open the Warehousing tab, and click New Warehouse.
5. Add a set of source databases to the new warehouse by typing their names into the
Source Databases field. As you start typing, the type-ahead will list databases that
match the characters you are typing.
6. Supply your Bluemix credentials; your IBM ID and password.
Page 3
Loading JSON data into dashDB from Cloudant
7. Provision the new dashDB instance with the Create Warehouse button
With Create Warehouse a number of things happen:




Cloudant creates a new dashDB service in your Bluemix account.
Cloudant scans the selected source databases to understand the document
structure in each.
Cloudant creates tables in the new dashDB database to represent these
documents.
Cloudant copies the document data into these new tables in dashDB.
8. Optional: On the Warehousing tab, monitor the load progress.
The progress indicator shows the number of documents being copied and the colorcoded progress.



Blue status indicates that documents are still be processed initially.
Green status indicates that Cloudant has loaded as many documents into
dashDB as are currently in the source databases. Updated documents or new
document revisions automatically update the corresponding records in
dashDB in real-time replication.
Red status indicates that there were problems either during the initial load or
the ongoing replication.
9. Click Visit warehouse to launch the dashDB console.
Page 4
Loading JSON data into dashDB from Cloudant
Step 2: Work with the tables in your dashDB instance
1. Click Go to your tables in the dashDB console.
2. Select the Schema and then the Table Name to inspect the Table Definition and
view the created database tables.
Note: Tables are created in a Schema with a name identical to the dashDB instance
name. The schema is selected by default, but there are other sample schemas
available in the default dashDB instance. Make sure to select the right schema to
find the tables.
Page 5
Loading JSON data into dashDB from Cloudant
3. Select the Browse Data tab to view the data populated into the tables.
Note: The Warehousing process may have created multiple tables for a single
Cloudant database. All tables are prefixed with the capitalized database name, for
example GEO_.
4. Optional: Inspect the _OVERFLOW table.
Note: This table is created to capture warnings and exceptions that may happen
during load. There is one OVERFLOW table for every source database (for example
GEO_OVERFLOW).
Page 6
Loading JSON data into dashDB from Cloudant
Step 3: Stop Cloudant replication, rescan, or delete the dashDB instance
1. Log back into your Cloudant dashboard.
2. Stop the database load with the Stop action.
Note: Since the dashDB load from Cloudant is real-time replication, the load will
never stop automatically. Even if all documents have long been processed, the Stop
action is necessary to disconnect from the Cloudant changes feed.
3. Optional: Reload the dashDB instance with the Rescan action
Note: Sometimes the selected Cloudant database changes considerably in structure
and documents no longer fit in the created dashDB tables. The rescan action is
useful to drop and recreate those tables and reload all documents again. The rescan
action therefore deletes all data previously loaded into dashDB, but does not delete
the instance itself.
4. Optional: Drop the dashDB instance with the Delete action.
Note: This will de-provision the dashDB instance and delete all data in it - even data
that has been created manually or loaded outside of Cloudant.
Page 7
Loading JSON data into dashDB from Cloudant
Appendix: Access the Cloudant Account
Use one of the following procedures to access the Cloudant dashboard.
From cloudant.com

Either sign-in (https://cloudant.com/sign-in/) or sign-up (https://cloudant.com/signup/) for a Cloudant account. Once signed in, you will brought directly to the Cloudant
dashboard.
From Bluemix
1. Either log in to your Bluemix account at
https://console.ng.bluemix.net/home/auth/bluemix, or sign up for a new Bluemix
account at https://console.ng.bluemix.net/registration.
2. View the Catalog.
3. In the Data Management category, locate and select the Cloudant NoSQL DB
Service, and then provide the following information:



For App, select Leave unbound.
For Service name, type a name for the instance.
For Selected Plan, select the Shared plan.
4. Click Create to create the service.
5. In the Bluemix Dashboard, in the Services section, select the Cloudant NoSQL DB
service instance to open the service detail view.
6. Click Launch to open the Cloudant dashboard.
Page 8