Data Scientist Tutorial

This section takes the first time user through the DKube workflow using a simple model and dataset. The MNIST model is used to provide a simple, successful initial experience.

General Workflow

The workflow demonstrated in this example is as follows:

  • Load the program into the Workspace store
  • Load the dataset into the Dataset store
  • Create a DKube Notebook and open a Jupyter notebook
  • Create a Training Run
  • View & deploy the resulting Model
  • Use a simple Inference test on the trained model

Load Program

Load the MNIST program from a GitHub repository into the DKube store from the Workspaces menu by selecting “+Workspace”.

_images/Data_Scientist_Workspace_Blank.jpg

The fields should be filled in as follows, then select “Add Workspace”.

Name mnist
url https://github.com/oneconvergence/dkube-examples/tree/master/tensorflow/classification/mnist/digits/classifier/program
Tag mnist
_images/Data_Scientist_Workspace_mnist.jpg

This will create the mnist workspace.

_images/Data_Scientist_Workspace_Success.jpg

Load Dataset

Load the MNIST dataset from a GitHub repository into the DKube store from the Datasets menu by selecting “+Dataset”.

_images/Data_Scientist_Dataset_Blank.jpg

The fields should be filled in as follows, then select “Add Dataset”.

Name mnist
url https://github.com/oneconvergence/dkube-examples/tree/master/tensorflow/classification/mnist/digits/classifier/data
Tag mnist
_images/Data_Scientist_Dataset_mnist.jpg

This will create the mnist dataset.

_images/Data_Scientist_Dataset_Success.jpg

Create Notebook

Create a Notebook from the Notebooks menu to experiment with the program by selecting “+ Notebook”.

_images/Data_Scientist_Notebook_Blank.jpg

The fields should be filled in as follows, then select “Submit”.

Name mnist
Tag mnist
Workspace Select mnist program
Dataset Select mnist dataset

All the other fields should be left in their default state.

_images/Data_Scientist_Notebook_mnist.jpg

Note

The initial Notebook will take a few minutes to start. Follow-on Notebooks with the same TensorFlow version will start more quickly.

_images/Data_Scientist_Notebook_Success.jpg

While on this screen, start the default DKube notebook instance by selecting the “Start” icon.

_images/Data_Scientist_Notebook_DKube.jpg

Open Jupyter Notebook

Open a Jupyter notebook by selecting the Jupyter icon under “Actions” on the far right.

_images/Data_Scientist_Jupyter_mnist.jpg

Create Training Job

Create a Training Job from the Jobs menu to train the mnist model on the dataset and create a trained model.

_images/Data_Scientist_Job_Blank.jpg

The fields should be filled in as follows, then select “Submit”.

Name mnist
Tag mnist
Start-up script (in Container) python model.py
Workspace Select mnist program
Dataset Select mnist dataset

All the other fields should be left in their default state.

_images/Data_Scientist_Job_mnist.jpg

Note

The initial Job will take a few minutes to start. Follow-on Jobs with the same TensorFlow version will start more quickly.

_images/Data_Scientist_Job_Success.jpg

Open TensorBoard

While waiting for the Job to complete, start the TensorBoard pod within DKube by selecting the “Start” icon under TensorBoard on the right.

_images/Data_Scientist_Job_Complete.jpg

Once the Job is complete, open the TensorBoard window by selecting the TensorBoard icon.

_images/Data_Scientist_Job_TensorBoard.jpg

View and Deploy Model

A completed Training Job will create a trained model, which shows up in the Model menu.

_images/Data_Scientist_Model_Success.jpg

The model can be viewed by selecting the model instance name. From this screen the model can be deployed for inference by selecting the “Deploy” button.

_images/Data_Scientist_Model_Deploy.jpg

A popup screen will appear, and should be filled in as follows, then select “Deploy”.

Name mnist-inference
CPU Select CPU
_images/Data_Scientist_Model_Deploy_Popup.jpg

Inference Test

The trained model will be deployed for inference, and show up in the Inference Menu.

_images/Data_Scientist_Inference_Success.jpg

The inference is running at the url shown on the far right. Put this url into the copy buffer by selecting the copy icon at the far right.

The inference testing will happen from the special DKube notebook instance, available on the Notebooks menu. In order to navigate to the inference application, open a Jupyter notebook from the DKube instance.

_images/Data_Scientist_Notebook_DKube_Jupyter.jpg
_images/Data_Scientist_Notebook_DKube_Jupyter_Window.jpg
_images/Data_Scientist_Notebook_DKube_Jupyter_Tools.jpg

Note

The inference application must be opened in a new tab or a new window, as shown in the picture.

_images/Data_Scientist_Notebook_DKube_Jupyter_Infapp.jpg

The inference application will call up a screen where you fill in the necessary fields, then select “Predict”

Model Serving URL Paste in the server url from the Inference screen
Program Choose MNIST from dropdown menu
Testing Image Choose image file

An image file for inference test is available at http://oneconvergence.com/guide/downloads/3.png

The inference test application will run the prediction and show the output.

_images/Data_Scientist_Notebook_DKube_Jupyter_Inftest_Success.jpg