PrimeHub
v4.1
v4.1
  • Introduction
  • Installation
  • Tiers and Licenses
  • End-to-End Tutorial
    • 1 - MLOps Introduction and Scoping the Project
    • 2 - Train and Manage the Model
    • 3 - Compare, Register and Deploy the Model
    • 4 - Build the Web Application
    • 5 - Summary
  • User Guide
    • User Portal
    • Notebook
      • Notebook Tips
      • Advanced Settings
      • PrimeHub Notebook Extension
      • Submit Notebook as Job
    • Jobs
      • Job Artifacts
      • Tutorial
        • (Part1) MNIST classifier training
        • (Part2) MNIST classifier training
        • (Advanced) Use Job Submission to Tune Hyperparameters
        • (Advanced) Model Serving by Seldon
        • Job Artifacts Simple Usecase
    • Models
      • Manage and Deploy Model
      • Model Management Configuration
    • Deployments
      • Pre-packaged servers
        • TensorFlow server
        • PyTorch server
        • SKLearn server
        • Customize Pre-packaged Server
        • Run Pre-packaged Server Locally
      • Package from Language Wrapper
        • Model Image for Python
        • Model Image for R
        • Reusable Base Image
      • Prediction APIs
      • Model URI
      • Tutorial
        • Model by Pre-packaged Server
        • Model by Pre-packaged Server (PHFS)
        • Model by Image built from Language Wrapper
    • Shared Files
    • Datasets
    • Apps
      • Label Studio
      • MATLAB
      • MLflow
      • Streamlit
      • Tutorial
        • Create Your Own App
        • Create an MLflow server
        • Label Dataset by Label Studio
        • Code Server
    • Group Admin
      • Images
      • Settings
    • Generate an PrimeHub API Token
    • Python SDK
    • SSH Server Feature
      • VSCode SSH Notebook Remotely
      • Generate SSH Key Pair
      • Permission Denied
      • Connection Refused
    • Advanced Tutorial
      • Labeling the data
      • Notebook as a Job
      • Custom build the Seldon server
      • PrimeHub SDK/CLI Tools
  • Administrator Guide
    • Admin Portal
      • Create User
      • Create Group
      • Assign Group Admin
      • Create/Plan Instance Type
      • Add InfuseAI Image
      • Add Image
      • Build Image
      • Gitsync Secret for GitHub
      • Pull Secret for GitLab
    • System Settings
    • User Management
    • Group Management
    • Instance Type Management
      • NodeSelector
      • Toleration
    • Image Management
      • Custom Image Guideline
    • Volume Management
      • Upload Server
    • Secret Management
    • App Settings
    • Notebooks Admin
    • Usage Reports
  • Reference
    • Jupyter Images
      • repo2docker image
      • RStudio image
    • InfuseAI Images List
    • Roadmap
  • Developer Guide
    • GitHub
    • Design
      • PrimeHub File System (PHFS)
      • PrimeHub Store
      • Log Persistence
      • PrimeHub Apps
      • Admission
      • Notebook with kernel process
      • JupyterHub
      • Image Builder
      • Volume Upload
      • Job Scheduler
      • Job Submission
      • Job Monitoring
      • Install Helper
      • User Portal
      • Meta Chart
      • PrimeHub Usage
      • Job Artifact
      • PrimeHub Apps
    • Concept
      • Architecture
      • Data Model
      • CRDs
      • GraphQL
      • Persistence Storages
      • Persistence
      • Resources Quota
      • Privilege
    • Configuration
      • How to configure PrimeHub
      • Multiple Jupyter Notebook Kernels
      • Configure SSH Server
      • Configure Job Submission
      • Configure Custom Image Build
      • Configure Model Deployment
      • Setup Self-Signed Certificate for PrimeHub
      • Chart Configuration
      • Configure PrimeHub Store
    • Environment Variables
Powered by GitBook
On this page
  • API Token
  • Submit Notebook as a Job
  1. User Guide
  2. Notebook

PrimeHub Notebook Extension

PreviousAdvanced SettingsNextSubmit Notebook as Job

Last updated 10 days ago

PrimeHub Extension is an extension of Jupyter Notebook developed for PrimeHub. We plans to roll out more features to enhance user's ML workflow and experience from Notebook.

The latest images provided by InfuseAI are built with PrimeHub Extension and the corresponding environment to execute extension features smoothly. To Use these features, please make sure the running image is the latest one or is built on top of the latest one from InfuseAI. See the list.

How to build own images including PrimeHub Extensions? See the .

After opening a Notebook file (*.ipynb), there is PrimeHub extension on the menu bar of the Notebook. This document explains these extension features briefly.

API Token

You could open the PrimeHub Extension dropdown list to find the API Token setting:

Submit Notebook as a Job

The working group's Group Volume is required.

However, there are differences between jobs from Notebook and jobs from Job Submission directly:

  • A job submitted by PrimeHub Extension (a.k.a Notebook Job) always uses the directory where the notebook locates as its working directory, whereas a job submitted by Job Submission uses /home/jovyan instead.

  • A notebook has to locate inside a group volume.

Path, for example:

There is a group volume phusers and a notebook, my-notebook.ipynb, in the sub path experiment-1

/home/jovyan/phusers/experiment-1/my-notebook.ipynb

When the notebook my-notebook.ipynb is submitted into a job, the job takes /home/jovyan/phusers/experiment-1/ as the working directory and run all of cells of the notebook.

Job Artifacts with a Notebook Job

In terms of a Notebook Job, users could store outputs either in the group volume or in the Job Artifacts but different working directories as we mention above have to be taken into consideration. Since working directories are different, relative path in code may cause problems.

For example,

The code snippet works in a Job but not in a Notebook Job, because the working directory is at /home/jovyan and it put data relatively into /home/jovyan/artifacts which is a correct path.

mkdir -p artifacts/
cp my_model.h5 artifacts/
cp -r logs artifacts/

However, a Notebook Job starts at /home/jovyan/<group_volume>/path/to/ where the notebook locates, the relative path of same codes above becomes /home/jovyan/<group_volume>/path/to/artifcats incorrectly.

Hence, we encourage using absolute path in both of cases instead to avoid the mistake and the confusion.

mkdir -p /home/jovyan/artifacts/
cp my_model.h5 /home/jovyan/artifacts/
cp -r logs /home/jovyan/artifacts/

Job Artifacts is optional

We also encourage using group volume to store persistent data.

Using Job Artifacts is optional when we expect

  • Download artifacts from PrimeHub Job without opening a Notebook

  • Artifacts will be cleaned automatically

PrimeHub Extension needs an API Token to submit a Job, Please and setup it.

Users could submit their Notebook to the PrimeHub Jobs. See .

generate a API Token
Simple UseCase
repo