PrimeHub Notebook Extension
Last updated
Last updated
PrimeHub Extension is an extension of Jupyter Notebook developed for PrimeHub. We plans to roll out more features to enhance user's ML workflow and experience from Notebook.
The latest images provided by InfuseAI are built with PrimeHub Extension and the corresponding environment to execute extension features smoothly. To Use these features, please make sure the running image is the latest one or is built on top of the latest one from InfuseAI. See the list.
How to build own images including PrimeHub Extensions? See the repo.
After opening a Notebook file (*.ipynb), there is PrimeHub extension on the menu bar of the Notebook. This document explains these extension features briefly.
PrimeHub Extension needs an API Token to submit a Job, Please generate a API Token and setup it.
You could open the PrimeHub Extension dropdown list to find the API Token setting:
The working group's Group Volume is required.
Users could submit their Notebook to the PrimeHub Jobs. See Simple UseCase.
However, there are differences between jobs from Notebook and jobs from Job Submission directly:
A job submitted by PrimeHub Extension (a.k.a Notebook Job) always uses the directory where the notebook locates as its working directory, whereas a job submitted by Job Submission uses /home/jovyan instead.
A notebook has to locate inside a group volume.
Path, for example:
There is a group volume phusers and a notebook, my-notebook.ipynb, in the sub path experiment-1
When the notebook my-notebook.ipynb is submitted into a job, the job takes /home/jovyan/phusers/experiment-1/ as the working directory and run all of cells of the notebook.
In terms of a Notebook Job, users could store outputs either in the group volume or in the Job Artifacts but different working directories as we mention above have to be taken into consideration. Since working directories are different, relative path in code may cause problems.
For example,
The code snippet works in a Job but not in a Notebook Job, because the working directory is at /home/jovyan and it put data relatively into /home/jovyan/artifacts which is a correct path.
However, a Notebook Job starts at /home/jovyan/<group_volume>/path/to/ where the notebook locates, the relative path of same codes above becomes /home/jovyan/<group_volume>/path/to/artifcats incorrectly.
Hence, we encourage using absolute path in both of cases instead to avoid the mistake and the confusion.
We also encourage using group volume to store persistent data.
Using Job Artifacts is optional when we expect
Download artifacts from PrimeHub Job without opening a Notebook
Artifacts will be cleaned automatically
