PrimeHub
v4.1
v4.1
  • Introduction
  • Installation
  • Tiers and Licenses
  • End-to-End Tutorial
    • 1 - MLOps Introduction and Scoping the Project
    • 2 - Train and Manage the Model
    • 3 - Compare, Register and Deploy the Model
    • 4 - Build the Web Application
    • 5 - Summary
  • User Guide
    • User Portal
    • Notebook
      • Notebook Tips
      • Advanced Settings
      • PrimeHub Notebook Extension
      • Submit Notebook as Job
    • Jobs
      • Job Artifacts
      • Tutorial
        • (Part1) MNIST classifier training
        • (Part2) MNIST classifier training
        • (Advanced) Use Job Submission to Tune Hyperparameters
        • (Advanced) Model Serving by Seldon
        • Job Artifacts Simple Usecase
    • Models
      • Manage and Deploy Model
      • Model Management Configuration
    • Deployments
      • Pre-packaged servers
        • TensorFlow server
        • PyTorch server
        • SKLearn server
        • Customize Pre-packaged Server
        • Run Pre-packaged Server Locally
      • Package from Language Wrapper
        • Model Image for Python
        • Model Image for R
        • Reusable Base Image
      • Prediction APIs
      • Model URI
      • Tutorial
        • Model by Pre-packaged Server
        • Model by Pre-packaged Server (PHFS)
        • Model by Image built from Language Wrapper
    • Shared Files
    • Datasets
    • Apps
      • Label Studio
      • MATLAB
      • MLflow
      • Streamlit
      • Tutorial
        • Create Your Own App
        • Create an MLflow server
        • Label Dataset by Label Studio
        • Code Server
    • Group Admin
      • Images
      • Settings
    • Generate an PrimeHub API Token
    • Python SDK
    • SSH Server Feature
      • VSCode SSH Notebook Remotely
      • Generate SSH Key Pair
      • Permission Denied
      • Connection Refused
    • Advanced Tutorial
      • Labeling the data
      • Notebook as a Job
      • Custom build the Seldon server
      • PrimeHub SDK/CLI Tools
  • Administrator Guide
    • Admin Portal
      • Create User
      • Create Group
      • Assign Group Admin
      • Create/Plan Instance Type
      • Add InfuseAI Image
      • Add Image
      • Build Image
      • Gitsync Secret for GitHub
      • Pull Secret for GitLab
    • System Settings
    • User Management
    • Group Management
    • Instance Type Management
      • NodeSelector
      • Toleration
    • Image Management
      • Custom Image Guideline
    • Volume Management
      • Upload Server
    • Secret Management
    • App Settings
    • Notebooks Admin
    • Usage Reports
  • Reference
    • Jupyter Images
      • repo2docker image
      • RStudio image
    • InfuseAI Images List
    • Roadmap
  • Developer Guide
    • GitHub
    • Design
      • PrimeHub File System (PHFS)
      • PrimeHub Store
      • Log Persistence
      • PrimeHub Apps
      • Admission
      • Notebook with kernel process
      • JupyterHub
      • Image Builder
      • Volume Upload
      • Job Scheduler
      • Job Submission
      • Job Monitoring
      • Install Helper
      • User Portal
      • Meta Chart
      • PrimeHub Usage
      • Job Artifact
      • PrimeHub Apps
    • Concept
      • Architecture
      • Data Model
      • CRDs
      • GraphQL
      • Persistence Storages
      • Persistence
      • Resources Quota
      • Privilege
    • Configuration
      • How to configure PrimeHub
      • Multiple Jupyter Notebook Kernels
      • Configure SSH Server
      • Configure Job Submission
      • Configure Custom Image Build
      • Configure Model Deployment
      • Setup Self-Signed Certificate for PrimeHub
      • Chart Configuration
      • Configure PrimeHub Store
    • Environment Variables
Powered by GitBook
On this page
  • Background
  • Goal
  • Requirements
  • Implementation
  • Bootstrap
  • Simplify the PrimeHub Installation
  • Migration From 1.x
  1. Developer Guide
  2. Design

Meta Chart

Primehub meta chart is a helm chart to install Primehub.

| PrimeHub meta chart is used by default since v2.0.0

Background

In 1.x, to install PrimeHub, we need to install these charts.

Chart
Description

keycloak

The identify server solution.

primhub-prerequisite

The CRD, CRUD, ConfigMaps for jupyterhub customizatino.

primehub-console

The primehub UI and graphql server

jupyterhub

The solution to run multi-user jupyter servers on kubernetes

admin-notebook

The primehub operation notebook.

Besides these, we still need a bootstrap script to setup the keycloak.

Goal

Aggregate all the primehub relative charts into one meta chart.

Chart
Description

keycloak

The identify server solution.

primehub

The chart to install primehub

Requirements

  1. Single chart to install PrimeHub

  2. Single helm install can install PrimeHub. No extra bootstrap script is required.

  3. The keycloak can be installed externally.

  4. Provide a way to migrate to meta chart.

Implementation

  1. Create a primehub chart at modules/charts/primehub

  2. Move primehub-prerequisite, primehub-console, jupyterhub, admin-notebook into the charts folder of the primehub chart

  3. Merge all the old charts helm values to helm/primehub/*.yaml.gotmpl

  4. Provide a migration script to merge helm overrides value files to the new primehub helm override value file.

Bootstrap

Primehub highly integrating keycloak. As primehub is installed, there are a lot of keycloak resources neeed to be created. Bootstrap script is designed to do this job.

In the meta chart, the bootstrap script is put in a kubernetes job and run whenever helm installation or upgrade.

There are two responsibility for bootstrap script.

  1. Initiate or upgrade the keycloak settings, including realm, clients, everyone group. Whenever the helm release is upgraded, the bootstrap script would run an idempotent update against the client and make sure the protocol mappers and redirect URIs are configured correctly.

    Note that the client attributes would not updated in the current version. If there are new added attribute, please make sure to enhance the logic here.

  2. Create default resources in the very beginning installation, including default users/group and default CRDs, like cpu-only instance type and base notebook image.

Currently, the default resource creation is only run when there are no user in the target realm.

Simplify the PrimeHub Installation

In old bootstrap script, it create the keycloak clients, output the client secret to .env file, and then use .env to install primehub-console and jupyterhub. The multi-stages make the primehub installation diffcult.

In the new meta-chart, the bootstrap is run in the job and the client secrets are generated and put in the kuberentes secret. It makes the installation process much simpler.

Migration From 1.x

Please follow the instructions in the migration document in ${PRIMEHUB}/modules/upgrade/migrate-v2.0/migrate-v2.0.md

PreviousUser PortalNextPrimeHub Usage

Create a bootstrap job in the chart template and it is triggered by

helm install and upgrade