PrimeHub Usage

PrimeHub Usage provides administrators a overall insight of the usage of the PrimeHub.

Usage is about allocated resources, not about actual utilization. For example, when an user opens an Jupyter notebook, the record of the allocated resources is logged in the usage data, even if the user doesn't run any program actually on it. The each record includes the lifetime of a pod, and CPU/GPU/Memory are allocated/occupied for a pod.


  • PrimeHub administrator can download monthly usage report (CSV format)


  • Actual utilization is not covered in this scope.

The utilization describes the ratio of actual utilized resources to allocated resources.


To enable PrimeHub Usage, set the usage.enabled to true.

PathDescriptionDefault Value


If the PrimeHub Usage is enabled



PrimeHub Usage is made of five components:

  • Usage

    • API: a Rest API to query data from the usage database

    • Prober: a watcher to save pod events in the usage database

  • Database: a Postgresql database saves data of a pvc created by StatefulSet

  • Reporting: a cronjob to generate monthly reports daily, it generates two reports (this month and last month) each time.

  • Monitor: similar to Prober, but it only monitors pod events if it is not updated recently. If a pod hasn't been changed for a while, it will mark the pod finished. In general case, the finished state should be handled by Prober. However, we do a final check in a separated process to deal with edge cases.


apiVersion: v1
kind: Pod
  annotations: '{"component": "jupyter", "component_name": "jupyter-foo",
      "group": "phusers", "user": "foo", "instance_type": "cpu-1"}'
  creationTimestamp: "2020-08-31T17:51:31Z"
    app: jupyterhub
    chart: jupyterhub-0.9-dev

A prober watches pod events and filtering events in specific namespace (e.g. hub) with annotation It defines the lifetime of a pod between

  • A pod when scheduled

  • Terminated time of the last container


Usage API is an internal API consumed by GraphQL.

Available months

curl http://primehub-usage-api/report/monthly

Get report from a month

curl http://primehub-usage-api/report/monthly/2020/9

Legacy resources migration

A cluster might have lots of resources created before PrimeHub Usage enabled. There is a tool to migrate legacy resources by patching their annotation.

There is a in the prober pod:

kubectl -n hub exec -it primehub-usage-prober-it-is-an-example --

After execution, it generates patch commands if some resources are needed to patch:

Please review the commands before applying them.

# patch jupyter pod: jupyter-foo
kubectl -n hub patch pod jupyter-foo --type='json' -p '[{"op": "add", "path": "/metadata/annotations/", "value": "{\"component\": \"jupyter\", \"component_name\": \"jupyter-foo\", \"group\": \"phusers\", \"user\": \"foo\", \"instance_type\": \"cpu-1\"}"}]'

# patch phdeployment deployment: tmp-1gawm, it might restart pods if anything have changed
kubectl -n hub patch deployment tmp-1gawm --type='json' -p '[{"op": "add", "path": "/spec/template/metadata/annotations/", "value": "{\"component\": \"deployment\", \"component_name\": \"tmp-1gawm\", \"group\": \"model-deployment-test-group\", \"user\": \"ericy\", \"instance_type\": \"cpu-tiny\"}"}]'

Last updated