PrimeHub Usage
Last updated
Last updated
PrimeHub Usage provides administrators a overall insight of the usage of the PrimeHub.
Usage is about allocated resources, not about actual utilization. For example, when an user opens an Jupyter notebook, the record of the allocated resources is logged in the usage data, even if the user doesn't run any program actually on it. The each record includes the lifetime of a pod, and CPU/GPU/Memory are allocated/occupied for a pod.
PrimeHub administrator can download monthly usage report (CSV format)
Actual utilization is not covered in this scope.
The utilization describes the ratio of actual utilized resources to allocated resources.
To enable PrimeHub Usage, set the usage.enabled
to true
.
Path | Description | Default Value |
---|---|---|
PrimeHub Usage is made of five components:
Usage
API: a Rest API to query data from the usage database
Prober: a watcher to save pod events in the usage database
Database: a Postgresql database saves data of a pvc created by StatefulSet
Reporting: a cronjob to generate monthly reports daily, it generates two reports (this month and last month) each time.
Monitor: similar to Prober, but it only monitors pod events if it is not updated recently. If a pod hasn't been changed for a while, it will mark the pod finished. In general case, the finished state should be handled by Prober. However, we do a final check in a separated process to deal with edge cases.
A prober watches pod events and filtering events in specific namespace (e.g. hub
) with annotation primehub.io/usage
. It defines the lifetime of a pod between
A pod when scheduled
Terminated time of the last container
Usage API is an internal API consumed by GraphQL.
A cluster might have lots of resources created before PrimeHub Usage enabled. There is a tool to migrate legacy resources by patching their primehub.io/usage
annotation.
There is a primehub-usage-legacy-pods-helper.py
in the prober pod:
After execution, it generates patch commands if some resources are needed to patch:
Please review the commands before applying them.
usage.enabled
If the PrimeHub Usage is enabled
false