IBM/k8s-custom-scheduler

Name: k8s-custom-scheduler

Owner: International Business Machines

Description: k8s-custom-scheduler

Created: 2017-12-13 15:37:44.0

Updated: 2018-04-30 11:39:41.0

Pushed: 2018-01-29 17:06:39.0

Homepage: null

Size: 20

Language: Go

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Demonstrating Cloud Bursting using a Custom Kubernetes Scheduler.

This is an example demonstrating how to create a custom Kubernetes scheduler. The custom scheduler modifies the Kubernetes default scheduler so that a GPU related job can be provisioned in Nimbix cloud if the local cluster is unable to satisfy the GPU resource requests.

Project Organization

nimbix-app: Base image to be used for building application docker images scheduler: Custom K8s scheduler files for Kubernetes version - 1.8.3 deploy: Sample deployment files

How to build the custom scheduler.

clone https://github.com/IBM/k8s-custom-scheduler.git
8s-custom-scheduler/scheduler
er build -t nimbix-sched .

Deploy Nimbix scheduler on IBM Cloud Private (ICP)

While these instructions are specific for ICP, it should apply for any Kubernetes setup with minor modifications

1.Create a secret for the certificate files to access apiserver over https

In ICP, certificates file reside on /etc/cfc/conf in master node. Use the following command to create the secret

ctl create secret generic certs --from-file=kube-scheduler-config.yaml --from-file=kube-scheduler.crt --from-file=kube-scheduler.key

2.Build the images for custom scheduler and sample nimbix job

Use the Dockerfiles in scheduler and nimbix-app directory

3.Deploy the scheduler

Example deployment yaml is available at deploy/k8s-custom-sched.yaml. Update the yaml file with the MASTER_IP

ctl create -f k8s-custom-sched.yaml

4.Create appropriate role binding so that custom scheduler from system:kube-scheduler can modify pods from default namespace

ctl create rolebinding someRole --clusterrole=admin --user=system:kube-scheduler --namespace=default

5.Deploy a sample GPU job using the custom scheduler

Example yaml is available at deploy/sample-job.yaml. Update the yaml with your Nimbix USERNAME and APIKEY. The job will be provisioned to Nimbix cloud if resource requirement is not met in the local cluster

ctl create -f sample-job.yaml

Authors

Abhishek Dasgupta (abdasgupta@in.ibm.com)
Pradipta Kumar Banerjee (bpradipt@in.ibm.com)