IBM/deploy-graph-db-container

Name: deploy-graph-db-container

Owner: International Business Machines

Description: Host a graph database such as OrientDB on IBM Container Service using Kubernetes APIs

Created: 2017-08-16 14:05:02.0

Updated: 2018-04-17 10:28:00.0

Pushed: 2017-12-05 15:09:25.0

Homepage: https://developer.ibm.com/code/patterns/cloud-host-graph-database-using-kubernetes

Size: 4077

Language: null

GitHub Committers

UserMost Recent Commit# Commits

Other Committers

UserEmailMost Recent Commit# Commits

README

Deploy OrientDB container on Bluemix Kubernetes

Read this in other languages: ???.

Graph databases, such as OrientDB, store data in a graph structure consisting of nodes, edges and properties. Graph databases, by design, allow simple and fast retrieval of complex hierarchical structures in a much more efficient manner than relational databases. Gremlin is a standardised graph traversal language for retrieving data from graph databases (the way SQL is for RDBMS).

In this journey we show you how to quickly deploy OrientDB on Bluemix Container Service, so that you can leverage it for your team's development and test purposes.

IBM Bluemix Container Service combines Docker and Kubernetes to deliver powerful tools to automate the deployment, operation, scaling, and monitoring of containerized apps over a cluster of independent compute hosts by using the Kubernetes APIs.

This journey gives you step by step instructions for:

Flow

architecture/Flow diagram

  1. Log in to Bluemix CLI and initialize Bluemix Container Service plugin.
  2. Set context for Kubernetes CLI by downloading Bluemix Kubernetes configuration files and setting KUBECONFIG environment variable.
  3. Save desired OrientDB password in Kubernetes secret.
  4. Configure persistent storage for OrientDB volumes.
  5. Deploy OrientDB container and NodePort service to Kubernetes cluster.
  6. Access OrientDB dashboard by using the public IP address of worker node and the NodePort mapped to OrientDB's HTTP port.
Watch the Overview Video

Kubernetes Concepts Used
Included Components
Steps
  1. Setup Bluemix Kubernetes Cluster
  2. 1.1 Install CLI for Bluemix and Kubernetes
  3. 1.2 Log in to the Bluemix CLI and initialize Bluemix Container Service plugin
  4. 1.3 Create your Bluemix Kubernetes cluster
  5. 1.4 Point the Kubernetes CLI to your Bluemix Kubernetes cluster
  6. Deploy OrientDB service into Kubernetes clusters
  7. 2.1 Copy OrientDB Kubernetes configuration scripts
  8. 2.2 Save desired OrientDB password in Kubernetes secret
  9. 2.3 Configure persistent storage for OrientDB volumes
  10. 2.4 Deploy OrientDB into Kubernetes cluster
  11. 2.5 View a local version of the Kubernetes dashboard
  12. 2.6 Open OrientDB dashboard
  13. Import a public database and explore it using OrientDB Dashboard and Gremlin console
  14. 3.1 Import a public database
  15. 3.2 Explore schema and data (vertices/edges) using OrientDB dashboard
  16. 3.3 Open Gremlin/OrientDB console and run queries

Troubleshooting

Step 1. Setup Bluemix Kubernetes Cluster
1.1 Install CLI for Bluemix and Kubernetes

Set up Bluemix and Kubernetes CLI as per instructions in https://console.bluemix.net/docs/containers/cs_tutorials.html#cs_cluster_tutorial. The steps are repeated here for quick reference.

1.2 Log in to the Bluemix CLI and initialize Bluemix Container Service plugin
1.3 Create your Bluemix Kubernetes cluster

Bluemix allows you to create a free cluster that comes with 2 CPUs, 4 GB memory, and 1 worker node. This is called lite cluster and allows you to get familiar with and test Kubernetes capabilities. However they lack capabilities like persistent NFS file-based storage with volumes.

To setup your cluster for maximum availability and capacity, Bluemix allows you to create a fully customizable, production-ready cluster called standard cluster. Standard clusters allow highly available cluster configurations such as a setup with two clusters that run in different regions, each with multiple worker nodes. Please see https://console.bluemix.net/docs/containers/cs_planning.html#cs_planning_cluster_config to review other options for highly available cluster configurations.

A detailed comparison of capabilities of lite and standard clusters is given in https://console.bluemix.net/docs/containers/cs_planning.html#cs_planning.

1.4 Point the Kubernetes CLI to your Bluemix Kubernetes cluster

Configure your Kubernetes CLI environment to point to your Bluemix Kubernetes cluster as below.

Step 2. Deploy OrientDB service into Kubernetes clusters
2.1 Copy OrientDB Kubernetes configuration scripts

Clone or download the OrientDB Kubernetes configuration scripts to your user home directory.

t clone https://github.com/IBM/deploy-graph-db-container.git

Navigate to the source directory

 deploy-graph-db-container
2.2 Save desired OrientDB password in Kubernetes secret

Create a new file called password.txt in the same directory and put your desired OrientDB password inside password.txt (Could be any string with ASCII characters).

We need to make sure password.txt does not have any trailing newline. Use the following command to remove possible newlines.

 -d '\n' <password.txt >.strippedpassword.txt && mv .strippedpassword.txt password.txt

Put OrientDB password in Kubernetes secret

bectl create secret generic orientdb-pass --from-file=password.txt
et "orientdb-pass" created
2.3 Configure persistent storage for OrientDB volumes

OrientDB docker image requires following directories to be volume mounted so as to persist data across container delete/relaunch.

entdb/databases
entdb/backup

If you are using Bluemix standard Kubernetes cluster, then you can leverage dynamic volume provisioning which allows storage volumes to be created on-demand. To use this feature, update the value of volume.beta.kubernetes.io/storage-class annotation in orientdb.yaml to one of the NFS file-based storage classes supported in Bluemix: ibmc-file-bronze or ibmc-file-silver or ibmc-file-gold. Also change accessModes to ReadWriteMany and increase storage request to say 20GB.

: PersistentVolumeClaim
ersion: v1
data:
me: orientdb-pv-claim
bels:
service: orientdb
type: pv-claim
notations:
volume.beta.kubernetes.io/storage-class: "ibmc-file-gold"
:
cessModes:
ReadWriteMany
sources:
requests:
  storage: 20Gi
notations:

In case you are using Bluemix lite Kubernetes cluster, where NFS file storage is not supported, you can instead use hostPath PersistentVolume. A hostPath PersistentVolume uses a file or directory on the Node to emulate network-attached storage. To create a hostPath PersistentVolume, review local-volumes.yaml and run kubectl apply command.

t local-volumes.yaml
ersion: v1
: PersistentVolume
data:
me: "pv-volume"
bels:
type: local
:
pacity:
storage: "5Gi"
cessModes:
- "ReadWriteOnce"
stPath:
path: /tmp
rsistentVolumeReclaimPolicy: Recycle

Create hostPath persistent volume

bectl apply -f local-volumes.yaml
istentvolume "pv-volume" created
2.4 Deploy OrientDB into Kubernetes cluster

Run the OrientDB Kubernetes configuration script in the cluster. When the deployment and the service are created, OrientDB is available as a service for users.

bectl apply -f orientdb.yaml
istentvolumeclaim "orientdb-pv-claim" created
oyment "orientdbservice" created
ice "orientdbservice" created

The orientdb.yaml script creates a Kubernetes deployment for OrientDB container. The OrientDB password is fetched from the Kubernetes secret created in Step 1.2 above. Similarly the persistent volumes configured in Step 1.3 above are used as the persistent storage for OrientDB volumes. The corresponding snippet from orientdb.yaml script is shown below.

: Deployment
ersion: extensions/v1beta1
data:
me: orientdbservice
bels:
service: orientdb
:
plicas: 1
mplate:
metadata:
  name: orientdbservice
  labels:
    service: orientdb
    type: container-deployment
spec:
  containers:
  - name: orientdbservice
    image: orientdb:2.2.26
    env:
    - name: ORIENTDB_ROOT_PASSWORD
      valueFrom:
        secretKeyRef:
          name: orientdb-pass
          key: password.txt
    ports:
    - containerPort: 2424
      name: port-binary
    - containerPort: 2480
      name: port-http
    volumeMounts:
    - mountPath: /orientdb/databases
      name: orientdb-data
      subPath: databases
    - mountPath: /orientdb/backup
      name: orientdb-data
      subPath: backup
  volumes:
  - name: orientdb-data
    persistentVolumeClaim:
      claimName: orientdb-pv-claim

The orientdb.yaml script also exposes OrientDB ports (HTTP: 2480 and binary: 2424) to the internet by creating a Kubernetes service of type NodePort as shown in the snippet below.

: Service
ersion: v1
data:
me: orientdbservice
bels:
service: orientdb
type: nodeport-service
:
pe: NodePort
lector:
service: orientdb
type: container-deployment
rts:
protocol: TCP
port: 2424
name: binary
protocol: TCP
port: 2480
name: http
2.5 View a local version of the Kubernetes dashboard.

Launch your Kubernetes dashboard with the default port 8001.

bectl proxy

Open the following URL in a web browser to see the Kubernetes dashboard. http://localhost:8001/ui Kubernetes Dashboard

In the Workloads tab, you can see the resources that you created. When you are done exploring the Kubernetes dashboard, use CTRL+C to exit the proxy command.

2.6 Open OrientDB dashboard

Get information about the deployed OrientDB service to see which NodePort was assigned for OrientDB's HTTP port 2480.

bectl describe service orientdbservice
:           orientdbservice
space:      default
ls:         service=orientdb
        type=nodeport-service
tations:        kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"service":"orientdb","type":"nodeport-service"},"name":"orientdbservice","na...
ctor:       service=orientdb,type=container-deployment
:           NodePort
        10.10.10.177
:           binary  2424/TCP
Port:       binary  32039/TCP
oints:      172.xx.xxx.xx:2424
:           http    2480/TCP
Port:       http    31420/TCP
oints:      172.xx.xxx.xx:2480
ion Affinity:   None
ts:         <none>

Get the public IP address for the worker node in the cluster.

 cs workers mycluster

                                               Public IP        Private IP     Machine Type   State    Status   Version
-hou02-pa85736d86a8f24324806f9b83d24960e5-w1   173.xxx.xx.xxx   10.47.64.200   free           normal   Ready    1.7.4_1502

Open a browser and check out the OrientDB dashboard with the following URL.

://<Public_IP_address>:<HTTP_NodePort>/studio/index.html#/

OrientDB Dashboard

Step 3. Import a public database and explore it using OrientDB Dashboard and Gremlin console
3.1 Import a public database
3.2 Explore schema and data (vertices/edges) using OrientDB dashboard
3.3 Open Gremlin/OrientDB console and run queries

Troubleshooting

References

License

Apache 2.0

This work is supported by the National Institutes of Health's National Center for Advancing Translational Sciences, Grant Number U24TR002306. This work is solely the responsibility of the creators and does not necessarily represent the official views of the National Institutes of Health.