openshift/online-analytics

Name: online-analytics

Owner: OpenShift

Description: An application for reporting OpenShift cluster usage analytics

Created: 2017-09-08 17:28:08.0

Updated: 2018-05-14 15:32:54.0

Pushed: 2018-03-26 19:52:18.0

Homepage: null

Size: 48890

Language: Go

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Data Analytics Integration

This application syncs OpenShift user activity data with external analytics systems.

Usage

Using Woopra with default IDs

ew-app -n openshift-infra -f ansible/roles/oso_analytics/files/user-analytics.yaml \
OOPRA_ENABLED="true" \
OOPRA_ENDPOINT="http://www.woopra.com/track/ce" \
OOPRA_DOMAIN="YOURDOMAIN" \
SER_KEY_STRATEGY="[name/uid/annotation]" \
SER_KEY_ANNOTATION="[desired annotation]" \
OG_LEVEL=5
LUSTER_NAME="<int/stg/prod/..>"

The CLUSTER_NAME parameter is used to distinguish different environments (such as INT, STG, or a local test cluster). Its value by default is “kubernetes”.

Note the 2 new flags USER_KEY_STRATEGY and USER_KEY_ANNOTATION. Key strategy refers to the keying method used for users in Woopra:

• name will key users by their user.Name
• uid will key users by their user.UID
• annotation will key users by an annotation specified in USER_KEY_ANNOTATION (this flag is only required if using strategy=annotation)

The annotation previously used in Devpreview was openshift.io/online-managed-id

Prometheus metrics enablement

The following flags enable Prometheus metrics gathering:

ETRICS_PORT="8080" \
ETRICS_COLLECT_RUNTIME="true" \
ETRICS_COLLECT_WOOPRA="true" \
ETRICS_COLLECT_QUEUE="true" \

The metrics are available at METRICS_PORT/metrics

METRICS_COLLECT_RUNTIME enables various runtime, Go, and process metrics provided by Prometheus

METRICS_COLLECT_WOOPRA enables Woopra latency metrics at /analytics_woopra_latency_seconds

METRICS_COLLECT_QUEUE enables analytics on the controller's internal events processing queue available at /analytics_queue_size_events, which returns the current size of the queue, and /analytics_events_handled, which returns the total number of events processed since the start of the controller.

Local development and testing

Using a Local Endpoint

ew-app -n openshift-infra -f templates/user-analytics.yaml \
OOPRA_ENABLED="false" \
OCAL_ENDPOINT_ENABLED="true" \
OG_LEVEL=5

Note! This will log to glog.V(5). A user must be logged in for an analytic to be counted.

Generate analytics by logging in and creating a project w/ basic app.

 login
entication required for https://10.240.0.2:8443 (openshift)
name: foo
word:
n successful.

don't have any projects. You can try to create a new project, by running

oc new-project <projectname>

 new-project test
using project "test" on server "https://10.240.0.2:8443".

can add applications to this project with the 'new-app' command. For example, try:

oc new-app centos/ruby-22-centos7~https://github.com/openshift/ruby-ex.git

uild a new example application in Ruby.
 new-app centos/ruby-22-centos7~https://github.com/openshift/ruby-ex.git

Building

Build and test with make:

• make will run vendor dependencies and then install
• make build will compile the binary for the application
• make test will compile and run the unit tests
• make test-integration will compile and run the integration tests against an OpenShift master

Analytic Events

The following events are observed by a controller and sent to an analytics provider via Basic Authenticated GET w/ encoded URL.

User

user_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | user_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |

user_deleted

| | parameter | v3 field | description | |———–|———–|———-|————| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | user_deleted | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Deleted | ce_timestamp | obj.DeletionTimestamp | in milliseconds |

pod_failed

| | parameter | v3 field | description | |———–|———–|———-|————| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | pod_failed | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Failed | ce_timestamp | pod.Status.Condition.LastTransitionTime | in milliseconds | | Failed | ce_reason | pod.Status.Condition.Reason | in milliseconds |

ReplicationController

replicationcontroller_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | replicationcontroller_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds | | Replica Count | ce_replica_count | obj.Spec.Replicas | The number of replicas of a pod |

PersistentVolumeClaim

persistentvolumeclaim_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | persistentvolumeclaim_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds | | Capacity | ce_capacity | obj.Spec.Capacity | The requested storage capacity | | Access Modes | ce_access_modes | stringify(obj.Spec.AccessModes | The requested access modes for storage |

persistentvolumeclaim_bound

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | persistentvolumeclaim_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Bound Date | ce_timestamp | obj.Condition.LastTransitionTime | in milliseconds | | Request Capacity | ce_requested_capacity | obj.Spec.Capacity | The requested storage capacity | | Request Access Modes | ce_requested_access_modes | stringify(obj.Spec.AccessModes | The requested access modes for storage | | Actual Capacity | ce_actual_capacity | obj.Status.Capacity | The actual storage capacity of the backing volume | | Actual Access Modes | ce_actual_access_modes | stringify(obj.Status.AccessModes | The actual storage capacity of the backing volume |

deploymentconfig_failed

| | parameter | v3 field | description | |———–|———–|———-|————| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | deploymentconfig_failed | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Failed | ce_timestamp | ? | ? |

build_failed

| | parameter | v3 field | description | |———–|———–|———-|————| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | build_failed | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Failed | ce_timestamp | ? | ? |

Pod

pod_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | pod_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |

ReplicationRontroller

replicationrontroller_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | replicationrontroller_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |

PersistentVolumeClaim

persistentvolumeclaim_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | persistentvolumeclaim_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |

Secret

secret_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | secret_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |

Service

service_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | service_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |

Namespace

namespace_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | namespace_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |

Deployment

deployment_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | deployment_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |

Route

route_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | route_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |

Build

build_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | build_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |

RoleBinding

rolebinding_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | rolebinding_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |

Template

template_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | template_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |

ImageStream

imagestream_created

| | parameter | v3 field | description | |———–|—————|———————–|————————-| | ID | cv_id | user.annotation[“openshift.io/online-managed-id”] | User Identifier for analytics | | Event | event | imagestream_created | Analytic event identifier | | Name | ce_name | obj.Name | v3 object name (but not identifier/UID) | | Namespace | ce_namespace | obj.Namespace | object's project/namespace | | Created | ce_timestamp | obj.CreationTimestamp | in milliseconds |