HewlettPackard/Jenkins-stats

Name: Jenkins-stats

Owner: Hewlett Packard Enterprise

Description: Tool for gathering job data from the Jenkins CI system (https://jenkins-ci.org/) and generating summary stat reports and graphs. These reports cover metrics such as success/failure rates and job duration.

Created: 2018-01-03 19:12:02.0

Updated: 2018-05-10 07:50:12.0

Pushed: 2018-01-05 12:00:40.0

Homepage:

Size: 201

Language: Python

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

© Copyright 2017 Hewlett Packard Enterprise Development LP

jenkins-stats

Introduction

Tool for gathering job data from the Jenkins CI system (https://jenkins-ci.org/) and generating summary stat reports and graphs. These reports cover metrics such as success/failure rates and job duration.

Used initially for jobs triggered by OpenStack Zuul (https://docs.openstack.org/infra/zuul/) but should work with other Jenkins jobs too.

Screenshots

Example output with overall and per-slave success/failure rates

Example output with overall and per-slave job durations

Installation

Using virtualenv

clone <url to jenkins-stats repo>
 apt install virtualenv
ualenv ~/venv/jenkins-stats
venv/jenkins-stats/bin/activate
install -r requirements.txt

Running

Running as script

t_jenkins_stats.py -s <jenkins url> -j <jenkins job name> -o <output directory>

e.g.

t_jenkins_stats.py -s https://www.example.net/jenkins/ -j my-jenkins-job -o /var/tmp/jenkins
rting on jobs between 10:55:06 30-Nov-2017 and 10:55:06 14-Dec-2017
/tmp/jenkins/my-jenkins-job.json not found, creating new data_file
 0 builds from /var/tmp/jenkins/my-jenkins-job.json
enkins-job is a new job, querying all builds
ing new record for build 1121
ing new record for build 1122
ing new record for build 1123

e .html
e /var/tmp/jenkins/my-jenkins-job.html

You can then view the report by opening /var/tmp/jenkins/my-jenkins-job.html in your browser.

Running as container

Build container

• Set VERSION to the version you want to use for your image (https://semver.org/)
• build-args can be dropped if you're not afflicted by a proxy

rt VERSION=0.0.2
er build --build-arg version=${VERSION} --build-arg http_proxy=$http_proxy --build-arg https_proxy=$https_proxy -t zingstats/jenkins-stats:${VERSION} -t zingstats/jenkins-stats:latest .

Run container

er run -v $(pwd)/output:/var/www/html/jenkins/stats -e JENKINS_URL=<jenkins url> -e JENKINS_JOB=<jenkins job name> zingstats/jenkins-stats:${VERSION}

Testing

TODO (gasp!)

Design

General strategy

1. Retrieve jenkins job data from Jenkins server.
2. Write data to a file in JSONL (https://en.wikipedia.org/wiki/JSON_Streaming#Line_delimited_JSON)
3. Load data into Pandas DataFrames
4. Resample to days or hours
5. Tabulate and plot

Roadmap / Potential TODO

1. Add proper testing.
2. Move from JSONL to a db?
3. Split gathering and analysis into separate tools?

License

This project is licensed under the Apache 2.0 license. Please see LICENSE for more info.

Contributing and feature requests

Contributing: You know the drill. Fork it, branch it, change it, commit it, and pull-request it. We are passionate about improving this project, and glad to accept help to make it better. However, keep the following in mind:

• You must sign a Contributor License Agreement first. Contact one of the authors (from Hewlett Packard Enterprise) for details and the CLA.
• We reserve the right to reject changes that we feel do not fit the scope of this project, so for feature additions, please open an issue to discuss your ideas before doing the work.

Feature Requests: If you have a need that is not met by the current implementation, please let us know (via a new issue). This feedback is crucial for us to deliver a useful product. Do not assume we have already thought of everything, because we assure you that is not the case.