Name: sarjitsu
Owner: distributed-system-analysis
Description: dockerized setup for visualizing System Activity Report (SAR) data.
Created: 2016-04-28 22:48:05.0
Updated: 2018-03-29 08:49:56.0
Pushed: 2018-03-19 13:26:56.0
Homepage: http://arcolife.github.io/blog/2016/06/06/sarjitsu-a-project-on-visualizing-your-systems-activity/
Size: 2752
Language: Python
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Sarjitsu ingests a SAR (System Activity Reporter) binary data file (located under /var/log/sa/) ..and produces dynamic visualizations based on Grafana. The name is inspired from 'SAR + jistsu', so to speak, unleashing the power of SAR data.
You could also refer to this blog post on Sarjitsu to know more about this app and catch glimpses of some screenshots.
The app is composed of following containerized instances:
1) datasource: Used to store timeseries data and metadata. A full-text search engine powered by elasticsearch
2) metricstore: PostgreSQL powered, used by the frontend (Grafana) to store metadata about dashboards, datasource and users.
3) frontend: Powered by Grafana, a dynamic visualization frontend which sources data from elasticsearch, and stored metadata in metricstore.
4) middleware: A Python-Flask powered API that talks to backend and metricstore; builds scriped dashboards
5) backend: A ~~NodeJS (rewritten in Flask)~~ Python-Flask powered web app, which runs the web interface
for users to upload a SA binary file and obtain visualizations.
6) caching: Redis based caching solution for the web app.
7) proxy: Nginx based proxy for web app.
Application flow is explained in detail in the section APP FLOW below.
Prerequisites: docker-compose
Copy env.example to .env. Then, run $ docker-compose up --build -d
t: `docker-compose up -d`
nup: `docker-compose rm` or `docker-compose rm --all`
down: `docker-compose down`
art: `docker-compose restart`
: `docker-compose kill`
Get your openshift cluster up and running. Refer to ose-deployment.md
Once it's up, do this:
mpose convert -f docker-compose.yml --provider openshift -o openshift/templates/
create -f openshift/templates/
or this:
mpose up --provider openshift
ther of:
get pods,svc -o wide
get pods -w
get pods --show-all=false
Once all the pods are running, do this:
expose svc nginx
tionally, supply --hostname='' ..if a DNS hostname resolution entry is reserved for this application.
Sarjitsu is accessible through Nginx container's URL / IP address.
If deployed through docker-compose, access the nginx IP.
If you've used default settings from env.example, the URL should be http://0.0.0.0:8001/
If deployed through openshift, use the exposed nginx route.
Defaults to sample URL -> http://nginx-myproject.<your IP address>.xip.io/
One could experiment with sample SA binaires included in test/ folder.
You could also upload files through the commandline tool vizit, from under utils/ folder. This is useful when you're working out of a remote server and unable to access SA binaries (since sarjitsu's web interface requires selection of files from your local machine).
For using vizit, you need to fulfill requirements as follows:
cd utils/
virtualenv venv -p python3
source venv/bin/activate
pip3 install -r requirements.txt
Now, edit conf/vizit.cfg under utils/ and update with sarjitsu server's nginx URL/address and Port.
./vizit [ options ] [ <datafile/dir path> ]
fault: $ ./vizit <sa01 path> <sa02 path> ..
tions are:
[-f path to a single SA binary file.]
[-d path to directory with a bunch of SA binary files in it.]
[-r path to pbench results dir. Recursively locates all sar.data]
Some examples:
./vizit -f <path to sa binary file>"
./vizit -d <path to dir w/ sa binaries>"
./vizit -r <pbench latest results dir>
A sample of various outputs from vizit tool is present in docs/vizit_output_sample.txt
# iptables -F
to flush the IP tables in the server where it is running. Otherwise check your firewall
settings. Or the proxy environment settings, like that of Nginx, if you've
routed your application in that fashion.Building docker images on first run would take some time, as the images are pulled from dockerhub, customized & built; then packages are installed and so on..
If you have custom HOST(s) configured, they should match the following versions (for sarjitsu compatibility reasons):
Elasticsearch < 2.0 and > 1.5 (containerized version: 1.7.3)
Grafana > 2.5 and <= 3.0 (containerized version: 3.0.1-1)
Postgres == 9.5 (containerized version: 9.5 (dockerhub latest) ..utilizes UPSERT feature introduced in this version)
Without docker-compose (WARNING: not recommended / supported anymore), a container can be started using the following approach:
er rm -f elastic_jitsu
er build -t sarjitsu_elasticsearch --build-arg ES_PORT=9200 .
er run --name elastic_jitsu -p 9601:9200 -d sarjitsu_elasticsearch
R single line:
er rm -f elastic_jitsu; docker build -t saitsu_elasticsearch --build-arg ES_PORT=9200 . && docker run --name elastic_jitsu -p 9601:9200 -d sarjitsu_elasticsearch
upply additional args if needed: -v /sys/fs/cgroup:/sys/fs/cgroup:ro --privileged
lthough there's a new method out there for running containers without systemd. Checkout the links:
ttps://developers.redhat.com/blog/2016/09/13/running-systemd-in-a-non-privileged-container/
ttps://github.com/RHsyseng/container-rhel-examples/tree/master/starter-systemd
To delete the pods / services as well as the openshift cluster itself, do this:
delete service,dc,is,bc datasource frontend metricstore middleware nginx redis web
mpose down --provider=openshift
cluster down
We've included a script that automates following steps lib/backend/activate_dev_mode.
But as of now, it only works with Fedora and Ubuntu OS. For other Operating Systems, refer below:
cp env.example .envdocker-compose up -d.docker-compose kill web and make sure rest are running docker-compose psOutput should be like this:
Command State Ports
---------------------------------------------------------------------------------------
itsu_datasource_1 /docker-entrypoint.sh elastic Up 0.0.0.0:9200->9200/tcp
itsu_frontend_1 /docker-entrypoint.sh graf ... Up 0.0.0.0:3000->3000/tcp
itsu_metricstore_1 container-entrypoint run-p ... Up 0.0.0.0:5432->5432/tcp
itsu_middleware_1 /docker-entrypoint.sh api_ ... Up 0.0.0.0:5000->5000/tcp
itsu_nginx_1 /docker-entrypoint.sh prox ... Up 0.0.0.0:8001->8001/tcp
itsu_redis_1 docker-entrypoint.sh redis ... Up 0.0.0.0:6379->6379/tcp
itsu_web_1 /docker-entrypoint.sh backend Exit 137
` sarjitsu/lib/backend/` and run this: '
sticSearch]
= 0.0.0.0
= 9200
tings]
x_prefix = sarjitsu
x_version = 1
_action_count = 2000
er_of_shards = 5
er_of_replicas = 1
fana]
board_url = http://0.0.0.0:3000
url = http://0.0.0.0:5000/db/create/
conf/sar-index.cfg
'DEBUG = True' > src/config_local.py
'CFG_PATH = "'$(realpath conf/sar-index.cfg)'"' >> src/config_local.py
For Ubuntu OS:
apt-get install libgpgme11 libgpgme11-dev python3-gpgme python3-dev
For Fedora OS:
dnf install python3-devel gpgme-devel gpgme
pip dependencies:ualenv -p python3 venv
ce venv/bin/activate
install -r requirements.txt
redis hostname: echo '0.0.0.0 redis' | sudo tee -a /etc/hostslib/backend:rt VOS_CONFIG_PATH="$(realpath conf/sar-index.cfg)"
rc/
rver.py
From docs/sarjitsu_architecture.png:
Following steps involved in visualizing SA binary file:
Sarjitsu's frontend service segragates data into various panels, based on params like CPU, Disk, Network usage.
Description of those parameters could be obtained in detail by running the command
$ man sar on a linux terminal. Or you could read about them here on the official man page for sar command
Files generated in RHEL 5 OS (old enough) ..upto the ones generated by the latest sysstat version.
What sarjitsu does, is it gives you the unique ability throw in any version of
sa binary files from your system's /var/log/sa/ folder, to produce visualizations
with all the SAR params supported till date. This makes it much easier for a user
to go see what's wrong / different about the system behavior by instantly getting
access to all the data indexed in a nice NoSQL based full-text search engine and
a dynamic visualization playground. It further simplifies this process by providing
a web interface to upload these files too!
Sarjitsu also automatically detects the time range of the sa files to display the time-series visualizations and names the dashboards based on the nodename of your system.
It is not tied to a specific machine, but is an independent solution, as a web app.
Sarjitsu is scalable since it keeps the datasource, frontend and backend separately and is based on completely scalable and portable solution, i.e., Elasticsearch, Postgres, Grafana et al, in their respective containerized environments using Docker.
Please use Github issues list and its features for contributions. Click here list of issues. If you think there's a bug in sarjitsu, you're welcome to open an issue here on github and could submit PRs for the same. Any refactoring suggestions/PRs to the app are also welcome.
The default GitHub Issues and Pull Requests interface.
Refer to GPL v3 LICENSE file included in the repo, for reference.