discourse/syslogstash

Name: syslogstash

Owner: Discourse

Description: Feed syslog messages from a UNIX socket to logstash

Created: 2015-12-02 00:51:32.0

Updated: 2018-04-21 01:49:34.0

Pushed: 2018-04-21 01:49:38.0

Homepage: null

Size: 55

Language: Ruby

GitHub Committers

UserMost Recent Commit# Commits

Other Committers

UserEmailMost Recent Commit# Commits

README

Syslogstash is intended to provide a syslog-compatible socket for one or more applications to send their syslog messages to. The messages are then parsed and sent to a logstash server for posterity. No more needing to run a syslog server that writes to a file, just to have a second program that reads those files again. With syslogstash, everything is in one neat little package.

If you're running a containerised environment, there's a reasonable chance you've got multiple things that want to log to syslog, but you want to keep them organised and separate. That's easy: just run multiple syslogstash instances, one per “virtual syslog socket” you want to provide. Multiple containers can share the same socket, they'll just share a logstash connection and have the same metadata / extra tags.

For maximum flexibility, you can optionally feed the syslog messages to one or more other “downstream” sockets, and/or print all the log messages to stdout for ad-hoc “local” debugging.

Installation

It's a gem:

gem install syslogstash

There's also the wonders of the Gemfile:

gem 'syslogstash'

If you're the sturdy type that likes to run from git:

rake install

Or, if you've eschewed the convenience of Rubygems entirely, then you presumably know what to do already.

Docker

Published image at https://hub.docker.com/r/discourse/syslogstash/

To build a new Docker image, run rake docker:build. A rake docker:push will push out a new release.

Usage

Syslogstash is configured by means of environment variables. At the very least, syslogstash needs to know where logstash is (LOGSTASH_SERVER), and the socket to listen on for syslog messages (SYSLOG_SOCKET). You specify those on the command line, like so:

LOGSTASH_SERVER=logstash-json \
  SYSLOG_SOCKET=/dev/log \
  syslogstash

The full set of environment variables, and their meaning, is described in the “Syslogstash Configuration” section, below.

Logstash server setup

The logstash server(s) you send the collected messages to must be configured to listen on a TCP port with the json_lines codec. This can be done quite easily as follows:

  tcp {
    port  => 5151
    codec => "json_lines"
  }

Adjust the port number to taste.

Signals

There are a few signals that syslogstash recognises, to control various aspects of runtime operation. They are:

Use with Docker

For convenience, syslogstash is available in a Docker container, discourse/syslogstash:v2. It requires a bit of gymnastics to get the syslog socket from the syslogstash container to whatever container you want to capture syslog messages from. Typically, you'll want to share a volume between the two containers, tell syslogstash to create its socket there, and then symlink /dev/log from the other container to there.

For example, you might start the syslogstash container like this:

docker run -v /srv/docker/syslogstash:/syslogstash \
  -e LOGSTASH_SERVER=logstash-json \
  -e SYSLOG_SOCKET=/syslogstash/log.sock \
  discourse/syslogstash:v2

Then use the same volume in your other container:

docker run -v /srv/docker/syslogstash:/syslogstash something/funny

In the other container's startup script, include the following command:

ln -sf /syslogstash/log.sock /dev/log

… and everything will work nicely.

If you feel like playing on nightmare mode, you can also mount the log socket directly into the other container, like this:

docker run -v /srv/docker/syslogstash/log.sock:/dev/log something/funny

This allows you to deal with poorly-implemented containers which run software that logs to syslog but doesn't provide a way to override where /dev/log points. However, due to the way bind mounts and Unix sockets interact, if the syslogstash container restarts for any reason, you also need to restart any containers that have the socket itself as a volume. If you can coax your container management system into satisfying that condition, then you're golden.

Syslogstash Configuration

All configuration of syslogstash is done by placing values in environment variables. The environment variables that syslogstash recognises are listed below.

Contributing

Bug reports should be sent to the Github issue tracker. Patches can be sent as a [Github pull request](https://github.com/discourse/syslogstash/pulls].

Licence

Unless otherwise stated, everything in this repo is covered by the following copyright notice:

Copyright (C) 2015, 2018 Civilized Discourse Construction Kit Inc.

This program is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License version 3, as
published by the Free Software Foundation.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.
This work is supported by the National Institutes of Health's National Center for Advancing Translational Sciences, Grant Number U24TR002306. This work is solely the responsibility of the creators and does not necessarily represent the official views of the National Institutes of Health.