Name: Pulse
Owner: gophergala2016
Description: Machine Learning for your Logs!!!
Created: 2016-01-22 19:23:33.0
Updated: 2018-05-02 08:50:44.0
Pushed: 2016-01-25 00:55:47.0
Homepage: http://pulse.clemsonopoly.com
Size: 495
Language: Go
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Log pulse learns from your log files. It uses a machine learning algorithm that Michael Dropps came up with. It is a go package that can be consumed and used for use for anyone. The package itself just reads lines of strings and returns what it thinks is out of place. That way when you are trying to find that error in your logs, you don't spend hours searching and looking. We have made a simple application around it to show case it's ability.
The application is simple. If you run it with no flags or arguments it will read the PulseConfig.toml file and read those files listed there. If you include arguments but no flags then the arguments must be filepaths to logs you want to read. EX LogPulse someFile.log anotherFile.log waitHereIsAnother.log.
LogPulse accepts one flag -api. It accepts a file on an endpoint in the body and runs the algorithm. It will email the user when it is done with all the anomalies it could find (we are using MailGun). If you wanted to run local you could supply an SMTP config file (location is set in PulseConfig.toml and must be a toml file). This is were the credentials are so you are able to send emails locally. You could have the SMTP config file setup and run LogPulse without the -api flag and it would send emails as well. If no email option is set it will save all emails (subject and body) to the output file that is specified in the PulseConfig.toml
To use the algorithm just import the package as such!
import "github.com/gophergala2016/Pulse/pulse"
This package exposes the Run(chan string, func(string)) function. You just need to create a channel that you are going to use. It does require that it is passed in line by line as well. The func(string) is a function that is called whenever an unusual string comes by. It is highly recommended that if this is being written to a file to buffer a few strings before you write. Then when you have read all strings dump the rest of the buffer in the file.
Installing is as simple as:
go get github.com/gophergala2016/Pulse/LogPulse
go run main.go <Path/to/File>
The PulseConfig.toml needs to be located in the same directory as your executable. The file should look similar to this:
ist = [
oData/kern.log.1",
oData/kern.log.2"
lList = [
euser@example.org",
eConley@example.org",
reAwesome@example.org"
utFile = "PulseOut.txt"
Config = "SMTP.toml"
= 8080
LogList is a list of strings. This is where the log files are located that you want pulse to read.
EmailList is also a list of strings. But this is everyone that you want to email when something is unusual
OutputFile is just a string. It is where the emails are sent if you do not setup an SMTP server (don't have SMTPConf file).
SMTPConfig is the location of you SMTP credentials (explained below).
Port is the port on which the API server will listen on.
The SMTP.toml can be anywhere you want it as long as the application can read the file. It is where all the required information is to send email to the SMTP server. It should look like:
ver]
= "smtp.server.com"
= 25
r]
Name = "user@server.com"
Word = "LovelyPassword"
[Server] is a table with Host and Port
Host is the where the server is listening to receive emails to send.Port is the port on which the server is listening[User] is also a table but with UserName and PassWord
UserName is the email address at which the email is sending from.PassWord is the password for the user that is sending the email