hortonworks/fieldeng-scythe

Name: fieldeng-scythe

Owner: Hortonworks Inc

Description: Time Series Library

Created: 2017-07-14 19:29:45.0

Updated: 2017-11-02 20:14:35.0

Pushed: 2018-01-02 14:50:15.0

Homepage: null

Size: 146

Language: Scala

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

fieldeng-scythe

Time Series Library

| Time | TagName | Value | | ———-|:———–:| —–:| | 12:00 | signal1 | 0 | | 12:01 | signal1 | 0.5 | | 12:02 | signal1 | 1 | | 12:03 | signal1 | 0.5 | | 12:04 | signal1 | 1 |

Interpolation

val rs = new LinearInterpolation.interpolateDate(sig1Raw, sig2Raw)

Start Spark shell with dependencies

k-shell --jars target/scythe-0.0.1-SNAPSHOT.jar

Load Sample data from CSV

rt spark.implicits._
rt spark.sqlContext._

df = spark.read.format("com.databricks.spark.csv")
ption("header", "true")
ption("delimiter", ",")
ption("inferSchema", "true")
oad("src/test/resources/example.csv")

rintSchema
how

Interpolate one tag against the other

rt com.hortonworks.scythe.cronus.Helper
ch = Helper
map = ch.interpolate ("mpg1", List("mpg1", "mpg2"), "function", ds)

Interpolate values for tag “ss” to align to “ps”

map = ch.interpolate (
s", List("ps", "ss"), 
agName", "yyyy-MM-dd HH:mm", df


values.foreach { println }

Downsample a signal

hourlyResampled = new Sample().downSample("H", "AVG", df).select("avg(value)").collect

Pattern Matching

95% precision

val rs = new PatternMatching.findPattern(.05, p, s)