functional-streams-for-scala/fs2

Name: fs2

Owner: functional-streams-for-scala

Description: Compositional, streaming I/O library for Scala

Created: 2013-03-06 17:34:15.0

Updated: 2018-01-16 16:48:58.0

Pushed: 2018-01-11 09:31:40.0

Homepage:

Size: 6266

Language: Scala

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

FS2: Functional Streams for Scala (previously 'Scalaz-Stream')

Quick links:

• About the library
• How to get latest version
• API docs (fs2-core), API docs (fs2-io)
• Docs and getting help

About the library

FS2 is a streaming I/O library. The design goals are compositionality, expressiveness, resource safety, and speed. Here's a simple example of its use:

rt cats.effect.{IO, Sync}
rt fs2.{io, text}
rt java.nio.file.Paths

fahrenheitToCelsius(f: Double): Double =
 - 32.0) * (5.0/9.0)

converter[F[_]](implicit F: Sync[F]): F[Unit] =
.file.readAll[F](Paths.get("testdata/fahrenheit.txt"), 4096)
.through(text.utf8Decode)
.through(text.lines)
.filter(s => !s.trim.isEmpty && !s.startsWith("//"))
.map(line => fahrenheitToCelsius(line.toDouble).toString)
.intersperse("\n")
.through(text.utf8Encode)
.through(io.file.writeAll(Paths.get("testdata/celsius.txt")))
.compile.drain

t the end of the universe...
u: Unit = converter[IO].unsafeRunSync()

This will construct a F[Unit], converter, which reads lines incrementally from testdata/fahrenheit.txt, skipping blank lines and commented lines. It then parses temperatures in degrees Fahrenheit, converts these to Celsius, UTF-8 encodes the output, and writes incrementally to testdata/celsius.txt, using constant memory. The input and output files will be closed upon normal termination or if exceptions occur.

At the end it's saying that the effect F will be of type cats.effect.IO and then it's possible to invoke unsafeRunSync(). You can choose a different effect type or your own as long as it implements cats.effect.Sync.

The library supports a number of other interesting use cases:

• Zipping and merging of streams: A streaming computation may read from multiple sources in a streaming fashion, zipping or merging their elements using an arbitrary Tee. In general, clients have a great deal of flexibility in what sort of topologies they can define–source, sink, and effectful channels are all first-class concepts in the library.
• Dynamic resource allocation: A streaming computation may allocate resources dynamically (for instance, reading a list of files to process from a stream built off a network socket), and the library will ensure these resources get released upon normal termination or if exceptions occur.
• Nondeterministic and concurrent processing: A computation may read from multiple input streams simultaneously, using whichever result comes back first, and a pipeline of transformations can allow for nondeterminism and queueing at each stage.

Documentation and getting help

• The official guide is a good starting point for learning more about the library.
• The FAQ has frequently asked questions. Feel free to open issues or PRs with additions to the FAQ!
• Also feel free to come discuss and ask/answer questions in the gitter channel and/or on StackOverflow using the tag FS2.

Blog posts and other external resources are listed on the Additional Resources page.

Where to get the latest version

• API docs (fs2-core 0.10.0-RC1) API docs (fs2-io 0.10.0-RC1)

The 0.10 release is almost complete and will be released when Cats 1.0 is released. Milestone builds are available now. The 0.10 migration guide summarizes the differences between 0.10 and 0.9. To get 0.10, add the following to your SBT build:

vailable for Scala 2.11, 2.12
aryDependencies += "co.fs2" %% "fs2-core" % "0.10.0-RC1" // For cats 1.0.1 and cats-effect 0.8

ptional I/O library
aryDependencies += "co.fs2" %% "fs2-io" % "0.10.0-RC1"

The 0.9 release is out and we recommend upgrading. You may want to first read the 0.9 migration guide if you are upgrading from 0.8 or earlier. To get 0.9, add the following to your SBT build:

vailable for Scala 2.11, 2.12
aryDependencies += "co.fs2" %% "fs2-core" % "0.9.7"

ptional I/O library
aryDependencies += "co.fs2" %% "fs2-io" % "0.9.7"

The fs2-core library is also supported on Scala.js:

vailable for Scala 2.11.8, 2.12.0
aryDependencies += "co.fs2" %%% "fs2-core" % "0.9.7"

API docs:

• The core library
• The io library, FS2 bindings for NIO-based file I/O and TCP/UDP networking

The previous stable release is 0.8.4 (source).

Projects using FS2

If you have a project you'd like to include in this list, either open a PR or let us know in the gitter channel and we'll add a link to it here.

• doobie: Pure functional JDBC built on fs2.
• fs2-crypto: TLS support for fs2.
• fs2-http: Http server and client library implemented in fs2.
• http4s: Minimal, idiomatic Scala interface for HTTP services using fs2.
• fs2-kafka: Simple client for Apache Kafka.
• fs2-rabbit: Stream-based client for RabbitMQ built on top of Fs2.
• scodec-stream: A library for streaming binary decoding and encoding, built using fs2 and scodec.
• scodec-protocols: A library for working with libpcap files. Contains many interesting pipes (e.g., working with time series and playing back streams at various rates).
• streamz: A library that supports the conversion of Akka Stream Sources, Flows and Sinks to and from FS2 Streams, Pipes and Sinks, respectively. It also supports the usage of Apache Camel endpoints in FS2 Streams and Akka Stream Sources, Flows and SubFlows.
• fs2-zk: Simple Apache Zookeeper bindings for fs2.
• fs2-reactive-streams: A reactive streams implementation for fs2.
• circe-fs2: Streaming JSON manipulation with circe.
• fs2-elastic: Simple client for Elasticsearch

Related projects

FS2 has evolved from earlier work on streaming APIs in Scala and Haskell and in Scala. Some influences:

• Machines, a Haskell library by Ed Kmett, which spawned scala-machines
• The FP in Scala stream processing library developed for the book FP in Scala
• Reflex, an FRP library in Haskell, by Ryan Trinkle
• There are various other iteratee-style libraries for doing compositional, streaming I/O in Scala, notably the scalaz/iteratee package and iteratees in Play.

Presentations, Blogs, etc.

See Additional resources.

Acknowledgments

Special thanks to YourKit for supporting this project's ongoing performance tuning efforts with licenses to their excellent product.