RoaringBitmap/gocroaring

Name: gocroaring

Owner: Roaring bitmaps: A better compressed bitset

Description: Go wrapper for CRoaring

Created: 2016-08-09 21:55:18.0

Updated: 2017-12-29 10:58:55.0

Pushed: 2017-09-25 14:11:31.0

Homepage: http://roaringbitmap.org/

Size: 343

Language: C

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

gocroaring

Go wrapper for CRoaring (a C/C++ implementation at https://github.com/RoaringBitmap/CRoaring)

Roaring bitmaps are used by several important systems:

• Apache Lucene and derivative systems such as Solr and Elastic,
• Metamarkets' Druid,
• Apache Spark,
• Netflix Atlas,
• LinkedIn Pinot,
• OpenSearchServer,
• Cloud Torrent,
• Whoosh,
• Pilosa,
• Microsoft Visual Studio Team Services (VSTS),
• eBay's Apache Kylin.

Roaring bitmaps are found to work well in many important applications:

Use Roaring for bitmap compression whenever possible. Do not use other bitmap compression methods (Wang et al., SIGMOD 2017)

The original java version can be found at https://github.com/RoaringBitmap/RoaringBitmap

There is a native Go version at https://github.com/RoaringBitmap/roaring

This code is licensed under Apache License, Version 2.0 (ASL2.0).

Copyright 2016 by the authors.

References

• Daniel Lemire, Owen Kaser, Nathan Kurz, Luca Deri, Chris O'Hara, François Saint-Jacques, Gregory Ssi-Yan-Kai, Roaring Bitmaps: Implementation of an Optimized Software Library arXiv:1709.07821
• Samy Chambi, Daniel Lemire, Owen Kaser, Robert Godin, Better bitmap performance with Roaring bitmaps, Software: Practice and Experience Volume 46, Issue 5, pages 709?719, May 2016 http://arxiv.org/abs/1402.6407 This paper used data from http://lemire.me/data/realroaring2014.html
• Daniel Lemire, Gregory Ssi-Yan-Kai, Owen Kaser, Consistently faster and smaller compressed bitmaps with Roaring, Software: Practice and Experience (accepted in 2016, to appear) http://arxiv.org/abs/1603.06549

Dependencies

None in particular.

Naturally, you also need to grab the roaring code itself:

• go get github.com/RoaringBitmap/gocroaring

Example

Here is a simplified but complete example:

age main

rt (
"fmt"

"github.com/RoaringBitmap/gocroaring"


 main() {
// example inspired by https://github.com/fzandona/goroar
fmt.Println("==roaring==")
rb1 := gocroaring.New(1, 2, 3, 4, 5, 100, 1000)
rb1.RunOptimize() // improves compression
fmt.Println("Cardinality: ", rb1.Cardinality())
fmt.Println("Contains 3? ", rb1.Contains(3))

rb2 := gocroaring.New()
rb2.Add(3, 4, 1000)
rb2.RunOptimize() // improves compression

rb1.And(rb2)
// prints {3,4,1000}
fmt.Println(rb1)

rb3 := gocroaring.New(1, 5)
rb3.Or(rb1)

    // prints 1, 3, 4, 5, 1000
    i := rb3.Iterator()
    for i.HasNext() {
      fmt.Println(i.Next())
    }
    fmt.Println()

fmt.Println(rb3.ToArray())
fmt.Println(rb3)

rb4 := gocroaring.FastOr(rb1, rb2, rb3) // optimized way to compute unions between many bitmaps
fmt.Println(rb4)

// next we include an example of serialization
buf := make([]byte, rb1.SerializedSizeInBytes())
rb1.Write(buf) // we omit error handling
newrb, _ := gocroaring.Read(buf)
if rb1.Equals(newrb) {
    fmt.Println("I wrote the content to a byte stream and read it back.")
}

fmt.Println(rb1.Stats()) // show the cardinality and the numbers of each type of container used.

Documentation

Current documentation is available at http://godoc.org/github.com/RoaringBitmap/gocroaring

Mailing list/discussion group

https://groups.google.com/forum/#!forum/roaring-bitmaps

Compatibility with Java RoaringBitmap library

You can read bitmaps in Go, Java, C, C++ that have been serialized in Java, Java, C, C++.