RoaringBitmap/real-roaring-datasets

Name: real-roaring-datasets

Owner: Roaring bitmaps: A better compressed bitset

Description: for benchmarking other implementations, just the datasets from https://github.com/RoaringBitmap/RoaringBitmap/tree/master/real-roaring-dataset/src/main/resources/real-roaring-dataset

Created: 2016-12-19 22:18:24.0

Updated: 2018-05-12 09:42:16.0

Pushed: 2016-12-19 22:30:22.0

Homepage:

Size: 45219

Language: null

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Real data sets for bitmap testing

See also https://github.com/RoaringBitmap/CRoaring/tree/master/benchmarks/realdata for uncompressed .txt versions.

Essentially, each file represents a set of integer values. You can create bitmaps out of these files.

In many cases, the description of the data sets is provided in :

• Samy Chambi, Daniel Lemire, Owen Kaser, Robert Godin, Better bitmap performance with Roaring bitmaps, arXiv:1402.6407. http://arxiv.org/abs/1402.6407

To be used with software published on http://roaringbitmap.org/

Files starting with the prefix “dimension” were prepared by Xavier Léauté from a Druid dump.

---

Some bitsets from a real world example, that can be used to transform them into BitSets

• bitsets_1925630_96.gz # rows with long[], in this case: long[1925630][96]

Can be deserialized by first reading an int, that is the amout of rows to come (e.g. 1925630 rows) A row is read by first reading an int, the amount of longs to come (e.g. 96 longs), and then reading those longs. Used DataInputStream to write this.

See also BitSetUtilBenchmark.