efficient/fasst

Name: fasst

Owner: Efficient Computing at Carnegie Mellon

Description: Source code for our OSDI 2016 paper

Created: 2016-11-02 15:18:29.0

Updated: 2018-05-02 06:31:02.0

Pushed: 2018-04-21 14:55:14.0

Homepage: null

Size: 29798

Language: C++

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

FaSST

Fast, scalable, and simple transactions over RDMA. For historical reasons, FaSST is called HoTS in this codebase. This is the source code for our USENIX OSDI paper.

Required hardware and software

• InfiniBand HCAs
  • RoCE HCAs have not been tested, but should work with minor modifications. See is_roce() usage in HERD for details.
• Ubuntu 12.04+
• Mellanox OFED 2.4+
• memcached, libmemcached-dev, libmemcached-tools
• libnuma-dev (numactl-devel on CentOS)
• Boost 1.6 (earlier and newer versions not tested)
• Depends on python-dev and libbz2-dev
• libpapi-dev
• gcc-5.3 (C++11 support required; gcc-4.8 works but can have ~10% lower performance)

Required settings

All benchmarks run in a symmetric setting with multiple machines (i.e., there are no designated server or client machines. Every benchmark is contained in one directory.

• Benchmark parameters are contained in a json file in the experiment directory.
• Modify HRD_REGISTRY_IP in run-servers.sh to the IP address of server 0's machine. This machine runs a memcached instance that is used as a queue pair registry.
• Allocate hugepages on the NIC's socket at the server machine. On our machines, the NIC is attached to socket 0, so all benchmark scripts bind allocated memory and threads to socket 0. On Ubuntu systems, create 8192 hugepages on socket 0 using:

   echo 8192 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
  

Code structure

| Directory | Description | | ————- | ————- | | libhrd | An RDMA management and communication library. | | rpc | FaSST RPCs. This should depend only on libhrd and hots.h. | | mica2 | A C++ implementation of a MICA hash table. | | datastore | Interface of FaSST datastores, and RPC handlers for MICA.| | tx | The transaction protocol. | | logger | The commit record logger for transactions. | | mappings | Helper functions to determine primary/backup/log machine numbers. | | app | Benchmarks (TATP, SmallBank, RPC microbenchmark, packet loss test). | | drivers | Modified libmlx4 and libmlx5 drivers with the cheap RECV posting optimization.| | loss\_study | The code and output logs used in our InfiniBand packet loss study. | | other | Contains a DCT benchmark. |

Building the benchmarks

Each benchmark in the app directory contains a Makefile to build the benchmark.

Instructions to run on Emulab's Apt cluster:

• Set HRD_REGISTRY_IP to node-1's IP address
• Reduce HRD_MAX_INLINE to 60 bytes
• Reduce num_ports to 1
• Change the thread-to-core binding from thread i -> core 2 * i to thread i -> core i

Acknowledgements

Hyeontaek Lim's MICA code is used directly and in a modified way.

Contact

Anuj Kalia (akalia@cs.cmu.edu)

License

    Copyright 2016, Carnegie Mellon University

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.