neo4j-contrib/boltkit

Name: boltkit

Owner: Neo4j Contrib

Description: Toolkit for Neo4j 3.0+ driver authors

Created: 2016-07-15 14:35:13.0

Updated: 2018-04-26 14:58:46.0

Pushed: 2018-04-26 14:58:44.0

Homepage:

Size: 219

Language: Python

GitHub Committers

UserMost Recent Commit# Commits

Other Committers

UserEmailMost Recent Commit# Commits

README

Boltkit

Boltkit is a collection of tools and resources for Neo4j 3.0+ driver authors.

Contents

Installation

The package can either be installed globally or within a virtualenv. Installation makes available several command line tools, the names of which all start with either bolt or neoctrl.

install --upgrade boltkit
Demo Driver

This file contains both a fully-working Neo4j driver as well as a step-by-step tutorial for how to implement a driver in any language. To view the code and tutorial in a terminal, use:

 $(python -c "from boltkit import driver; print(driver.__file__)")
Statement Runner

Example:

run "UNWIND range(1, 10) AS n RETURN n"
Stub Bolt Server

The stub Bolt server can be used as a testing resource for client software. Scripts can be created against which unit tests can be run without the need for a full Neo4j server.

A server script describes a conversation between client and server in terms of the messages exchanged. An example can be seen in the test/scripts/count.bolt file.

When the server receives a client message, it will attempt to match that against the next client message in the script; if found, this line of the script will be consumed. Then, any server messages that follow will also be consumed and sent back. When the client closes its connection, the server will shut down. If any script lines remain, the server will exit with an error status; if none remain it will exit successfully. After 30 seconds of inactivity, the server will time out and shut down with an error status.

Scripting

Scripts generally consist of alternating client (C:) and server (S:) messages. Each message line contains the message name followed by its fields, in JSON format.

Some messages, such as INIT and RESET, can be automatically (successfully) consumed if they are not relevant to the current test. For this use a script line such as !: AUTO INIT.

An example:

UTO INIT
UTO RESET

UN "RETURN {x}" {"x": 1}
ULL_ALL
UCCESS {"fields": ["x"]}
ECORD [1]
UCCESS {}
Command Line Usage

To run a stub server script:

stub 7687 test/scripts/count.bolt

To run a Cypher command against the stub server:

run "UNWIND range(1, 10) AS n RETURN n"
Java Test Usage

The stub server can be used from any environment from which command line tools can be executed. To use from Java, first construct a wrapper for the server:

rt java.io.IOException;
rt java.util.ArrayList;
rt java.util.List;

rt static java.lang.Thread.sleep;
rt static java.util.Arrays.asList;
rt static java.util.Collections.singletonList;

ic class StubServer

// This may be thrown if the driver has not been closed properly
public static class ForceKilled extends Exception {}

private Process process = null;

private StubServer( String script, int port ) throws IOException, InterruptedException
{
    List<String> command = new ArrayList<>();
    // This assumes the `boltstub` command is available on the path
    command.addAll( singletonList( "boltstub" ) );
    command.addAll( asList( Integer.toString( port ), script ) );
    ProcessBuilder server = new ProcessBuilder().inheritIO().command( command );
    process = server.start();
    sleep( 500 );  // might take a moment for the socket to start listening
}

public static StubServer start( String script, int port ) throws IOException, InterruptedException
{
    return new StubServer( script, port );
}

public int exitStatus() throws InterruptedException, ForceKilled
{
    sleep( 500 );  // wait for a moment to allow disconnection to occur
    try
    {
        return process.exitValue();
    }
    catch ( IllegalThreadStateException ex )
    {
        // not exited yet
        process.destroy();
        process.waitFor();
        throw new ForceKilled();
    }
}

Then, assuming you have a valid script available, create a test to use the stub:

t
ic void shouldBeAbleRunCypher() throws StubServer.ForceKilled, InterruptedException, IOException

// Given
StubServer server = StubServer.start( "/path/to/resources/return_x.bolt", 7687 );
URI uri = URI.create( "bolt://localhost:7687" );
int x;

// When
try ( Driver driver = GraphDatabase.driver( uri ) )
{
    try ( Session session = driver.session() )
    {
        Record record = session.run( "RETURN {x}", parameters( "x", 1 ) ).single();
        x = record.get( 0 ).asInt();
    }
}

// Then
assertThat( x, equalTo( 1 ) );

// Finally
assertThat( server.exitStatus(), equalTo( 0 ) );
Neo4j Controller

The Neo4j controller module comprises a set of scripts for downloading, starting, stopping and configuring Neo4j servers. These scripts should work for any 3.0+ server version and can pull artifacts from local file system, TeamCity and AWS server (credentials might be required).

Module also contains neoctrl-cluster and neoctrl-multicluster commands that give ability to install, start and stop Neo4j Causal clusters and Neo4j Multi-clusters. Neo4j cluster commands support 3.1+ enterprise Neo4j versions only. And Neo4j multi-cluster commands support 3.4+ enterprise Neo4j versions only.

Standalone Server
neoctrl-download 
e: neoctrl-download [-h] [-e] [-v] version [path]

load a Neo4j server package for the current platform.

ple:
octrl-download -e 3.1.0-M09 $HOME/servers/
octrl-download -e 3.3 $HOME/servers/

tional arguments:
rsion           Neo4j server version
th              download destination path (default: .)

onal arguments:
, --help        show this help message and exit
, --enterprise  select Neo4j Enterprise Edition (default: Community)
, --verbose     show more detailed output

ronment variables:
ST_HOST         name of distribution server (default: dist.neo4j.org)
AMCITY_HOST     name of build server
AMCITY_USER     build server user name
AMCITY_PASSWORD build server password
S_ACCESS_KEY_ID aws access key id
S_SECRET_ACCESS_KEY aws secret access key
OCTRL_LOCAL_PACKAGE local package source is always preferred if configured


CITY_* environment variables are required to download snapshot servers.
* environment variables are used to access enterprise distribution servers.

rt bugs to drivers@neo4j.com
neoctrl-install 
e: neoctrl-install [-h] [-e] [-v] version [path]

load and extract a Neo4j server package for the current platform.

ple:
octrl-install -e 3.1.0-M09 $HOME/servers/

tional arguments:
rsion           Neo4j server version
th              download destination path (default: .)

onal arguments:
, --help        show this help message and exit
, --enterprise  select Neo4j Enterprise Edition (default: Community)
, --verbose     show more detailed output

neoctrl-download for details of supported environment variables.

rt bugs to drivers@neo4j.com
neoctrl-start 
e: neoctrl-start [-h] [-v] [home]

t an installed Neo4j server instance.

ple:
octrl-start $HOME/servers/neo4j-community-3.0.0

tional arguments:
me           Neo4j server directory (default: .)

onal arguments:
, --help     show this help message and exit
, --verbose  show more detailed output

rt bugs to drivers@neo4j.com
neoctrl-stop 
e: neoctrl-stop [-h] [-v] [-k] [home]

 an installed Neo4j server instance.

ple:
octrl-stop $HOME/servers/neo4j-community-3.0.0

tional arguments:
me           Neo4j server directory (default: .)

onal arguments:
, --help     show this help message and exit
, --verbose  show more detailed output
, --kill     forcefully kill the instance

rt bugs to drivers@neo4j.com
neoctrl-create-user 
e: neoctrl-create-user [-h] [-v] [home] user password

te a new Neo4j user.

ple:
octrl-create-user $HOME/servers/neo4j-community-3.0.0 bob s3cr3t

tional arguments:
me           Neo4j server directory (default: .)
er           name of new user
ssword       password for new user

onal arguments:
, --help     show this help message and exit
, --verbose  show more detailed output

rt bugs to drivers@neo4j.com
neoctrl-configure 
e: neoctrl-configure [-h] [-v] [home] key=value [key=value ...]

te Neo4j server configuration.

ple:
octrl-configure . dbms.security.auth_enabled=false

tional arguments:
me           Neo4j server directory (default: .)
y=value      key/value assignment

onal arguments:
, --help     show this help message and exit
, --verbose  show more detailed output

rt bugs to drivers@neo4j.com
neoctrl-set-initial-password 
e: neoctrl-set-initial-password [-h] password [home]

 the initial password of the initial admin user ('neo4j').

ple:
octrl-set-initial-password newPassword $HOME/servers/neo4j-community-3.0.0

tional arguments:
ssword    password for the admin user
me        Neo4j server directory (default: .)

onal arguments:
, --help  show this help message and exit

rt bugs to drivers@neo4j.com
Cluster
e: neoctrl-cluster [-h] {install,start,stop} ...

ate Neo4j causal cluster.

onal arguments:
, --help            show this help message and exit

lable sub-commands:
art                 Start the causal cluster located at the given path
op                  Stop the causal cluster located at the given path
stall               Download, extract and configure causal cluster

nstall,start,stop}  commands are available

neoctrl-download for details of supported environment variables.

rt bugs to drivers@neo4j.com
neoctrl-cluster install 
e: neoctrl-cluster install [-h] [-v] [-c CORE_COUNT]
                           [-r READ_REPLICA_COUNT] [-i INITIAL_PORT]
                           -p PASSWORD
                           version [path]

load, extract and configure causal cluster

ple:
octrl-cluster install [-v] [-c 3] -p init_password 3.1.0 $HOME/cluster/

tional arguments:
rsion               Neo4j server version
th                  download destination path (default: .)

onal arguments:
, --help            show this help message and exit
, --verbose         show more detailed output
 CORE_COUNT, --cores CORE_COUNT
                    number of core members in the cluster (default 3)
 READ_REPLICA_COUNT, --read-replicas READ_REPLICA_COUNT
                    number of read replicas in the cluster (default 0)
 INITIAL_PORT, --initial-port INITIAL_PORT
                    initial port number for all used ports on all cluster
                    members. Each next port will simply be an increment of
                    the previous one (default 20000)
 PASSWORD, --password PASSWORD
                    initial password of the initial admin user ('neo4j')
                    for all cluster members (always required)

neoctrl-download for details of supported environment variables.

rt bugs to drivers@neo4j.com
neoctrl-cluster start 
e: neoctrl-cluster start [-h] [-t TIMEOUT] [path]

t the causal cluster located at the given path

ple:
octrl-cluster start $HOME/cluster/

tional arguments:
th                  causal cluster location path (default: .)

onal arguments:
, --help            show this help message and exit
 TIMEOUT, --timeout TIMEOUT
                    startup timeout in seconds (default: 120)

neoctrl-download for details of supported environment variables.

rt bugs to drivers@neo4j.com
neoctrl-cluster stop 
e: neoctrl-cluster stop [-h] [-k] [path]

 the causal cluster located at the given path

ple:
octrl-cluster stop $HOME/cluster/

tional arguments:
th        causal cluster location path (default: .)

onal arguments:
, --help  show this help message and exit
, --kill  forcefully kill all instances in the cluster

neoctrl-download for details of supported environment variables.

rt bugs to drivers@neo4j.com
Multi-Cluster
e: neoctrl-multicluster [-h] {install,start,stop} ...

ate Neo4j multi-cluster.

onal arguments:
, --help            show this help message and exit

lable sub-commands:
art                 Start the multi-cluster located at the given path
op                  Stop the multi-cluster located at the given path
stall               Download, extract and configure multi-cluster

nstall,start,stop}  Commands are available

neoctrl-download for details of supported environment variables.
neoctrl-multicluster install 
e: neoctrl-multicluster install [-h] [--path PATH] -p PASSWORD [-v] -d
                                DATABASE
                                version

load, install and configure multi-cluster

ple:
octrl-multicluster install 3.4.0 [--path $HOME/multi-cluster/] -p pAssw0rd [-v] -d '{"london": {"c": 3, "r": 2}, "malmo": {"c": 5, "i": 9001}}'

tional arguments:
rsion               Neo4j server version

onal arguments:
, --help            show this help message and exit
path PATH           download destination path (default: .)
 PASSWORD, --password PASSWORD
                    initial password of the initial admin user ('neo4j')
                    for all cluster members
, --verbose         show more detailed output
 DATABASE, --database DATABASE
                    a json string describing the multi-cluster structure
                    in the form of {database_name: {c:core_size,
                    r:read_replica_size, i:initial_port},...} defaults:
                    core_size=3, read_replia_size=0, initial_port=20000
                    e.g. '{"london": {"c": 3, "r": 2}, "malmo": {"c": 5,
                    "i": 9001}}'

neoctrl-download for details of supported environment variables.

rt bugs to drivers@neo4j.com
neoctrl-multicluster start 
e: neoctrl-multicluster start [-h] [-v] [-t TIMEOUT] [path]

t the multi-cluster located at the given path

ple:
octrl-multicluster start [-v] $HOME/cluster/

tional arguments:
th                  multi-cluster location path (default: .)

onal arguments:
, --help            show this help message and exit
, --verbose         show more detailed output
 TIMEOUT, --timeout TIMEOUT
                    startup timeout for each cluster inside the
                    multicluster in seconds (default: 180)

neoctrl-download for details of supported environment variables.

rt bugs to drivers@neo4j.com
neoctrl-multicluster stop 
e: neoctrl-multicluster stop [-h] [-v] [-k] [path]

 the multi-cluster located at the given path

ple:
octrl-multicluster stop [-v] $HOME/cluster/

tional arguments:
th           multi-cluster location path (default: .)

onal arguments:
, --help     show this help message and exit
, --verbose  show more detailed output
, --kill     forcefully kill all instances in the multi-cluster. Default:
             false

neoctrl-download for details of supported environment variables.

rt bugs to drivers@neo4j.com
This work is supported by the National Institutes of Health's National Center for Advancing Translational Sciences, Grant Number U24TR002306. This work is solely the responsibility of the creators and does not necessarily represent the official views of the National Institutes of Health.