icgc-dcc/es-loader

Name: es-loader

Owner: ICGC DCC

Description: Tool for initializing Elasticsearch indexes and loading data

Created: 2017-02-23 21:00:10.0

Updated: 2017-02-23 22:18:15.0

Pushed: 2017-03-30 18:06:52.0

Homepage: null

Size: 96

Language: Java

GitHub Committers

User	Most Recent Commit	# Commits
Junjun Zhang	2017-03-18 19:56:26.0	1
Du?an Andri?	2017-03-30 18:06:46.0	25

Other Committers

User	Email	Most Recent Commit	# Commits

README

Elasticsearch Loader

Facilitates the creation of Elasticsearch indexes, the loading of type mappings, and indexing of data.

The loader will parse the mappings files to construct a DAG representation of parent-child relationships to ensure the loading of mappings in the correct order.

Based on the original work done by @btiernay and @slovit

Build

In this directory run the following in the shell:

clean package

This will produce a JAR file target/es-loader-<version>.jar

Run

To see all the options the utility accepts run the JAR without any arguments:

 -jar es-loader-0.0.1-SNAPSHOT.jar 
beust.jcommander.ParameterException: The following options are required:     --es-uri     --input-dir     --config-dir 
e: <main class> [options]
tions:
    --concurrency
   How many threads to use for index loading.
   Default: 0
    --config-dir
   Config directory which contains mappings and settings.
    --es-uri
   Elasticsearch URI to index to.
    --file-prefix
   When scanning through contents of the input directory, process only files
   that start with the prefix
    --index-name-prefix
   Create index with the following prefix
    --input-dir
   Input directory which contains documents to load.
    --skip-init
   Skip indices initialization.
   Default: false

--es-uri, --config-dir, and --input-dir are mandatory. The other ones are optional.