Name: json-avro-converter
Owner: Allegro Tech
Description: JSON to Avro conversion tool designed to make migration to Avro easier.
Created: 2016-02-05 11:56:25.0
Updated: 2018-01-18 19:03:07.0
Pushed: 2017-10-20 12:16:41.0
Homepage: null
Size: 135
Language: Groovy
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
JSON to Avro conversion tool designed to make migration to Avro easier. Includes simple command line validator.
Apache Avro ships with some very advanced and efficient tools for reading and writing binary Avro but their support for JSON to Avro conversion is unfortunately limited and requires wrapping fields with type declarations if you have some optional fields in your schema. This tool is supposed to help with migrating project from using JSON to Avro without having to modify JSON data if it conforms to the JSON schema.
ndencies {
compile group: 'tech.allegro.schema.json2avro', name: 'converter', version: '0.2.5'
rt tech.allegro.schema.json2avro.converter.AvroConversionException;
rt tech.allegro.schema.json2avro.converter.JsonAvroConverter;
rt org.apache.avro.generic.GenericData;
rt org.apache.avro.Schema;
vro schema with one string field: username
ng schema =
"{" +
" \"type\" : \"record\"," +
" \"name\" : \"Acme\"," +
" \"fields\" : [{ \"name\" : \"username\", \"type\" : \"string\" }]" +
"}";
ng json = "{ \"username\": \"mike\" }";
AvroConverter converter = new JsonAvroConverter();
onversion to binary Avro
[] avro = converter.convertToAvro(json.getBytes(), schema);
onversion to GenericData.Record
ricData.Record record = converter.convertToGenericDataRecord(json.getBytes(), new Schema.Parser().parse(schema));
onversion from binary Avro to JSON
[] binaryJson = converter.convertToJson(avro, schema);
xception handling
ng invalidJson = "{ \"username\": 8 }";
{
converter.convertToAvro(invalidJson.getBytes(), schema);
tch (AvroConversionException ex) {
System.err.println("Caught exception: " + ex.getMessage());
A command line tool for validating your JSON/Avro documents against the schema.
To bundle the tool into single executable JAR:
adlew :validator:shadowJar
-jar validator/build/libs/json2avro-validator-{version}.jar --help
Running Validator with --help
option will print help message listing all possible arguments.
Sample Avro schema and messages can be found in:
validator/src/test/resources/user.avcs
validator/src/test/resources/user.json
validator/src/test/resources/user.avro
You can validate your JSON to Avro conversion:
-jar json2avro-validator.jar -s user.avcs -i user.json
If everything will process correctly, the process will end with zero status code.
You can convert the Avro binary data into JSON by setting mode -m avro2json
option:
-jar json2avro-validator.jar -s user.avcs -i user.avro -m avro2json
If you would like to know how message will look like after encoding and decoding invoke:
-jar json2avro-validator.jar -s user.avcs -i user.json -m json2avro2json
json-avro-converter is published under Apache License 2.0.