Name: groonga-schema
Owner: Groonga Project
Description: Ruby library and tool that processes Groonga schema
Created: 2016-08-02 06:26:56.0
Updated: 2016-08-05 02:49:50.0
Pushed: 2016-08-16 07:04:09.0
Homepage: null
Size: 67
Language: Ruby
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
groonga-schema
Groonga-schema is a Ruby library and tool to processes Groonga's schema.
% gem install groonga-schema
Here are command lines provided by groonga-schema:
groonga-schema-diff: It reports difference between 2 schema.groonga-schema-diffgroonga-schema-diff reports difference between 2 schema:
oonga-schema-diff FROM_SCHEMA TO_SCHEMA
The output of groonga-schema-diff is a Groonga command list. It
means that you can apply difference by processing the output of
groonga-schema-diff by Groonga. The relation of them are similar to
diff and patch.
The following example shows about it.
Here are sample schema:
current.grn:
e_create Logs TABLE_NO_KEY
mn_create Logs timestamp COLUMN_SCALAR ShortText
new.grn:
e_create Logs TABLE_NO_KEY
mn_create Logs timestamp COLUMN_SCALAR Time
In the current.grn schema, Logs.timestamp column's value type is
ShortText. In the new.grn schema, it's Time.
Here is the output of groonga-schema-diff:
oonga-schema-diff current.grn new.grn
mn_create --flags "COLUMN_SCALAR" --name "timestamp_new" --table "Logs" --type "Time"
mn_copy --from_name "timestamp" --from_table "Logs" --to_name "timestamp_new" --to_table "Logs"
mn_rename --name "timestamp" --new_name "timestamp_old" --table "Logs"
mn_rename --name "timestamp_new" --new_name "timestamp" --table "Logs"
mn_remove --name "timestamp_old" --table "Logs"
The output Groonga command list does the followings:
Create a new column Logs.timestamp_new. The value type of the new column is Time not ShortText.
Copy data to Logs.timestamp_new from Logs.timestamp.
Rename Logs.timestamp to Logs.timestamp_old.
Rename Logs.timestamp_new to Logs.timestamp.
Remove Logs.timestamp_old.
It means that the output Groonga command list supports data migration.
Here is a sample database to show data migration:
oonga DB_PATH dump
e_create Logs TABLE_NO_KEY
mn_create Logs timestamp COLUMN_SCALAR ShortText
--table Logs
d","timestamp"],
2016-08-16 00:00:01"],
2016-08-16 00:00:02"],
2016-08-16 00:00:03"],
2016-08-16 00:00:04"],
2016-08-16 00:00:05"]
You can apply the change by the following command lines:
oonga-schema-diff current.grn new.grn > diff.grn
oonga --file diff.grn DB_PATH
Or:
oonga-schema-diff current.grn new.grn | groonga DB_PATH
Here is the sample database after applying the changes:
oonga DB_PATH dump
e_create Logs TABLE_NO_KEY
mn_create Logs timestamp COLUMN_SCALAR Time
--table Logs
d","timestamp"],
471273201.0],
471273202.0],
471273203.0],
471273204.0],
471273205.0]
Logs.timestamp column's value type is changed to Time from
ShortText and data are also converted.
You can also use groonga-schema-diff to remote databases.
The following command line shows difference between databases served
at http://192.168.0.1:10041 and http://192.168.0.2:10041/:
oonga-schema-diff \
'http://192.168.0.1:10041/d/dump?dump_records=no' \
'http://192.168.0.2:10041/d/dump?dump_records=no'
You can apply the output Groonga command list by groonga-client
command provided by groonga-client-cli gem:
oonga-client --host 192.168.0.1 diff.grn
The following command synchronizes schema at
http://192.168.0.1:10041/ with schema at http://192.168.0.1:10042/:
oonga-schema-diff \
'http://192.168.0.1:10041/d/dump?dump_records=no' \
'http://192.168.0.2:10041/d/dump?dump_records=no' |
oonga-client --host 192.168.0.1
After this command line, the following command line outputs nothing
because there are no difference between schema at
http://192.168.0.1:10041/ and http://192.168.0.1:10042/:
oonga-schema-diff \
'http://192.168.0.1:10041/d/dump?dump_records=no' \
'http://192.168.0.2:10041/d/dump?dump_records=no'
NOTE: You should use database carefully while applying changes. Because some tables and columns may be removed while applying changes. If you touch removed tables and/or columns from another threads, Groonga may be crashed. It's better that you reduce the max number of threads to 1 while applying changes like the following:
ho thread_limit 1 | groonga-client --host 192.168.0.1
oonga-schema-diff \
'http://192.168.0.1:10041/d/dump?dump_records=no' \
'http://192.168.0.2:10041/d/dump?dump_records=no' |
oonga-client --host 192.168.0.1
ho thread_limit 8 | groonga-client --host 192.168.0.1
NOTE: You can't use the thread_limit technique with groonga-httpd
because groonga-httpd is multi-process model not multi-threading
model. You need to reduce the number of workers by changing
worker_processes to 1 in groonga-httpd.conf and reload the
configuration file. You also need to increase the number of workers
after you apply the changes.
TODO…
LGPLv2.1 or later. See doc/text/lgpl-2.1.txt for details.
(Kouhei Sutou has a right to change the license including contributed patches.)