Name: cassandra-cloud-backup
Owner: Google Cloud Platform
Description: Cassandra backups to Google Cloud Storage
Created: 2016-03-04 15:38:47.0
Updated: 2018-04-18 09:32:06.0
Pushed: 2018-01-05 21:04:39.0
Homepage: https://cloud.google.com/solutions/google-cloud-storage-for-cassandra-disaster-recovery
Size: 45
Language: Shell
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Shell script for creating and managing Cassandra Backups using Google Cloud Storage.
Google Cloud SDK installed with gsutil utility configured for authentication to An existing Google Cloud Storage bucket Linux system with BASH shell. Cassandra 2+
./cassandra-cloud-backup.sh [ options ] < command>
Take a full snapshot, gzip compress it with nice level 15, use the /var/lib/cassandra/backups directory to stage the backup before uploading it to the GCS Bucket, and clear old incremental and snapshot files
./cassandra-cloud-backup.sh -b gs://cassandra-backups123/ -zCc -N 15 -d /var/lib/cassandra/backups backup
Do a dry run of a full snapshot with verbose output and create list of files that would have been copied
./cassandra-cloud-backup.sh -b gs://cassandra-backups123/ -vn backup
Backup and bzip2 compress copies of the most recent incremental backup files since the last incremental backup
./cassandra-cloud-backup.sh -b gs://cassandra-backups123/ -ji -d /var/lib/cassandra/backups backup
Restore a backup without prompting from given bucket path and keep the old files locally
./cassandra-cloud-backup.sh -b gs://cass-bk123/backups/host01/snpsht/2016-01-20_18-57/ -fk -d /var/lib/cassandra/backups restore
List inventory of available backups stored in Google Cloud Store
./cassandra-cloud-backup.sh -b gs://cass-bk123 inventory
List inventory of available backups stored in Google Cloud Store for a different server
./cassandra-cloud-backup.sh -b gs://cass-bk123 inventory -a testserver01
Flags:
-a, –alt-hostname
Specify an alternate server name to be used in the bucket path construction. Used
to create or retrieve backups from different servers
-B, backup
Default action is to take a backup
-b, –gcsbucket Google Cloud Storage bucket used in deployment and by the cluster.
-c, –clear-old-ss
Clear any old SnapShots taken prior to this backup run to save space
additionally will clear any old incremental backup files taken immediately
following a successful snapshot. this option does nothing with the -i flag
-C, –clear-old-inc
Clear any old incremental backups taken prior to the the current snapshot
-d, –backupdir
The directory in which to store the backup files, be sure that this directory
has enough space and the appropriate permissions
-D, –download-only
During a restore this will only download the target files from GCS
-f, –force
Used to force the restore without confirmation prompt
-h, –help
Print this help message.
-H, –home-dir
This is the $CASSANDRA_HOME directory and is only used if the data_directories, commitlog_directory,
or the saved_caches_directory values cannot be parsed out of the yaml file.
-i, –incremental
Copy the incremental backup files and do not take a snapshot. Can only
be run when compression is enabled with -z or -j
-j, –bzip
Compresses the backup files with bzip2 prior to pushing to Google Cloud Storage
This option will use additional local disk space set the --target-gz-dir
to use an alternate disk location if free space is an issue
-k, –keep-old
Set this flag on restore to keep a local copy of the old data files
Set this flag on backup to keep a local copy of the compressed backup and schema dump
-l, –log-dir
Activate logging to file 'CassandraBackup${DATE}.log' from stdout
Include an optional directory path to write the file
Default path is /var/log/cassandra
-n, –noop
Will attempt a dry run and verify all the settings are correct
-N, –nice
Set the process priority, default 10
-p
The Cassandra User Password if required for security
-r, restore
Restore a backup, requires a --gcsbucket path and optional --backupdir
-s, –split-size
Split the resulting tar archive into the configured size in Megabytes, default 100M
-S, –service-name
Specify the service name for cassandra, default is cassandra use to stop and start service
-T, –target-gz-dir
Override the directory to save compressed files in case compression is used
default is --backupdir/compressed, also used to decompress for restore
-u
The Cassandra User account if required for security
-U, –auth-file
A file that contains authentication credentials for cqlsh and nodetool consisting of
two lines:
CASSANDRA_USER=username
CASSANDRA_PASS=password
-v –verbose
When provided will print additional information to log file
-y, –yaml
Path to the Cassandra yaml configuration file
default: /etc/cassandra/cassandra.yaml
-z, –zip
Compresses the backup files with gzip prior to pushing to Google Cloud Storage
This option will use additional local disk space set the --target-gz-dir
to use an alternate disk location if free space is an issue
30 1 * * * /path_to_scripts/cassandra-cloud-backup.sh -z -N10 -b gs://cass-bk123-vCcj -d /var/lib/cassandra/backups > /var/log/cassandra/$(date +\%Y\%m\%d\%H\%M\%S)-fbackup.log 2>&1
0 * * * * /path_to_scripts/cassandra-cloud-backup.sh -b -N10 gs://cass-bk123 -vjiz -d /var/lib/cassandra/backups > /var/log/cassandra/$(date +\%Y\%m\%d\%H\%M\%S)-ibackup.log 2>&1
The script must be run with sufficient privileges to be able to stop/start processes and create/delete directories and files within the working directories.
The restore command is designed to perform a simple restore of a full snapshot. In the event that you want to restore incremental backups you should start by restoring the last full snapshot prior to your target incremental backup file and manually move the files from each incremental backup in chronological order leading up to the target incremental backup file. The schema dump is included in the snapshot backups, but if necessary it must also be restored manually.
Snapshots are taken at the system level, the script currently does not support backup or restore of an individual keyspace or columnfamily.
In order to enable incremental backups, the incremental_backups option has to be set to true in the cassandra.yaml file.
Copyright 2016 Google Inc. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS-IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
This is not an official Google product.