Name: mirrorbits
Owner: Team Kodi
Description: Mirrorbits is a geographical download redirector written in Go for distributing files efficiently across a set of mirrors.
Forked from: etix/mirrorbits
Created: 2017-04-20 21:30:49.0
Updated: 2018-03-17 19:08:11.0
Pushed: 2018-03-17 19:08:10.0
Size: 523
Language: Go
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Mirrorbits is a geographical download redirector written in Go for distributing files efficiently across a set of mirrors. It offers a simple and economic way to create a Content Delivery Network layer using a pure software stack. It is primarily designed for the distribution of large-scale Open-Source projects with a lot of traffic.
Yes! Mirrorbits is already running in production at
Yet some things might change before the 1.0 release. If you intend to deploy Mirrorbits in a production system it is advised to notify the author first so we can help you to make any transition as seamless as possible!
Optional:
You can either get a prebuilt version or choose to build it yourself.
get -u github.com/etix/mirrorbits
$GOPATH/src/github.com/etix/mirrorbits
ke install
The resulting executable should now be in your $GOPATH/bin directory.
If you plan to use the web UI be sure to copy the templates into your system (usually in /usr/share/mirrorbits).
A sample configuration file can be found here.
Option | description
—— | ———–
Repository | Path to your own copy of the repository
Templates | Path containing the templates
GoogleMapsAPIKey | Google Maps API key that you can obtain here.
OutputMode | auto: based on the Accept header content
redirect: do an HTTP redirect to the destination
json: return a JSON formatted document (also known as API mode)
ListenAddress | Local address and port to bind
Gzip | Use gzip compression for the JSON responses
RedisAddress | Address and port of the Redis database
RedisPassword | Password to access the Redis database
RedisDB | Database index
RedisSentinelMasterName | Name of the redis-sentinel cluster
RedisSentinels | List of redis-sentinel hosts
LogDir | Path to the directory where to save log files
TraceFileLocation | Relative path to a trace file (from the repository root) containing a Unix timestamp regularly updated
GeoipDatabasePath | Path to the geoip databases
ConcurrentSync | Maximum number of server sync (rsync/ftp) do to simultaneously
ScanInterval | Interval between rsync/ftp synchronizations (in minutes)
CheckInterval | Interval between mirrors health's checks (in minutes)
RepositoryScanInterval | Interval between scans of the local repository (in minutes, 0 to disable)
Hashes | List of file hashes to computes (SHA1, SHA256, MD5)
DisallowRedirects | Disable any mirror trying to do an HTTP redirect
WeightDistributionRange | Multiplier of the distance to the first mirror to find other possible mirrors in order to distribute the load
DisableOnMissingFile | Disable a mirror if an advertised file on rsync/ftp appears to be missing on HTTP
MaxLinkHeaders | Amount of backup mirror locations returned in HTTP headers
CheckSumHeaders | Add headers for enabled Hashes
Fallbacks | A list of possible mirrors to use as fallback if a request fails or if the database is unreachable. These mirrors are not tracked by mirrorbits. It is assumed they have all the files available in the local repository.
Mirrorbits is a self-contained application and can act, at the same time, as the server and the cli.
To run the server:
orbits -D
Additional options can be found with `mirrobits -help`.
To run the cli:
orbits help
Add a mirror:
orbits add -ftp="ftp://ftp.mirrors.example/myproject/" -http="http://ftp.mirrors.example/myproject/" mirrors.example
Enable the mirror:
orbits enable mirrors.example
By appending ?mirrorlist to any file served by mirrorbits, you'll be able to get some useful realtime informations about the given file. You can see a live example here.
Mirror statistics are available by querying mirrorbits with the ?mirrorstats argument. You can see a live example here.
Note: Clustering support has been added recently and should be treated as experimental.
Multiple instances of mirrorbits can be started simultanously on different servers, discovery of other nodes should be automatic as long as all the instances are connected to the same redis server. In addition to the clustering it is advised to use redis-sentinel to monitor the database and gracefuly handle failover.
Mirrorbits has a mode called seamless binary upgrade to upgrade the server executable at runtime without service disruption. Once the binary has been replaced on the filesystem just issue the following command in the cli:
orbits upgrade
y_set_header X-Forwarded-For $remote_addr;
The best place to discuss about mirrorbits is to join the #VideoLAN IRC channel on Freenode.
For the latest news, you can follow @mirrorbits on Twitter.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.