Name: stsci-bot
Owner: Space Telescope Science Institute
Description: A prototype :robot: for stsci, based on the astropy-bot
Forked from: astropy/astropy-bot
Created: 2017-11-01 16:09:54.0
Updated: 2017-11-01 16:54:04.0
Pushed: 2018-03-12 15:05:36.0
Size: 150
Language: Python
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
This is a GitHub bot written using Flask for the Astropy project which can be installed via GitHub integrations. We use the concept of flask “blueprints” to separate the different functionality. Each blueprint is defined in a separate file inside changebot/blueprints. The bot includes the following functionality:
Check pull requests whenever they are changed. For now the checks only include making sure the changelog is consistent with the milestone and the labels but this could be expanded in future. This is defined in pull_request_checker.py.
Check for issues labeled as “Close?” that have become stale, and close them (with a warning period). This is defined in stale_issues.py.
Check for pull requests that have become stale, and close them (with a warning period). This is defined in stale_pull_requests.py.
More details about the bot and its components is provided in the sections below.
For astropy-bot, the astrochangebot.herokuapp.com server is a GitHub
app that is set up on Heroku.
We use the auto-deploy functionality in Heroku to re-deploy the app anytime
a change is made to this repository.
To create your own instance of the bot, first, if you don't already have a Heroku account, create a free one there. When you see the option to create a new app, select it (ignore the “add to pipeline” option). Give a name to your app; You need to select a name that is not already taken and it does not have to be the same as the bot's name here.
You should now be on the “Deploy” section. Again, ignore the pipeline option. Select Github as “Deployment Method”. Enter the relevant GitHub organization or account that the bot resides in (this should be automatically populated if you have given Heroku access to your GitHub account) and type in the bot's repository name (either this bot or a forked version of it).
If you want to enable automatic deployment from a selected branch of the repository, click the “Enable Automatic Deploys” button. This will pick up changes to the given branch and re-deploy the bot as needed. For most cases, you don't need the “wait for CI to pass before deploy” option as the bot is already tested here.
For the first time, you also need to manually deploy the bot by clicking “Deploy Branch”.
Once it is successfully deployed, go to “Settings” tab of the bot and you can customize its behavior using “Config Vars”. This is the only custom configuration on Heroku and can be set through the Heroku admin interface, as mentioned. The main required environment variables (also see “Authentication” section below) are::
GITHUB_APP_INTEGRATION_ID, which should be set to the integration ID
provided by GitHub app (see “GitHub settings” section below) under
“General Settings”, specifically “About… ID”. For astropy-bot,
the integration ID is currently 3400 (this is not a secret).
GITHUB_APP_PRIVATE_KEY, which is generated by the GitHub app
(see “GitHub settings” section below).
This private key should look like:
-BEGIN RSA PRIVATE KEY-----
e random characters>
-END RSA PRIVATE KEY-----
The whole key, including the BEGIN and END header and footer
should be pasted into the field. To obtain this key for astropy-bot
or one that you did not set up, please contact bot administrator.
CRON_TOKEN, which should be set to a random string known
only to the person or people who need to run cron jobs manually.
Some of the functionality in astropy-bot is not intended to be
triggered by GitHub but instead by cron jobs.
However, we don't want just anyone to be able to run these cron jobs, so
this environment variable is the shared secret/password that allows certain
actions to be triggered.
If you don't need the cron jobs, this needs to be set anyway due to the
way webapp.py is currently coded, so just set to some string like
“N/A”.
In addition to these environment variables, there are several others required for specific components of the bot, as described in “Stale issue and pull request checkers” section below. These must also be set whether you wish to use them or not.
NOTE: If you are creating a new instance of this bot from your fork
using a different bot name, it is important that you replace astropy-bot
with your new bot's name in the production branch of your fork.
Otherwise, the bot will not able to edit its comments. However, you don't
have to modify the https://github.com/astropy/astropy-bot/issues URLs
because your fork will not have its own issues tracker.
To add the bot to your own organization or account, go to your GitHub organization or account URL (not the repository) and then its settings. Then, click on “Developer settings” at the very bottom of the left navigation bar and the “New GitHub App” button on top right.
Give your bot a “GitHub App name” as you want it to appear on GitHub activities. Under “Homepage URL”, enter the GitHub repository URL where the bot code resides (either here or your fork, as appropriate).
For the User authorization callback URL, it should be in the format of
http://<heroku-bot-name>.herokuapp.com/callback. For astropy-bot,
this is http://astrochangebot.herokuapp.com/callback.
For the Webhook URL, it should be in the format of
http://<heroku-bot-name>.herokuapp.com/hook. For astropy-bot,
this is http://astrochangebot.herokuapp.com/hook
You can ignore “Setup URL” and “Webhook secret”. It would be useful to provide a description of what your bot intends to do but not required.
The permissions of the app should be read/write access to Commit statuses, Issues, and Pull requests. Once you have checked these options, you will see extra “Subscribe to events” entries that you can check as well. For the events, it should be sufficient to only check Status, Issue comment, Issues, Pull request, Pull request review, and Pull request review comment.
It is up to you to choose whether you want to allow your GitHub app here to be installed only on your account or by any user or organization.
Once you have clicked “Create GitHub App” button, you can go back to the app's “General” settings and upload a logo, which is basically a profile picture of your bot.
For astropy-bot, don't be tempted to re-generate a key from the
GitHub app settings here. There is a key that is already generated that
we should use; Ask @astrofrog if you would like to know the key.
However, for your own bot, you can generate a private key for the app to avoid
GitHub anonymous API limits; Save this key somewhere private and secure.
For Astropy, the GitHub app for this bot is defined in the Astropy organization settings (not everyone will be able to see this).
Go to https://github.com/apps/<github-app-name>; For this bot, it is
https://github.com/apps/astropy-bot . Then, click on the big green “Install”
button. You can choose to install the bot on all or select repositories
under your account or organization. It is recommended to only install it for
select repositories by start typing a repository name and let auto-completion
do the hard work for you (repeat this once per repository). Once you are done,
click “Install”.
After a successfull installation, you will be taken to a
https://github.com/settings/installations/<installation-number> page.
This page is also accessible from your account or organization settings in
“Applications”, specifically under “Installed GitHub Apps”.
You can change the installation settings by clicking the “Configure”
button next to the listed app, if desired.
For Astropy, the app can be added to specific repositories under the Your installations tab, by clicking on the gearbox next to Astropy, which goes to this page (not everyone will be able to see this).
The pull request checker is intended to be triggered by GitHub whenever there
is a change in a pull request. Currently the main code that gets run when this
happens is the check_changelog_consistency function, which makes sure that
the changelog is consistent with the pull request number, labels, and milestone.
The main logic is defined in
pull_request_checker.py and
changelog_helpers.py
If CHANGES.rst, CHANGES, or CHANGES.md file does not exist in the
repository, bot will not comment at all.
For repositories that do not use a changelog, this checker can be disabled
by defining changelog_check: false in .astropybot.yml at the top
level of the repository.
The components to check for stale issues and pull requests are intended to be triggered by a cron job rather than from GitHub. The main reasons for this are because these tend to do more API calls than the single pull request checker (because e.g. all open pull requests need to be queried) and because by definition we are looking for inactivity rather than activity.
These components can be configured via the following environment variables on Heroku:
STALE_ISSUE_CLOSE, which should be set to TRUE or FALSE. If
FALSE, this doesn't check for whether issues labeled as Close?
should be closed, only whether the warning about closing should be posted.
The intent is that this should always be TRUE except the first time that
this is run or if you never want your stale issues be closed.
This requires that you have a Close? label (case sensitive; include the
question mark) on each of your repositories that your bot watches.
STALE_ISSUE_WARN_SECONDS, which is the time in seconds from when an
issue was labeled as Close? in order to be warned that it will be closed.
If you never want this warning, set it to a very high number
or do not apply the Close? label.
The recommended value is 12960000, which is approximately 5 months.
STALE_ISSUE_CLOSE_SECONDS, which is the time in seconds from when an
issue was labeled as Close? in order to be considered ready for closing.
This should be larger than STALE_ISSUE_WARN_SECONDS.
This setting is only relevant if STALE_ISSUE_CLOSE is set to TRUE.
The recommended value is 15552000, which is approximately 6 months.
STALE_PULL_REQUEST_CLOSE, which should be set to TRUE or FALSE.
If FALSE, this doesn't check for whether pull requests should be closed,
only whether the warning about closing should be posted. The intent is that
this should always be TRUE except the first time that this is run or
if you never want your stale pull requests be closed.
For repositories that do not want to ever close stale pull requests,
this feature can be permanently disabled by defining
autoclose_stale_pull_request: false in .astropybot.yml at the top
level of the repository.
STALE_PULL_REQUEST_WARN_SECONDS, which is the time in seconds from the
last commit in order to be warned that it will be closed.
If you never want this warning, set it to a very high number.
The recommended value is 12960000, which is approximately 5 months.
STALE_PULL_REQUEST_CLOSE_SECONDS, which is the time in seconds from the
last commit in order to be considered ready for closing unless
a keep-open is attached to the pull request.
This should be larger than STALE_PULL_REQUEST_WARN_SECONDS. This
setting is only relevant if STALE_PULL_REQUEST_CLOSE is set to TRUE.
The recommended value is 15552000, which is approximately 6 months.
For issues, removing the Close? label and adding it back resets the clock. For pull requests, adding a new commit resets the clock, while adding the keep-open label means that this pull request will not be touched by the bot.
To run these checks, you can access http://astrochangebot.herokuapp.com/close_stale_issues and http://astrochangebot.herokuapp.com/close_stale_pull_requests using a POST request and with the following parameters encoded in JSON:
'repository': the name of the repository to run the checks for,
including the owner (e.g. astropy/astropy)'cron_token': this should be the same as the CRON_TOKEN
environment variable'installation': this should be the installation ID
(see the Authentication section). For the Astropy organization
repositories, this is 36238.An example for how to do this with the requests package is:
rt requests
= {'repository': 'astrofrog/test-bot',
'cron_token': 'theactualtoken',
'installation': '36238'}
ests.post(URL, json=data)
The different components of the bot interact with GitHub via a set of helper
classes that live in
changebot/github/github_api.py.
These classes are RepoHandler, IssueHandler, and
PullRequestHandler. It is possible to try these out locally, at least for
the parts of the GitHub API that do not require authentication.
For example, the following should work:
from changebot.github.github_api import RepoHandler, IssueHandler, PullRequestHandler
repo = RepoHandler('astropy/astropy')
repo.get_issues('open', 'Close?')
5, 5193, 4842, 4549, 4058, 3951, 3845, 2603, 2232, 1920, 1024, 435, 383, 282]
issue = IssueHandler('astropy/astropy', 6597)
issue.labels
g', 'coordinates']
pr = PullRequestHandler('astropy/astropy', 6606)
pr.labels
hancement', 'Refactoring', 'testing', 'Work in progress']
pr.last_commit_date
374526.0
However since these are being run un-authenticated, you may quickly run into the GitHub public API limits. If you are interested in authenticating locally, see the Authentication section below.
Code-wise, the authentication of the app is handled in the
changebot/github/github_auth.py file -
the main function from there that is used in
changebot/github/github_api.py is the
github_request_headers function which returns headers for requests
that contain the appropriate tokens.
In some cases, you may want to test the bot locally as if it was running on Heroku. In order to do this you will need to make sure you have all the environment variables described above set correctly.
The main ones to get right as far as authentication is concerned are as follows (see “Heroku settings” section above for further descriptions):
GITHUB_APP_INTEGRATION_IDGITHUB_APP_PRIVATE_KEYThe last thing you will need is an Installation ID - a GitHub app can be linked to different GitHub accounts, and for each account or organization, it has a unique ID. You can find out this ID by going to Your installations and then clicking on the settings box next to the account where you have a test repository you want to interact with. The URL of the page you go to will look like:
https://github.com/settings/installations/36238
In this case, 36238 is the installation ID. Provided you set the environment variables correctly, you should then be able to do e.g.:
from changebot.github.github_api import IssueHandler
issue = IssueHandler('astrofrog/test-bot', 5, installation=36238)
issue.submit_comment('I am alive!')
Use this power wisely! (And avoid testing out things on the main Astropy repos…)
Note: Authentication will not work properly if you have a .netrc file
in your home directory, so you will need to rename this file temporarily.
This app requires Python 3.6 to run, and dependencies are listed in
requirements.txt