Name: chicago-justice
Owner: datamade
Description: Chicago Justice Project
Forked from: chicago-justice-project/chicago-justice
Created: 2017-12-25 01:54:28.0
Updated: 2017-12-25 01:54:30.0
Pushed: 2017-12-06 02:51:40.0
Homepage: null
Size: 9317
Language: Python
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
The easiest way to install PostgreSQL for Mac is with a prebuilt Postgres installation, like Postgres.app.
Alternatively, you may use Homebrew:
install postgres
services start postgresql
The version of PostgreSQL provided in most distros' repositories should be adequate and can be installed through your distro's package manager.
Ubuntu 16.04:
apt-get update
apt-get install postgresql
Arch Linux:
pacman -S postgresql
-u postgres initdb --locale $LANG -E UTF8 -D '/var/lib/postgres/data'
systemctl start postgresql.service
Once PostgreSQL is installed and running, you can create the database you'll use locally for this app.
As a user with Postgres database privileges:
tedb cjpdb
The name of the database (e.g., cjpdb) may be anything you choose, but
keep track of what you name it along with the user and password we're about to
create. You'll need these for setting up your virtual environment.
Create the Postgres user and give it a password:
teuser --interactive --pwprompt
Finally, grant privileges on the database you just created to the user you just
created. For instance, if we created database cjpdb and the user cjpuser:
-d postgres -c "GRANT ALL ON DATABASE cjpdb TO cjpuser;"
Certain settings are read from environment variables. There are two ways you
can set variables: 1) Use a .env file in the root directory; 2) setup a
python virtual environment and use virtualenv's postactivate and
predeactivate hooks. Both methods are detailed below.
.env environment variable fileAn example .env file is provided. You should copy it:
env-example .env
Then, you can edit the file in your preferred editor.
Alternatively, you can create a virtual environment to house the environment variables and the app's dependencies.
If not already installed, install python's virtualenv and
virtualenvwrapper:
install virtualenv virtualenvwrapper
r ~/.virtualenvs
Add the following to your .bashrc file:
rt WORKON_HOME=~/.virtualenvs
ce /usr/local/bin/virtualenvwrapper.sh
Find out the path to your python installation:
h python
Create your working environment, naming it whatever you'd like (e.g.,
cjp_dev), where usr/local/bin/python is whatever path the previous command
returned:
rtualenv --python=/usr/local/bin/python cjp_dev
You may now use workon cjp_dev and deactivate to activate and deactivate
the virtual environment. Setup hooks so that when the virtual environment is
activated, the proper environment variables will be set. Be sure to substitute
cjp_dev, cjpdb, cjpuser, and cjppassword with your setup. You can also
generate a unique secret key with something like this Django Secret Key Generator
Add the following to ~/.virtualenvs/cjp_dev/bin/postactivate:
rt DJANGO_SETTINGS_MODULE="cjp.settings.local"
rt DATABASE_NAME="cjpdb"
rt DATABASE_USER="cjpuser"
rt DATABASE_PASSWORD="cjppassword"
rt SECRET_KEY='#&ubnzmo6$-0nk7i&hmii=e$7y-)nv+bm#&ps)6eq@!k+n-nq5'
To make sure these variables are unset upon deactivating the virtual
environment, add the following to ~/.virtualenvs/cjp_dev/bin/predeactivate:
t DJANGO_SETTINGS_MODULE
t DATABASE_NAME
t DATABASE_USER
t DATABASE_PASSWORD
t SECRET_KEY
With the environment variables set, we're now ready to install the necessary dependencies:
install -r requirements.txt
nage.py migrate
nage.py loaddata category news_source
nage.py runserver
nage.py runscrapers
To run a single scraper, enter the scraper name as an argument, e.g.:
nage.py runscrapers crains
This is a standard Django project. The following information is specific to an Ubuntu based deployment using Nginx with Gunicorn. This information will not necessarily apply to all deployment setups. For details on deploying Django applications, see the Django deployment documentation: https://docs.djangoproject.com/en/1.10/howto/deployment/
Current deployment is done via reverse-proxied Nginx with gunicorn running the application. To achieve this, be sure gunicorn is running the application on a port. How you do this will vary according to your system. On Ubuntu, consider using Upstart to manage Gunicorn.
An example upstart config is available in conf/etc/init/chicagojustice.conf.
The application can then be managed via: sudo service chicagojustice [start|stop|restart]
Your Nginx configuration should contain information to proxy requests to the application port. Example nginx config is
available in conf/etc/nginx/default.
The code is deployed via git repository. Deployment of code changes should
simply require git pull inside the application repository, and likely
sudo service chicagojustice restart (please check the name of the service with what
is in the upstart configs under /etc/init)
In some cases (ie. model changes) a schema migration is required. Migrate via
./manage.py migrate. Be sure to source the virtual environment before running
migrations. See the Django docs for details on schema migrations:
https://docs.djangoproject.com/en/1.10/topics/migrations/
To copy static files into place for production, you must run
python manage.py collectstatic.
The script dumpArticleTables.sh currently runs every 24 hours. This script exports the article and category tables in CSV format, then packs them into a tar archive in /home/sftp_users/files.
Users in the sftp_users group can access /home/sftp_users via SFTP only. These users do not have shell access and cannot access any other directories.
To create a user in this group, use the command adduser --home /home/sftp_users/files --ingroup sftp_users.
The current SSHD config for this group is as follows:
h Group sftp_users
ForceCommand internal-sftp
PasswordAuthentication yes
ChrootDirectory /home/sftp_users
PermitTunnel no
AllowAgentForwarding no
AllowTcpForwarding no
X11Forwarding no