datamade/chi-councilmatic

Name: chi-councilmatic

Owner: datamade

Description: :eyes: keep tabs on Chicago city council

Created: 2015-05-19 20:09:47.0

Updated: 2018-04-04 18:26:37.0

Pushed: 2018-04-04 18:34:27.0

Homepage: https://chicago.councilmatic.org/

Size: 22222

Language: Python

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Chicago Councilmatic

Keep track of what Chicago City Council is doing.

Setup

Install OS level dependencies:

• Python 3.4
• PostgreSQL 9.4 +

Install app requirements

We recommend using virtualenv and virtualenvwrapper for working in a virtualized development environment. Read how to set up virtualenv.

Once you have virtualenvwrapper set up,

rtualenv chi-councilmatic
clone https://github.com/datamade/chi-councilmatic.git
hi-councilmatic
install -r requirements.txt

Afterwards, whenever you want to use this virtual environment to work on chi-councilmatic, run workon chi-councilmatic

OPTIONAL: install django-councilmatic locally
If you plan on making changes to core councilmatic features (as opposed to Chicago-specific stuff), you'll want to install django-councilmatic locally instead of installing from pypi.

.
clone https://github.com/datamade/django-councilmatic.git
jango-councilmatic
on setup.py develop
./chi-councilmatic

Create your settings file

ouncilmatic/settings_deployment.py.example councilmatic/settings_deployment.py

Then edit councilmatic/settings_deployment.py:

• DATABASES['default']['USER'] should be your username
• if you're setting up councilmatic for local development, use a dummy cache by setting CACHES['default']['BACKEND'] to 'django.core.cache.backends.dummy.DummyCache'. if you're deploying, leave it as is

Setup your database

Before we can run the website, we need to create a database.

tedb chi_councilmatic

Then, run migrations

on manage.py migrate

Create an admin user - set a username & password when prompted

on manage.py createsuperuser

Importing data from the open civic data api

Run the loaddata management command. This will take a while, depending on volume (Chicago has ~70k bills, & it takes ~1 hour to load the data).

on manage.py import_data

By default, the import_data command is smart about what it looks at on the OCD API. If you already have bills loaded, it won't look at everything on the API - it'll look at the most recently updated bill in your database, see when that bill was last updated on the OCD API, & then look through everything on the API that was updated after that point. If you'd like to load things that are older than what you currently have loaded, you can run the import_data management command with a --delete option, which removes everything from your database before loading.

The import_data command has some more nuance than the description above, for the different types of data it loads. If you have any questions, open up an issue and pester us to write better documentation.

Setup CSS, Images, and other static resources

on manage.py collectstatic

Running Chicago Councilmatic locally

on manage.py runserver

navigate to http://localhost:8000/

Setup Search

Install Open JDK or update Java

On Ubuntu:

do apt-get update
do apt-get install openjdk-7-jre-headless

On OS X:

1. Download latest Java from http://java.com/en/download/mac_download.jsp?locale=en

2. Follow normal install procedure

3. Change system Java to use the version you just installed:

    mv /usr/bin/java /usr/bin/java16
    ln -s /Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java /usr/bin/java
   

Download & setup Solr

 http://mirror.sdunix.com/apache/lucene/solr/4.10.4/solr-4.10.4.tgz
-xvf solr-4.10.4.tgz
 cp -R solr-4.10.4/example /opt/solr

py schema.xml for this app to solr directory
 cp solr_scripts/schema.xml /opt/solr/example/solr/collection1/conf/schema.xml

Run Solr

xt, start the java application that runs solr
 this in another terminal window & keep it running
 you see error output, somethings wrong
opt/solr/example
 java -jar start.jar

Index the database

ck in the chi-councilmatic directory:
on manage.py rebuild_index

OPTIONAL: Install and configure Jetty for Solr

Just running Solr as described above is probably OK in a development setting. To deploy Solr in production, you'll want to use something like Jetty. Here's how you'd do that on Ubuntu:

 apt-get install jetty

ckup stock init.d script
 mv /etc/init.d/jetty ~/jetty.orig

t init.d script suggested by Solr docs
 cp solr_scripts/jetty.sh /etc/init.d/jetty
 chown root.root /etc/init.d/jetty
 chmod 755 /etc/init.d/jetty

d Solr specific configs to /etc/default/jetty
 cp solr_scripts/jetty.conf /etc/default/jetty

ange ownership of the Solr directory so Jetty can get at it
 chown -R jetty.jetty /opt/solr

art up Solr
 service jetty start

lr should now be running on port 8983

Regenerate Solr schema

While developing, if you need to make changes to the fields that are getting indexed or how they are getting indexed, you'll need to regenerate the schema.xml file that Solr uses to make it's magic. Here's how that works:

on manage.py build_solr_schema > solr_scripts/schema.xml
olr_scripts/schema.xml /opt/solr/solr/collection1/conf/schema.xml

In order for Solr to use the new schema file, you'll need to restart it.

Using Solr for more than one Councilmatic on the same server

If you intend to run more than one instance of Councilmatic on the same server, you'll need to take a look at this README to make sure you're configuring things properly.

Team

• David Moore - project manager
• Forest Gregg - Open Civic Data (OCD) and Legistar scraping
• Cathy Deng - data models and loading
• Derek Eder - front end
• Eric van Zanten - search and dev ops

Errors / Bugs

If something is not behaving intuitively, it is a bug, and should be reported. Report it here: https://github.com/datamade/chi-councilmatic/issues

Copyright

Copyright (c) 2015-17 Participatory Politics Foundation and DataMade. Released under the MIT License.