Name: councilmatic-starter-template
Owner: datamade
Description: :clipboard: Starter code & documentation for new councilmatic instances
Created: 2016-02-22 22:42:06.0
Updated: 2018-04-27 12:23:21.0
Pushed: 2018-03-16 21:22:18.0
Size: 44
Language: Python
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
This repo provides starter code and documentation for new Councilmatic instances.
The councilmatic family is a set of web apps for keeping tabs on city representatives and their legislative activity.
Councilmatic started as a Code for America project by Mjumbe Poe, who designed the earliest version of a Councilmatic site ? Councilmatic Philadelphia. DataMade then implemented Councilmatic in New York City (in partnership with the Participatory Politics Foundation), Chicago (in partnership with the Sunlight foundation), and Los Angeles (in partnership with the Los Angeles County Metropolitan Transportation Authority).
To simplify redeployment of Councilmatic instances, DataMade identified two necessities: (1) Open Civic Data, a data standard for describing people, organizations, events, and bills, and (2) the abstraciton of core functionality into a separate app. And so, DataMade built django-councilmatic ? a django app with base functionality, common across all cities, which implements the OCD data standard.
With this template, you can create a Councilmatic site for your own city, built using the django-councilmatic app. Read on!
You need data about your city in the Open Civic Data API.
How you get your data into an instance of the OCD API is up to you. What does DataMade do? We use scrapers, which run nightly to update the API by scraping data from Legistar-backed sites operated by the cities for which we built Councilmatic. Your city may be running a Legistar-backed site, and if so, you can checkout python-legistar-scraper and the pupa framework to get a head start on scraping.
If you need examples of how to customize your scraper, look at scrapers-us-municipal as well as DataMade, which hosts several municipal-level scrapers. You can find information about what cities and other governmental bodies are already covered on the OCD DataMade site.
NOTE: This guide focuses on setting up your app for development. It does not discuss in detail the process of finding, scraping, and importing city data or deploying your site.
clone https://github.com/datamade/councilmatic-starter-template.git yourcity_councilmatic
We recommend using virtualenv and virtualenvwrapper for working in a virtualized development environment. Read how to set up virtualenv.
Once you have virtualenvwrapper set up, do the following in your bash profile:
rtualenv councilmatic
ourcity_councilmatic
install -r requirements.txt
Afterwards, whenever you want to use this virtual environment, run:
on yourcity_councilmatic
Inside the git repository that you cloned above, you should see a folder called city. Rename this folder to something that makes sense for your project, e.g., “chicago.”
ity chicago
Now, update the main settings file - councilmatic/settings.py. First, in INSTALLED_APPS, add the name of the folder that you just renamed:
ALLED_APPS = (
...
~'city',~
'chicago',
Then, change the TIME_ZONE. You can use this list to find the correct formatting.
_ZONE = 'America/Chicago'
Look for councilmatic/settings_jursidiction.py. This settings file tells your Councilmatic instance how to populate different parts of the UI and get fresh data from the OCD API. The following table explains why and how to adjust these values.
| Name | Description | Example value | Optional? |
|---|---|---|---|
| OCD_JURISDICTION_ID |
For scrapers hosted on the Datamade OCD API, you can find the jurisdiction id here. Otherwise, look at the |
ocd-jurisdiction/country:us/state:il/place:chicago/government | No |
| OCD_CITY_COUNCIL_ID |
Set either
You can find the name and id of your city council on the Datamade OCD API or by using the same path to your own OCD API. |
ocd-organization/ef168607-9135-4177-ad8e-c1f7a4806c3a | Only if you set OCD_CITY_COUNCIL_NAME |
| OCD_CITY_COUNCIL_NAME | See OCD_CITY_COUNCIL_ID | Chicago City Government | Only if you set OCD_CITY_COUNCIL_ID |
| CITY_COUNCIL_NAME | Display name for the legislative body you'll be tracking. | Chicago City Council | No |
| LEGISLATIVE_SESSIONS | A list of years that tells Councilmiatic when a new body is elected and which of these sessions you have data for in your OCD API. | ['2007', '2011', '2015'] | No |
| CITY_NAME | Complete, pretty name of the jurisdiction | New York City | No |
| CITY_NAME_SHORT | Shortened version of the jurisdiction name | NYC | No |
| CITY_VOCAB | A dictionary telling Councilmatic how to refer to different entities in the jurisdiction | (See individual values below) | No |
| CITY_VOCAB['MUNICIPAL_DISTRICT'] | The name of the city sections, which could be a "ward", "sector", "district", etc. | Ward | No |
| CITY_VOCAB['SOURCE'] | The name of the entity responsible for the source data that your Councilmatic will be based upon. | Chicago City Clerk | No |
| CITY_VOCAB['COUNCIL_MEMBER'] | The name of a member of the legislative body that your Councilmatic tracks. | Alderman | No |
| CITY_VOCAB['COUNCIL_MEMBERS'] | Plural form of the name of a member of the legislative body your Councilmatic tracks. | Aldermen | No |
| CITY_VOCAB['EVENTS'] | The name of the meetings that members of the legislative body attend. | Meetings | No |
| APP_NAME | The name of the folder where your jurisdiction specific code lives. | chicago | No |
| SITE_META |
Dictionary containing values stored in meta tags.
|
(See individual values below) | Yes |
| SITE_META['site_name'] |
Used for the og:name, og:title, and twitter:title meta tags
|
Chicago Councilmatic | Yes |
| SITE_META['site_desc'] |
Used for the description, og:description, and twitter:description meta tags
|
City Council, demystified. Keep tabs on Chicago legislation, aldermen, & meetings. | Yes |
| SITE_META['site_author'] |
Used for the author meta tag
|
DataMade | Yes |
| SITE_META['site_url'] | Used to build absolute URLs for the meta tags. | https://chicago.councilmatic.org | Yes |
| SITE_META['twitter_site'] |
Used for twitter:site meta tag.
|
@councilmatic | Yes |
| SITE_META['twitter_creator'] |
Used for twitter:creator meta tag.
|
@DataMadeCo | Yes |
Look for councilmatic/settings_deployment.py.example. Make a copy of it so that you can customize it for your city:
ouncilmatic/settings_deployment.py.example councilmatic/settings_deployment.py
This file is important! It's where you keep the parts of your Councilmatic that should not end up in version control (e.g., passwords, database connection strings, etc., which will vary based on where the app is running and which you probably won't want other people to know). Most of these variables are generic Django settings, which you can look up in the Django docs; but some of these variables require customization. This table shows only the variables that you need to customize.
| Name | Description | Optional? |
|---|---|---|
| HAYSTACK_CONNECTIONS | Dictionary of connections used by Haystack. More on that here. | No |
| FLUSH_KEY | Used by the `flush` view in `django-councilmatic` to clear Django's cache. | No |
| DISQUS_SHORTNAME | Will cause Disqus comment threads to be attached to individual bill pages. | Yes |
| ANALYTICS_TRACKING_CODE | Google Analytics property ID | Yes |
| HEADSHOT_PATH | Absolute path to where the data loader will save images of members of the legislative body that are often available through Legistar. | Yes |
| EXTRA_APPS | List of additional django apps that you want to use with your custom councilmatic | Yes |
Before we can run the website, we need to create a database.
tedb yourcity_councilmatic
Then, run migrations. (Be sure that you are “working on” the correct virtual environment.)
on manage.py migrate --no-initial-data
Create an admin user. Set a username and password when prompted.
on manage.py createsuperuser
Get an image (suggested 310x310), and transform it in device-specific favicon images using this site. Then, move the images into <city_app>/static/images/icons.
The django-councilmatic app comes with a import_data management command, which populates bills, people, committees, and events, loaded from the OCD API. You can explore the nitty-gritty of this code here. Note: Earlier releases of django-councilmatic (< 0.7) use loaddata, instead of import_data.
Running import_data will take a while, depending on volume (e.g., NYC may require around half an hour).
on manage.py import_data
By default, the import_data command carefully looks at the OCD API; it is a smart management command. If you already have bills loaded, it will not look at everything on the API - it will look at the most recently updated bill in your database, see when that bill was last updated on the OCD API, and then look through everything on the API that was updated after that point. If you'd like to load things that are older than what you currently have loaded, you can run the import_data management command with a --delete option, which removes everything from your database before loading.
The import_data command has some more nuance than the description above, for the different types of data it loads. If you have any questions, open up an issue and pester us to write better documentation.
Now, you are ready to run your unique version of Councilmatic!
on manage.py runserver
Navigate to http://localhost:8000/.
On a Councilmatic site, users can search bills according to given query parameters. To power our searches, we use Solr, an open source tool, written in Java.
Requirements: Open JDK or Java
On Ubuntu:
do apt-get update
do apt-get install openjdk-7-jre-headless
On OS X:
Download latest Java from http://java.com/en/download/mac_download.jsp?locale=en
Follow normal install procedure
Change system Java to use the version you just installed:
mv /usr/bin/java /usr/bin/java16
ln -s /Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java /usr/bin/java
Download & setup Solr
http://archive.apache.org/dist/lucene/solr/4.10.4/solr-4.10.4.tgz
-xvf solr-4.10.4.tgz
cp -R solr-4.10.4/example /opt/solr
py schema.xml for this app to solr directory
cp solr_scripts/schema.xml /opt/solr/solr/collection1/conf/schema.xml
Run Solr
xt, start the java application that runs solr
this in another terminal window & keep it running
you see error output, somethings wrong
opt/solr
java -jar start.jar
Index the database
ck in the nyc-councilmatic directory:
on manage.py rebuild_index
OPTIONAL: Install and configure Jetty for Solr
Just running Solr as described above is probably OK in a development setting. To deploy Solr in production, you'll want to use something like Jetty. Here's how you'd do that on Ubuntu:
apt-get install jetty
ckup stock init.d script
mv /etc/init.d/jetty ~/jetty.orig
t init.d script suggested by Solr docs
cp solr_scripts/jetty.sh /etc/init.d/jetty
chown root.root /etc/init.d/jetty
chmod 755 /etc/init.d/jetty
d Solr specific configs to /etc/default/jetty
cp solr_scripts/jetty.conf /etc/default/jetty
ange ownership of the Solr directory so Jetty can get at it
chown -R jetty.jetty /opt/solr
art up Solr
service jetty start
lr should now be running on port 8983
Regenerate Solr schema
While developing, if you need to make changes to the fields that are getting indexed or how they are getting indexed, you'll need to regenerate the schema.xml file that Solr uses to make it's magic. Here's how that works:
on manage.py build_solr_schema > solr_scripts/schema.xml
olr_scripts/schema.xml /opt/solr/solr/collection1/conf/schema.xml
In order for Solr to use the new schema file, you'll need to restart it.
Using Solr for more than one Councilmatic on the same server
If you intend to run more than one instance of Councilmatic on the same server, you'll need to take a look at this README to make sure you're configuring things properly.
If something is not behaving intuitively, it is a bug, and should be reported. Report it here.
We welcome your ideas and feedback! Here's how to make a contribution:
Copyright (c) 2015 Participatory Politics Foundation and DataMade. Released under the MIT License.