Name: syria
Owner: NPR visuals team
Description: With Syria engulfed in civil war, here are four stories of families struggling to stay together.
Created: 2015-01-29 20:48:25.0
Updated: 2018-05-23 16:25:25.0
Pushed: 2018-05-23 16:25:24.0
Homepage: http://apps.npr.org/syria/
Size: 739
Language: JavaScript
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
Copyright 2015 NPR. All rights reserved. No part of these materials may be reproduced, modified, stored in a retrieval system, or retransmitted, in any form or by any means, electronic, mechanical or otherwise, without prior written permission from NPR.
(Want to use this code? Send an email to nprapps@npr.org!)
This is a photo-driven layout built for an NPR story about Syrian refugees.
The following things are assumed to be true in this documentation.
For more details on the technology stack used with the app-template, see our development environment blog post.
The project contains the following folders and important files:
confs
– Server configuration files for nginx and uwsgi. Edit the templates then fab <ENV> servers.render_confs
, don't edit anything in confs/rendered
directly.data
– Data files, such as those used to generate HTML.fabfile
– Fabric commands for automating setup, deployment, data processing, etc.etc
– Miscellaneous scripts and metadata for project bootstrapping.jst
– Javascript (Underscore.js) templates.less
– LESS files, will be compiled to CSS and concatenated for deployment.templates
– HTML (Jinja2) templates, to be compiled locally.tests
– Python unit tests.www
– Static and compiled assets to be deployed. (a.k.a. “the output”)www/assets
– A symlink to an S3 bucket containing binary assets (images, audio).www/live-data
– “Live” data deployed to S3 via cron jobs or other mechanisms. (Not deployed with the rest of the project.)www/test
– Javascript tests and supporting files.app.py
– A Flask app for rendering the project locally.app_config.py
– Global project configuration for scripts, deployment, etc.copytext.py
– Code supporting the Editing workflowcrontab
– Cron jobs to be installed as part of the project.public_app.py
– A Flask app for running server-side code.render_utils.py
– Code supporting template rendering.requirements.txt
– Python requirements.static.py
– Static Flask views used in both app.py
and public_app.py
.Node.js is required for the static asset pipeline. If you don't already have it, get it like this:
install node
https://npmjs.org/install.sh | sh
Then bootstrap the project:
yria
rtualenv syria
install -r requirements.txt
install
update
Problems installing requirements? You may need to run the pip command as ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install -r requirements.txt
to work around an issue with OSX.
Project secrets should never be stored in app_config.py
or anywhere else in the repository. They will be leaked to the client if you do. Instead, always store passwords, keys, etc. in environment variables and document that they are needed here in the README.
Large media assets (images, videos, audio) are synced with an Amazon S3 bucket specified in app_config.ASSETS_S3_BUCKET
in a folder with the name of the project. (This bucket should not be the same as any of your app_config.PRODUCTION_S3_BUCKETS
or app_config.STAGING_S3_BUCKETS
.) This allows everyone who works on the project to access these assets without storing them in the repo, giving us faster clone times and the ability to open source our work.
Syncing these assets requires running a couple different commands at the right times. When you create new assets or make changes to current assets that need to get uploaded to the server, run `fab assets.sync
`. This will do a few things:
Unfortunantely, there is no automatic way to know when a file has been intentionally deleted from the server or your local directory. When you want to simultaneously remove a file from the server and your local environment (i.e. it is not needed in the project any longer), run `fab assets.rm:"www/assets/file_name_here.jpg"
`
A site can have any number of rendered pages, each with a corresponding template and view. To create a new one:
templates
directory. Ensure it extends _base.html
.app.py
. Decorate it with a route to the page name, i.e. @app.route('/filename.html')
.html
and do not start with _
will automatically be rendered when you call fab render
.A flask app is used to run the project locally. It will automatically recompile templates and assets on demand.
on $PROJECT_SLUG
app
Visit localhost:8000 in your browser.
IMPORTANT NOTE: This project relies on an outdated method to access content Google Spreadsheets. For now, the connection has been disabled (in this commit), and the project instead pulls from a spreadsheet stored in www/assets/copy.xlsx
. Run fab assets.sync
to download this file (and other media files) from the assets rig.
This app uses a Google Spreadsheet for a simple key/value store that provides an editing workflow.
View the sample copy spreadsheet.
This document is specified in app_config
with the variable COPY_GOOGLE_DOC_KEY
. To use your own spreadsheet, change this value to reflect your document's key (found in the Google Docs URL after &key=
).
A few things to note:
key
, there is expected to be a column called value
and rows will be accessed in templates as key/value pairsThe app template is outfitted with a few fab
utility functions that make pulling changes and updating your local data easy.
To update the latest document, simply run:
text.update
Note: text.update
runs automatically whenever fab render
is called.
At the template level, Jinja maintains a COPY
object that you can use to access your values in the templates. Using our example sheet, to use the byline
key in templates/index.html
:
OPY.attribution.byline }}
More generally, you can access anything defined in your Google Doc like so:
OPY.sheet_name.key_name }}
You may also access rows using iterators. In this case, the column headers of the spreadsheet become keys and the row cells values. For example:
or row in COPY.sheet_name %}
ow.column_one_header }}
ow.column_two_header }}
ndfor %}
When naming keys in the COPY document, pleaseattempt to group them by common prefixes and order them by appearance on the page. For instance:
e
ne
t_header
t_body
t_url
load_label
load_url
This project contains an experiment to remove some of the copy editing from the spreadsheet.
Each chapter of the project is managed as a Markdown file in www/assets
(e.g. www/assets/greece
).
Single photos or series of photos are managed in the Google sheet named 'photos' and embedded in the copy with a simple Jinja token:
text...
hotos('greece3') }}
This project adds a simple new image optimization feature to the app template.
To compress and optimize JPEG image assets in www/assets/img
run:
assets.crunch
If there is a file called myphoto.jpg
, the optimizer will create
desktop-myphoto.jpg
(1140px wide) and mobile-myphoto.jpg
(768px wide). In turn the image
loader in app.js
will pick the optimally sized image for the device and the container the
image appears in.
To add new image sizes or tweak compression parameters, edit crunch_images.sh
.
Sometimes, our projects need to read data from a Google Doc that's not involved with the COPY rig. In this case, we've got a class for you to download and parse an arbitrary Google Doc to a CSV.
This solution will download the uncached version of the document, unlike those methods which use the “publish to the Web” functionality baked into Google Docs. Published versions can take up to 15 minutes up update!
First, export a valid Google username (email address) and password to your environment.
rt APPS_GOOGLE_EMAIL=foo@gmail.com
rt APPS_GOOGLE_PASS=MyPaSsW0rd1!
Then, you can load up the GoogleDoc
class in etc/gdocs.py
to handle the task of authenticating and downloading your Google Doc.
Here's an example of what you might do:
rt csv
etc.gdoc import GoogleDoc
read_my_google_doc():
doc = {}
doc['key'] = '0ArVJ2rZZnZpDdEFxUlY5eDBDN1NCSG55ZXNvTnlyWnc'
doc['gid'] = '4'
doc['file_format'] = 'csv'
doc['file_name'] = 'gdoc_%s.%s' % (doc['key'], doc['file_format'])
g = GoogleDoc(**doc)
g.get_auth()
g.get_document()
with open('data/%s' % doc['file_name'], 'wb') as readfile:
csv_file = list(csv.DictReader(readfile))
for line_number, row in enumerate(csv_file):
print line_number, row
_my_google_doc()
Google documents will be downloaded to data/gdoc.csv
by default.
You can pass the class many keyword arguments if you'd like; here's what you can change:
See etc/gdocs.py
for more documentation.
Python unit tests are stored in the tests
directory. Run them with fab tests
.
With the project running, visit localhost:8000/test/SpecRunner.html.
Compile LESS to CSS, compile javascript templates to Javascript and minify all assets:
on syria
render
(This is done automatically whenever you deploy to S3.)
If you want to test the app once you've rendered it out, just use the Python webserver:
ww
on -m SimpleHTTPServer
staging master deploy
You can deploy to EC2 for a variety of reasons. We cover two cases: Running a dynamic web application (public_app.py
) and executing cron jobs (crontab
).
Servers capable of running the app can be setup using our servers project.
For running a Web application:
app_config.py
set DEPLOY_TO_SERVERS
to True
.app_config.py
set DEPLOY_WEB_SERVICES
to True
.fab staging master servers.setup
to configure the server.fab staging master deploy
to deploy the app.For running cron jobs:
app_config.py
set DEPLOY_TO_SERVERS
to True
.app_config.py
, set INSTALL_CRONTAB
to True
fab staging master servers.setup
to configure the server.fab staging master deploy
to deploy the app.You can configure your EC2 instance to both run Web services and execute cron jobs; just set both environment variables in the fabfile.
Cron jobs are defined in the file crontab
. Each task should use the cron.sh
shim to ensure the project's virtualenv is properly activated prior to execution. For example:
* * * ubuntu bash /home/ubuntu/apps/syria/repository/cron.sh fab $DEPLOYMENT_TARGET cron_jobs.test
To install your crontab set INSTALL_CRONTAB
to True
in app_config.py
. Cron jobs will be automatically installed each time you deploy to EC2.
The cron jobs themselves should be defined in fabfile/cron_jobs.py
whenever possible.
Web services are configured in the confs/
folder.
Running fab servers.setup
will deploy your confs if you have set DEPLOY_TO_SERVERS
and DEPLOY_WEB_SERVICES
both to True
at the top of app_config.py
.
To check that these files are being properly rendered, you can render them locally and see the results in the confs/rendered/
directory.
servers.render_confs
You can also deploy only configuration files by running (normally this is invoked by deploy
):
servers.deploy_confs
Sometimes it makes sense to run a fabric command on the server, for instance, when you need to render using a production database. You can do this with the fabcast
fabric command. For example:
staging master servers.fabcast:deploy
If any of the commands you run themselves require executing on the server, the server will SSH into itself to run them.
The Google Analytics events tracked in this application are:
|Category|Action|Label|Value|Custom 1|Custom 2|
|——–|——|—–|—–|——–|——–|
|syria|tweet|location
||||
|syria|facebook|location
||||
|syria|email|location
||||
|syria|new-comment||||
|syria|open-share-discuss||||
|syria|close-share-discuss||||
|syria|summary-copied||||
|syria|featured-tweet-action|action
||tweet_url
|
|syria|featured-facebook-action|action
||post_url
|