Name: parallel
Owner: Python for Data
Description: null
Created: 2016-03-23 19:23:23.0
Updated: 2018-02-25 16:22:48.0
Pushed: 2016-07-11 04:27:21.0
Homepage: null
Size: 442
Language: Jupyter Notebook
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Students will walk away with a high-level understanding of both parallel problems and how to reason about parallel computing frameworks. They will also walk away with hands-on experience using a variety of frameworks easily accessible from Python.
Knowledge of Python and general familiarity with the Jupyter notebook are assumed. This is generally aimed at a beginning to intermediate audience.
For the first half, we will cover basic ideas and common patterns encountered when analyzing large data sets in parallel. We start by diving into a sequence of examples that require increasingly complex tools. From the most basic parallel API: map, we will cover some general asynchronous programming with Futures, and high level APIs for large data sets, such as Spark RDDs and Dask collections, and streaming patterns. For the second half, we focus on traits of particular parallel frameworks, including strategies for picking the right tool for your job. We will finish with some common challenges in parallel analysis, such as debugging parallel code when it goes wrong, as well as deployment and setup strategies.
Install Anaconda
Update select packages
Everyone:
conda install -c conda-forge ipyparallel ujson
pip install snakeviz
Python 2 users:
conda install futures
Linux/Mac users:
conda install -c quasiben spark
Test your installation:
python -c 'import concurrent.futures, ipyparallel, dask, jupyter, pyspark'