metakgp/mcmp

Name: mcmp

Owner: metakgp

Description: My Choice My Prof

Created: 2015-10-04 18:51:54.0

Updated: 2017-04-01 11:29:38.0

Pushed: 2017-03-29 12:49:14.0

Homepage: http://metakgp.github.io/mcmp/

Size: 443

Language: Python

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

My Choice My Prof

Searching for professors from given fields is easier than you thought. Reverse search enables to find professors from the field, say you need a list of all the professors working in nanomaterials. What's next? Keep reading.

The repository holds the source for the crawler to generate data for mcmp ` Professor Reverse Search`.

Prerequisites : Install Scrapy. Follow Scrapy Install.

p install scrapy

Get started :

• Clone the repo as :

t clone https://github.com/metakgp/mcmp.git

• Checkout to the source code branch :

checkout source
everse

• To run the crawler with spider-config as in rsspider.py and store data as json, use :

  py crawl reverse -o prof_details.json -t json
  

Contributing to mcmp

Please use issues page to report any bugs or file feature requests.

Developing mcmp

We love PR's. Before we begin developing, a short note:

• scrapy will scrape items written in reverse/reverse/items.py

• find scraping logic (= rules for scraping from html dump) in reverse/reverse/spiders/rsspiders.py

• departments.json contains list of all departments.

• prof_details.json uses professor.json to populate itself. (Itself, huh? Oh yes, yes you do it. Code is just a manifestation of your brilliant mind. :D )

Search Box on webpage

makelists.py processes prof_details.json and creates finallist.json which is then used to create the fuzzy search, search box here. If you want to modify the webpage for mcmp, checkout the gh-pages branch. More about github pages.