sfbrigade/datasci-open-data-search

Name: datasci-open-data-search

Owner: SFBrigade

Description: This project aims to use analytics of search text to improve the user experience at https://data.sfgov.org/

Created: 2016-06-28 18:39:57.0

Updated: 2016-11-17 04:01:55.0

Pushed: 2017-08-24 03:13:30.0

Homepage: null

Size: 9427

Language: Jupyter Notebook

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Open Data Search

This project is a part of the Data Science Working Group at Code for San Francisco. Other DSWG projects can be found at the main GitHub repo.

– Project Status: Active

Project Intro/Objective

The purpose of this project is to use analytics and topic modelling of search text to improve the user experience at https://data.sfgov.org/

Partner

• SF Open Data
• https://datasf.org/opendata/
• Partner contact: Jason Lally, @jasonlally

Methods Used

• Data Analysis
• Descriptive Statistics/Data Visualization
• Natural Language Processing
• Word2Vec Modelling

Technologies

• R
• Python
• Pandas, Spacy

Project Description

Th major goals of the project are as follows:

• Clean and process search terms and categorize search terms by quality
• Utilize Natural Language Processing and Topic Modelling on valid search terms and cluster terms to determine potential demand for data sources
• Provide actionable insights to improve search functionality on the site

Needs of this project

• NLP/Topic Modelling Expertise

Getting Started

1. Clone this repo, for help see this tutorial.

2. Data is being kept here
   

3. Data processing/transformation script is Data Combiner

   • Script that combines raw data from .tsv files into a single .csv file
   • Search Data Processing Jupyter Notebook - Notebook that cleans, processes and categorizes search terms

Featured Notebooks/Analysis

• Exploration of Search Google Analytics

• Search Data Modelling Juptyer Notebook

  • Notebook with vectorization of search terms using pre-trained word2vec model

Contributing DSWG Members

Team Lead (Contact): Rocio S Ng (@Rocio)

Other Members:

|Name | Slack Handle | |———|—————–| |Bao Lin Liu | @jbaolinliu | |Scott Brenstuhl| @scott_brenstuhl |

Contact

• If you haven't joined the SF Brigade Slack, you can do that here.
  
• Our slack channel is #datasci-open-data_src
• Feel free to contact team leads with any questions or if you are interested in contributing!