GoogleCloudPlatform/unstructured-text-demo

Name: unstructured-text-demo

Owner: Google Cloud Platform

Description: null

Created: 2016-08-12 20:09:34.0

Updated: 2018-01-22 18:47:32.0

Pushed: 2017-09-29 19:13:59.0

Homepage: null

Size: 57

Language: JavaScript

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Wikipedia and the Google Cloud Natural Language API

This demo contains code to detect entities and sentiment of a dump of Wikipedia, as well as a webapp that queries the resultant BigQuery table.

Introduction

This demo is split up into two parts:

• The tools directory contains a Google Cloud Dataflow pipeline that starts from a Wikipedia XML dump and ends with a table in Google BigQuery containing all detected entities in Wikipedia, by way of the Google Cloud Natural Language API.
• The app directory contains a Google App Engine app that uses Wikipedia's Mediawiki API to fetch a given Wikipedia page dynamically, and uses the Natural Language API to highlight all the entities detected. It can also display a related-entities graph, by querying the BigQuery table created above.

Prerequisites

• Set up a project in the Google Cloud Console
• Enable the Natural Language API

Setup

• For tools/, see tools/README.md

• For app/:

• Install the AppEngine Python SDK

• Download a service account key and set the GOOGLE_APPLICATION_CREDENTIALS environment variable, as detailed here.

• Install the dependencies into a lib/ directory:

    $ cd app/
    $ pip install -r requirements.txt -t lib/
  

Running

• For tools/, see tools/README.md

• For app/:

• Run the local test server:

    $ cd app/
    $ dev_appserver.py .
  

• Visit the site at http://localhost:8080/

Disclaimer

This is not an official Google product.