Name: nodejs-dataproc
Owner: Google APIs
Description: Google Cloud Dataproc is a managed Apache Spark and Apache Hadoop service that lets you take advantage of open source data tools for batch processing, querying, streaming, and machine learning.
Created: 2017-12-18 23:27:04.0
Updated: 2018-05-22 18:33:00.0
Pushed: 2018-05-22 18:32:59.0
Homepage: https://cloud.google.com/dataproc/
Size: 853
Language: JavaScript
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Node.js idiomatic client for Cloud Dataproc.
Cloud Dataproc is a managed Apache Spark and Apache Hadoop service that lets you take advantage of open source data tools for batch processing, querying, streaming, and machine learning.
Read more about the client libraries for Cloud APIs, including the older Google APIs Client Libraries, in Client Libraries Explained.
Table of contents:
Select or create a Cloud Platform project.
Enable billing for your project.
Enable the Google Cloud Dataproc API.
Set up authentication with a service account so you can access the API from your local workstation.
npm install --save @google-cloud/dataproc
rocess.env.GCLOUD_PROJECT ||
rocess.env.GOOGLE_APPLICATION_CREDENTIALS
row new Error(
'Usage: GCLOUD_PROJECT=<project_id> GOOGLE_APPLICATION_CREDENTIALS=<path to json key> node #{$0}'
t dataproc = require('@google-cloud/dataproc');
t client = new dataproc.v1.ClusterControllerClient({
optional auth parameters.
t projectId = process.env.GCLOUD_PROJECT;
terate over all elements.
t region = 'global';
t request = {
ojectId: projectId,
gion: region,
nt.listClusters(request).then(responses => {
nst resources = responses[0];
nsole.log('Total resources:', resources.length);
r (let i = 0; i < resources.length; i += 1) {
console.log(resources[i]);
r obtain the paged response.
t options = {autoPaginate: false};
t callback = responses => {
The actual resources in a response.
nst resources = responses[0];
The next request if the response shows that there are more responses.
nst nextRequest = responses[1];
The actual response object, if necessary.
const rawResponse = responses[2];
r (let i = 0; i < resources.length; i += 1) {
console.log(resources[i]);
(nextRequest) {
// Fetch the next page.
return client.listClusters(nextRequest, options).then(callback);
nt.listClusters(request, options).then(callback);
nt.listClustersStream(request).on('data', element => {
nsole.log(element);
The Cloud Dataproc Node.js Client API Reference documentation also contains samples.
This library follows Semantic Versioning.
This library is considered to be in alpha. This means it is still a work-in-progress and under active development. Any release is subject to backwards-incompatible changes at any time.
More Information: Google Cloud Platform Launch Stages
Contributions welcome! See the Contributing Guide.
Apache Version 2.0
See LICENSE