Name: searchConsoleR
Owner: Zapier
Description: R interface with Google Search Console API v3, including Search Analytics.
Forked from: MarkEdmondson1234/searchConsoleR
Created: 2016-12-01 21:06:01.0
Updated: 2017-01-24 13:20:05.0
Pushed: 2016-09-09 07:41:53.0
Homepage: http://code.markedmondson.me/searchConsoleR/
Size: 1683
Language: R
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
R interface with Google Search Console (formally Google Webmaster Tools) API v3.
Check out the news for latest updates.
Install dependency googleAuthR from CRAN:
all.packages("googleAuthR")
ary(googleAuthR)
Install searchConsoleR 0.2.0 from CRAN:
all.packages("searchConsoleR")
ary(searchConsoleR)
If you want the development version of searchConsoleR on Github:
ools::install_github("MarkEdmondson1234/searchConsoleR")
ary(searchConsoleR)
Authentication can be done locally or within a Shiny app. See a very bare bones example here: https://mark.shinyapps.io/searchConsoleRDemo/
search_analytics() - download Google SEO data into an R dataframe.list_websites() - list websites in your Google Search Console.add_website() - add a website to your Google Search Console.delete_website() - delete a website from your Google Search Console.list_sitemaps() - list sitemaps recognised in Google Search Console.add_sitemap() - add sitemap URL location to Google Search Console.delete_sitemap() - remove sitemap URL location in Google Search Console.crawl_errors() - list various types of crawl errors googlebot has found.list_crawl_error_samples() - get a list of example URLs with errors.error_sample_url() - show details about an example URL error (for example, links to a 404 URL)fix_sample_url() - mark a URL as fixed.scr_auth() - main authentication function. Works locally and within a Shiny environment.Work flow always starts with authenticating with Google:
ary(searchConsoleR)
auth()
Your browser window should open up and go through the Google sign in OAuth2 flow. Verify with a user that has Search Console access to the websites you want to work with.
Check out the documentation of any function for a guide on what else can be done.
rchConsoleR
If you authenticate ok, you should be able to see a list of your websites in the Search Console via:
ebsites <- list_websites()
ebsites
We'll need one unique `sc_websites$siteUrl` for the majority of the other functions.
Most people will find the Search Analytics most useful. All methods from the web interface are available.
Here is an example query, which downloads the top 100 rows of queries per page for the month of July 2015, for United Kingdom desktop web searches:
desktop_queries <-
search_analytics("http://example.com",
"2015-07-01", "2015-07-31",
c("query", "page"),
dimensionFilterExp = c("device==DESKTOP","country==GBR"),
searchType="web", rowLimit = 100)
For a lot more details see:
rch_analytics
You can get more than the standard 5000 rows via batching. There are two methods available, one via a API call per date, the other using the APIs startRow parameter.
The date method gets more impressions for 0 click rows, the batch method is quicker but gets just rows with clicks.
Specify a rowLimit when batching - if using method byDate this will be the limit it fetches per day, and currently needs to be over 5000 to work. (Issue #17 will fix this).
0 <- search_analytics("http://www.example.co.uk",
dimensions = c("date","query","page","country"),
rowLimit = 200000,
walk_data = "byBatch")
hing data via method: byBatch
test0 has 13063 rows
<- search_analytics("http://www.example.co.uk",
dimensions = c("date","query","page","country"),
walk_data = "byDate")
hing data via method: byDate
test has 419957 rows
m(test0$clicks)
12866
m(test$clicks)
12826
m(test$impressions)
1420217
m(test0$impressions)
441029
Here is an example for downloading daily data and exporting to .csv
script to download and archive Google search analytics
emo of searchConsoleR R package.
ersion 1 - 10th August 2015
ark Edmondson (http://markedmondson.me)
ary(searchConsoleR)
hange this to the website you want to download data for. Include http
ite <- "http://copenhagenish.me"
ata is in search console reliably 3 days ago, so we donwnload from then
oday - 3 days
t <- Sys.Date() - 3
ne days data, but change it as needed
<- Sys.Date() - 3
hat to download, choose between data, query, page, device, country
load_dimensions <- c('date','query')
hat type of Google search, choose between 'web', 'video' or 'image'
<- c('web')
ther options available, check out ?search_analytics in the R console
uthorize script with Search Console.
irst time you will need to login to Google,
ut should auto-refresh after that so can be put in
uthorize script with an account that has access to website.
auth()
irst time stop here and wait for authorisation
et the search analytics data
<- search_analytics(siteURL = website,
startDate = start,
endDate = end,
dimensions = download_dimensions,
searchType = type)
o stuff to the data
ombine with Google Analytics, filter, apply other stats etc.
rite a csv to a nice filename
name <- paste("search_analytics",
Sys.Date(),
paste(download_dimensions, collapse = "",sep=""),
type,".csv",sep="-")
e.csv(data, filename)
This parameter is used in search_analytics to filter the result.
Filter using this format: `filter operator expression`
Filter can be one of:
country,devicepagequeryOperator can be one of `~~, ==, !~, !=` where the symbols mean:
~~ : 'contains',== : 'equals',!~ : 'notContains',!= : 'notEquals'Expression formatting:
`pageor ``query``` is free text.`country` must be the three letter country code as per the the ISO 3166-1 alpha-3 standard. e.g. USA, GBR = United Kingdom, DNK = Denmark`device` must be one of: 'MOBILE', 'DESKTOP' or 'TABLET'You can have multiple `AND` filters by putting them in a character vector. The below looks for desktop searches in the United Kingdom, not showing the homepage and not including queries containing 'brandterm'.
evice==DESKTOP","country==GBR", "page!=/home", "query!~brandterm")
R``` filters aren't yet supported in the API.
sing your own Google API project
efault `searchConsoleR` uses its own Google API project to grant requests, but if you want to use your own keys:
et up your project in the [Google API Console](https://developers.google.com/identity/sign-in/web/devconsole-project) to use the search console v3 API.
For local use
lick 'Create a new Client ID', and choose "Installed Application".
ote your Client ID and secret.
odify these options after `searchConsoleR` has been loaded:
`options("searchConsoleR.client_id" = "YOUR_CLIENT_ID")`
`options("searchConsoleR.client_secret" = "YOUR_CLIENT_SECRET")`
For Shiny use
lick 'Create a new Client ID', and choose "Web Application".
ote your Client ID and secret.
dd the URL of where your Shiny app will run, as well as your local host for testing including a port number. e.g. https://mark.shinyapps.io/searchConsoleRDemo/ and http://127.0.0.1:4624
n your Shiny script modify these options:
`options("searchConsoleR.webapp.client_id" = "YOUR_CLIENT_ID")`
`options("searchConsoleR.webapp.client_secret" = "YOUR_CLIENT_SECRET")`
un the app locally specifying the port number you used e.g. `shiny::runApp(port=4624)`
r deploy to your Shiny Server that deploys to web port (80 or 443).