This tutorial is to learn how to authenticate to the Google Search Console API with OAuth2 and Python.
I will show the steps to:
- Connect to the developer console,
- Get Your API Keys and authenticate to the API OAuth2
- Make your first API call.
Follow this tutorial if you want to learn how to use Google Search Console API with Python.
To learn more, view my complete guide on Python for SEO.
Navigation Show
Make Your First Google Search Console API call with Python
There are many things that you can do with the Search Console API like getting search traffic and extracting indexed pages.
In this post, we will make a very basic API call: get validated properties.
The steps to use Google Search Console API in Python are:
- Get Your Google Search Console API Key
- Import Libraries
- Authenticate to the API
- Make Your API Call
Step 1: Get Your Google Search Console API Key
You will need an API key to be able to connect to the Google Search Console API.
How to Get Google Search Console API Keys
What is an API Key?
An API key is like your username and password to access the API.
Follow this guide to help you get the detailed steps to get your Google Search Console API key.
Otherwise, here are the simplified steps to get your Google Search Console API keys.
- Go toGoogle’s developers console, and sign-in;
- Go to “Dashboard” and click “Enable APIs and Services” ;
- Search for “Google Search Console API” and enable the API;
- Go to the “credential” tab, click on “create credential” and select “OAuth Client ID”;
- Click on “configure the consent screen ” and give a name to your product;
- Choose “Other” as the application type and click create;
- Copy the client ID and client Secret or go-on and save to download the JSON file.
Download your API key first, and then you’ll be ready to connect to the Search Console API with Python.
Step 2: Import Libraries
To run the OAuth 2.0 authentication you will need to install and import all those libraries.
Alternatively, you can clone the Github Repository that I made and run:
$ pip install -r requirements.txt
import argparseimport httplib2import requestsfrom collections import defaultdictfrom dateutil import relativedeltafrom googleapiclient.discovery import buildfrom oauth2client import clientfrom oauth2client import filefrom oauth2client import tools
Step 3. Authenticate to the API
Now, we will log in to the API using OAuth. Two options are possible. Using a JSON file (recommended), or using the API client_id and client_secret.
Choose your preferred solution.
Connect Using a JSON API Key
The authorize_creds() function will check the credentials file and define a .dat file to save authorised credentials so you don’t have to go through the login process each time.
def authorize_creds(creds,authorizedcreds='authorizedcreds.dat'): ''' Authorize credentials using OAuth2. ''' print('Authorizing Creds') # Variable parameter that controls the set of resources that the access token permits. SCOPES = ['https://www.googleapis.com/auth/webmasters.readonly'] # Path to client_secrets.json file CLIENT_SECRETS_PATH = creds # Create a parser to be able to open browser for Authorization parser = argparse.ArgumentParser( formatter_class=argparse.RawDescriptionHelpFormatter, parents=[tools.argparser]) flags = parser.parse_args([]) # Creates an authorization flow from a clientsecrets file. # Will raise InvalidClientSecretsError for unknown types of Flows. flow = client.flow_from_clientsecrets( CLIENT_SECRETS_PATH, scope = SCOPES, message = tools.message_if_missing(CLIENT_SECRETS_PATH)) # Prepare credentials and authorize HTTP # If they exist, get them from the storage object # credentials will get written back to the 'authorizedcreds.dat' file. storage = file.Storage(authorizedcreds) credentials = storage.get() # If authenticated credentials don't exist, open Browser to authenticate if credentials is None or credentials.invalid: credentials = tools.run_flow(flow, storage, flags) # Add the valid creds to a variable # Take the credentials and authorize them using httplib2 http = httplib2.Http() # Creates an HTTP client object to make the http request http = credentials.authorize(http=http) # Sign each request from the HTTP client with the OAuth 2.0 access token webmasters_service = build('searchconsole', 'v1', http=http) # Construct a Resource to interact with the API using the Authorized HTTP Client. print('Auth Successful') return webmasters_serviceRun OAuth 2.0
To run the code, simply add your credentials and run authorize_creds(). Once this is done you will be able to run Google Search Console API using the webmasters_service variable.
if __name__ == '__main__': creds = 'client_secrets.json' webmasters_service = authorize_creds(creds)
The if name equals main line checks whether you are running the module or importing it. If you are importing it, authorize_creds() will not run.
Step 4: Make Your API Call
Now the glory!
Let’s see what websites we have validated in Google Search Console using the API.
site_list = webmasters_service.sites().list().execute()verified_sites_urls = [s['siteUrl'] for s in site_list['siteEntry'] if s['permissionLevel'] != 'siteUnverifiedUser' and s['siteUrl'][:4] == 'http']for site_url in verified_sites_urls: print( site_url)
Full Code
Here is the full code to let you make your first API call with Python.
oauth.py
#!/usr/bin/env python# oauth.py# Authorize credentials using OAuth2.# @author: Jean-Christophe Chouinard. # @role: Sr. SEO Specialist at SEEK.com.au# @website: jcchouinard.com# @LinkedIn: linkedin.com/in/jeanchristophechouinard/ # @Twitter: twitter.com/@ChouinardJCimport argparseimport httplib2import requestsfrom collections import defaultdictfrom dateutil import relativedeltafrom googleapiclient.discovery import buildfrom oauth2client import clientfrom oauth2client import filefrom oauth2client import toolsdef authorize_creds(creds,authorizedcreds='authorizedcreds.dat'): ''' Authorize credentials using OAuth2. ''' print('Authorizing Creds') # Variable parameter that controls the set of resources that the access token permits. SCOPES = ['https://www.googleapis.com/auth/webmasters'] # Path to client_secrets.json file CLIENT_SECRETS_PATH = creds # Create a parser to be able to open browser for Authorization parser = argparse.ArgumentParser( formatter_class=argparse.RawDescriptionHelpFormatter, parents=[tools.argparser]) flags = parser.parse_args([]) # Creates an authorization flow from a clientsecrets file. # Will raise InvalidClientSecretsError for unknown types of Flows. flow = client.flow_from_clientsecrets( CLIENT_SECRETS_PATH, scope = SCOPES, message = tools.message_if_missing(CLIENT_SECRETS_PATH)) # Prepare credentials and authorize HTTP # If they exist, get them from the storage object # credentials will get written back to the 'authorizedcreds.dat' file. storage = file.Storage(authorizedcreds) credentials = storage.get() # If authenticated credentials don't exist, open Browser to authenticate if credentials is None or credentials.invalid: credentials = tools.run_flow(flow, storage, flags) # Add the valid creds to a variable # Take the credentials and authorize them using httplib2 http = httplib2.Http() # Creates an HTTP client object to make the http request http = credentials.authorize(http=http) # Sign each request from the HTTP client with the OAuth 2.0 access token webmasters_service = build('searchconsole', 'v1', http=http) # Construct a Resource to interact with the API using the Authorized HTTP Client. print('Auth Successful') return webmasters_service# Create Function to execute your API Requestdef execute_request(service, property_uri, request): return service.searchanalytics().query(siteUrl=property_uri, body=request).execute() if __name__ == '__main__': creds = 'client_secrets.json' webmasters_service = authorize_creds(creds) get_properties.py
#!/usr/bin/env pythonfrom oauth import authorize_creds, execute_requestdef get_property_list(webmasters_service): ''' Get a list of validated properties from GSC ''' site_list = webmasters_service.sites().list().execute() # Filter for verified websites verified_sites_urls = [s['siteUrl'] for s in site_list['siteEntry'] if s['permissionLevel'] != 'siteUnverifiedUser' and s['siteUrl'][:4] == 'http'] return verified_sites_urlsif __name__ == '__main__': creds = 'client_secrets.json' webmasters_service = authorize_creds(creds) verified_sites_urls = get_property_list(webmasters_service)
Alternative OAuth2 connection: Client_Id and Client_Secrets
You might want to connect to the API simply by adding your client secrets and client id instead of using the JSON credential file.
Here is the code below:
Make sure that you don’t forget to add your own client_id and client_secrets. You can view the notebook.
parser = argparse.ArgumentParser( formatter_class=argparse.RawDescriptionHelpFormatter, parents=[tools.argparser])flags = parser.parse_args([])flow = client.OAuth2WebServerFlow(client_id='XXXXXXXXXXXXXXXXXXXXXXXXXX.apps.googleusercontent.com', client_secret='XXXXXXXXXXXXXXXXXXXXX', scope=SCOPES)storage = file.Storage('searchconsole.dat')credentials = storage.get()if credentials is None or credentials.invalid: credentials = tools.run_flow(flow, storage, flags)# Create an httplib2.Http object and authorize it with our credentialshttp = credentials.authorize(http=httplib2.Http())Search Console API Connection Errors
Error 403: access_denied in Search Console API
The Error 403: access_denied error when using Google Search Console API means that the user that validated the connection to the API does not have access to either the Google Search Console property or the Google Search Console API.
The error message goes something like this:
Access blocked: [PROJECT NAME] has not completed the Google verification process[PROJECT NAME] has not completed the Google verification process. The app is currently being tested and can only be accessed by developer-approved testers. If you think you should have access, contact the developer.If you are a developer of [PROJECT NAME], see error details.Error 403: access_denied
The Access blocked 403 error can happen if the email that you used is not allowed:
- access level for the property: go to Settings > Users and permissions in Google Search Console and verify that you have accessed
- access to the API: go to Google cloud console > OAuth Consent Screen > Test Users and add your email to the allows user list. This often happens when the app has just been created and pending being tested by Google.
OSError: [Errno 48] Address already in use
The OSError: [Errno 48] Address already in use error in Google Search Console API means thatthe Port that you try to call is already taken by another running process.
This often occur when the connection to the OAuth Consent screen is still open and you try to reopen a new connection to the API. Fix this by shutting down your Jupyter Notebook in Python or killing the process like that.
$ ps -fA | grep python$ kill PROCESS_NAMEValueError: Client secrets must be for a web or installed app
The ValueError: Client secrets must be for a web or installed app error in Google Search Console generally means that you are using the JSON generated under the wrong type of credentials (e.g. JSON created under Service accounts and not under OAuth 2.0 client ID)s.
Error 400: invalid_request
The Error 400: invalid_request error in Google Search Console API generally means that you have a malformed URL in your API requests. Verify the URL in your browser to check if you are fetching the right API endpoint, and using all the required parameter inside your request. Check how to use OAuth2 for more information. This happened to me when I was using the Google Analytics Data API credentials file instead of the Google Search Console JSON file. Also happens when the redirect URI is wrong or the Out-of-Band (OOB) flow migration hasn’t been done by the API wrapper.
Click on the “Error details” link to verify the part of the URL that is malformed.
Conclusion
We are done. You have successfully connected to Google Search Console, calling the API With Python. Now we will learn how to Get All Your Search traffic With Google Search Console API.
4.7/5 - (7 votes)
Related posts:
- Backup Google Search Console Data Into MySQL With Python
- Find Keyword Cannibalization Using Google Search Console and Python
- Google Search Console API with Python (Video Example)
- Authorise Requests to GSC API Using OAuth 2.0
