Home » A Google Webmaster Tools API PHP Example – Using Search Analytics API to download search analytics data as CSV with the new OAuth 2.0 method

A Google Webmaster Tools API PHP Example – Using Search Analytics API to download search analytics data as CSV with the new OAuth 2.0 method


This is a Webmaster Tools API PHP Example to download your search analytics data as CSV with the new OAuth 2.0 method. As you probably have learned, Google announced that they will be discontinuing their old CSV download scripts for information on queries & rankings and replacing it with their new Search Analytics API. Since there’s a ton of Stack Overflow posts that seem to be unanswered on how to do this, I figured I’d post my approach to getting a CSV of my Search Analytics data like I’m accustomed to getting through the webmaster tools admin panel.

First thing we will do is create an application with these steps: https://developers.google.com/webmaster-tools/v3/how-tos/authorizing#APIKey

You’re going to add credentials for a Service account and select P12 as your key type. You’ll then receive the .p12 file that we’ll use to authenticate with OAuth 2.0 in the steps below. These are the screens you’ll see:

Google API Service AccountGoogle API P12 Key

 

 

 

 

 

 

At the end of this you’ll also get a service account email address. Copy that address down so you can put it into the example below. After you’ve got your app setup, follow all the instructions in these links to grant access to your app through the appropriate Google services (just going to link to Webmaster Tools and Google Analytics for this tutorial):

Webmaster Tools:  https://developers.google.com/webmaster-tools/v3/quickstart/quickstart-oacurl?hl=en

Google Analytics:  https://developers.google.com/analytics/devguides/reporting/core/v3/quickstart/service-php

Now we need to download the Google API PHP Client Library for PHP. Make a directory and place your .p12 file you downloaded from earlier inside. Open up a terminal, go to that directory, and clone the git repo they have to pull their latest Google API PHP Client Library:

That should download the google-api-php-client folder into your project’s directory. Now for the tricky part… writing the functions to retrieve the data. I took an object-oriented approach to this because I plan to turn this class into a library that pulls all the analytics and webmaster tools data I need – feel free to hack this up to be procedural if that’s all you need.

Here’s the beginning to my class that has a couple methods to grab the data from the API and create a CSV:

Now all you have to do is run it via the command line (or in a web browser if you want), and we have a beautiful CSV of the top 50 search queries (which can be changed with the second parameter in the get_searchanalytics() method) in a file called test.csv in my project’s directory. I know this is a relatively new subject so I’m going to post this prematurely without editing this a bunch so people can get the help they need on Stack Overflow and such.

A lot of this has been trial and error and digging into their Webmaster class for the methods, but if you have issues or questions feel free to comment and I’ll try to help you navigate this API as best I can.

Note: If you get a 403 Forbidden error, make sure you approve your app with this link: https://console.developers.google.com/flows/enableapi?apiid=webmasters&credential=client_key

Update: (courtesy of Jakob from the comments below)

Since the article was written Google has updated the main branch of their API, and a lot of code was removed from it because it has to be rewritten, so Webmasters class, which was in src/Google/Service/Webmasters.php is not there any longer.

However, if you browse their github page, they tell you that there is a separately maintained branch, marked as v1-master right here:
https://github.com/google/google-api-php-client/tree/v1-master

And that is the one you should use for this project. Don’t forget to update file references accordingly.

One more update. When you create a service account, it will ask if you want an App Engine default service account or Compute Engine default service account. The one you need to select in this case is the Compute Engine default service account.

27 comments on “A Google Webmaster Tools API PHP Example – Using Search Analytics API to download search analytics data as CSV with the new OAuth 2.0 method

  1. Ben on

    Hello,

    I’ve trying the same method to obtain query analytics but i always got the same error. I also tried to fetch using your class, same issue here :/

    The raised error is :
    Fatal error: Class ‘Google_Service_Webmasters_SearchAnalyticsQueryRequest’ not found

    If you have any idea or maybe if you faced the same issue i will be so glad 🙂

    Best regards,

    Reply
    • Jeff Hays on

      Hey Markus,

      Yeah, if you notice at the bottom of my tutorial I have a little section in blue that has a link for you in the event you’re getting a 403 error like you describe. Click that link and make sure your app is approved for this API call. Here it is for your convenience:

      Note: If you get a 403 Forbidden error, make sure you approve your app with this link: https://console.developers.google.com/flows/enableapi?apiid=webmasters&credential=client_key

      Let me know if you continue to have issues.

      Reply
        • Jeff Hays on

          Hmm, odd – when I got a 403 error that was the thing that fixed it for me. What I would recommend is to try and delete the app and start all over again from the instructions (more Google’s instructions than mine, but go through the links in order). Tough to troubleshoot auth errors – I think I deleted and recreated my app (and granted API permissions and updated the googleservice account email) like 3 times before I figured out the API access wasn’t being granted. I added my googleservice account to my Google Analytics settings as well but I think that’s only if you need that API, not just the search console.

          Reply
          • queseo on

            Hi!
            I have the same problem.
            I go the link: https://console.developers.google.com/flows/enableapi?apiid=webmasters&credential=client_key
            And i go there https://console.developers.google.com/apis/credentials/domainverification to add the domain.
            I try with other codes and i have the same access problem =(
            The code and parameters i put:

          • Jeff Hays on

            Hey Queseo,

            I’ll be working towards an updated example with the new API libraries from Google at some point this week. I’ll post back when this is updated. Thanks for your patience.

      • Jason on

        I found that I was confused… I am site owner with the same Google login email as I am using for the developers console, and thought that was enough, but had to add the serviceacc: XXXX407@developer.gserviceaccount.com as a restrticted user. Which, when you think about it is logical since the serviceacc: XXXX407@developer.gserviceaccount.com is for a specific app and being owner of a site doesn’t necessarily mean I would want to give access to that site to all my apps… Hope this a) makes sense and b) helps someone

        Reply
  2. Sven on

    Hi Jeff,

    thank for the example. As pointed out by Karl this code does not work anymore. Do you have an estimate for an updated version?

    thanks,
    sven

    Reply
    • Jeff Hays on

      Hey Sven,

      Sorry for the delay – things have been crazy coming into the new year. I’ll try to pull the updated API libraries from Google and rework this example some time this week if I can. Thanks for your patience! If you have already figured out an update for the new libraries, let me know and I’ll make you an author so you can update things and/or write your own findings here. I’ll post back when I have an updated working example for you (this example still works for my solution at work because we’re on the old libraries).

      Reply
    • Jeff Hays on

      Awesome! This is exactly the reason I post these solutions – when there’s nothing else out there that was helpful to me to get up and running quickly. Thanks for posting back! I’ll have an updated solution with the new API libraries from Google shortly as well.

      Reply
  3. Jakob on

    People are quick to complain before doing any research. Here’s the thing – since the article was written Google has updated the main branch of their API, and a lot of code was removed from it because it has to be rewritten, so Webmasters class, which was in src/Google/Service/Webmasters.php is not there any longer.
    However, if you browse their github page, they tell you that there is a separately maintained branch, marked as v1-master right here:
    https://github.com/google/google-api-php-client/tree/v1-master
    And that is the one you should use for this project. Don’t forget to update file references accordingly.

    One more update. When you create a service account, it will ask if you want an App Engine default service account or Compute Engine default service account. The one you need to select in this case is the Compute Engine default service account.

    P.S.: Jeff, thank you so much for the work you’ve done, this code and instructions save a lot of trial and error and frustration. Up to date you are pretty much the only one who took the time and effort to make this.

    Reply
    • Jeff Hays on

      Thank you for finding the time to research their updated API and leave constructive and helpful comments. I often just get too busy to revisit solutions/tutorials that break when a library is updated. I’ll be updating this article to add your comments for other people that are having issues. You rock Jakob! 🤓

      Reply
  4. Stephan on

    Deprecated: google-api-php-client’s autoloader was moved to vendor/autoload.php in 2.0.0. This redirect will be removed in 2.1. Please adjust your code to use the new location. in /usr/www/users/tauchrk/seobox/reports/gwt/google-api-php-client/src/Google/autoload.php on line 19

    Fatal error: Class ‘Google_Auth_AssertionCredentials’ not found in /usr/www/users/tauchrk/seobox/reports/gwt/smtest.php on line 25

    I don`t find the error

    Reply
    • Jeff Hays on

      Hey Daniel,

      Great question! I actually just included that because I needed it for the project I was working on that inspired me to write this tutorial and then someone also asked me to include it so they knew the class to include. Short answer: you are absolutely correct and that part of this tutorial is unnecessary 🙂

      Reply
  5. Madhushalini on

    I am attempting to pull a list of the companies that my company has registered in their Google Webmaster Tools. I am doing this using PHP and the Google API Client Library for PHP (https://developers.google.com/api-client-library/php/). I have gotten everything going, except for the fact that it gives me an empty array as a result. My code is below and any help would be greatly appreciated.

    The appropriate values are filled in for client id, account, etc. and everything authenticates appropriately, as I used the same code to access the books API

    Reply
  6. matts on

    Hi Jeff,

    Thanks for the post.It’s indeed useful for poeple like me who are novice.I am new to use Google Search API using php.I tried to download google api library and started working step by step through the process given above.I changed the code to accomodate json file and I am trying to run above code.I am getting below error.
    “code”: 403,
    “message”: “The request is missing a valid API key.”,
    Somewhere I am doing mistake and I don’t know what I have done wrong.
    1.I went to API console and enable google search console API,compute engine api,Created a service account and selected compute engine and created my service account and got .json file and also the keyid .And in code used $client->setAuthConfig(‘.json’); and $client->setScopes([‘https://www.googleapis.com/auth/analytics.readonly’]); but Iam getting above error while trying to run the php.
    Could you please help me debug.

    Reply
    • Jeff Hays on

      Hey Matt,

      It definitely sounds like part of your API credentials and/or the that are problematic here, and I remember this being kinda tricky to figure out initially when I got the same error. It looks like you need to specify the fully qualified path to your .json file. Your example above should be more like: $client->setAuthConfig(‘/path/to/your/key.json’);

      If that isn’t the problem, did you try this part that I mentioned in my tutorial?

      Note: If you get a 403 Forbidden error, make sure you approve your app with this link: https://console.developers.google.com/flows/enableapi?apiid=webmasters&credential=client_key

      If not, try to approve that there and try again. I believe I generated new .json or .p12 files after doing that, but it’s been awhile so I can’t fully remember the process I took on the App Approval side of things. Hopefully this helps.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *