Wednesday, 27 January 2016

How to Create RSS Feeds for Google Search Results

Google Alerts, you probably know this, offer an easy way for you to create RSS feeds from the Google search results of any query. This is a good option if you are looking to monitor when new web pages are indexed by Google that match your search query.


Google Search RSS Feed

If you have never created feeds with Google Alerts earlier, here's a quick primer. Type your search keyword, expand options, choose "Automatic" for sources, choose "All Results" for "How Many" and set "RSS Feed" as the Delivery Method. Google will create an RSS feed of web search results that you can subscribe in Feedly or another RSS Reader.

Please see the Google Alerts tutorial to write more advanced search queries.

Better RSS Feeds for Google Search

A big limitation with feeds created using the "Google Alerts" approach is that you've limited control over the feed and it won't include search results from all over the web. In fact, the first time you create a feed, it is likely to be empty and results would be added as Google discovers and indexes new content for that search query.

There's an alternate method for creating RSS feeds for Google Search results and, though the initial setup takes few extra steps, you've enough options to fine tune the search query and have more meaniningful search results in the feed. You can even have RSS feeds for Google Images.

  1. Go to Google Custom Search and create a new search engine. If you would like to search the entire web, just like Google search, put *.com as the site to search, edit your CSE and under Sites to Search section, select the option that says "Search the entire web but emphasize included sites."
  2. Make a note of the Google CSE id which will be something like xxxx:yyy – click the Search Engine ID button under Details to know your CSE ID.
  3. Go to Google Developers Console, sign-in with your Google Account and create a new Project. Give your project any name – like Google Search RSS Feed – and click the Create Project button.
  4. Next go to the API & Auth link in the sidebar and click on APIs. Here search for "Custom Search API" and enable the API.
  5. Under the same API & Auth group, click Credentials – Add Creditials – Browser Key and click the Create button. You'll get the API key now.

Google Search API Credentials
Now we have all the ingredients required to create our RSS feed for Google Search. The basic feed URL will be something like this:

https://www.googleapis.com/customsearch/v1?alt=atom&cx=xxx:yyy&key=abc&q=query

Replace xxx:yy with the Search Engine ID, abc with the API key and query with the actual search query. If your query has multiple words, separate them with a + symbol (like Digital+Inspiration). There are tons of other search parameters to the Feed URL. For instance:

  • googlehost=google.de (to return results from Google Germany)
  • searchType=image (search images only, the default is web pages)
  • safe=high (filter adult content from search results)
  • dateRestrict = d10 (return results published in the last 10 days)

To give you an example, the following feed URL will fetch family safe pages from Google India, published in the last 2 weeks for a query "Make In India"

https://www.googleapis.com/customsearch/v1?alt=atom&cx=xxx:yyy&key=abc&q=Make+In+India&dateRestrict=w2&googlehost=google.in&safe=high

Advanced users can create simple web apps that use the Custom Search API to monitor search results by email or they can scrape Google search in a spreadsheet. The only limitation is that the Search API has a daily quota (100 requests per day) so you should not share the feed or the API key with other users. However, if you enable billing inside the Google API Console, the free limit will be upgraded to 200 search queries per day.


No comments:

Post a Comment