Need private packages and team management tools?Check out npm Teams »

google-search-results-nodejs

1.3.0 • Public • Published

Google Search Results Node.js

Build Status

This NodeJS module is designed to scrape and parse Google, Bing and Baidu results using SERP API. Feel free to fork this repository to add more backends.

This Ruby Gem is meant to scrape and parse Google results using SerpApi. The following services are provided:

Serp API provides a script builder to get you started quickly.

The full documentation is available here.

Requirement

  • ES6 basic understanding
  • NodeJS coding skills
  • Node 7+ and NPM installed

Installation

NPM 7+

$ npm install google-search-results-nodejs

Link to npm package

Quick start

const GSR = require('google-search-results-nodejs')
const client = new GSR.GoogleSearchResults("Your Private Key")
client.json({
 q: "Coffee", 
 location: "Austin, TX"
}, (result) => {
  console.log(result)
})

This example runs a search about "coffee" using your secret api key.

The Serp API service (backend)

  • searches on Google using the client: q = "coffee"
  • parses the messy HTML responses
  • return a standardizes JSON response The class GoogleSearchResults
  • Format the request to Serp API server
  • Execute GET http request
  • Parse JSON into Ruby Hash using JSON standard library provided by Ruby Et voila..

Alternatively, you can search:

  • Bing using BingSearchResults class
  • Baidu using BaiduSearchResults class

See the playground to generate your code.

Example

How to set SERP API key

The Serp API key can be set globally using a singleton pattern.

const GSR = require('google-search-results-nodejs')
let client = new GSR.GoogleSearchResults("Your Private Key")

The Serp API key can be provided for each request

const GSR = require('google-search-results-nodejs')
let client = new GSR.GoogleSearchResults()
let result = client.json({
 api_key: "Your private key",
 q: "Coffee",            // search query
 location: "Austin, TX", // location 
}, (data) => {
  console.log(data)
})

Search API capability

query_params = {
  q: "query",
  google_domain: "Google Domain", 
  location: "Location Requested", 
  device: device,
  hl: "Google UI Language",
  gl: "Google Country",
  safe: "Safe Search Flag",
  num: "Number of Results",
  start: "Pagination Offset",
  api_key: "Your SERP API Key",  // https://serpapi.com/dashboard
  tbm: "nws|isch|shop",
  tbs: "custom to be search criteria",
  async: true|false,   // allow async query
  output: "json|html", // output format
}
 
const GSR = require('google-search-results-nodejs')
const client = new GSR.GoogleSearchResults()
 
// create a callback
callback = (data) => {
 console.log(data)
}
 
// Show result as JSON
client.json(query_params, callback)
 
// Show result as HTML file
client.html(query_params, callback)

(the full documentation)[https://serpapi.com/search-api]

see below for more hands on examples.

Example by specification

We love true open source, continuous integration and Test Drive Development (TDD). We are using RSpec to test our infrastructure around the clock to achieve the best QoS (Quality Of Service).

The directory test/ includes specification/examples.

Set your api key.

export API_KEY="your secret key"

Run all tests npm test

Location API

const client = new GSR.GoogleSearchResults(api_key)
client.location("Austin", 3, (data) => {
  console.log(data)
})

it prints the first 3 location matching Austin (Texas, Texas, Rochester)

[ { id: '585069bdee19ad271e9bc072',
    google_id: 200635,
    google_parent_id: 21176,
    name: 'Austin, TX',
    canonical_name: 'Austin,TX,Texas,United States',
    country_code: 'US',
    target_type: 'DMA Region',
    reach: 5560000,
    gps: [ -97.7430608, 30.267153 ],
    keys: [ 'austin', 'tx', 'texas', 'united', 'states' ] },
  ...]

Search Archive API

The first search result returns a search_id which can be provided to get the search result from the archive.

var client = new GSR.GoogleSearchResults(api_key)
client.json({q: "Coffee", location: "Portland" }, (search_result) => {
  // search in archive for the search just returned
  client.search_archive(search_result.search_metadata.id, (archived_search) => {
    console.log(archived_search)
  })
})

it prints the search from the archive.

Account API

const client = new GSR.GoogleSearchResults(api_key)
client.account((data) => {
  console.log(data)
})

it prints your account information.

Promise and callback

This API was developped using basic callback to handle response. And exception are just throw away with interceptor.

if you want to take advantage of the promise to block the request. here is how I will do.

const util = require('util')
 
function getJson(parameter, resolve, reject) {  
  const client = new gsr.GoogleSearchResults(api_key)
  try {
    client.json(parameter, resolve)
  } catch (e) {
    reject(e)
  }
}
 
const blockFn = util.promisify(getJson)
blockFn[util.promisify.custom](parameter).then((data) => {
  expect(data.local_results[0].title.length).toBeGreaterThan(5)
  done()
}).catch((error) => {
  console.error(error)
  done()
})

Reference:

Coding style

This API is using callback to run in non-blocking code. Here we are trying to follow the spirit of NodeJS. For reference you can read this article:

For pratical example, you can see the test located under test/.

Conclusion

Serp API supports Google Images, News, Shopping and more.. To enable a type of search, the field tbm (to be matched) must be set to:

  • isch: Google Images API.
  • nws: Google News API.
  • shop: Google Shopping API.
  • any other Google service should work out of the box.
  • (no tbm parameter): regular Google client.

The field tbs allows to customize the search even more.

The full documentation is available here.

Contributing

Contributions are welcome, feel to submit a pull request!

To run the tests:

export API_KEY="your api key"
make test

Install

npm i google-search-results-nodejs

DownloadsWeekly Downloads

877

Version

1.3.0

License

ISC

Unpacked Size

20 kB

Total Files

15

Last publish

Collaborators

  • avatar