Google Search Results JAVA API via SerpApi

Overview

Google Search Results JAVA API

test

This Java package enables to scrape and parse Google, Bing and Baidu search results using SerpApi. Feel free to fork this repository to add more backends.

This project is an implementation of SerpApi in Java 7. This code depends on GSON for efficient JSON processing. The HTTP response are converted to JSON Object.

An example is provided in the test. @see src/test/java/GoogleSearchImplementationTest.java

The full documentation is available here.

Requirements

Runtime:

  • Java / JDK 8+ (https://www.java.com/en/download/) Older version of java do not support HTTPS protocol. The SSLv3 is buggy which leads to Java raising this exception: javax.net.ssl.SSLHandshakeException

For development:

Quick start

To get started with this project in Java. We provided a fully working example.

git clone https://github.com/serpapi/google_search_results_java.git
cd google_search_results_java/demo
make run api_key=<your private key>

Note: You need an account with SerpApi to obtain this key from: https://serpapi.com/dashboard

file: demo/src/main/java/demo/App.java

public class App {
    public static void main(String[] args) throws SerpApiSearchException {
        if(args.length != 1) {
            System.out.println("Usage: app <secret api key>");
            System.exit(1);
        }

        String location = "Austin,Texas";
        System.out.println("find the first Coffee in " + location);

        // parameters
        Map<String, String> parameter = new HashMap<>();
        parameter.put("q", "Coffee");
        parameter.put("location", location);
        parameter.put(GoogleSearch.SERP_API_KEY_NAME, args[0]);

        // Create search
        GoogleSearch search = new GoogleSearch(parameter);

        try {
            // Execute search
            JsonObject data = search.getJson();

            // Decode response
            JsonArray results = (JsonArray) data.get("local_results");
            JsonObject first_result = results.get(0).getAsJsonObject();
            System.out.println("first coffee: " + first_result.get("title").getAsString() + " in " + location);
        } catch (SerpApiSearchException e) {
            System.out.println("oops exception detected!");
            e.printStackTrace();
        }
    }
}

This example runs a search about "coffee" using your secret api key.

The Serp API service (backend)

  • searches on Google using the query: q = "coffee"
  • parses the messy HTML responses
  • return a standardized JSON response The class GoogleSearch
  • Format the request to Serp API server
  • Execute GET http request
  • Parse JSON into Ruby Hash using JSON standard library provided by Ruby Et voila..

Alternatively, you can search:

  • Bing using BingSearch class
  • Baidu using BaiduSearch class

See the playground to generate your code. https://serpapi.com/playground

Example

How to set SERP API key

The Serp API key can be set globally using a singleton pattern.

GoogleSearch.serp_api_key_default = "Your Private Key"
search = GoogleSearch(parameter)

Or the Serp API key can be provided for each query.

search = GoogleSearch(parameter, "Your Private Key")

Example with all params and all outputs

query_parameter = {
  "q": "query",
  "google_domain": "Google Domain",
  "location": "Location Requested",
  "device": device,
  "hl": "Google UI Language",
  "gl": "Google Country",
  "safe": "Safe Search Flag",
  "num": "Number of Results",
  "start": "Pagination Offset",
  "serp_api_key": "Your SERP API Key",
  "tbm": "nws|isch|shop",
  "tbs": "custom to be search criteria",
  "async": true|false,    // allow async request - non-blocker
  "output": "json|html",  // output format
}

query = GoogleSearch.new(query_parameter)
query.parameter.put("location", "Austin,Texas")

String html_results = query.getHtml()
JsonObject json_results = query.getJson()

Example by specification

We love true open source, continuous integration and Test Drive Development (TDD). We are using RSpec to test our infrastructure around the clock to achieve the best QoS (Quality Of Service).

The directory test/ includes specification/examples.

To run the test: gradle test

Location API

GoogleSearch search = new GoogleSearch(new HashMap<String, String());
JsonArray locationList = search.getLocation("Austin", 3);
System.out.println(locationList.toString());

it prints the first 3 location matching Austin (Texas, Texas, Rochester)

Search Archive API

Let's run a search to get a search_id.

Map<String, String> parameter = new HashMap<>();
parameter.put("q", "Coffee");
parameter.put("location", "Austin,Texas");

GoogleSearch search = new GoogleSearch(parameter);
JsonObject result = search.getJson();
int search_id = result.get("search_metadata").getAsJsonObject().get("id").getAsInt();

Now let retrieve the previous search from the archive.

JsonObject archived_result = search.getSearchArchive(search_id);
System.out.println(archived_result.toString());

it prints the search from the archive.

Account API

Get account API

GoogleSearch.serp_api_key_default = "Your Private Key"
GoogleSearch search = new GoogleSearch();
JsonObject info = search.getAccount();
System.out.println(info.toString());

it prints your account information.

Build project

How to build from the source ?

You must clone this repository.

git clone https://github.com/serpapi/google_search_results_java.git

Build the jar file.

gradle build

Copy the jar to your project lib/ directory.

cp build/libs/google_search_results_java.jar path/to/yourproject/lib

How to test ?

make test

Conclusion

This service supports Google Images, News, Shopping. To enable a type of search, the field tbm (to be matched) must be set to:

  • isch: Google Images API.
  • nws: Google News API.
  • shop: Google Shopping API.
  • any other Google service should work out of the box.
  • (no tbm parameter): regular Google Search.

The full documentation is available here.

Issue

SSL handshake error.

symptom

javax.net.ssl.SSLHandshakeException

cause

SerpApi is using HTTPS / SSLv3. Older JVM version do not support this protocol because it's more recent.

solution

Upgrade java to 1.8_201+ (which is recommended by Oracle).

  • On OSX you can switch versino of Java.
export JAVA_HOME=`/usr/libexec/java_home -v 1.8.0_201`
java -version

Changelog

  • 2.0.1 update gradle 6.7.1
  • 2.0 refractor API : suffix SearchResults renamed Search
  • 1.4 Add support for Yandex, Yahoo, Ebay
  • 1.3 Add support for Bing and Baidu
  • 1.2 Add support for location API, account API, search API

Source

Author

Victor Benarbia - [email protected]

You might also like...

A universal types-preserving Java serialization library that can convert arbitrary Java Objects into JSON and back

A universal types-preserving Java serialization library that can convert arbitrary Java Objects into JSON and back, with a transparent support of any kind of self-references and with a full Java 9 compatibility.

Dec 30, 2021

A simple java JSON deserializer that can convert a JSON into a java object in an easy way

JSavON A simple java JSON deserializer that can convert a JSON into a java object in an easy way. This library also provide a strong object convertion

Mar 18, 2022

Java with functions is a small java tools and utils library.

Java with functions is a small java tools and utils library.

Oct 14, 2022

Set of support modules for Java 8 datatypes (Optionals, date/time) and features (parameter names)

Overview This is a multi-module umbrella project for Jackson modules needed to support Java 8 features, especially with Jackson 2.x that only requires

Dec 23, 2022

A modern JSON library for Kotlin and Java.

Moshi Moshi is a modern JSON library for Android and Java. It makes it easy to parse JSON into Java objects: String json = ...; Moshi moshi = new Mos

Dec 31, 2022

A fast JSON parser/generator for Java.

A fast JSON parser/generator for Java.

fastjson Fastjson is a Java library that can be used to convert Java Objects into their JSON representation. It can also be used to convert a JSON str

Dec 31, 2022

JSON to JSON transformation library written in Java.

Jolt JSON to JSON transformation library written in Java where the "specification" for the transform is itself a JSON document. Useful For Transformin

Dec 30, 2022

Java JsonPath implementation

Jayway JsonPath A Java DSL for reading JSON documents. Jayway JsonPath is a Java port of Stefan Goessner JsonPath implementation. News 10 Dec 2020 - R

Jan 4, 2023

A streaming JsonPath processor in Java

JsonSurfer - Let's surf on Json! Why JsonSurfer Streaming No need to deserialize entire json into memory. JsonPath Selectively extract json data by th

Dec 12, 2022
Comments
  • fixed the Makefile in demo && fixed the readme

    fixed the Makefile in demo && fixed the readme

    Hi maintainers,

    There was a bug in the Makefile under demo directory. I was able to fix it and also updated the commands in the readme file for quick installation. Following were the errors I was getting before the fix. It was failing at make run command. Since there was only one file to be copied, it was copying it as a file called "lib" rather than copying a file under the newly created "lib" directory. The quick installation commands are also updated in the Makefile under demo directory so you should be able to execute it properly.

    Screenshot from 2020-09-18 16-14-51

    Screenshot from 2020-09-18 16-16-36

    It runs successfully post fix as in the following screenshot.

    Screenshot from 2020-09-18 16-17-56

    Please review and accept my pull request.

    Thank you.

    opened by albertcoder 1
Releases(2.0.3)
Owner
SerpApi
API to get search engine results with ease.
SerpApi
Bu repositorie Google'ın tamamıyla ücretsiz olarak sunduğu Android Kotlin Fundamentals içeriğindeki projelerin tamamıdır.

Bu repositorie Google'ın tamamıyla ücretsiz olarak sunduğu Android Kotlin Fundamentals içeriğindeki projelerin tamamıdır. Kotlin ile Android geliştiriciliğine başlayacaklara önerilir.

Serkan Alc 15 Dec 2, 2022
Jakarta money is a helpful library for a better developer experience when combining Money-API with Jakarta and MicroProfile API.

jakarta-money Jakarta money is a helpful library for a better developer experience when combining Money-API with Jakarta and MicroProfile API. The dep

Money and Currency API | JavaMoney 19 Aug 12, 2022
Lean JSON Library for Java, with a compact, elegant API.

mJson is an extremely lightweight Java JSON library with a very concise API. The source code is a single Java file. The license is Apache 2.0. Because

Borislav Iordanov 77 Dec 25, 2022
Elide is a Java library that lets you stand up a GraphQL/JSON-API web service with minimal effort.

Elide Opinionated APIs for web & mobile applications. Read this in other languages: 中文. Table of Contents Background Documentation Install Usage Secur

Yahoo 921 Jan 3, 2023
Core part of Jackson that defines Streaming API as well as basic shared abstractions

Overview This project contains core low-level incremental ("streaming") parser and generator abstractions used by Jackson Data Processor. It also incl

FasterXML, LLC 2.1k Jan 1, 2023
null 5 Jan 11, 2022
A declarative API to handle Android runtime permissions.

PermissionsDispatcher Fully Kotlin/Java support Special permissions support 100% reflection-free PermissionsDispatcher provides a simple annotation-ba

PermissionsDispatcher 11.1k Jan 5, 2023
dOOv (Domain Object Oriented Validation) a fluent API for type-safe bean validation and mapping

dOOv (Domain Object Oriented Validation) dOOv is a fluent API for typesafe domain model validation and mapping. It uses annotations, code generation a

dOOv 77 Nov 20, 2022
Convert Java to JSON. Convert JSON to Java. Pretty print JSON. Java JSON serializer.

json-io Perfect Java serialization to and from JSON format (available on Maven Central). To include in your project: <dependency> <groupId>com.cedar

John DeRegnaucourt 303 Dec 30, 2022
A Java serialization/deserialization library to convert Java Objects into JSON and back

Gson Gson is a Java library that can be used to convert Java Objects into their JSON representation. It can also be used to convert a JSON string to a

Google 21.7k Jan 8, 2023