Inelegant

Internet ∪ Open Source

Inelegant header image 2

Undocumented JSON API for Google Search (+ Perl Client)

December 22nd, 2006 · No Comments

The news of Goggle discontinuing their SOAP API reminded me of the JSON API their SearchMash product exposes. This is, until they close this loophole, superior to their “AJAX API” because it doesn’t require a developer key and isn’t rate limited.

To get a JSON representation of a search result use the following URL form: http://www.searchmash.com/results/{query}, e.g. http://www.searchmash.com/results/perl. An i parameter dictates the result number to start from, and an n parameter specifies how many results to return. For example, http://www.searchmash.com/results/perl?i=5&n=2 shows result 5 and 6 for perl.

The traditional way to access the SOAP API in Perl used Net::Google. The synopsis for that module is:

use Net::Google;
use constant LOCAL_GOOGLE_KEY => “********************************”;

my $google = Net::Google->new( key => LOCAL_GOOGLE_KEY );
my $search = $google->search();

# Search interface

$search->query(qw(aaron straup cope));
$search->lr(qw(en fr));
$search->starts_at(5);
$search->max_results(15);

map { print $_->title() . \n; } @{ $search->results() };

# or…

foreach my $r ( @{ $search->response() } ) {
    print “Search time :” . $r->searchTime() . \n;

    # returns an array ref of Result objects
    # the same as the $search->results() method
    map { print $_->URL() . \n; } @{ $r->resultElements() };
}

Using the JSON module we can use the SearchMash API to search from the command line like so:

#!/usr/bin/perl
use strict;
use JSON;
use LWP::Simple;
use URI::Escape;
my $query = uri_escape( $ARGV[0] ) or die “Usage: $0 ;
my $results = get “http://searchmash.com/results/$query
    or die “Query failed”;
my $json = new JSON;
my $obj  = $json->jsonToObj($results);
for my $r ( @{ $obj->{results} } ) {

    for ( $r->{title}, $r->{snippet} ) {
        s{?br?>}{}g;
    }
    print “* “
        . $r->{title} . \n
        . $r->{snippet} . \n<"
        . $r->{rawUrl} . “>\n\n;
}

(15 lines long before Perltidying).

For a query of perl this prints:

  • Perl.com: The Source for Perl — perl development, conferences News site run by O'Reilly. Contains documentation, weekly articles, Perl development summaries, blogs, and links to a variety of resources.

  • Downloading the Latest Version of Perl Instructions on downloading a Perl interpreter for your computer platform.

  • The Perl Directory - perl.org Run by the Perl Foundation with the aim of being “the central directory of all things Perl”. Lists news, applications, documentations, communities, …

  • CPAN The Comprehensive Perl Archive Network, the gateway to all things Perl. The canonical location for Perl code and modules.

  • Perl - Wikipedia, the free encyclopedia Provides language history, function information and code examples.

  • ActiveState - Dynamic Tools for Dynamic Languages ActiveState Open source technology-based tools and solutions for Perl, PHP, Python, Tcl, Ruby, on AIX, HP-UX, Linux, Mac OS X, Solaris, and Windows.

  • ActiveState - ActivePerl free Perl open source binary language … ActiveState's quality-assured, distribution of Perl on Linux, Mac OS X, Solaris and Windows. ActiveState offers professional programming tools for Perl, …

  • PERL — Practical Extraction and Report Language This web document is a re-organized version of the “perl.1″ man page for PERL version 4. (I will not be producing a revised version for Perl5. …

  • Perl Monks Contains tutorials, discussion forums, Perl poetry, obfuscated code, and a large code repository.

  • modperl: Welcome to the modperl world modperl brings together the full power of the Perl programming language … modperl gives you a persistent Perl interpreter embedded in your web server. …

This could be easily wrapped in an OO interface to produce a drop-in replacement for Net::Google, in fact, the interface could be even simpler.

(Perl golf has never been my strong point, but the shortest implementation I could manage is:

perl -MJSON -MLWP::Simple -e
'$r=(JSON->new)->jsonToObj(get("http://searchmash.com/results/$ARGV[0]"));for(@{$r->{results}}){print $_->{url}."\n"}' perl

).

EDIT: I’ve written a basic OO Perl module for interfacing with this API: Perl Interface for SearchMash.com’s JSON API (Alpha).

Tags: perl · cli · api · web · google · post · search · json · soap

Leave a Comment