Hacking on Open APIs - Digital Web

Welcome back to our two-part series on Application Programming Interfaces: APIs. If you’re not familiar with what APIs are—or maybe you are but you’re still not sure what all the fuss is about—then I’d recommend starting with part one of this series: APIs and Mashups For The Rest Of Us. This time around, we’ll dispense with the lengthy introductions and dive straight in. We’ll quickly introduce the three main techniques that we’ll be making use of, then play with a few examples.

When working with third-party APIs, it’s always important to check incoming and outgoing request data. You don’t want those APIs to become yet another attack vector for those looking to exploit your application.

We are going to use PHP for our code examples, although much of what we’ll be saying will be relevant to whatever language you’re using. Dynamic languages such as Python and Ruby are particularly well-suited to quickly hacking with APIs, and their interactive consoles make experimentation easy. Don’t be put off by all the code examples either—the easiest way of working out what is going on is to try them yourself.

HTTP

The first thing we need to understand is HTTP. HTTP stands for Hypertext Transfer Protocol, and is the protocol used to transport much of the web’s content; if you visit a URL in your browser, it is usually an HTTP request which is sent to the server and an HTTP response which brings the web page back to your browser. HTTP is how we are going to talk to those distant APIs, and how they are going to talk back to us. (I would thoroughly recommend flipping through the HTTP 1.1 Specification at least once.)

When sending an HTTP request, there are several HTTP verbs that we can use: GET, POST, PUT, and DELETE are all quite common. The HTTP specification defines these verbs, as well as a number of others, along with their intended uses and a number of response codes. The right ones to use will be defined by the API with which you are communicating.

First we are going to send a simple request to an API and get a response back—the ‘Hello World’ of API development, if you like. In PHP, we could do this in a number of ways, including using the get_file_contents function, or using the curl extension. On this occasion, I’ve decided to use the PEAR HTTP_Request module, which is going to make our life easier later on. If you already have PEAR installed, then you should be able to just type pear install http_request on the command line, but if not, you can find more details about PEAR and installing modules on the PEAR website.

<?php

require('HTTP/Request.php');

$url = "http://example.com";

$request =& new HTTP_Request($url);
$request->sendRequest();
$response = $request->getResponseBody();

?>

All we’re doing here is preparing a new request object using the URL provided by our API (www.example.com, on this occasion), then making a request and storing the information we get back in the $response variable. By default, HTTP_Request makes a request using the GET verb. We can change that to POST using the following setMethod method:

<?php

require('HTTP/Request.php');

$url = "http://example.com";

$request =& new HTTP_Request($url);
$request->setMethod(HTTP_REQUEST_METHOD_POST);
$request->sendRequest();
$response = $request->getResponseBody();

?>

So now we can talk to our distant API —but what are we likely to get back? Different APIs will give you different things in response to your request, but the majority provide a response using either XML or JSON. Some APIs will even give you the option of which flavor you prefer. We’ll look at simple ways of parsing both.

XML Parsing

Parsing XML is a huge subject, and there are lots of ways of doing it, but for the moment we’ll use SimpleXML—which was introduced in PHP5—as it provides a very simple interface for us to get started.

<?php

$xml = simplexml_load_string($response);
print_r($xml)

?>

Here we create a new SimpleXML object called $xml using the string stored in $response. Now we can either print the object out using the handy print_r function to get a look inside, or use simple object syntax to access the data. For example, if the XML has a root node named <response>, we can access its contents with $xml->response. For a more in-depth look at SimpleXML, see our recent article, PHP and XML Sitting in a Tree.

JSON Parsing

JSON is the new kid on the block and, happily, removes much of the complexity of working with XML. JSON is a subset of JavaScript, proposed by Douglas Crockford, and intended for data storage. As of PHP5.2, there is native JSON parsing support built into PHP, but if you’re using an earlier version, we can use the PEAR JSON module instead.

<?php

require('Services/JSON.php');

$json = new Services_JSON();
$data = $json->decode($response);
print_r($data);

?>

The decode method returns a PHP object based on the JSON response. In the same manner as the XML example above, we can now use $json->response to access the contents of the <response> JSON element (assuming that is what our root node is named).

Libraries

If you are looking to work with a widely used and stable API, such as Flickr or Amazon, you are likely to find a number of libraries which already do the job. In these cases you can save quite a bit of time by not reinventing the wheel—instead, focus your efforts on what it is you are trying to do with the API, rather than the low level conversations between code.

I’d recommend taking a look at the Amazon S3 library from neurofuzzy.net and phpFlickr to name but two. If you’re working in PHP, you’ll also find a number of web services have PEAR libraries that can prove a good starting point.

Extending Twitter

For newer services, or those with evolving APIs, we are more likely to have to roll our own mini libraries. With that in mind, we are going to have a look at the Twitter API.

For those that have yet to catch the bug, Twitter is a way of keeping in touch with friends and acquaintances via the web, instant messaging, and text messages (be warned though, it’s highly addictive). In order to work through the following examples, you’ll need to have a Twitter account set up already, or you can set up a new one just to experiment with the code. It’s free and only takes a moment.

Twitter is yet another one of those services where you have to make connections between yourself and your friends. There is not currently an official way of finding out which people have marked you as a friend but whom you have not connected with (one of your ‘followers’, in social-network parlance). This can be an interesting way of expanding your network and making new friends, so we’re going to create the functionality ourselves with help from the API.

The Twitter API provides a small number of methods, the names and details of which can be found in the API documentation, including two that we are going to make use of: Friends and Followers. These API calls will return a list of your friends and followers, respectively. All we then need to do is compare the two lists and output people who are in the followers list but not in the friends list.

Both of the methods return the list of people in the same way, so first we’ll create a function named getPeople() to wrap up the request and return an array of individual people:

<?php

require('HTTP/Request.php');
require('Services/JSON.php');

function getPeople($url,$username,$password) {
  $json = new Services_JSON();
  $request =& new HTTP_Request($url);
  $request->setBasicAuth($username, $password);
  $request->sendRequest();
  $response = $request->getResponseBody();
  return $json->decode($response);
}

The above function simply bundles up a few things we’ve seen already; namely sending an HTTP request and then converting the JSON returned into an array of PHP objects. In order to use the above function, we have to provide a valid email address and password for Twitter. We’ll also define a couple of blank arrays which we’ll use later on.

$username = 'your email address';
$password = 'your password';

$friends = array();
$followers = array();

The people objects we get back from Twitter have all sorts of information in them that we don’t need—location, latest status, avatar, and so on—which will make our comparison harder, so we’ll filter this data down into two simpler arrays, $friends and $followers, after we call our getPeople() function. We use a foreach loop to extract the name and url of each person.

// Send an API request for a list of all my friends in JSON
$friends_response = getPeople('http://twitter.com/statuses/friends.json',$username,$password);
// Loop through the object and create a simple array of name and url
foreach ($friends_response as $friend) {
  $friends[$friend->screen_name] = array($friend->name, $friend->url);
}

// Send an API request for a list of all my followers
$followers_response = getPeople('http://twitter.com/statuses/followers.json',$username,$password);
// Loop through the object and create an array that matches the structure of $friends
foreach ($followers_response as $follower) {
  $followers[$follower->screen_name] = array($follower->name, $follower->url);
}

Now we’ve got two separate arrays we can look for the differences. The PHP function array_diff_key is exactly what we’re looking for here. It takes two arrays and returns us an array of items only found in the first array. array_diff_key was introduced in PHP5 but if you’re still using PHP4 there are a number of solutions in the notes on the function description page on php.net.

$only_followers = array_diff_key($followers,$friends);

And that’s it—we now have an array containing the name and URL of everyone who has added us as a friend in Twitter, but that we haven’t added as a friend of ours. All that is left is to output the details of the array in the form of a list:

echo '<ul>';
  foreach ($only_followers as $name => $details) {
    echo '<li>';
    echo '<a href="http://twitter.com/' . $name . '">' . $details[0] . '</a>';
    echo ' - ';
    echo '<a href="' . $details[1] . '">' . $details[1] . '</a>';
    echo '</li>';
  }
echo '</ul>';
?>

For the sake of simplicity, the above snippets don’t include any error handling—the final app below includes a small amount of checking and even a helpful message. When working with third-party APIs, it’s always important to check incoming and outgoing request data. You don’t want those APIs to become yet another attack vector for those looking to exploit your application. (In the real world, you’d probably make this a little more object-oriented, and abstract out some of the configuration settings and the like, but that’s for another day.)

<?php

require('HTTP/Request.php');
require('Services/JSON.php');

function getPeople($url,$username,$password) {
  $json = new Services_JSON();
  $request =& new HTTP_Request($url);
  $request->setBasicAuth($username, $password);
  if (!PEAR::isError($request->sendRequest())) {
    $response = $request->getResponseBody();
    $people = $json->decode($response);
    return $people;
  } else {
    return 0;
  }
}

$username = 'your email address';
$password = 'your password';

$friends = array();
$followers = array();

$friends_response = getPeople('http://twitter.com/statuses/friends.json',$username,$password);
if ($friends_response) {
  foreach ($friends_response as $friend) {
    $friends[$friend->screen_name] = array($friend->name, $friend->url);
  }
}

$followers_response = getPeople('http://twitter.com/statuses/followers.json',$username,$password);
if ($followers_response) {
  foreach ($followers_response as $follower) {
    $followers[$follower->screen_name] = array($follower->name, $follower->url);
  }
}

$only_followers = array_diff_key($followers,$friends);

if (count($only_followers) != 0) {
  echo '<ul>';
  foreach ($only_followers as $name => $details) {
    if ($details[1]) {
      echo '<li><a href="http://twitter.com/' . $name . '">' . $details[0] .
      '</a> - <a href="' . $details[1] .
      '">' . $details[1] . '</a></li>';
    } else {
      echo '<li><a href="http://twitter.com/' . $name .
      '">' . $details[0] . '</a></li>';
    }
  }
  echo '</ul>';
} else {
  echo '<p>no followers who are not already friends!</p>';
}

?>

Hopefully this quick-and-dirty example gives you a glimpse into how easy it is not just to start playing around with open APIs, but to build something that people might actually find useful. Once you’re proficient at throwing HTTP requests around and playing with JSON and XML, and the world is your API.

HTTP

XML Parsing

JSON Parsing

Libraries

Extending Twitter

Further Reading