PHP CURL Tutorial

In this lesson we're going to learn about PHP cURL, which stands for Client URL. We’ll refer to it as cURL from here on out. cURL is a library that lets you make HTTP requests in PHP.

In this tutorial, we’ll use cURL to load a simple page into a web browser and output it. Then we’ll load search results from Amazon and match every image in the results, storing the URLs in an array variable. Finally, we’ll loop through the array to output the images in a browser. This fun example lets you see what’s possible with a PHP cURL library.

Creating a cURL Request

There are four main steps to creating a cURL request. First, we create the cURL resource with the cURL initialize function. Next, we set cURL options using the cURL setopt function, then run cURL or execute the HTTP request with the cURL execute function. Finally, we can close the cURL resource with cURL close function to free up the resources.

// 1) Create cURL resource $curl = curl_init(); // 2) Set cURL options curl_setopt($curl, CURLOPT_URL, 'http://www.google.com'); // 3) Run cURL (execute http request) curl_exec($curl); // 4) Close cURL resource curl_close($curl);

Using cURL to Load a Webpage

Now we’re going to create a cURL request to load a webpage we specify. As outlined above, create the variable cURL and initialize the resource. The cURL is going to have the datatype curl resource, which allows us to use functions specifically designed for that resource. One of those functions is setopt. Use setopt with the constant CURLOPT_URL, then specify a URL (in this case google.com). Execute the command with exec to see that it loads Google into a web browser.

$curl = curl_init(); //$curl is going to be data type curl resource curl_setopt($curl, CURLOPT_CURL, 'http://www.google.com'); curl_exec($curl);

To change the URL and direct to Amazon instead of Google, there’s one important thing to add. Notice that Amazon uses the HTTPS protocol, so in order to load this page there’s an additional cURL option that needs to be set. To navigate to a page using a secure protocol, set the option SSL_VERIFY_PEER to false. Now when you execute your cURL command it dynamically loads the Amazon homepage.

$curl = curl_init(); //$curl is going to be data type curl resource curl_setopt($curl, CURLOPT_CURL, 'https://www.amazon.com'); curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); curl_exec($curl);

Directing to an Amazon Search

To create an Amazon search with a URL, there is a line of text to append to the base URL specifying field keywords. The screenshot below shows the URL with the field keywords and the resulting Amazon page with a search of PC video games from 2016.

With our goal to extract and print all the images from this search in a browser, we’ll need the image URLs. After executing the Amazon search, use Google Chrome’s Developer Tools > Inspect to find the block of code that corresponds to the first image. By clicking on the image you can find the block of code responsible for displaying it. Hovering over the first URL in the block displays the image to confirm that it is the URL we want. Save that URL for use later.

One additional option we want to use is RETURNTRANSFER, true. This stores the result in a variable instead of automatically printing it out. Using the echo function, you can print the stored result.

$curl = curl_init(); //$curl is going to be data type curl resource curl_setopt($curl, CURLOPT_CURL, 'https://www.amazon.com'); curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($curl, CURLOPT_RETURNTRANSFER, true); $result = curl_exec($curl); echo $result; curl_close($curl);

Instead of pointing to the Amazon home page, let's point to the search we did earlier. To make everything more easily edited, create a new variable for the search string, specifying the desired Amazon search. Then create a URL variable, adding the search string to the base Amazon URL. Modify the cURL option to point to the URL variable, and check that when the command is executed it brings the Amazon search up in a browser.

$curl = curl_init(); //$curl is going to be data type curl resource $search_string = "pc video games 2016"; $url = "https://www.amazon.com/s/field-keywords=$search_string"; curl_setopt($curl, CURLOPT_CURL, $url); curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($curl, CURLOPT_RETURNTRANSFER, true); $result = curl_exec($curl); echo $result; curl_close($curl);

Matching Images

The goal we’ve been working towards, to print out all the images from an Amazon search in a browser, requires identifying the URLs of all the images from the Amazon search, not just the one we’ve already found with the Developer Tools. To do this we’ll use the preg_match_all function.

Note that if you look at more of the image URLs, there is a constant beginning and end, but there is an identifier in the middle that will change with each unique image. The screenshot below shows a few of the images from an Amazon search with the unique image names shown under the transparent yellow block.

To have our function match all the images, we will use preg_match_all with a regular expression. A new video on regular expressions will be posted later, so you don’t need to fully understand this piece of the code now. For now, understand that the function as shown below will match everything but spaces, so it will match all of the image URLs. The image URLs are stored as an array in the matches variable.

$curl = curl_init(); //$curl is going to be data type curl resource $search_string = "pc video games 2016"; $url = "https://www.amazon.com/s/field-keywords=$search_string"; curl_setopt($curl, CURLOPT_CURL, $url); curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($curl, CURLOPT_RETURNTRANSFER, true); $result = curl_exec($curl); preg_match_all("!https://images-na.ssl-images-amazon.com/images/I/[^\s]*?._AC_US160_.jpj!", $result, $matches); print_r($matches); curl_close($curl);

If we print the matches variable and use “view page source” we see the array of image URLs in a readable format.

There is one last piece of code to make sure the array doesn’t contain duplicate images. Use array_unique to reduce the array to only the unique entries. Now the code will match and print only the unique image URLs. One thing to note is that when reducing the array to only its unique values, the array keys are changed. Use the function array_values around the function array_unique to correct them.

$curl = curl_init(); //$curl is going to be data type curl resource $search_string = "pc video games 2016"; $url = "https://www.amazon.com/s/field-keywords=$search_string"; curl_setopt($curl, CURLOPT_CURL, $url); curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($curl, CURLOPT_RETURNTRANSFER, true); $result = curl_exec($curl); preg_match_all("!https://images-na.ssl-images-amazon.com/images/I/[^\s]*?._AC_US160_.jpg!", $result, $matches); $images = array_values(array_unique($matches[0])); print_r($images); curl_close($curl);

Looping and Displaying Images in a Browser

With our code running the Amazon search and finding the resulting image URLs, the only remaining piece of the puzzle is printing all those images out in a browser. With a for loop you can loop through all the image URLs and use the IMG HTML tag to display them. The loop cycles through and grabs each URL to use it as the source for the IMG HTML tag. Replace the print_r call of your images array with the for loop to display the images instead of their URLs.

for ($i = 0; $i < count ($images); $i++) { echo "<div style='float: left; margin: 10 0 0 0; '>"; echo "<img src='$images[$i]'><br />"; echo "</div>"; }

To test it out a little further, you can change your search string and search instead for movies 2015 and see it output those images instead.

If you enjoyed this tutorial, leave a comment and like, share, and subscribe to Clever Techie!