ihateonions.uk
Return Home
Programming
21st December, 2016

Sorry, No API For Auto-Trader? I Will Just Web Scrape Then!

Wednesday evening i was approached by my MD at work who spoke about a new site for a client who’s recently become a car dealer and signed up to Autotrader.co.uk.

Apparently the client wasn’t happy with the styling of his new site and wanted a new bespoke build on his own hosting and domain. The current site was provided by auto trader themselves, as part of the dealer partnership means you can have all your vehicles also displayed on your website as well as Auto traders website. This sounds attractive but it does have its boundaries.

The way Auto trader do this is replace your site with one of their pre-made themes which have the relevant information and code to connect and pull the dealers car information from the database. This stops the need for them to create and maintain an API.

So i called them up and after a while waiting and speaking to a fool i finally got the answer i was after. ‘No we don’t offer an API and you cannot connect to any of our data sources!’

So i replied, ‘Second thoughts, not to worry surly theres an option to add my own stylesheet to override the themes style’. Once again, No was the answer.

Wasn’t the greatest of help and i still didn’t have a way to build the site.

After a while thinking i remember a previous project i worked on a few years back which was to pull latest news headlines into a phone app. We started by using simple web scraping tools via the DOM. This was it, all i needed todo is write a new program to pull relevant information from the loaded dealers page. Solved!

So that’s what I’ve done. The code below is nice and simple and only shows the basics of how you can pull information from not just AutoTrader but any website you want. This is nothing new or special but when i had a quick search online, no one really displayed the full working code for a basic “scarp”.

Well here it is.
$html = file_get_contents('http://www.integrityvehiclesales.co.uk.razsor.autotrader.co.uk/used-cars'); $auto_doc = new DOMDocument(); libxml_use_internal_errors(TRUE); //turn off libxml errors nif(!empty($html)){ //if anything returns back to me $auto_doc->loadHTML($html); libxml_clear_errors(); //clean the autotrader shit $auto_xpath = new DOMXPath($auto_doc); //Get all my info on cars etc here. find the classes and style them inclding image tags $auto_row = $auto_xpath->query('//div[@class="title-wrapper"]'); if($auto_row->length > 0){ foreach($auto_row as $row){ echo $row->nodeValue . " "; } } }