Website scraping, crawling, data mining & extraction, parsing and reporting
I've been engaged in web scraping since 2006 and making it easier for others. I'm providing all kind of web scraping services in shortest possible time and at a reasonable cost.
Whether you need a big list of email addresses from some directories or a list of real estate properties or a price comparison tool to extract prices from your competitor stores, I can handle that in shortest possible time and deliver expected results. I develop tools to scrape text data, email, phone, audio, video or image from websites, feeds (RSS / ATOM) and social networks. Not only the scraped data, I will also deliver the tool that can help you forever in doing the data extraction.
For scraping / crawling - I mainly use PHP and Perl. But I may use any other technology that is better for the respective job.
For data parsing / filtering - I use Perl Compatible Regular Expressions (PCRE) along with some document parsers for better performance.
For reporting - I mainly prefer database, CSV, XML, PDF, JSON or plaintext output. But anything suggested / required by the customer is also doable.
Being a freelance web application engineer, I also provide other services required within / beside any assigned scraping project
SOME OF MY WORKS & EXPERIENCES
ONLINE JOB AGGREGATING
Developed 3 (three) job listing sites by scraping jobs from other sites. One sample: http://joblance.info is a job site which aggregates online job lists from some other popular job sources like oDesk, GAF (Freelancer.com), Get A Coder, ScriptLance, etc.
PRICE COMPARISON SCRIPTS
I've developed services / scripts that can scrape product information along with pricing data from buy.com, amazon.com, dell.com, google products, etc. and saves to database for comparison purposes. Any custom solution can be given as per the requirements.
I've scraped many business directories and chambers websites for contact info like name, email address, telephone number, address, etc. My scraper logs into (when needed) a site with given login info and harvest for data. Scraped data are saved to database and/or outputted as CSV, XML or PDF.
T-SHIRT DATA SCRAPER
I've developed a t-shirt scraping project which scraps t-shirt info (with images) from some 18+ t-shirt related websites and saves the data (with image) to wordpress database as blog posts. CRON jobs run to do these automatically. You may check the site: http://www.t-shirtguru.com
MAILBOX GRABBER FOR YAHOO, GMAIL, HOTMAIL & POP/IMAP
I've experiences in developing a complete email client that aggregates yahoo, gmail, pop/imap emails along with attachments, etc. This work required building complex scraping scripts that can email boxes with proper session cookies behavior. Later I've built a facebook application of this email client.
RSS PARSER MODULE FOR DRUPAL & WORDPRESS
Developed a complex RSS Parser and aggregator module for Drupal that can scrape given feeds and create nodes with proper versioning. It doesn't only merge RSS feeds but also can hanle duplicate items according to the setup in the backend. The module is fully manageable from the backend. Later we developed a similar plugin for WordPress.
REAL ESTATE SCRAPER
Developed some 15+ scraping tools to scrape & deploy real estate listings. My scraper scrapes both data & images regularly and keeps the destination site up-to-date. My experience includes scraping data from http://www.beachfrontrealty.net/ and http://www.houseandco.co.uk and many property sites.
TWITTER SCRAPER & AGGREGATOR
I've developed a facebook application to aggregate multiple twitter accounts information into one facebook fan page. This required scraping twitter accounts and then saving all tweets in the mysql database. You may take a look: click here
I've developed many other scrapers like media scraper that scrapes 1000s media files from a given site, link scraper that scrapes all links from a given website, cigars scraper that scrapes cigar info from some 20+ cigars websites, cars scraper that scrapes car info from car dealers websites and many more. Please contact for any custom scraping services.
GET IN TOUCH WITH ME
PHONE+880 1714 131963
HOUSE # 160