WP Web Scraper
An easy to implement professional web scraper for WordPress. This can be used to display real time data from any websites directly into your posts, pages or sidebar. Use this to include real time stock quotes, cricket or soccer scores or any other generic content. The scraper is built using time tested libraries cURL for scrapping and phpQuery for parsing HTML. Features include:
- Can be easily implemented using the button in the post / page editor.
- Configurable caching of scraped data. Cache timeout in minutes can be defined in minutes for every scrap.
- Custom Useragent header for your scraper can be set for every scrap.
- Scrap output can be displayed thru custom template tag, shortcode in page, post and sidebar (text widget).
- Other configurable settings like cURL timeout, disabling shortcode etc.
- Error handling – Silent fail, standard error, custom error message or display expired cache.
- Option to clear or replace a certain regex pattern from the scrap before output.
- Built-in module for stock market data (NSE, BSE and NASDAQ supported currently), other markets to follow.
Download and usage
The plugin is already downloaded 33,266 times. The download count displayed is not static. It is dynamically scraped from this URL with a refresh rate of 60 minutes. The CSS selector used is ‘.last-child td’.