Scraping gets hard--fast--when you need deal with dynamic content (read: javascript), pagination, and changing page structures. In other words, most popular websites built in the last 10 years.

Learn how we built ParseHub to cope with these problems. Try the same techniques in python with the open-source Scrapy framework.