Extract data from site that scrolls to load more data


#1

How can I scrape data from a site which loads more data on the scroll down.
In example the site I want to get data from doesn’t have separate pages but when you scroll to the bottom of the page it loads more data. It does this continuously. Any help would be greatly appreciated! Thank you


#2

There are a couple of articles that deal with this. I’m in the middle of scraping a bunch of sites with infinite scroll. Here are the articles. Post the URL and we might be able to help you more:

https://guide.import.io/dealing-with-infinite-scroll.html