Infinite Scroll


#1

How to scrap data from a webpage with INFINITE scroll and closed API?


#2

I am also interested to know how to scrap data with Jquery/Javascript infinite scroll → NO /page-2 in the Url


#3

Same here, there are many such sites like twitter where the data is huge but the extractor gets only the top results in the run.


#4

Hey guys,

In the normal help sections theres an explanation that is quite elaborate:

https://guide.import.io/dealing-with-infinite-scroll.html

I hope this helps!


#5

Thanks Leadgen, unable to find the pagenum with twitter:(


#6

Can you post the site you are trying to scrape so I can take a look at it?


#7

I’m struggling with the same issue. I read the help section on Infinite Scroll but Twitter doesn’t seem to use page numbers or php. Here’s the Twitter feed I’m trying to scrape: https://twitter.com/ensbot?lang=en

Any help would be greatly appreciated.