I have set up a crawler, to get the title and price information of items on Amazon UK
In the 'Where to crawl?' field, I have removed:www.amazon.co.uk/ and replaced it with:www.amazon.co.uk/Pet-Supplies-Food-Animals/b/ref=sd_allcat_ps_t2?ie=UTF8&node=340840031
but when I run the crawler, it doesn't return any results.
What I am doing wrong?
I will not be surprised if amazon are actively stopping this from happening depending upon ip address and various other factors.
Depending upon your usecase this data is available via amazon product API.
I hope this helps.
Okay, I'll see what can be done via the API. Thanks.