Hands-Free Web Scraping for Hands-On Marketers
Subtitle: Tips for dealing with “Load More”
- Visit your target website
- Click Listly’s extension icon and select LISTLY WHOLE / LISTLY PART
- Copy CSS Selector of “Load More” button
- Click Edit Setting on Databoard
- Paste CSS Selector to auto-click field
- Refresh data sources
Marketers spend hours on routine tasks from market research to competitor price monitoring. Well, there’s no getting around the need for marketing automation. Above all, we cannot talk about it without web scraping. By the end of this read, you’ll be ready to gather data from any website using “Load More” buttons.
In this tutorial, we’ll collect women’s new product information from the H&M website.
The website displays only 36 products at a time, and you can load more items once you click the “LOAD MORE PRODUCTS” button.
Normally you should repeat mouse clicks until there are no new products left, but you can automate this process in just 10 seconds with Listly.
Click LISTLY Whole and you’ll see that only 36 items were collected.
Move your mouse to the “LOAD MORE PRODUCTS” button, right-click, and select Inspect.
You’ll be able to see some of the code is highlighted in blue, right-click again and select Copy and Copy selector.
Go back to your Databoard and click add setting.
There you go, paste the element CSS selector to the following field. You can also change the number of times to repeat the auto-click action.
- Repeat N Times: The number of times to repeat the auto-click action
- Wait after clicking: The amount of time that is given to Listly’s web scraping service after clicking a “Load More” button
Here, we set the auto-click action to repeat 23 times and changed the wait time to 5 seconds.
All right, now it’s time to hit the refresh data button!
What do you say? Every single new item on the H&M website was collected in just one page.
You know what? You can also download the data by simply clicking the EXCEL button on your Databoard!