BlaBlaPriceBot, the price bot of BlaBlaPrice
BlaBlaPriceBot is the bot of BlaBlaPrice, a request-for-quotes service run by EVERLOGIC LLC (Ukraine). It reads the prices of products on online shops' own pages, so the price index shows buyers current prices and links to the shops.
What the bot collects
Only the public price, the currency, whether the product is in stock, the product name and the page address, taken from the page's structured data (schema.org). The bot does not copy descriptions, photos or reviews and collects no personal data.
How often and how fast
It visits at night. Each page is read about once a week. One site gets at most one request every 5 seconds, or less often when its robots.txt asks for a longer Crawl-delay, and the bot makes at most two requests at a time in total. It uses conditional requests (ETag, Last-Modified) and compression, and stops for two weeks on a site that refuses it.
robots.txt
The bot follows robots.txt: the rules written for BlaBlaPriceBot, otherwise the rules for all robots (User-agent: *). It reads robots.txt again every day. It identifies itself as:
BlaBlaPriceBot/1.0 (+https://blablaprice.com/bot)
How to stop the bot
Add these lines to robots.txt, and the bot will not open the site from the next day:
User-agent: BlaBlaPriceBot
Disallow: /
Contact
Questions about the bot, or a request to remove a shop: support@blablaprice.com. A company can also be removed from our lists with this form: