VeBot is Ve’s ecommerce crawling bot (alternatively known as a “spider” or “crawler”). Crawling is the process by which VeBot discovers new products and changes to existing products, so that they can be added or updated in the Ve product database.
VeBot will crawl your product pages to obtain key product data (such as product titles, prices and images) that can be used by VeApps.
VeBot accesses URLs associated with existing records in our product database, and records any changes to the fields already being captured by VeConnect. VeBot can also use Sitemap data provided by webmasters, to discover new products on your site.
As well as being distributed on several machines to enhance performance, VeBot’s crawl frequency is intelligently managed to avoid overloading your site’s servers. To request a change in the crawl frequency, please contact your account manager.
VeConnect is a Javascript technology, meaning it can only capture data when a visitor hits a product page on your site. VeBot is able to crawl these pages at a predetermined frequency, to ensure the VeApps you have enabled on your site are not using out of date product information.
The simplest method is to check the user agent of the bot in question, in this case it should be VeBot.
As a first step, make sure the VeConnect tag is installed site-wide. In addition to this, publishing an up to date sitemap is a great way to assist VeBot in discovering any new products on your site.
