Bot traffic describes machine-based traffic to your website. Some sources believe bots account for over 40% of internet traffic. While the term carries a negative connotation, not all bot traffic is malicious or harmful. Bot traffic takes many forms, some good and some bad:
- Good Traffic:
- Search engine data gathering
- SEO application data gathering
- Script based/API initiated automation
- Bad Traffic:
- Malicious scripted attacks (DDos)
- Web scraping bots
- Script based/API initiated automation
- Malicious penetration testing
Bot traffic is necessary for many search engines to operate and sites to function optimally. It's not possible–nor desirable–to eliminate bot traffic entirely. However, you can use Noibu to detect bot traffic and take steps to reduce or deter malicious activity.
Detecting Bot Traffic
While it's difficult to detect bot traffic through a session video, Noibu has assembled a list of detection methods that might identify a bot-controlled session. This can help you discern human-run sessions from bot-run sessions. When reviewing a session, look out for the following anomalies:
Issue Information:
- 100% bounce rate or 0% conversion rates (usually on the "On Site" step)
- Illegal invocation errors
Session Information:
- Large amounts of sessions with single page visits (1 page count)
- Large amounts of sessions with extremely outdated browsers (e.g. Chrome v85) and/or OS (e.g. Windows 7)
- Large amounts of Linux sessions
- IP addresses that map to a "Data center", "Proxy server", "VPN server", etc.
Session Recording:
- The cursor is at the furthest point in the top-left corner of the screen
- Mouse movements are linear, with no curvature
- Mouse movements are instantaneous, moving rapidly from point to point
- Mouse clicks are positioned at the direct centre of each element
- Multiple identical sessions
Managing Bot Traffic
While you can't get rid of all bot traffic, consult the suggestions below to minimize malicious activity on your website:
- Limit the number of requests allowed
- Use a robots.txt file
- Block IP addresses from accessing resources
- Use third-party resources to help limit traffic
Learn more about iFrame Tracking and Whitelisting the Noibu IP Address.