Bot Management

TrafficDefender Bot Management

Non-human traffic (also known as automated traffic or bots) is an increasing proportion of traffic on any web based system. Typically accounting for around 40% of all traffic and up to 80% of traffic in extreme situations. Some of this is negative. Bot attacks can result in security compromises, fraud, competitor advantage and additional overhead on servers.

Equally, it is important not to be humanist! Some non-human traffic (such as Googlebot), monitoring tools and, increasingly, IoT devices or automated assistants (e.g. Alexa) deliver large amounts of value to your systems. In the middle there is a range of non-human traffic that may be positive or negative depending on your industry and business model, such as price scrapers.

TrafficDefender Bot Management provides an effective means of bot defence, allowing you to accurately identify different types of automated traffic and implement an effective bot management process. Wide ranging industry data from TrafficDefender customers and industry data sources prove that by using our industry leading traffic management system, you are able to optimise the experience of all users – whether human or non-human.

Benefits

Reduced Costs

Serving any traffic accrues costs, be they direct costs such as bandwidth and infrastructure or indirect such as lost ad revenue or increased third party usage. Non-human traffic and bot traffic drives these costs up. TrafficDefender Bot Management allows you to get control over these wasted costs.

Maximised Revenue

Your web systems are built to maximise the revenue that you gain from your users and the traffic that flows through your system. Some non-human traffic/bots help with this process (e.g. search engines), but others actively work against you (such as competitor price scraping, ad fraud and content theft). TrafficDefender Bot Management will ensure that you get control over your non-human traffic to maximise the revenue you gain from your customers and minimise the impact of malicious bot activity.

Fraud Reduction

Non-human traffic is a common source of fraudulent activity such as ad fraud (where ad revenue is fraudulently earned by automated traffic), account takeover or creation (where fake user accounts are created or legitimate accounts compromised to take advantage of bonuses or other features associated with accounts), or credit card fraud (where bots are used to execute transactions using stolen credit card details). Taking control of automated bot traffic with TrafficDefender Bot Management enables you to minimise this activity.

Security

There are a constant stream of automated attacks traversing the internet, looking to exploit known vulnerabilities in web based systems to cause damage to your business and the personal security of your customers. TrafficDefender Bot Management allows to to mitigate the risks of these bots.

Brand Reputation

Re-sellers are an important route to market for many businesses, but re-sellers can also be damaging to your brand, misrepresenting what they are selling on your behalf or marking your prices up out of line with your market positioning. TrafficDefender Bot Management allows you to control the levels of re-seller activity.

Content Protection

Your content is very valuable. A lot of time, money and expertise is invested in creating it. Be it journalistic content or product descriptions, being able to quickly gather content gives competitors an unfair advantage. TrafficDefender Bot Management protects your intellectual property from competitors and other third parties attempting to harvest your content.

Accurate Analytics

In some situations, non-human traffic or bot attacks will be recorded as human engagements in your web analytics. Effectively this invalidates your digital marketing and ecommerce analytics, making it more difficult to reliably understand the performance of your systems or the effectiveness of your marketing strategies and tactics. TrafficDefender Bot Management gives you visibility and control over the flow of non-human traffic into your systems, enabling you to reduce this discrepancy.

Industry Knowledge and Machine Learning

TrafficDefender Bot Management has a reliable and accurate identification capability for non-human traffic, which is based on our the expertise and analysis of our Data Scientist team and our proprietary machine learning technology. By combining a wide range of customer and industry data sources with complex machine learning techniques, TrafficDefender Bot Management is always one step ahead of the evolving challenges of accurate bot identification.

Empowering You

TrafficDefender’s reporting and dashboard gives you real-time insight into the nature of your web traffic and gives you the power to makes decisions based on the information provided. TrafficDefender Bot Management empowers you to take control of your non-human traffic both proactively and reactively in real-time.

Inline vs Offline

Technical or practical constraints may prevent you from putting the full Trafficdefender Web Traffic Management solution in front of your system. However, that does not stop you from being able to take advantage of the advanced non-human traffic analysis offered by TrafficDefender Bot Management. Bot Management offers an offline mode that enables you to transmit log files for processing in near real-time, with recommendations returned to your system on the likelihood of those connections being a bot.

Easy to Integrate

TrafficDefender requires no code changes or deployments to your infrastructure, which means you can be up and running within minutes. Simply re-point your DNS or CDN to TrafficDefender and away you go. If you do not want an inline system, TrafficDefender Bot Management can be made available as an offline system.

Accuracy of Detection

TrafficDefender Bot Management offers industry leading levels of accuracy in the recognition of non-human traffic, providing you with information about why identification of the traffic as non-human was made. TrafficDefender Bot Management is not a black box solution, we feel an important element of any web traffic management system is making decisions on how to handle traffic on the widest range of information available.

Better Than A WAF

Web Application Firewalls (WAFs) are valuable tools in protecting systems from security threats, but their value in protection from bot threats is limited. WAFs value is in pattern matching against requests that try and exploit known patterns that can trigger weaknesses in the web system. However, non-human traffic executes legitimate paths that are identical to those executed by human traffic. Identifying non-human traffic requires identification of patterns of multiple requests, across sessions and multiple sessions, across users to identify traffic that is different from human traffic. WAFs identify security threats not non-human traffic.

Features

Industry Data

TrafficDefender Bot Management brings a wide range of data from across industries to inform the identification of traffic as non-human. This data includes where the traffic is coming from, how it is identifying itself and also, the patterns and nature of traffic that make up non-human traffic.

Real-Time Statistics and Control

TrafficDefender Bot Management allows you to see, in real-time, the levels of non-human traffic passing through your system, with detailed breakdown about the nature of that traffic and the ability to take action to mitigate the effects of an attack. All changes take effect instantly, which means an attack and can quickly and easily be defended against.

Accurate Identification

TrafficDefender Bot Management uses a combination of data sources, analysis techniques and machine learning to ensure that bot identification is as accurate as possible.

Easy Integration

TrafficDefender Bot Management requires no code changes or deployments to your infrastructure, which means you can be up and running within minutes. Simply re-point your DNS or CDN to TrafficDefender and away you go.

Flexible Management and Response

TrafficDefender Bot Management allows full control over how traffic is handled in a range of manners that suit your business. Identify the nature of traffic and how that should be handled and make changes in real-time.

Enterprise Class, High Availability

A system’s or website’s availability is key to its success, that’s why all of our clients are given their own dedicated infrastructure in our highly available AWS cloud. With all our implementations making use of multi-datacentre failover and fail open technology, our services and your site’s uptime is guaranteed.

24/7 Protection

Guarantee your site’s availability all with all-year round protection. Visitor Management protects sites from sudden or unexpected spikes in traffic by automatically queuing as soon as your site gets to full capacity.

End-to-End Support

Our dedicated support team work closely with clients and are available 24 hours a day, 365 days a year.

Common Questions

Can I Just Block the Bot IP Addresses On My Firewall?

One way of dealing with bot traffic is by simply creating a blacklist of IP addresses on a firewall, CDN or other device. Traditionally this was a fairly common way of handling this problem. However, it is a very limited solution and suffers from several key issues.

  • It is always reactive. Adding in the IP addresses of bot attacks is retrospective because the config update is typically done after the attack is identified. Therefore it only contains details of the IP addresses of past attacks, whereas typically automated traffic will regularly rotate IP addresses
  • It needs constant maintenance to ensure that new threats are added to the list as they are discovered. It also needs a method of validating if the added IP addresses are still valid
  • It usually only allows for hard blocking, that is dropping the connections of any requests coming from matched IP addresses. This allows little scope for giving friendly messages back to any false positive users, or any of the other ways of dealing with non-human traffic offered by effective bot management tools

Any good bot management tool should address all of these limitations to provide a valid amount of insight and control of non-human traffic on your system.

Will A Web Application Firewall (WAF) Stop Bot Attacks?

WAFs are effective and useful tools as part of any secure web based system. However, they are solving different problems to those posed by non-human traffic or bot attacks.

WAFs are designed to look for patterns within requests that are targeted at exploiting security weaknesses within systems. Non-human traffic is usually legitimate requests, but made in a different pattern to standard human users. To effectively identify non-human traffic it is necessary to look at the nature and patterns of requests that are being made, and compare them to those being made by human users.

WAFs may help stop basic bot traffic but will not identify any more sophisticated traffic sources, which are becoming ever more common.

Book A Free Traffic Audit