How to Block Bot Traffic in Express With No Extra Infra

If you've built an Express API or web app and you're seeing bot traffic in your logs — scrapers, vulnerability scanners, AI training crawlers — you have five reasonable ways to deal with it. None require Cloudflare, AWS WAF, or any infrastructure outside your existing app.

For broader context on the firewall layer, see the SecureNow Firewall page.

The five approaches, ranked

1. Static blocklist + allowlist (5 lines of middleware) 2. User-agent filtering (10 lines, mostly imperfect) 3. Rate limiting (express-rate-limit) 4. IP-reputation firewall (one preload flag) 5. Full bot management (commercial — DataDome, PerimeterX, etc.)

For most applications, option 4 covers 90% of the value at near-zero setup cost. Walk through them in order.

Approach 1: static blocklist

If you have a known list of bad IPs (from your logs, your security team, or a public threat feed), reject them at the top of the middleware chain:

const blockedIps = new Set([
  '185.220.101.42',
  '185.220.102.245',
  // ... your list
]);

app.use((req, res, next) => {
  const ip = req.headers['x-forwarded-for']?.split(',')[0].trim() || req.socket.remoteAddress;
  if (blockedIps.has(ip)) return res.status(403).send('Forbidden');
  next();
});

Works fine for small static lists. Falls apart when bots rotate IPs (which they all do).

Approach 2: user-agent filtering

Most legitimate crawlers identify themselves honestly:

const BOT_UAS = /sqlmap|nikto|acunetix|masscan|nmap|python-requests/i;

app.use((req, res, next) => {
  const ua = req.headers['user-agent'] || '';
  if (BOT_UAS.test(ua)) return res.status(403).send('Forbidden');
  next();
});

Catches the lazy bots. Doesn't catch sophisticated ones that fake their UA. Useful as one signal in a layered approach.

Important: allowlist the good crawlers explicitly. They make your site discoverable in search and AI:

const GOOD_BOTS = /googlebot|bingbot|gptbot|claudebot|perplexitybot|duckduckbot/i;

app.use((req, res, next) => {
  const ua = req.headers['user-agent'] || '';
  if (GOOD_BOTS.test(ua)) return next(); // always allow
  if (BOT_UAS.test(ua)) return res.status(403).send('Forbidden');
  next();
});

Approach 3: rate limiting

The standard library:

npm install express-rate-limit

import rateLimit from 'express-rate-limit';

const limiter = rateLimit({
  windowMs: 60 * 1000,
  max: 100, // 100 requests per minute per IP
  standardHeaders: true,
  legacyHeaders: false,
});

app.use(limiter);

This catches naive bot traffic that hits your server faster than humans would. It doesn't catch slow-rolling attacks or distributed scrapers. Tune the threshold to your normal traffic — typical web apps see 10–50 requests/min/IP from real users.

Approach 4: IP-reputation firewall

This is the option most teams should start with. One preload flag adds an IP firewall with 500k known-bad addresses, automatic Googlebot/GPTBot/ClaudeBot allowlisting, and refresh-on-the-hour updates.

npm install securenow

# In your start command:
node -r securenow/firewall-only app.js

That's the entire setup. The firewall-only preload doesn't add tracing, doesn't change your code, doesn't add latency beyond the IP lookup (sub-millisecond per request). On unreachable backend, it fails open — your app keeps serving traffic.

If you also want the application security monitoring layer, swap firewall-only for register:

node -r securenow/register app.js

That adds OpenTelemetry tracing, log capture, and AI investigation in addition to the firewall. Free tier covers 1 GB of telemetry per month.

Approach 5: full bot management

For sites with sophisticated bot threats — sneaker bots, ad fraud, competitive scraping at scale — commercial products like DataDome, PerimeterX, and HUMAN add behavioral fingerprinting, TLS fingerprinting, and JavaScript challenges. They're effective and expensive ($1K–$10K/month for moderate traffic).

Most Express apps don't need this. Start with approach 4; add commercial bot management only if approach 4 leaves obvious gaps.

What to actually deploy

For 90% of Express apps, the right setup is:

node -r securenow/firewall-only app.js

Plus, if you want belt-and-suspenders:

import rateLimit from 'express-rate-limit';
app.use(rateLimit({ windowMs: 60_000, max: 100 }));

That's the entire bot defense for most production apps. The static blocklist (approach 1) and user-agent filter (approach 2) become unnecessary because the IP-reputation firewall covers them at scale.

Verifying it works

Check that your firewall is active:

npx securenow firewall status

You'll see the count of currently-blocked IPs and the last update timestamp. To test a specific IP:

npx securenow firewall test-ip 185.220.101.42

Returns whether that IP is on the blocklist or allowlist, and which entry matched.

What this costs

Approaches 1–3: free, requires your engineering time. Approach 4 (firewall-only): free up to 1 GB/month telemetry. The firewall itself is free even on paid plans. Approach 5: $1,000–$10,000/month plus integration time.

For most teams, approach 4 is the obvious starting point and often the ending point too.

Related

• Free SecureNow Firewall page
• Detecting credential stuffing in Express in 30 lines
• Adding an IP firewall to Express without Cloudflare