AI-Powered Trace Analysis: Detecting Security Issues Hidden in Application Spans

Your WAF blocked 12,000 requests last month. Your SAST tool passed your last build with flying colors. Your penetration tester gave you a clean report. And somewhere in your production environment right now, a SQL injection payload is executing inside a database span that none of those tools can see.

This is the blind spot that perimeter-focused security creates. WAFs see HTTP requests. SAST tools see source code. Penetration tests see external behavior. But none of them see what actually happens inside your application after a request arrives—how it propagates through services, what queries it triggers, what outbound connections it makes, and what file paths it accesses. That internal behavior, captured in distributed traces, is exactly where sophisticated attacks hide.

Trace-based security analysis flips the detection model. Instead of guessing what might be malicious based on request signatures, it observes what the application actually does and applies AI to identify patterns that constitute real attacks.

Why Network-Perimeter Detection Falls Short

Web Application Firewalls operate on a fundamentally limited model. They inspect HTTP traffic at the edge of your infrastructure, matching request patterns against known attack signatures. The OWASP ModSecurity Core Rule Set documents hundreds of these patterns—and attackers know every one of them.

The evasion techniques are well-cataloged in the MITRE ATT&CK framework under Exploit Public-Facing Application. Encoding tricks, parameter pollution, chunked transfer manipulation, and polyglot payloads all bypass signature-based detection with depressing regularity. But even a perfect WAF has a structural limitation: it cannot see what happens after the request passes through.

Consider a simple example. An attacker sends a request to /api/search?q=laptop. The WAF sees a clean search query and lets it through. Inside the application, a vulnerable search function concatenates that input into a SQL query. The database span records the actual query: SELECT * FROM products WHERE name LIKE '%laptop' UNION SELECT username, password FROM users--%'. The injection was embedded in a way the WAF's pattern matching missed, but the database span captured the full query as it was executed.

This is not a contrived scenario. The Verizon 2024 Data Breach Investigations Report found that web application attacks remain the most common breach vector, and a significant percentage of those attacks bypass perimeter controls entirely. The problem is not that WAFs are useless—they catch enormous amounts of noise—but that they create a false sense of completeness.

How Distributed Traces Expose Attack Behavior

OpenTelemetry has become the industry standard for application instrumentation, and for good reason. Every request that enters your application generates a trace—a collection of spans representing each operation the application performs. A single HTTP request might produce dozens of spans: the HTTP handler, authentication middleware, authorization checks, database queries, cache lookups, outbound HTTP calls, file system operations, and message queue interactions.

Each span carries structured metadata: the operation name, duration, status code, and custom attributes. Database spans record the full SQL statement. HTTP client spans record the outbound URL. File system spans record the accessed path. Authentication spans record success or failure.

This is an extraordinarily rich security dataset that most organizations are already generating but not analyzing for threats. The traces flow into observability platforms for performance monitoring, and nobody looks at them through a security lens.

That is precisely the gap SecureNow fills. The platform ingests OpenTelemetry traces and applies AI-powered security analysis to every span, turning your existing observability data into a threat detection system.

The AI Analysis Process

SecureNow's trace analysis is not a static rule engine. When you trigger a security analysis from the trace details page, the platform sends the complete trace structure—every span, attribute, timing relationship, and status—to an AI model prompted with deep security analysis context.

The AI examines the trace holistically. It does not just scan for known bad strings. It evaluates the relationships between spans, the sequence of operations, the timing patterns, and the data flow across service boundaries. This enables detection of attack categories that signature-based tools structurally cannot catch.

What the AI Evaluates

The analysis covers multiple dimensions of each trace:

• Input propagation — how user-supplied data flows from the entry span through downstream operations, identifying tainted data paths
• Query construction — whether database spans contain dynamic SQL with user input rather than parameterized queries
• Outbound request patterns — whether HTTP client spans make requests to internal infrastructure, cloud metadata endpoints, or attacker-controlled hosts
• Authentication flow integrity — whether the sequence of auth-related spans follows expected patterns or reveals bypass conditions
• File path construction — whether file system spans access paths constructed from user input without proper validation
• Error propagation — whether error spans leak sensitive information back to the caller
• Timing anomalies — whether span durations indicate brute-force attempts, resource exhaustion, or race conditions

The result is a structured security report that identifies specific vulnerabilities, classifies their severity, and provides remediation guidance—all derived from what the application actually did, not what someone guessed it might do.

<!-- CTA:trial -->

Detecting SQL Injection in Database Spans

SQL injection remains the most dangerous web application vulnerability, ranked consistently in the OWASP Top 10. Trace analysis catches it with a level of precision that perimeter tools cannot match, because database spans record the query as it was actually executed by the database driver.

What It Looks Like in a Trace

A legitimate database span might record:

span.name: "SELECT products"
db.system: "postgresql"
db.statement: "SELECT id, name, price FROM products WHERE category_id = $1"
db.parameters: [42]

A SQL injection attempt produces a markedly different span:

span.name: "SELECT products"
db.system: "postgresql"
db.statement: "SELECT id, name, price FROM products WHERE category_id = 42 UNION SELECT table_name, column_name, null FROM information_schema.columns--"

The AI identifies that the query contains a UNION SELECT against system tables, the parameterization was bypassed (no $1 placeholder), and the query structure does not match the expected pattern for this operation. It flags this as a confirmed SQL injection with high confidence and recommends switching to parameterized queries in the affected code path.

Blind SQL Injection Through Timing

Even blind SQL injection, where the attacker cannot see query output directly, is visible in traces. The AI detects database spans with abnormal durations caused by SLEEP() or WAITFOR DELAY injections, or conditional timing differences that indicate boolean-based extraction. A database span that normally completes in 5 milliseconds suddenly taking 5,000 milliseconds is an unmistakable signal in trace data.

Detecting SSRF in Outbound HTTP Spans

Server-Side Request Forgery attacks exploit your application's ability to make outbound HTTP requests. The attacker manipulates a URL parameter to make your server fetch internal resources, cloud metadata, or arbitrary external targets. The MITRE ATT&CK technique T1090 documents this as a proxy technique that is increasingly common in cloud-native environments.

What It Looks Like in a Trace

A normal outbound HTTP span:

span.name: "HTTP GET"
http.url: "https://api.stripe.com/v1/charges"
http.method: "GET"
http.status_code: 200

An SSRF attempt:

span.name: "HTTP GET"
http.url: "http://169.254.169.254/latest/meta-data/iam/security-credentials/"
http.method: "GET"
http.status_code: 200

The AI recognizes that 169.254.169.254 is the AWS metadata service endpoint—a prime SSRF target. It also catches less obvious variants: requests to internal RFC 1918 addresses (10.x.x.x, 172.16.x.x, 192.168.x.x), DNS rebinding attempts where the resolved IP changes between spans, and requests to localhost or link-local addresses. The trace reveals whether the SSRF succeeded (status 200 vs. connection refused), what data was returned, and exactly which application code path made the request.

Detecting Authentication Bypass in Auth Spans

Authentication bypass is among the most critical vulnerability classes, addressed directly in the NIST Digital Identity Guidelines (SP 800-63). Trace analysis detects it by examining the sequence and outcome of authentication-related spans.

What It Looks Like in a Trace

A normal authenticated request produces a trace with a clear authentication sequence:

Span 1: HTTP POST /api/admin/users (entry)
  Span 2: auth.verify_token (duration: 3ms, status: OK)
    Span 3: auth.check_permissions (role: admin, status: OK)
      Span 4: db.query "SELECT * FROM users" (status: OK)

An authentication bypass might look like:

Span 1: HTTP POST /api/admin/users (entry)
  Span 2: db.query "SELECT * FROM users" (status: OK)

The authentication and authorization spans are entirely absent. The AI detects that a request to an admin endpoint reached the database without passing through any authentication middleware. This could indicate a missing middleware configuration, a route-level bypass, or a direct object reference vulnerability.

The AI also catches subtler patterns: authentication spans that succeed with invalid tokens (indicating broken verification logic), authorization spans that grant admin access to non-admin roles, and session spans that reference expired or revoked tokens.

Detecting Path Traversal in File System Spans

Path traversal attacks manipulate file paths to access resources outside the intended directory. The OWASP Path Traversal guide documents the canonical ../ technique, but modern variants use encoding, null bytes, and OS-specific path separators to evade input filters.

What It Looks Like in a Trace

A legitimate file access span:

span.name: "fs.read"
file.path: "/app/uploads/user-123/avatar.png"

A path traversal attempt:

span.name: "fs.read"
file.path: "/app/uploads/../../etc/passwd"

The AI resolves the path, recognizes that it escapes the intended upload directory, and flags it as a path traversal. It also catches encoded variants (%2e%2e%2f), null byte injections (avatar.png%00.jpg), and OS-specific patterns. Critically, the trace reveals whether the file access succeeded, what was returned, and the full request chain that led to the vulnerable file operation.

Understanding Attack Flow with Trace Visualization

Detecting an attack is only half the problem. Understanding how it propagated through your system is essential for effective remediation. SecureNow provides three visualization modes for trace data, each optimized for a different analytical perspective.

Tree View

The tree view displays the parent-child relationship between spans, showing exactly how the initial request decomposed into downstream operations. For security analysis, this reveals the call path from the entry point to the vulnerable operation—making it clear which middleware was bypassed, which service propagated tainted data, and where validation should have occurred but did not.

Gantt View

The Gantt view shows spans on a timeline with their duration and overlap. This is critical for detecting timing-based attacks (blind SQL injection, race conditions, resource exhaustion) and understanding the performance impact of attack traffic. When a database span takes 100x longer than normal, the Gantt chart makes it immediately visible.

Timeline View

The timeline view provides a chronological sequence of events across the entire trace. For multi-step attacks—where the attacker first enumerates endpoints, then probes authentication, then exploits a vulnerability—the timeline shows the full attack progression as a coherent narrative.

Triggering AI Security Analysis

The analysis workflow is straightforward. From any trace detail page in SecureNow, you trigger the AI security analysis with a single action. The platform packages the full trace context—every span, attribute, and relationship—and sends it to the AI model for security review.

The AI returns a structured report identifying any security issues found, their severity, the specific spans involved, and remediation recommendations. This report is stored alongside the trace, creating a security audit trail that maps directly to application behavior.

For deeper investigation, you can pivot from a flagged trace into SecureNow's forensics system to query for similar patterns across your entire trace dataset. Ask a question like "show me all traces where the database span contains UNION SELECT in the last 7 days" and get instant results without writing SQL.

When combined with trace-level supply chain analysis, you get a comprehensive view of both first-party vulnerabilities and third-party dependency risks—all from the same OpenTelemetry data you are already collecting.

<!-- CTA:demo -->

Building a Trace-First Security Practice

Adopting trace-based security analysis does not require ripping out your existing security stack. It layers on top of it, covering the blind spots that perimeter tools leave exposed. Here is a practical adoption path for security teams.

Step 1: Ensure Comprehensive Instrumentation

The quality of trace analysis depends directly on the quality of your traces. Ensure your OpenTelemetry instrumentation covers database operations (with db.statement attributes), outbound HTTP calls (with full URLs), authentication operations, and file system access. Most OpenTelemetry auto-instrumentation libraries capture these by default.

Step 2: Establish Behavioral Baselines

Before you can detect anomalous spans, you need to understand normal ones. Use SecureNow's API Map Discovery to build a comprehensive map of your application's endpoints and their expected behavior patterns.

Step 3: Integrate Analysis Into Incident Response

When a security alert fires, make trace analysis the first investigative step. Instead of guessing what happened from access logs, examine the actual application behavior that triggered the alert. The trace tells you definitively what the attacker did, what succeeded, and what data was accessed.

Step 4: Convert Patterns to Alert Rules

When the AI identifies an attack pattern in a trace, convert that pattern into a persistent alert rule. SecureNow's SQL-based alert system can monitor for specific span attributes, query patterns, or behavioral signatures on a scheduled basis—turning every detected attack into a permanent detection capability.

The Limits of Perimeter Thinking

The security industry spent two decades building increasingly sophisticated walls around applications. WAFs, API gateways, DDoS protection, bot management—all of it focused on the perimeter. And all of it assumes that if a request looks clean at the boundary, it is clean.

Modern applications are distributed, polyglot, and composed of dozens of interacting services. An attack that looks innocuous at the API gateway might become dangerous when Service A passes it to Service B, which constructs a database query with it, which returns data that Service C writes to a file. The only way to see that full chain is in the trace.

Trace-based security analysis does not replace your WAF. It catches what your WAF cannot—and provides the application-level evidence that transforms alerts from "something might have happened" into "here is exactly what happened, in which service, at which line of code."

The traces are already flowing through your infrastructure. The question is whether you are analyzing them for security, or leaving that visibility on the table.

Frequently Asked Questions

What is trace-based security analysis?

Trace-based security analysis examines the distributed traces generated by applications (via OpenTelemetry) to detect security issues at the span level — revealing attack patterns that network-level tools miss.

What types of attacks can AI trace analysis detect?

AI trace analysis can detect SQL injection attempts, SSRF, authentication bypass, path traversal, privilege escalation, data exfiltration patterns, and anomalous API usage hidden within application spans.

How is trace analysis different from WAF detection?

WAFs inspect HTTP requests at the network perimeter. Trace analysis examines application behavior after the request is processed — catching attacks that bypass WAF rules or exploit application logic.

Does SecureNow require code changes for trace analysis?

SecureNow works with standard OpenTelemetry instrumentation. If your application already generates traces, SecureNow can analyze them without additional code changes.