Buzeli
buzeliSoluções Digitais
Costs

$342/month in unnecessary egress: how to identify clients bypassing the CDN with a Security Group audit

Published on May 3, 2026

The context: multi-tenant platform with mandatory CDN

The platform hosts dozens of WordPress sites on shared and dedicated EC2 instances. The security and cost model depends on all sites receiving traffic through the CDN — GoCache, Cloudflare, or Akamai, depending on the client. Each instance's Security Group allows ports 80 and 443 only for the IP ranges of the CDN in use.

When DNS correctly points to the CDN and the Security Group is aligned, EC2 egress is minimal — content is served by the CDN from cache, without hitting the origin on every request. AWS data transfer out costs only appear when the origin is accessed, not on cache hits.

Correctly configured CDN = near-zero egress on EC2. CDN configured but DNS bypassing it = full egress on EC2, cache costs on CDN, two costs for the same content.

The signal: $342 from a single domain in February

The monthly data transfer analysis via CloudWatch NetworkOut (more precise than Cost Explorer for isolating instances) revealed an anomaly in the ranking:

Copy
# CloudWatch NetworkOut data — February 2026
# Total infrastructure: 12,095 GB / $1,088.65

Egress ranking:
#1 cliente-loja.com.br       3,803 GB   $342.29   (31% of total)
#2 clientportal.example.com       1,775 GB   $159.75
#3 ...
...
#66 (last instance)              0.1 GB    $0.01

The second place had 1,775 GB. The first had 3,803 GB — more than double the second, and 31% of all egress from 66 instances. For a multi-tenant platform where no instance should have that volume, the number was the clearest signal that something was wrong.

The confirmation: GoCache with 0 GB served

The next check was GoCache analytics for the domain cliente-loja.com.br:

Copy
# GoCache Analytics API query — February 2026
# Domain: cliente-loja.com.br

Bandwidth served by GoCache: 0.00 GB
CDN requests: 0
Cache hit ratio: N/A

Zero gigabytes. The domain was configured on GoCache — the account existed, the rules were there — but the CDN had not served a single byte that entire month. All traffic was going directly to EC2, completely bypassing the CDN.

Two possible causes would explain this pattern: either DNS wasn't pointing to GoCache, or the instance's Security Group still had the World SG (0.0.0.0/0) open alongside the CDN SG, keeping the origin directly accessible.

The root cause: World SG coexisting with CDN SG

The Security Group audit starts with analyzing the SG combination on each instance. The relevant SGs on the platform:

Copy
# Platform Security Groups
sg-0aaaa1111aaaa1111  "World"      → opens 80/443 to 0.0.0.0/0 (unrestricted access)
sg-0bbbb2222bbbb2222  "GoCache"    → allows 80/443 only for GoCache IPs
sg-0cccc3333cccc3333  "Cloudflare" → allows 80/443 only for Cloudflare IPs
sg-0dddd4444dddd4444  "Akamai"     → allows 80/443 only for Akamai IPs

The expected state for an instance protected by CDN is: only the CDN SG in use, without the World SG. The cliente-loja.com.br instance had both: GoCache SG + World SG. With the World SG present, any IP on the internet could access EC2's port 443 directly — and since DNS pointed to EC2 (not GoCache), all traffic arrived directly.

DNS verification confirmed the bypass:

Copy
# Check domain DNS
dig cliente-loja.com.br +short
# Returns: 198.51.100.106  ← EC2 public IP, not GoCache IP (198.51.100.x)

# GoCache IPs start with 198.51.100.x
# If DNS resolves to the EC2 IP, the CDN is being bypassed

The full audit: 22 instances with GoCache SG

The audit didn't stop at one instance. The process was applied to all 22 instances with the GoCache SG configured, cross-referencing the SG with actual DNS resolution:

Copy
# List all instances with GoCache SG
aws ec2 describe-instances   --filters "Name=instance.group-id,Values=sg-0bbbb2222bbbb2222"   --query 'Reservations[].Instances[].[InstanceId,PublicIpAddress,Tags[?Key==`Name`].Value|[0]]'   --output table

# For each instance, check if DNS points to GoCache or directly to EC2 IP
for DOMAIN in $(domain-list); do
  DNS_IP=$(dig +short $DOMAIN | head -1)
  EC2_IP=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=*${DOMAIN}*"     --query 'Reservations[0].Instances[0].PublicIpAddress' --output text)

  if [ "$DNS_IP" = "$EC2_IP" ]; then
    echo "BYPASS: $DOMAIN → $DNS_IP (direct to EC2)"
  elif echo "$DNS_IP" | grep -q "^198\.51\.100\."; then
    echo "OK: $DOMAIN → $DNS_IP (GoCache)"
  else
    echo "CHECK: $DOMAIN → $DNS_IP (verify)"
  fi
done

Results of the audit on 22 instances with GoCache SG:

19 instances: DNS pointing to GoCache (198.51.100.x) — correct configuration.

3 instances: DNS pointing to Cloudflare (104.21.x.x, 172.67.x.x), but SG configured for GoCache — the wrong SG was allowing GoCache traffic that wasn't arriving, while the Cloudflare SG was not present.

In addition to the CDN bypass, the audit also revealed 6 instances with a redundant World SG:

Copy
# Instances with World SG + CDN SG (redundant World SG)
# After verifying DNS and confirming CDN is active:

Domain                     EC2 IP          DNS          Action
webserver-app01.example    203.0.113.99     GoCache      remove World SG
webserver-app02.example    203.0.113.117  GoCache      remove World SG
webserver-app03.example    203.0.113.184   GoCache      remove World SG
webserver-app04.example    203.0.113.70    GoCache      remove World SG
webserver-app05.example    203.0.113.52    GoCache      remove World SG
webserver-app06.example    203.0.113.236  Cloudflare   remove World SG + GoCache SG, add Cloudflare SG

The fixes applied

For the 5 instances with redundant World SG (DNS pointing to GoCache, World SG unnecessary), removal was done in batch:

Copy
# Remove World SG from instances where CDN is active
for INSTANCE in i-0aaaa1111aaaa1111 i-0bbbb2222bbbb2222 i-0cccc3333cccc3333                 i-0dddd4444dddd4444 i-0eeee5555eeee5555; do
  # List current SGs, excluding the World SG
  NEW_SGs=$(aws ec2 describe-instances --instance-ids $INSTANCE     --query 'Reservations[].Instances[].NetworkInterfaces[0].Groups[].GroupId'     --output text | tr '	' '
' | grep -v 'sg-0aaaa1111aaaa1111')

  # Apply new SG list (without World SG)
  aws ec2 modify-instance-attribute --instance-id $INSTANCE     --groups $(echo $NEW_SGs | tr '
' ' ')

  echo "$INSTANCE: World SG removed"
done

For the instance with GoCache SG but Cloudflare DNS, the fix was swapping the GoCache SG for the Cloudflare SG:

Copy
# Swap GoCache SG for Cloudflare SG
INSTANCE="i-0ffff6666ffff6666"

# Remove World SG (sg-0aaaa1111aaaa1111) and GoCache SG (sg-0bbbb2222bbbb2222)
# Add Cloudflare SG (sg-0cccc3333cccc3333)
aws ec2 modify-instance-attribute --instance-id $INSTANCE   --groups sg-0cccc3333cccc3333

Additional finding: obsolete IPs in the GoCache SG

During the audit, the currency of the IP list within the GoCache SG itself was also verified. The result revealed a second layer of risk:

Copy
# Count IPs in GoCache SG vs official list
IPs in SG:       31
Official IPs:    25
Difference:       6 extra IPs not listed on the official GoCache site

# Extra IPs found (confirmed obsolete by GoCache support)
198.51.100.13/32   — old standalone IP
198.51.100.152/29  — old /29
198.51.100.72/29   — old /29
198.51.100.64/26   — old /26
198.51.100.192/29  — old /29
198.51.100.24/29 — old /29

Obsolete IPs in the CDN SG are a silent risk: if one of those blocks is reassigned to a third party, any IP in that range can access the origin directly as if it were the CDN. GoCache support confirmed the 6 blocks were indeed obsolete and could be removed. The 11 corresponding rules (6 IPs × ports 80 and 443) were revoked:

Copy
# Revoke obsolete GoCache SG rules
aws ec2 revoke-security-group-ingress   --group-id sg-0bbbb2222bbbb2222   --security-group-rule-ids sgr-xxx1 sgr-xxx2 sgr-xxx3 sgr-xxx4 sgr-xxx5                              sgr-xxx6 sgr-xxx7 sgr-xxx8 sgr-xxx9 sgr-xxx10 sgr-xxx11

# Verify final count
aws ec2 describe-security-groups --group-ids sg-0bbbb2222bbbb2222   --query 'SecurityGroups[0].IpPermissions | length(@)'
# 25 — matches official GoCache list

The methodology: CloudWatch vs CDN analytics

The detection pattern that worked here is applicable to any platform with CDN — Cloudflare, CloudFront, Fastly, GoCache:

1. Measure actual EC2 egress: CloudWatch NetworkOut per instance. Cost Explorer consolidates per account, not per instance — use the API directly for granular data.

2. Cross-reference with CDN analytics: For each domain with high egress, check the bandwidth served by the CDN in the same period. 0 GB on CDN + high EC2 egress = bypass.

3. Confirm via DNS: If DNS resolves to the EC2 IP (not the CDN IP), the bypass is confirmed at the DNS level.

4. Check Security Groups: Even with DNS on the CDN, an open World SG allows direct access. Bots and scanners always test the direct IP — and if the SG allows it, they serve traffic without going through the CDN.

Copy
# Bypass detection script — CloudWatch vs CDN analytics
# Applies to GoCache, Cloudflare, CloudFront

for DOMAIN in $(cdn-domain-list); do
  EC2_EGRESS=$(get_cloudwatch_network_out $DOMAIN $MONTH)   # GB
  CDN_BANDWIDTH=$(get_cdn_bandwidth $DOMAIN $MONTH)         # GB

  RATIO=$(echo "$CDN_BANDWIDTH / $EC2_EGRESS" | bc -l)

  if (( $(echo "$RATIO < 0.1" | bc -l) )); then
    echo "BYPASS ALERT: $DOMAIN — CDN served only $(echo "$RATIO * 100" | bc -l)% of traffic"
    echo "  EC2 egress:    ${EC2_EGRESS} GB"
    echo "  CDN bandwidth: ${CDN_BANDWIDTH} GB"
  fi
done

Financial impact and the scale of the problem

For cliente-loja.com.br, the direct identified cost was $342.29/month in unnecessary egress. On a platform with dozens of clients, the multiplier effect is significant. Even without calculating each case individually, the 3 domains with incorrect SG identified in the audit represented similar potential leakage.

CDN without correct DNS is a double cost: you pay for the CDN that serves nothing, and you pay for EC2 egress that serves everything. The Security Group audit is the second step — but the first is cross-referencing CloudWatch NetworkOut with CDN analytics.

The lesson for multi-tenant platforms

Platforms hosting multiple clients under the same security model need periodic audits of the alignment between DNS, CDN, and Security Groups. The three layers derive from independent decisions — a client changes CDN, DNS is updated, but the SG is left behind. Or a World SG is temporarily added for diagnostics and never removed.

Recommended frequency: monthly DNS × SG × CDN analytics alignment audit. The cost of identifying a bypass is one hour of work. The cost of not identifying it is visible on next month's bill.

Related articles