Network Security Review
When to Use
- Designing network isolation for a new data platform
- Auditing existing security groups for overly permissive rules
- Enabling TLS/mTLS for all inter-service communication
- Investigating a potential lateral movement incident
- Planning VPC peering or PrivateLink connectivity
Kubernetes NetworkPolicy — Default Deny + Explicit Allow
# Step 1: Block all ingress to namespace by default
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
namespace: airflow
spec:
podSelector: {} # applies to ALL pods
policyTypes:
- Ingress
- Egress
---
# Step 2: Allow only specific traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: airflow-scheduler
namespace: airflow
spec:
podSelector:
matchLabels:
component: scheduler
policyTypes:
- Ingress
- Egress
ingress:
- from:
- podSelector:
matchLabels:
component: webserver
ports:
- port: 8793 # Airflow scheduler port
egress:
# Allow scheduler to connect to metadata DB
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: postgres
ports:
- port: 5432
# Allow scheduler to connect to Kafka
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kafka
ports:
- port: 9092
- port: 9093 # Kafka TLS
# Allow DNS resolution
- to:
- namespaceSelector: {}
ports:
- port: 53
protocol: UDP
- port: 53
protocol: TCP
AWS Security Group Audit
#!/bin/bash
# Find security groups with 0.0.0.0/0 ingress rules
aws ec2 describe-security-groups \
--filters "Name=ip-permission.cidr,Values=0.0.0.0/0" \
--query 'SecurityGroups[*].{
ID: GroupId,
Name: GroupName,
VPC: VpcId,
Rules: IpPermissions
}' \
--output json | jq '.[] |
"SG: \(.ID) (\(.Name)) in VPC \(.VPC)
Ports: \(.Rules | map("\(.FromPort)-\(.ToPort)/\(.IpProtocol)") | join(", "))"'
# Find RDS instances with public access
aws rds describe-db-instances \
--query "DBInstances[?PubliclyAccessible==\`true\`].{
ID: DBInstanceIdentifier,
Endpoint: Endpoint.Address
}" \
--output table
# Find MSK clusters with unauthenticated access
aws kafka list-clusters-v2 \
--query "ClusterInfoList[?ClusterState=='ACTIVE'].{
Name: ClusterName,
Auth: ClientAuthentication
}" | jq '.[] | select(.Auth.Unauthenticated.Enabled == true)'
TLS Configuration
Kafka mTLS
# MSK cluster: require TLS in transit
encryption_info {
encryption_in_transit {
client_broker = "TLS" # TLS_PLAINTEXT or TLS
in_cluster = true
}
}
# Java Kafka client with mTLS
spring:
kafka:
bootstrap-servers: kafka:9093
ssl:
trust-store-location: classpath:kafka-truststore.jks
trust-store-password: ${KAFKA_TRUSTSTORE_PASSWORD}
key-store-location: classpath:kafka-keystore.jks
key-store-password: ${KAFKA_KEYSTORE_PASSWORD}
properties:
security.protocol: SSL
PostgreSQL TLS
# Require SSL for all PostgreSQL connections
import psycopg2
conn = psycopg2.connect(
host="postgres.internal",
database="airflow",
user="airflow",
password=os.environ["DB_PASSWORD"],
sslmode="verify-full", # reject if cert doesn't match
sslcert="/certs/client.crt",
sslkey="/certs/client.key",
sslrootcert="/certs/ca.crt",
)
# Enforce SSL in PostgreSQL server
# postgresql.conf:
# ssl = on
# ssl_cert_file = 'server.crt'
# ssl_key_file = 'server.key'
# pg_hba.conf:
# hostssl all all 0.0.0.0/0 scram-sha-256
Istio Service Mesh mTLS (Zero-Trust)
# Enable strict mTLS across entire mesh
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: data-platform
spec:
mtls:
mode: STRICT # reject all plaintext traffic
---
# Define allowed traffic (Authorization Policy)
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: trino-access
namespace: trino
spec:
selector:
matchLabels:
app: trino-coordinator
rules:
- from:
- source:
principals:
- "cluster.local/ns/airflow/sa/airflow-worker"
- "cluster.local/ns/dbt/sa/dbt-runner"
to:
- operation:
ports: ["8080", "8443"]
VPC Network Design
# Production: data services only in private subnets
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "~> 5.0"
cidr = "10.0.0.0/16"
private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
public_subnets = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
database_subnets = ["10.0.201.0/24", "10.0.202.0/24", "10.0.203.0/24"]
# S3/Glue/ECR via VPC endpoints (no internet)
enable_s3_endpoint = true
enable_glue_endpoint = true
enable_ecr_dkr_endpoint = true
enable_ecr_api_endpoint = true
# No public IPs for data services
map_public_ip_on_launch = false
}
# Egress: restrict outbound traffic
resource "aws_security_group" "egress_restricted" {
name = "${local.prefix}-egress-restricted"
vpc_id = module.vpc.vpc_id
egress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"] # HTTPS only (for package downloads)
}
# No blanket 0.0.0.0/0 egress
}
VPC Flow Logs Analysis
# Enable VPC flow logs
aws ec2 create-flow-logs \
--resource-type VPC \
--resource-ids vpc-1234567890abcdef0 \
--traffic-type ALL \
--log-destination-type cloud-watch-logs \
--log-group-name /vpc/flow-logs \
--deliver-logs-permission-arn arn:aws:iam::123456789012:role/flowlogsRole
# Athena query: find suspicious connections to unusual ports
SELECT
srcaddr, dstaddr, dstport, action, packets, bytes,
from_unixtime(start) AS start_time
FROM vpc_flow_logs
WHERE action = 'REJECT'
AND dstport NOT IN (80, 443, 5432, 9092, 9093)
AND start > to_unixtime(current_timestamp - interval '1' hour)
ORDER BY packets DESC
LIMIT 50;
# Find data exfiltration patterns (large outbound transfers)
SELECT
srcaddr, dstaddr, SUM(bytes) AS total_bytes
FROM vpc_flow_logs
WHERE direction = 'egress'
AND dstaddr NOT LIKE '10.%' -- non-RFC1918 destinations
AND start > to_unixtime(current_timestamp - interval '24' hour)
GROUP BY srcaddr, dstaddr
HAVING SUM(bytes) > 100000000 -- > 100MB
ORDER BY total_bytes DESC;
Private Endpoints vs VPC Peering
| Pattern | Use Case | Security |
|---|
| VPC Endpoints (AWS PrivateLink) | AWS services (S3/RDS/MSK) | Traffic stays in AWS network |
| VPC Peering | Connect two VPCs (same/cross-account) | Transitive routing blocked |
| Transit Gateway | Hub-and-spoke, many VPCs | Centralized routing |
| PrivateLink (custom service) | Expose internal service to consumer VPC | No VPC peering required |
# AWS PrivateLink for RDS
resource "aws_vpc_endpoint" "rds" {
vpc_id = module.vpc.vpc_id
service_name = "com.amazonaws.us-east-1.rds"
vpc_endpoint_type = "Interface"
subnet_ids = module.vpc.private_subnets
security_group_ids = [aws_security_group.vpc_endpoints.id]
private_dns_enabled = true
}
Network Security Checklist
[ ] All data services in private subnets (no public IPs)
[ ] Security groups: no 0.0.0.0/0 ingress except load balancers
[ ] Kubernetes NetworkPolicy: default-deny in all namespaces
[ ] Kafka: TLS required (client_broker = TLS)
[ ] PostgreSQL: SSL required (hostssl only in pg_hba.conf)
[ ] Trino/ClickHouse: HTTPS endpoints only
[ ] VPC endpoints for AWS services (S3/ECR/Glue/SSM)
[ ] VPC flow logs enabled and analyzed weekly
[ ] No SSH (port 22) open to 0.0.0.0/0
[ ] mTLS via Istio or Linkerd for inter-service communication
[ ] DNS: private hosted zones, no public record for internal services
Anti-Patterns
0.0.0.0/0 ingress on any port — even SSH on jump hosts should be restricted to VPN CIDR; open internet access to any data service is critical risk.- No NetworkPolicy in Kubernetes — any compromised pod can reach any other pod and any database; deploy default-deny before any workloads.
- Plaintext Kafka — credentials and messages transmitted in clear text on network; always enforce TLS between clients and brokers.
- Database accessible from public subnets — RDS/ClickHouse should never be in public subnets; use a bastion host or VPN for admin access.
- No VPC flow logs — lateral movement and data exfiltration go undetected; enable flow logs before any production traffic.
References
- Kubernetes NetworkPolicy:
kubernetes.io/docs/concepts/services-networking/network-policies/
- Istio security:
istio.io/docs/concepts/security/
- AWS VPC endpoints:
docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html
- Related skills:
[[infra-kubernetes-security-audit]], [[infra-rbac-audit]], [[infra-aws-data-platform-review]]