Unified Threat Detection on AWS: Security Hub, GuardDuty, and Automated EC2 Isolation

In Project 2, GuardDuty Runtime Monitoring generated three findings against the EKS cluster: a crypto miner execution at severity 8.0, a fileless execution at 5.0, and a suspicious command at 1.0. All three appeared in the GuardDuty console and stayed there. No ticket opened, no instance quarantined, no engineer paged unless they happened to be looking at the right console at the right time.

That is the gap this project fills. A finding that sits in a dashboard until a human notices it isn’t a response, it’s a log. The question is what it takes to close the loop automatically: GuardDuty sees the threat, Security Hub normalises it, EventBridge routes it, Step Functions executes the response, and the affected EC2 instance is network-isolated before the attacker can pivot.

The Architecture

The pipeline connects four AWS services:

GuardDuty finding (HIGH/CRITICAL, AwsEc2Instance)
  → Security Hub (normalises + aggregates)
    → EventBridge rule (ProductName: GuardDuty, WorkflowState: NEW)
      → Step Functions state machine
          Step 1: Parse finding, extract InstanceId
          Step 2: Get or create isolation SG (zero inbound, zero outbound)
          Step 3: Swap instance SGs, tag original SGs on instance
          Step 4: Notify via SNS with finding summary and restore command

Security Hub is the normalisation layer between GuardDuty and EventBridge, receiving findings directly once both services are active in the same account. EventBridge can filter on Security Hub finding attributes (severity, product name, resource type, workflow state) in ways GuardDuty’s own event bus doesn’t expose. GuardDuty’s native events lack a ProductName or WorkflowState field, so filtering to only NEW GuardDuty findings requires going through Security Hub. That combination routes GuardDuty runtime findings precisely, without picking up FSBP compliance checks.

The Terraform boundary covers the response pipeline: EventBridge rule, Step Functions state machine, four Lambda functions, four IAM roles, SNS topic, SQS dead-letter queue, and the isolation security group. Everything in the console boundary (Security Hub setup, GuardDuty Extended Threat Detection, Inspector, target EC2) is configured manually.

Why SG swap, not instance stop

Stopping the EC2 instance would end the threat, but it would also wipe the memory state of the running process: injected code, open file handles, active connections that forensics would want to examine. The isolation SG swap keeps the instance running, severs all network access, and tags the original SG IDs on the instance so the swap is reversible with one CLI command. With SSM interface endpoints, and allowing outbound traffic to them in the isolation SG, Session Manager access can survive isolation.

The trade-off: isolation takes the instance offline if it’s serving real production traffic. Not a concern in this lab. In production you’d check for a critical-infrastructure tag before executing the swap, or route instances needing manual approval through a separate Step Functions branch.

Why Step Functions, not a single Lambda

A single Lambda function could execute all four steps. Step Functions instead makes each step visible and auditable in the console: the execution graph shows which step ran, its input and output, and where it failed. For a workflow that runs automatically without operator involvement, that audit trail is the fastest way to understand what happened, without digging through per-Lambda CloudWatch Logs.

Bootstrap: Security Hub, GuardDuty Extended Threat Detection, Inspector

Security Hub needs enabling before anything routes through it. The two standards that matter here, CIS AWS Foundations Benchmark v3.0.0 and AWS Foundational Security Best Practices v1.0.0, activate automatically when you enable Security Hub with default standards. GuardDuty integration is automatic once both services are active in the same account and region.

GuardDuty Extended Threat Detection correlates individual findings into multi-stage attack sequences. A crypto miner execution followed by a credential access attempt followed by an unusual API call becomes a single Impact:EC2/CompromisedCredentials sequence rather than three separate LOW and MEDIUM findings an analyst has to correlate manually. Here the feature is enabled but produces zero sequences: a single xmrig execution on one instance doesn’t chain into a multi-stage pattern. Its value shows up in environments with sustained attack campaigns.

Inspector is enabled for EC2 scanning, with findings flowing into Security Hub automatically. The target EC2 runs Amazon Linux 2023 with no application packages beyond the base install, so Inspector returned no findings; AL2023 is well-patched at launch. A production instance running web application dependencies would generate findings here.

The target EC2: i-0aa9e4b30f294f173, t3.micro, Amazon Linux 2023, default VPC, launched with no key pair. Session Manager handles all access via the AmazonSSMManagedInstanceCore instance profile. In the default VPC this works over the public internet through the internet gateway, no SSM interface endpoints needed. The EKS nodes in Project 2 sat in private subnets and did need them.

Terraform: The Response Pipeline

The full pipeline is 18 resources. The most important design decisions are in the EventBridge event pattern and the Step Functions state machine definition.

The EventBridge rule:

{
  "source": ["aws.securityhub"],
  "detail-type": ["Security Hub Findings - Imported"],
  "detail": {
    "findings": {
      "ProductName": ["GuardDuty"],
      "Severity": { "Label": ["HIGH", "CRITICAL"] },
      "Resources": { "Type": ["AwsEc2Instance"] },
      "RecordState": ["ACTIVE"],
      "WorkflowState": ["NEW"]
    }
  }
}

The ProductName: GuardDuty filter is load-bearing and got added after the first test run. Without it, the rule matches every HIGH and CRITICAL finding Security Hub imports, including FSBP compliance checks. The EC2.9 check (“EC2 instances should not have a public IPv4 address”) fires HIGH against the target EC2 and re-fires on a refresh cycle. The first test run without the filter produced eight Step Functions executions in ten minutes, each isolating the already-isolated instance and tagging the isolation SG itself as the original SG to restore to. The ProductName filter stops compliance checks from triggering the pipeline.

The isolation Lambda stores original SGs before swapping:

ec2.create_tags(
    Resources=[instance_id],
    Tags=[
        {"Key": "IsolationStatus", "Value": "quarantined"},
        {"Key": "IsolatedAt", "Value": datetime.now(timezone.utc).isoformat()},
        {"Key": "FindingId", "Value": finding_id[:256]},
        {"Key": "OriginalSGs", "Value": ",".join(original_sgs)[:256]}
    ]
)
ec2.modify_instance_attribute(
    InstanceId=instance_id,
    Groups=[isolation_sg_id]
)

The 256-character truncation on tag values is intentional, since EC2 tag values have a 256-character limit. An instance with many SGs would overflow a comma-separated list of SG IDs. This lab has one SG per instance, so truncation never fires, but the guard is there.

The isolation SG is a Terraform resource with no ingress or egress rules. A security group with no rules denies all traffic by default. Creating it via Terraform means it exists before any finding fires, so the GetIsolationSG Lambda step retrieves it in milliseconds rather than waiting for an API call mid-execution.

The EventBridge target has a retry policy (3 attempts, 1-hour max age) and a dead-letter SQS queue. If Step Functions rejects the invocation (throttled, wrong ARN, transient error), the event retries and ultimately lands in the DLQ instead of being silently dropped.

terraform init && terraform apply

The Step Functions state machine has five states, including a CheckInstanceId choice state that fails gracefully if the finding lacks an EC2 resource. FSBP compliance findings targeting IAM roles or S3 buckets would slip through the ProductName filter if somehow misattributed; the choice state catches them before attempting modify_instance_attribute on a null instance ID.

Before testing, confirm the SNS email subscription. AWS sends a confirmation email on creation. Skip the link and the Notify step still succeeds from Step Functions’ perspective, but no email arrives.

Triggering the Pipeline

The first trigger attempt used cp /bin/busybox /tmp/xmrig, but AL2023 has no busybox, and the binary has to exist to execute. The real xmrig binary works for this trigger:

curl -L https://github.com/xmrig/xmrig/releases/download/v6.21.0/xmrig-6.21.0-linux-static-x64.tar.gz \
  -o /tmp/xmrig.tar.gz
tar -xzf /tmp/xmrig.tar.gz -C /tmp
/tmp/xmrig-6.21.0/xmrig --help 2>/dev/null || true

Only do this in an isolated test account you control. The binary is legitimate mining software; downloading and executing it in a production environment is a real incident, not a test.

The --help flag is enough here. GuardDuty’s Runtime Monitoring agent detects the execution via the execve syscall before any mining pool connection is made, which is why it fires on a harmless --help invocation. This is the behavioural, syscall-level detection that distinguishes Runtime Monitoring from network-based detection requiring an outbound connection.

The second trigger accesses EC2 instance metadata credentials via IMDSv2:

TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" \
  -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" \
  http://169.254.169.254/latest/meta-data/iam/security-credentials/

GuardDuty generated findings within ten minutes. The HIGH Impact:Runtime/CryptoMinerExecuted finding appeared against i-0aa9e4b30f294f173.

Security Hub imported it automatically.

The Step Functions execution triggered within 30 seconds of the finding landing in Security Hub. Status: SUCCEEDED.

The EC2 instance security group swapped from sg-0cb48731e99aa9b6b (utd-prod-sg-target) to sg-01af38bf25f06012c (utd-prod-sg-isolation).

The SNS email arrived within seconds of the Notify step completing.

Running xmrig again to retrigger GuardDuty produced no new pipeline execution. GuardDuty deduplicates findings: re-executing the same binary against the same instance increments the count on the existing finding rather than emitting a new WorkflowState: NEW event. The pipeline only fires on new findings, correct behaviour, since re-isolating an already-isolated instance would overwrite the OriginalSGs tag with the isolation SG ID, making rollback impossible. GuardDuty’s dedup window keeps the pipeline from running against itself.

To verify pipeline mechanics without waiting for a new GuardDuty finding, a test event went directly to EventBridge with a custom source and matching finding payload. The execution succeeded and the instance isolated cleanly.

Pipeline State After Testing

After restoring the EC2 to its original SG, the DLQ is empty (no failed EventBridge deliveries) and all Step Functions executions show SUCCEEDED.

Where This Has Limits

The pipeline responds to GuardDuty HIGH and CRITICAL findings targeting EC2 instances. It doesn’t respond to other resource types: IAM roles, S3 buckets, Lambda functions. Isolating an EC2 is a concrete, reversible action. The equivalent response for a compromised IAM role (revoke sessions, detach policies) or S3 bucket (block public access, restrict bucket policy) needs separate Lambda functions and different isolation logic per resource type.

GuardDuty dedup means the pipeline fires once per finding per resource, not once per attack event. If the attacker re-runs xmrig after the first isolation and the instance somehow gets re-exposed, GuardDuty won’t re-fire the existing finding type until the dedup window expires. The pipeline also has no idempotency guard: firing twice against the same instance overwrites the OriginalSGs tag with the isolation SG ID, breaking rollback. Production pipelines should check isolation state before executing the swap; a DynamoDB record keyed on instance ID is the standard pattern.

The isolation SG swap severs SSM access here because the isolation SG has zero egress and the default VPC has no SSM interface endpoints. After isolation, Session Manager can’t reach the instance. Forensic investigation then needs either a temporary narrow egress rule on the isolation SG (port 443 to the SSM endpoint ENI IPs) or offline EBS snapshot analysis. With SSM interface endpoints, you can preserve Session Manager access post-isolation by allowing outbound 443 to the VPCE ENIs in the isolation SG; SSM traffic routes through the endpoint, but the SG egress rule still applies and must explicitly permit it.

GuardDuty Extended Threat Detection produced zero attack sequences here. Single-vector attacks on a single instance don’t chain into multi-stage sequences. The feature earns its keep with sustained campaigns across multiple resources, where GuardDuty can correlate credential theft, lateral movement, and data exfiltration into one high-fidelity finding instead of a flood of low-severity alerts.

The loop open at the start of this post is now closed. A HIGH GuardDuty finding against an EC2 instance no longer needs a human to notice it, open a ticket, and manually pull a security group. It triggers an automated response in the time it takes Step Functions to complete four Lambda invocations. Multi-resource response logic, idempotency controls, cross-account routing, escalation thresholds: what you build on top is the same pattern extended, not a different one.