1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

One-Liner Network Troubleshooting on AWS: Why `/dev/tcp` Beats `telnet` and `nc` Every Time

1
Last updated at Posted at 2026-04-27

do you know this special command and meaning?

When your EC2 instance can't reach an RDS, ElastiCache, or external endpoint, most engineers reach for telnet, nc, or curl. But these tools may not be installed by default on minimal AMIs (Amazon Linux 2023, Ubuntu Minimal, Bottlerocket).

Use this Bash built-in instead — no installation required, works everywhere Bash runs:

echo -n "" > /dev/tcp/{host_or_ip}/{port}

Three possible outcomes tell you exactly what's wrong:

  1. Returns silently with exit code 0 → Connection successful
  2. Hangs, then times out → Check your VPC, Security Groups, NACLs, or routing
  3. Returns garbled characters or "connection refused" → The port is open, but the middleware isn't responding correctly

Why This Matters on AWS

AWS networking troubleshooting is hard not because the tools are bad, but because the layer where things break is rarely obvious. A failed connection from an EC2 instance to an RDS can be caused by any of the following:

  • VPC route table misconfiguration
  • Security Group on EC2 (egress) or RDS (ingress)
  • Network ACL on the subnet
  • Subnet that doesn't have a route to the destination
  • VPC endpoint or PrivateLink configuration
  • DNS resolution issue
  • The application itself not listening on the expected port
  • Wrong port specified in the application config

The traditional approach is to install diagnostic tools, run them, parse output. But on a fresh Amazon Linux 2023 instance, you don't even have telnet by default. You have to dnf install it. Same for nc. And in many production environments, you don't have package-installation rights.

/dev/tcp is a Bash special device that is always there if Bash is there, which on AWS is essentially everywhere.


How /dev/tcp Actually Works

/dev/tcp/<host>/<port> is not a real file on disk. It's a magic path that Bash recognizes and interprets as "open a TCP socket to this host on this port." When you redirect to it, Bash performs the TCP handshake on your behalf.

# Equivalent to opening a TCP connection
echo -n "" > /dev/tcp/example.com/443

The echo -n "" sends an empty string. We're not actually sending data — we just want to know if the socket can be opened.

Important: This works only in Bash, not in sh, dash, or POSIX-strict shells. Amazon Linux 2023's default shell for ec2-user is Bash, so this works out of the box. On Alpine-based containers (which use ash), you'll need a different approach.


Reading the Three Outcomes

Outcome 1: Silent return, exit code 0 → Connection OK

$ echo -n "" > /dev/tcp/google.com/443
$ echo $?
0

The socket was opened successfully. The host is reachable, the port is open, and a TCP handshake completed. The network path is fine. Your problem is somewhere else (application config, credentials, payload format).

Outcome 2: Hangs, then times out → Network issue

$ echo -n "" > /dev/tcp/10.0.5.42/3306
# ... waits 30+ seconds, then ...
-bash: connect: Connection timed out
-bash: /dev/tcp/10.0.5.42/3306: Connection timed out

The TCP handshake never completed. This is a network-layer problem. On AWS, check in this order:

  1. Security Group on the source (EC2): Does it allow outbound traffic to the destination port?
  2. Security Group on the destination (RDS, EC2, etc.): Does it allow inbound traffic from the source's Security Group or CIDR?
  3. Network ACL on both source and destination subnets
  4. Route table of the source subnet: is there a route to the destination?
  5. DNS: did <hostname> resolve to the IP you expected? Cross-check with dig <hostname> or getent hosts <hostname>

Use a timeout to avoid waiting 30+ seconds:

timeout 3 bash -c 'echo -n "" > /dev/tcp/10.0.5.42/3306'

If timeout exits with status 124, you timed out — confirming a network issue.

Outcome 3: Garbled characters or "Connection refused" → Middleware issue

$ echo -n "" > /dev/tcp/10.0.5.42/3306
# Output may be empty, or:
$ echo -n "GET / HTTP/1.0\r\n\r\n" > /dev/tcp/10.0.5.42/3306
J
5.7.42-log...wG)/SAdLM]wFa\Vu7yG/BlQwmysql_native_password

The TCP connection succeeded, but the response is unexpected garbage. The network is fine — the issue is at the application/middleware layer. Common causes:

  • You're connecting to the right host but the wrong port (e.g., MySQL on 3306 when you expected HTTP)
  • The middleware is misconfigured (wrong listener, wrong protocol)
  • A proxy or sidecar is intercepting the connection

Or you may see "Connection refused":

$ echo -n "" > /dev/tcp/10.0.5.42/3306
-bash: connect: Connection refused
-bash: /dev/tcp/10.0.5.42/3306: Connection refused

Connection refused means the host is reachable and the kernel responded — but nothing is listening on that port. The application is down, or it's listening on a different port than you think.


Why I Reach for This in Production AWS

I've debugged enough VPC connectivity issues to have an opinion: the layer at which a connection fails tells you almost everything.

The strength of /dev/tcp is that it isolates the TCP layer specifically. curl adds HTTP, TLS, and DNS noise. telnet is interactive and harder to script. nc is great but isn't always installed.

echo > /dev/tcp answers exactly one question: can I open a TCP socket to this address? That's almost always the question you actually have when something on AWS is broken.


A Practical AWS Debugging Recipe

Here's the workflow I run when an EC2 instance can't reach an RDS:

# Step 1: Does DNS resolve?
getent hosts mydb.cluster-xxx.ap-northeast-1.rds.amazonaws.com

# Step 2: Can I open a TCP socket?
timeout 3 bash -c 'echo -n "" > /dev/tcp/mydb.cluster-xxx.ap-northeast-1.rds.amazonaws.com/3306'
echo "Exit code: $?"

# Step 3: If step 2 timed out, dump SG/NACL state from inside the instance
curl -s http://169.254.169.254/latest/meta-data/security-groups
curl -s http://169.254.169.254/latest/meta-data/mac
# (Then cross-check Security Groups in the AWS Console or via the CLI)

# Step 4: From a known-good instance in the same VPC, repeat step 2.
# If it works there, the problem is your source — likely SG egress or NACL.
# If it fails there too, the problem is on the destination side.

This four-step recipe has resolved roughly 90% of the AWS connectivity issues I've seen in the field, including production incidents.


When NOT to Use /dev/tcp

A few caveats so you don't get burned:

  • Not in Alpine / BusyBoxash doesn't support /dev/tcp. Use nc or install Bash.
  • Not for UDP/dev/tcp is TCP-only. There is /dev/udp but it behaves differently (no handshake means you can't reliably tell if the destination received the packet).
  • Not for HTTPS handshake validation — TCP succeeding doesn't tell you whether TLS will work. Use openssl s_client -connect host:443 or curl -v for that.
  • Production scripts should use timeout — without it, a hung socket can stall your script for the kernel's default timeout (often 60+ seconds).

Related AWS Tools

For deeper diagnosis when /dev/tcp says "the network is broken," AWS provides several services:

  • VPC Reachability Analyzer — Console-based path tracing between two endpoints; tells you exactly which SG/NACL/route is blocking
  • VPC Flow Logs — Logs every accepted/rejected packet at the ENI level
  • AWS Network Manager — Cross-account, cross-region network topology
  • AWS X-Ray — When the issue is at the application layer

But these tools take minutes to set up. /dev/tcp takes one second. Start there.


Closing Thought

This Tip went viral on X (4,000+ views in days) because it surprised people. Many AWS engineers have been using telnet or nc for years without realizing Bash had a built-in alternative they could use anywhere.

The lesson isn't really about /dev/tcp. It's about how often we reach for installed tools when the OS already has what we need. The engineers I trust most are the ones who understand the layer below the cloud, because that layer is what AWS is built on.

An engineer who understands both the application and the infrastructure — that's the perspective that makes troubleshooting fast.

And ... you can study japanese by original content.


About me

Tsuyoshi Otsuki (@plustick) — AWS Ambassador (2025) × AWS Japan Top Engineer (2023–). I write about practical AWS, IaC, and the OS/network layer that AWS sits on top of. Find me on Speaker Deck and LinkedIn.

If this Tip saved you a debugging session, leave a 👍 or share it with a teammate.

1
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?