Troubleshooting GRE over IPsec Tunnel Down

Overview

When troubleshooting a GRE over IPsec tunnel, work from the outside in:

  1. Verify underlay connectivity.
  2. Verify IKE (Phase 1).
  3. Verify IPsec (Phase 2).
  4. Verify GRE tunnel status.
  5. Verify routing and tunnel reachability.
  6. Verify NAT and crypto ACLs.
  7. Generate traffic and watch counters.
  8. Use debugs if necessary.

1. Verify Underlay Connectivity

Confirm the routers can reach each other’s public IP addresses.

Ping Peer Public IP 

ping <peer-public-ip>

Trace Route 

traceroute <peer-public-ip>

Verify Route to Peer 

show ip route <peer-public-ip>

Expected Result

The peer’s public IP should be reachable and routed correctly.

Common Problems

  • Missing route
  • ISP outage
  • Firewall blocking traffic
  • Incorrect peer IP

2. Verify IKE Phase 1

For IKEv1:

show crypto isakmp sa

Healthy State 

QM_IDLE

Common States

State Meaning
QM_IDLE Phase 1 complete
MM_NO_STATE No response from peer
MM_WAIT_MSG Waiting on peer
MM_KEY_EXCH Key exchange in progress
DELETE SA being removed

MM_NO_STATE Usually Indicates

  • Peer unreachable
  • UDP/500 blocked
  • NAT issue
  • Incorrect peer IP
  • ISAKMP policy mismatch

3. Verify IPsec Phase 2 

show crypto ipsec sa

Important Counters 

pkts encaps:
pkts decaps:

Healthy Tunnel

Both counters increase.

Example:

pkts encaps: 12345
pkts decaps: 12340

Encaps Increasing, Decaps Zero

Example:

pkts encaps: 500
pkts decaps: 0

Possible causes:

  • Remote peer down
  • Return routing issue
  • Firewall blocking ESP
  • Incorrect crypto ACL on remote side

Both Counters Zero

Possible causes:

  • No interesting traffic
  • Crypto ACL mismatch
  • Tunnel traffic not matching policy

4. Verify GRE Tunnel Status 

show interface tunnel 0

Healthy 

Tunnel0 is up, line protocol is up

Tunnel Up / Protocol Down 

Tunnel0 is up, line protocol is down

Usually indicates:

  • GRE packets not returning
  • Peer tunnel interface down
  • GRE blocked

Tunnel Down / Protocol Down 

Tunnel0 is down, line protocol is down

Usually indicates:

  • Tunnel destination unreachable
  • Tunnel source interface issue
  • Missing route to peer

5. Verify Tunnel Configuration

Display tunnel configuration:

show run interface tunnel0

Example:

interface Tunnel0
 ip address 10.0.0.1 255.255.255.252
 tunnel source GigabitEthernet0/0
 tunnel destination 1.2.3.4

Verify Tunnel Destination Reachability 

ping <tunnel-destination-public-ip> source <outside-interface-ip>

Example:

ping 1.2.3.4 source 2.2.2.2

6. Verify GRE is Being Protected by IPsec

GRE uses protocol 47.

Example output:

IPSEC FLOW: permit 47 host 192.168.0.110 host <peer-public-ip>

Protocol Reference

Protocol Number
ICMP 1
TCP 6
UDP 17
GRE 47
ESP 50

7. Verify Crypto ACLs

Display access lists:

show access-lists

Example GRE ACL:

permit gre host <local-public-ip> host <remote-public-ip>

Equivalent:

permit 47 host <local-public-ip> host <remote-public-ip>

Common Problems

  • Wrong source IP
  • Wrong destination IP
  • ACL not mirrored on peer
  • GRE not included

8. Verify Tunnel Reachability

Ping across the tunnel.

Example:

ping 10.0.0.2 source 10.0.0.1

If tunnel IPs cannot communicate:

  • GRE not functioning
  • IPsec not passing traffic
  • Routing issue

9. Verify Dynamic Routing Protocols

EIGRP 

show ip eigrp neighbors

OSPF 

show ip ospf neighbor

BGP 

show ip bgp summary

Healthy EIGRP Example 

Address         Interface
10.0.0.2        Tu0

If no neighbors exist:

  • Tunnel not forwarding traffic
  • Routing protocol blocked
  • Authentication mismatch

10. Verify NAT Exemption

Display NAT configuration:

show run | include nat

Traffic destined for the VPN peer should not be NATed.

Symptoms of NAT Issues

  • MM_NO_STATE
  • Phase 2 failures
  • One-way traffic
  • Encaps increasing but decaps not increasing

11. Generate Traffic and Watch Counters

Display counters:

show crypto ipsec sa

Generate traffic:

ping <remote-lan-ip>

Check counters again:

show crypto ipsec sa

What to Look For

Result Meaning
Encaps increasing Local encryption working
Decaps increasing Remote traffic returning
Encaps only Return traffic missing
Neither increasing Traffic not matching crypto ACL

12. Useful Debug Commands

ISAKMP / IKE 

debug crypto isakmp

IPsec 

debug crypto ipsec

GRE 

debug tunnel

Disable Debugs 

undebug all

or

u all

Fast Triage Workflow

When a GRE over IPsec tunnel is reported down:

Step 1

Check Phase 1:

show crypto isakmp sa

Healthy:

QM_IDLE

Step 2

Check Phase 2:

show crypto ipsec sa

Verify:

encaps
decaps

Step 3

Check Tunnel Status:

show interface tunnel <number>

Interpret Results

Status Meaning
Up / Up Tunnel healthy
Up / Down GRE not returning
Down / Down Underlay issue

Step 4

Verify Route to Peer

show ip route <peer-public-ip>

Step 5

Verify Crypto ACL

show access-lists

Confirm GRE traffic matches:

permit gre host <local-public-ip> host <remote-public-ip>

Common Failure Scenarios

MM_NO_STATE

Most likely:

  • Peer unreachable
  • Firewall blocking UDP/500
  • Wrong peer IP

QM_IDLE But No Traffic

Most likely:

  • Crypto ACL mismatch
  • Routing issue
  • Tunnel traffic not matching policy

Encaps Increasing, Decaps Zero

Most likely:

  • Remote side down
  • Return routing issue
  • Firewall blocking ESP

Tunnel Up / Down

Most likely:

  • GRE packets not returning
  • Peer tunnel interface down
  • GRE blocked

Tunnel Down / Down

Most likely:

  • Tunnel destination unreachable
  • Missing route
  • Tunnel source interface down

High-Value Show Commands 

show crypto isakmp sa
show crypto ipsec sa
show interface tunnel <number>
show run interface tunnel <number>
show ip route <peer-public-ip>
show access-lists
show ip eigrp neighbors
show ip ospf neighbor

These commands alone will usually identify the failure domain within a few minutes.