Troubleshooting IPsec VPN tunnel establishment and traffic flow

1. Confirm the route to the protected destination.

   Enter the applicable VDOM and query the destination route:

   Copy CLI

   get router info routing-table detail <destination-IP>
   

   Example:

   Copy CLI

   DE_A (root) # get router info routing-table details 192.168.50.10
   Routing table for VRF=0
   Routing entry for 0.0.0.0/0
     Known via "static", distance 10, metric 0, best
     * vrf 0 10.35.0.254, via port1
   

   If the exit interface is a physical interface rather than the expected IPsec tunnel, Phase 1 or Phase 2 may be down, or a routing problem may exist. If several tunnels can reach the destination, isolate the affected tunnel before continuing.

2. Identify the tunnel and selector state.

   From the VDOM containing the VPN, run:

   Copy CLI

   get vpn ipsec tunnel summary
   

   Example:

   Copy CLI

   'to10.174.0.182' 10.174.0.182:0  selectors(total,up): 1/1  rx(pkt,err): 1921/0  tx(pkt,err): 69/2
   'to10.189.0.182' 10.189.0.182:0  selectors(total,up): 1/0  rx(pkt,err): 0/0  tx(pkt,err): 0/0
   

   In this example, to10.189.0.182 has zero selectors up and should be investigated.

3. Check Phase 1 status.

   Copy CLI

   diagnose vpn ike gateway list name <IPsec Tunnel Name>
   

   Example:

   Copy CLI

   diagnose vpn ike gateway list name to10.189.0.182
   
   vd: root/0
   name: to10.189.0.182
   version: 1
   interface: port9 10
   addr: 10.189.0.31:500 -> 10.189.0.182:500
   created: 15s ago
   IKE SA: created 1/1
   IPsec SA: created 0/0
   
   id/spi: 19576 a83334b3c66f871b/0000000000000000
   direction: responder
   status: connecting, state 3, started 15s ago
   

   Interpret status as follows:

   • established means Phase 1 is active; proceed to Phase 2 checks.
   • connecting means Phase 1 has not completed.

4. If Phase 1 is down, verify bidirectional gateway reachability.

   The gateway is normally the WAN-interface IP address. Set the local source and test the remote peer:

   Copy CLI

   execute traceroute-options source 10.189.0.31
   execute traceroute 10.189.0.182
   

   A ping may also be used where permitted.

5. Verify that IKE traffic is not blocked.

   Capture UDP port 500 and 4500 traffic to the peer:

   Copy CLI

   diagnose sniffer packet any "host 10.189.0.182 and (port 500 or port 4500)" 4 0 l
   

   Expected setup output:

   Copy CLI

   interfaces=[any]
   filters=[host 10.189.0.182 and (port 500 or port 4500)]
   

   If nattraversal is enabled in Phase 1 and FortiGate is behind NAT, capture with udp port 4500. Otherwise, use udp port 500. The IKE port can be configured when another device in the path blocks the default port.

6. Run IKE debugging for configuration mismatches.

   Clear any existing filter first as a best practice:

   Copy CLI

   diagnose vpn ike log filter clear
   

   The same clear command applies for v7.4.0 and above.

   Use this sequence where the dst-addr4 filter syntax applies:

   Copy CLI

   diagnose debug reset
   diagnose vpn ike log filter dst-addr4 10.189.0.182
   diagnose debug application ike -1
   diagnose debug console timestamp enable
   diagnose debug enable
   

   For v7.4.0 and above, use:

   Copy CLI

   diagnose debug reset
   diagnose vpn ike log filter rem-addr4 10.189.0.182
   diagnose debug application ike -1
   diagnose debug console timestamp enable
   diagnose debug enable
   

   Starting from v7.4.1, diagnose vpn ike log-filter dst-addr4 was changed to diagnose vpn ike log filter rem-addr4. Starting from v7.4.1, diagnose vpn ike log-filter src-addr4 was changed to diagnose vpn ike log filter loc-addr4.

   Both applicable filter command forms work as intended in their respective contexts. Filtering by name is useful when the remote-peer IP address is unknown. To filter multiple IPv4 remote gateways, use mrem-addr4. Display all filter choices with:

   Copy CLI

   diagnose vpn ike log filter ?
   

   Available output includes:

   Copy CLI

   list                   Display the current filter.
   clear                  Erase the current filter.
   vd                     Index of virtual domain. -1 matches all.
   name                   Phase1 name to filter by.
   ifindex                Index of the interface that IKE connection is negotiated over.
   loc-addr4              IPv4 local gateway address range to filter by.
   mloc-addr4             Multiple IPv4 local gateway address to filter by.
   rem-addr4              IPv4 remote gateway address range to filter by.
   mrem-addr4             Multiple IPv4 remote gateway address to filter by.
   loc-addr6              IPv6 local gateway address range to filter by.
   mloc-addr6             Multiple IPv6 local gateway address to filter by.
   rem-addr6              IPv6 remote gateway address range to filter by.
   mrem-addr6             Multiple IPv6 remote gateway addresses to filter by.
   dst-port               Destination port range to filter by.
   negate                 Negate existing setting of the specified filter parameter.
   

   A v7.4.x-style filtered debug can also be started directly with:

   Copy CLI

   diagnose vpn ike log filter rem-addr4 10.189.0.182
   diagnose debug application ike -1
   diagnose debug console timestamp enable
   diagnose debug enable
   

   In v7.6.3, IKE debugging can also be performed from the GUI by using the CLI diagnostics command.

   Other documented IKE filter options include:

   • rem-addr4: IPv4 remote gateway address range.
   • msrc-addr4: Multiple IPv4 source addresses.
   • mdst-addr4: Multiple IPv4 destination addresses.
   • msrc-addr6: Multiple IPv6 source addresses.
   • mdst-addr6: Multiple IPv6 destination addresses.

   Stop debugging after collecting the output:

   Copy CLI

   diagnose debug disable
   diagnose debug reset
   

7. Collect authentication-specific debug output when applicable.

   For remote-user authentication issues:

   Copy CLI

   diagnose debug application fnbamd -1
   

   For SAML authentication issues:

   Copy CLI

   diagnose debug application samld -1
   diagnose debug application eap_proxy -1
   

   Stop these debugs with:

   Copy CLI

   diagnose debug disable
   diagnose debug reset
   

8. Check Phase 2 when Phase 1 is established.

   Copy CLI

   diagnose vpn tunnel list name <phase1-name>
   

   Example:

   Copy CLI

   diagnose vpn tunnel list name to10.189.0.182
   list all ipsec tunnel in vd 0
   name=to10.189.0.182 ver=1 serial=2 10.189.0.31:0->10.189.0.182:0
   bound_if=10 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/8 options[0008]=npu
   proxyid_num=1 child_num=0 refcnt=10 ilast=25 olast=25 ad=/0
   stat: rxp=0 txp=0 rxb=0 txb=0
   dpd: mode=on-demand on=0 idle=20000ms retry=3 count=0 seqno=534
   natt: mode=none draft=0 interval=0 remote_port=0
   proxyid=to10.189.0.182 proto=0 sa=0 ref=1 serial=4
   src: 0:172.16.170.0/255.255.255.0:0
   dst: 0:192.168.50.0/255.255.255.0:0
   

   Interpret sa as follows:

   • sa=0: Selector mismatch or no traffic has initiated the SA.
   • sa=1: The IPsec SA matches and traffic exists between the selectors.
   • sa=2: Temporary state visible during IPsec SA rekey.

   View selector details with:

   Copy CLI

   get vpn ipsec tunnel details
   

   Compare the Phase 2 encryption and hashing algorithms on both peers. Run the IKE debug from step 6 to identify selector or proposal mismatches.

9. Review encryption and decryption counters on both peers.

   Confirm that both enc and dec packet counters increase. If enc rises while dec remains unchanged, FortiGate is transmitting but return traffic is not reaching it. This generally indicates one-way traffic or a remote-side/return-path problem.

10. For dial-up IPsec, list all Phase 1 connection instances.

diagnose vpn tunnel dialup-list <phase1-name>

11. Align debug and packet-capture timestamps.

Before taking packet captures, enable console time so debug messages can be correlated precisely with sniffer timestamps:

diagnose debug console time enable

12. Investigate a tunnel that is green in the GUI but inactive.

In the documented scenario, ICMP entered the internal interface but did not appear on the tunnel:

FortiGate-61F # diagnose sniffer packet any 'host 10.1.1.37 and icmp' 4 0 l
interfaces=[any]
filters=[host 10.1.1.37 and icmp]
2024-07-16 10:46:01.327162 internal in 192.168.1.251 -> 10.1.1.37: icmp: echo request
2024-07-16 10:46:02.331446 internal in 192.168.1.251 -> 10.1.1.37: icmp: echo request
2024-07-16 10:46:03.335546 internal in 192.168.1.251 -> 10.1.1.37: icmp: echo request

Debug flow showed no explicit drop and repeatedly ended after route lookup:

2024-07-16 10:47:54 id=20085 trace_id=4 func=iprope_dnat_check line=5191 msg="in-[internal], out-[]"
2024-07-16 10:47:54 id=20085 trace_id=4 func=iprope_dnat_check line=5204 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000"
2024-07-16 10:47:54 id=20085 trace_id=4 func=vf_ip_route_input_common line=2615 msg="find a route: flag=00000000 gw-10.1.1.37 via root"
2024-07-16 10:47:59 id=20085 trace_id=5 func=print_pkt_detail line=5746 msg="vd-root:0 received a packet(proto=1, 192.168.1.251:1->10.1.1.37:2048) from internal. type
=8, code=0, id=1, seq=288."

The tunnel still had a negotiated SA:

diagnose vpn tunnel list name Primary Tunnel

name=Primary Tunnel ver=1 serial=2 10.101.1.1:0->10.101.1.2:0 tun_id=10.101.1.2 dst_mtu=1500 dpd-link=on remote_location=0.0.0.0 weight=1
bound_if=5 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/520 options[0208]=npu frag-rfc  run_state=0 accept_traffic=1 overlay_id=0
proxyid_num=1 child_num=0 refcnt=4 ilast=0 olast=0 ad=/0
stat: rxp=11746 txp=32 rxb=1939048 txb=9861
dpd: mode=on-demand on=1 idle=10000ms retry=3 count=0 seqno=0
natt: mode=none draft=0 interval=0 remote_port=0
proxyid=Net-10.1.1.0-24 proto=0 sa=1 ref=2 serial=5
  src: 0:192.168.1.0/255.255.255.0:0
  dst: 0:10.1.1.0/255.255.255.0:0
  SA:  ref=4 options=10226 type=00 soft=0 mtu=1438 expire=42832/0B replaywin=2048
       seqno=1 esn=0 replaywin_lastseq=000001db itn=0 qat=0 hash_search_len=1
  life: type=01 bytes=0/0 timeout=42933/43200
  dec: spi=e6bb099b esp=aes key=16 4c3dd75b85f0be4c2d2b95e5dfe7e99d
       ah=sha1 key=20 f9a5dbc8360e4b58aff64e74fe40137cd89dc98e
  enc: spi=533e1c0c esp=aes key=16 4fddb780c9606c5c235737bd7d05c585
       ah=sha1 key=20 57cc8349171a97eaf230e565bb90853cefaca7ab
  dec:pkts/bytes=476/75272, enc:pkts/bytes=0

This output identifies source 10.101.1.1, destination and tunnel ID 10.101.1.2, bound interface 5, auto mode, no encapsulation, NPU and fragment-RFC options, and run_state=0. Traffic totals are 11746 received packets, 32 transmitted packets, 1939048 received bytes, and 9861 transmitted bytes. DPD is on-demand with a 10000ms idle time and three retries. NAT traversal is disabled. Proxy ID Net-10.1.1.0-24 covers 192.168.1.0/24 to 10.1.1.0/24; the SA uses MTU 1438, expires in 42832 seconds, and has a replay window of 2048. ESP uses AES with a 16-byte key, and AH uses SHA1 with a 20-byte key. The SA had 476 decrypted packets/75272 bytes but zero encrypted packets.

Check the link-monitor health-check configuration. In this scenario, the link monitor marked Primary Tunnel dead because the monitored peer was unreachable. Disabling the relevant link-monitor settings made the tunnel interface active.

13. Disable NPU offloading temporarily while troubleshooting.

NPU offloading is enabled by default. Offloaded packets may bypass CPU-based flow filters, making packet hits invisible during flow debugging. Check the current setting:

config vpn ipsec phase1-interface
    edit <name of the tunnel>
        show full | grep npu

Disable offloading:

config vpn ipsec phase1-interface
    edit <name of the tunnel>
         set npu-offload disable
    next
end

Disabling npu-offload resets the IPsec tunnel.

14. Collect detailed IKE status for TAC if needed.

diagnose vpn ike status detailed

15. Flush and renegotiate a tunnel when required.

Available commands are:

diagnose vpn tunnel flush <my-phase1-name>
diagnose vpn ike gateway clear name <my-phase1-name>
diagnose vpn ike gateway flush name <my-phase1-name>

16. Resolve unknown SPI events.

Unknown SPI messages mean FortiGate received ESP packets whose SPIs do not match an active IPsec tunnel. This can follow an SA rekey mismatch, FortiGate reboot, or network interruption. Flush the named tunnel to force negotiation and synchronize SPIs:

diagnose vpn tunnel flush name <tunnel_name>

Flushing interrupts VPN traffic. Confirm that Phase 2 SA lifetimes match on both peers and that Dead Peer Detection is enabled to reduce recurrence.

Repeat the route, summary, and tunnel-detail checks after remediation:

get router info routing-table detail <destination-IP>
get vpn ipsec tunnel summary
diagnose vpn ike gateway list name <IPsec Tunnel Name>
diagnose vpn tunnel list name <phase1-name>

Verify that:

• The protected destination routes through the intended IPsec interface.
• Phase 1 reports established.
• The expected selectors are up.
• Phase 2 normally reports sa=1 when traffic is present.
• Both enc and dec counters increase during a bidirectional traffic test.
• Packet captures show traffic entering and leaving the expected interfaces.
• The link monitor no longer marks the tunnel dead.
• Unknown SPI messages stop after renegotiation.

If NPU offloading was disabled only for troubleshooting, restore it after data collection with:

config vpn ipsec phase1-interface
    edit <name of the tunnel>
        set npu-offload enable
    next
end

This change resets the IPsec tunnel. If link-monitor settings were disabled, restore them only after correcting the monitored peer reachability or health-check configuration. Always terminate active debug sessions with:

diagnose debug disable
diagnose debug reset