Troubleshooting FortiGate IPsec VPN tunnel establishment and traffic flow

1. Confirm the route to the remote destination. Run the routing lookup from the VDOM where the VPN is configured:

get router info routing-table detail <destination-IP>

Example:

DE_A (root) # get router info routing-table details 192.168.50.10
Routing table for VRF=0
Routing entry for 0.0.0.0/0
  Known via "static", distance 10, metric 0, best
  * vrf 0 10.35.0.254, via port1

If the exit interface is a physical interface rather than the IPsec tunnel, Phase 1 or Phase 2 may be down, or the routing configuration may require further investigation. If several tunnels can reach the destination, first identify which one is failing.

2. Identify the affected tunnel and selector state. Enter the applicable VDOM and run:

get vpn ipsec tunnel summary

Example:

'to10.174.0.182' 10.174.0.182:0  selectors(total,up): 1/1  rx(pkt,err): 1921/0  tx(pkt,err): 69/2
'to10.189.0.182' 10.189.0.182:0  selectors(total,up): 1/0  rx(pkt,err): 0/0  tx(pkt,err): 0/0

Here, to10.189.0.182 has zero selectors up and should be investigated.

3. Check Phase 1 status. Run:

diagnose vpn ike gateway list name <IPsec Tunnel Name>

Example:

diagnose vpn ike gateway list name to10.189.0.182

vd: root/0
name: to10.189.0.182
version: 1
interface: port9 10
addr: 10.189.0.31:500 -> 10.189.0.182:500
created: 15s ago
IKE SA: created 1/1
IPsec SA: created 0/0

id/spi: 19576 a83334b3c66f871b/0000000000000000
direction: responder
status: connecting, state 3, started 15s ago

Focus on status:

• established means Phase 1 is active; proceed to Phase 2 checks.
• connected also indicates an active connection state.
• connecting means Phase 1 has not completed.

4. If Phase 1 is down, verify bidirectional gateway reachability. Test the remote peer from the relevant local WAN address:

execute traceroute-options source 10.189.0.31
execute traceroute 10.189.0.182

A ping can also be used where appropriate.

5. Confirm that IKE traffic is not blocked. Capture UDP port 500 and 4500 traffic to the peer:

diagnose sniffer packet any "host 10.189.0.182 and (port 500 or port 4500)" 4 0 l

Expected filter display:

interfaces=[any]
filters=[host 10.189.0.182 and (port 500 or port 4500)]

If nattraversal is enabled under Phase 1 and the FortiGate is behind NAT, capture udp port 4500. Otherwise, use udp port 500.

6. Align debug and packet-capture timestamps when correlating results. Before collecting captures, run:

diagnose debug console time enable

7. Clear any existing IKE filter before starting a new debug. This is the recommended practice:

diagnose vpn ike log filter clear

The same command applies for v7.4.0 and above.

8. Run an IKE debug using the command appropriate for the FortiOS release. For the original destination-address filter syntax, use:

diagnose debug reset
diagnose vpn ike log filter dst-addr4 10.189.0.182
diagnose debug application ike -1
diagnose debug console timestamp enable
diagnose debug enable

For v7.4.0 and above, use:

diagnose debug reset
diagnose vpn ike log filter rem-addr4 10.189.0.182
diagnose debug application ike -1
diagnose debug console timestamp enable
diagnose debug enable

Starting from v7.4.1, diagnose vpn ike log-filter dst-addr4 changed to diagnose vpn ike log filter rem-addr4. Also starting from v7.4.1, diagnose vpn ike log-filter src-addr4 changed to diagnose vpn ike log filter loc-addr4.

The following sequence can also be used:

diagnose vpn ike log filter rem-addr4 10.189.0.182
diagnose debug application ike -1
diagnose debug console timestamp enable
diagnose debug enable

Available filter concepts include:

• rem-addr4: IPv4 remote gateway address range.
• msrc-addr4: multiple IPv4 source addresses.
• mdst-addr4: multiple IPv4 destination addresses.
• msrc-addr6: multiple IPv6 source addresses.
• mdst-addr6: multiple IPv6 destination addresses.
• mrem-addr4: multiple IPv4 remote gateway addresses.
• name: Phase 1 name, useful when the remote peer IP is unknown.

Display all supported options with:

diagnose vpn ike log filter ?

Example output:

list                   Display the current filter.
clear                  Erase the current filter.
vd                     Index of virtual domain. -1 matches all.
name                   Phase1 name to filter by.
ifindex                Index of the interface that IKE connection is negotiated over.
loc-addr4              IPv4 local gateway address range to filter by.
mloc-addr4             Multiple IPv4 local gateway address to filter by.
rem-addr4              IPv4 remote gateway address range to filter by.
mrem-addr4             Multiple IPv4 remote gateway address to filter by.
loc-addr6              IPv6 local gateway address range to filter by.
mloc-addr6             Multiple IPv6 local gateway address to filter by.
rem-addr6              IPv6 remote gateway address range to filter by.
mrem-addr6             Multiple IPv6 remote gateway addresses to filter by.
dst-port               Destination port range to filter by.
negate                 Negate existing setting of the specified filter parameter.

In v7.6.3, the same IKE investigation can be initiated through the GUI by using the CLI diagnostics command.

9. Stop all debugging after collecting the required output. Run:

diagnose debug disable
diagnose debug reset

10. Collect authentication debugging when relevant. For remote-user authentication issues, run:

diagnose debug application fnbamd -1

For SAML authentication issues, run:

diagnose debug application samld -1
diagnose debug application eap_proxy -1

Stop these debugs with:

diagnose debug disable
diagnose debug reset

11. If Phase 1 is established, inspect Phase 2. Run:

diagnose vpn tunnel list name <phase1-name>

Example:

diagnose vpn tunnel list name to10.189.0.182
list all ipsec tunnel in vd 0
name=to10.189.0.182 ver=1 serial=2 10.189.0.31:0->10.189.0.182:0
bound_if=10 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/8 options[0008]=npu
proxyid_num=1 child_num=0 refcnt=10 ilast=25 olast=25 ad=/0
stat: rxp=0 txp=0 rxb=0 txb=0
dpd: mode=on-demand on=0 idle=20000ms retry=3 count=0 seqno=534
natt: mode=none draft=0 interval=0 remote_port=0
proxyid=to10.189.0.182 proto=0 sa=0 ref=1 serial=4
src: 0:172.16.170.0/255.255.255.0:0
dst: 0:192.168.50.0/255.255.255.0:0

Interpret the sa value as follows:

• sa=0: the selectors may not match, or no traffic has initiated the SA.
• sa=1: the IPsec SA matches and traffic exists between the selectors.
• sa=2: transient state visible during IPsec SA rekeying.

Phase 2 encryption and hashing algorithms can also be mismatched. View selector details with:

get vpn ipsec tunnel details

Use the IKE debugging procedure above to identify negotiation mismatches.

12. For a dial-up VPN, display all Phase 1 connection instances. Run:

diagnose vpn tunnel dialup-list <phase1-name>

13. When the interface is green but traffic does not pass, capture the affected traffic. Example:

FortiGate-61F # diagnose sniffer packet any 'host 10.1.1.37 and icmp' 4 0 l
interfaces=[any]
filters=[host 10.1.1.37 and icmp]
2024-07-16 10:46:01.327162 internal in 192.168.1.251 -> 10.1.1.37: icmp: echo request
2024-07-16 10:46:02.331446 internal in 192.168.1.251 -> 10.1.1.37: icmp: echo request
2024-07-16 10:46:03.335546 internal in 192.168.1.251 -> 10.1.1.37: icmp: echo request

A flow debug may show route resolution without a visible drop, for example:

2024-07-16 10:47:54 id=20085 trace_id=4 func=iprope_dnat_check line=5191 msg="in-[internal], out-[]"
2024-07-16 10:47:54 id=20085 trace_id=4 func=iprope_dnat_check line=5204 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000"
2024-07-16 10:47:54 id=20085 trace_id=4 func=vf_ip_route_input_common line=2615 msg="find a route: flag=00000000 gw-10.1.1.37 via root"
2024-07-16 10:47:59 id=20085 trace_id=5 func=print_pkt_detail line=5746 msg="vd-root:0 received a packet(proto=1, 192.168.1.251:1->10.1.1.37:2048) from internal. type
=8, code=0, id=1, seq=288."

14. Examine detailed tunnel counters and negotiated parameters. Example:

diagnose vpn tunnel list name Primary Tunnel

name=Primary Tunnel ver=1 serial=2 10.101.1.1:0->10.101.1.2:0 tun_id=10.101.1.2 dst_mtu=1500 dpd-link=on remote_location=0.0.0.0 weight=1
bound_if=5 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/520 options[0208]=npu frag-rfc  run_state=0 accept_traffic=1 overlay_id=0
proxyid_num=1 child_num=0 refcnt=4 ilast=0 olast=0 ad=/0
stat: rxp=11746 txp=32 rxb=1939048 txb=9861
dpd: mode=on-demand on=1 idle=10000ms retry=3 count=0 seqno=0
natt: mode=none draft=0 interval=0 remote_port=0
proxyid=Net-10.1.1.0-24 proto=0 sa=1 ref=2 serial=5
  src: 0:192.168.1.0/255.255.255.0:0
  dst: 0:10.1.1.0/255.255.255.0:0
  SA:  ref=4 options=10226 type=00 soft=0 mtu=1438 expire=42832/0B replaywin=2048
       seqno=1 esn=0 replaywin_lastseq=000001db itn=0 qat=0 hash_search_len=1
  life: type=01 bytes=0/0 timeout=42933/43200
  dec: spi=e6bb099b esp=aes key=16 4c3dd75b85f0be4c2d2b95e5dfe7e99d
       ah=sha1 key=20 f9a5dbc8360e4b58aff64e74fe40137cd89dc98e
  enc: spi=533e1c0c esp=aes key=16 4fddb780c9606c5c235737bd7d05c585
       ah=sha1 key=20 57cc8349171a97eaf230e565bb90853cefaca7ab
  dec:pkts/bytes=476/75272, enc:pkts/bytes=0

This output identifies source 10.101.1.1, destination and tunnel ID 10.101.1.2, bound interface 5, auto mode, no encapsulation, NPU and RFC fragmentation options, and run_state=0, indicating that the tunnel is not actively passing traffic. Statistics show 11746 received packets, 32 transmitted packets, 1939048 received bytes, and 9861 transmitted bytes. DPD is on-demand with a 10000ms idle interval and three retries. NAT traversal is disabled. Proxy ID Net-10.1.1.0-24 covers 192.168.1.0/24 to 10.1.1.0/24. The SA has MTU 1438, expiry 42832 seconds, and replay window 2048. ESP uses AES with a 16-byte key, while AH uses SHA1 with a 20-byte key.

Use the enc and dec counters as a quick bidirectional check. Increasing enc with unchanged dec means the FortiGate is transmitting but not receiving traffic from the peer. In this example, dec:pkts/bytes=476/75272 while enc:pkts/bytes=0, showing received/decrypted traffic but no encrypted outbound packets for this SA.

15. Inspect link-monitor health checks. A link monitor can mark the Primary Tunnel interface dead when its peer is unreachable, even while other VPN information appears valid. In the documented scenario, disabling the applicable link-monitor settings caused the tunnel interface to become active. Confirm that the monitored target is reachable and that the health check is appropriate before changing the configuration.

16. Disable NPU offloading temporarily during troubleshooting. NPU offloading is enabled by default and can prevent packets processed by the NPU from appearing in flow-filter output. Check its status with:

config vpn ipsec phase1-interface
    edit <name of the tunnel>
        show full | grep npu

Disable it with:

config vpn ipsec phase1-interface
    edit <name of the tunnel>
         set npu-offload disable
    next
end

Disabling NPU offloading resets the IPsec tunnel.

17. Collect additional detailed status for TAC when needed. Run:

diagnose vpn ike status detailed

18. Flush tunnel state when renegotiation is required. Available commands are:

diagnose vpn tunnel flush <my-phase1-name>
diagnose vpn ike gateway clear name <my-phase1-name>
diagnose vpn ike gateway flush name <my-phase1-name>

19. Resolve unknown SPI events by flushing the named tunnel. Unknown SPI messages mean FortiGate received ESP packets whose SPIs do not correspond to an active IPsec tunnel. Run:

diagnose vpn tunnel flush name <tunnel_name>

This forces renegotiation and SPI synchronization. The flush interrupts VPN traffic. To reduce recurrence, verify matching Phase 2 SA lifetimes and enable Dead Peer Detection.

Repeat the routing lookup and confirm that the IPsec tunnel is selected as the exit interface. Then verify:

get vpn ipsec tunnel summary

The expected selector count should show the required selectors up. Confirm Phase 1 with:

diagnose vpn ike gateway list name <IPsec Tunnel Name>

The status should be established or connected. Confirm Phase 2 with:

diagnose vpn tunnel list name <phase1-name>

The relevant proxy ID should normally show sa=1. Generate traffic between the configured source and destination selectors and confirm that the appropriate enc and dec packet/byte counters increase. If a link monitor is used, confirm its peer is reachable and the tunnel no longer reports a dead state. Packet captures should show traffic entering and leaving through the expected interfaces.

If NPU offloading was disabled only for troubleshooting, restore its prior state after data collection. The source does not provide the explicit re-enable command. Be aware that changing the NPU-offload option resets the IPsec tunnel. If link-monitor settings were changed, restore the prior settings if they are required after correcting the monitored-peer reachability issue.