Many stateful firewalls and other middleware appliances require that connections remain active or they will be removed from their state tables. The timers in these appliances are typically on the order of 5 to 15 minutes. In an effort to prevent this many sites have turned to the TCP keep-alive feature, reducing the system default from 2 hours to 5 or 10 minutes. This however works only if the application has set the SO_KEEPALIVE socket option.
It is possible to fool the middleware appliance by sending a keep-alive message that appears to come from the connection that is idle but that you don't want to time out. All that is necessary is to send a TCP segment with the correct source and destination IP addresses and the correct source and destination port and sequence numbers.
The command 3rd_party_keep_alive.pm will do this. The command takes 7 arguments
---------------------------------- keep_alive -------------------------------- -source_address: -destination_address: -source_port: 0 -destination_port: 0 -sequence_number: 0 -acknowledgment_number: 0 -verbose: 0 |
Where:
source_address
This is the source IP address in hex. For example 10.1.1.1 would be 0a010101. The hex format is used because that is the format that the dump_onetcb analyze_system request uses to display IP addresses.
destination_address
This is the destination IP address also in hex.
source_port
This is the source port number in decimal.
destination_port
This is the destination port number in decimal.
sequence_number
This is the current sequence number that the source of the connection is using in decimal.
acknowledgment_number
This is the current sequence number that the destination of the connection is using in decimal.
verbose
A value of 1 will print lines showing the arguments, See example 1.
Note that while the remote host should send an acknowledgment in response to the segment sent by this command, the command does NOT listen for it. This command will not diagnose a remote connection that has failed, its only function is to send a TCP segment that will fool middleware appliances into thinking that the connection is not idle. This may have the effect of maintaining a connection in the middleware appliance that has actually failed.
Typing in the IP addresses and port numbers is not hard but determining the sequence numbers is a little more cumbersome. The 3rd_party_keep_alive command macro makes things simpler; it takes only 3 arguments
----------------------------- 3rd_party_keep_alive --------------------------- PCB: SLEEP: 0 verbose: 0 |
Where:
PCB
This is the address of the Protocol Control Block (PCB) for the idle but not to be timed out connection. It can be found by executing the "netstat -PCB_addr" command, see example 1.
SLEEP
If set to 0 this command macro will execute once and terminate (example 1). If set to some positive number the command will sleep for that indicated number of minutes and then execute again. Basically sending a keep-alive segment every N minutes (example 2). When executing the command macro in a loop it will run forever or until it determines that the socket has been closed. It does that by checking that the socket state is ESTABLISHED and by comparing the IP addresses and port numbers of each iteration with the IP addresses and port numbers it recorded on the first iteration, see examples 3 and 4.
verbose
If set to a positive value this command macro will display the 3rd_party_keep_alive.pm command line that it will execute. If set to 0 it will not print the line. The analyze_system command which is executed will print out its messages regardless of the value of this argument. This argument is also passed to the 3rd_party_keep_alive.pm, see example 2.
This example shows the netstat command that will display the PCB address and an execution of the 3rd_party_keep_alive command macro. The first argument is the PCB address. The SLEEP argument is 0 so the command is only executed once. The verbose argument is set to 1 so you can see a display of the 3rd_party_keep_alive.pm command line and the informational line that the PM writes. Finally you see the output from the Wireshark tshark command showing first a packet from a client to a server and then from the server to the client. Then the server sending a packet that tshark recognizes as a TCP Keep-Alive and the client sending back a TCP Keep-Alive ACK
netstat -numeric -PCB_addr Active connections PCB Proto Recv-Q Send-Q Local Address Foreign Address (state) . . . . . 8abd8f40 tcp 0 0 164.152.77.217:9999 164.152.77.50:49053 ESTABLISH +ED . . . . . 3rd_party_keep_alive 8abd8f40 0 1 OpenVOS Release 17.1.1aa, analyze_system Release 17.1.1aa Current process is 1424, ptep 8BF8BA80, Noah_Davids.CAC as: as: as: as: as: as: as: as: 3rd_party_keep_alive.pm -source_address A4984DD9 -destination_address A4984D32 - +source_port 9999 -destination_port 49053 -sequence_number 3616216802 -acknowled +gment_number 2827825928 -verbose 1 Sending keep_alive to a4984d32,49053 from a4984dd9,9999 with sequence number 3616216801 (d78b16e1) and acknowledgment number 2827825928 +(a88d3308) C:\Windows\system32>"C:\Program Files\Wireshark\tshark" -i 2 -R "tcp.port == 9999" Capturing on Realtek PCIe GBE Family Controller 18.090902 164.152.77.50 -> 164.152.77.217 TCP 56 49053 > 9999 [PSH, ACK] Seq=1 Ack=1 Win=64240 Len=2 18.315854 164.152.77.217 -> 164.152.77.50 TCP 60 9999 > 49053 [ACK] Seq=1 Ack=3 Win=8192 Len=0 214.969532 164.152.77.217 -> 164.152.77.50 TCP 60 [TCP Keep-Alive] 9999 > 49053 [ACK] Seq=0 Ack=3 Win=128 Len=1 214.969551 164.152.77.50 -> 164.152.77.217 TCP 66 [TCP Keep-Alive ACK] 49053 > 9999 [ACK] Seq=3 Ack=1 Win=64240 Len=0 |
In this example I execute the 3rd_party_keep_alive.cm with the PCB address and a SLEEP value of 1. The example shows the initial execution and 3 iterations. The tshark output shows the keep-alive segments, the timestamp is from the start of the trace but you can see that the keep-alive segments, not counting the first one, are about 60 seconds apart. Since the verbose argument defaults to 0 the information lines from the command macro and the command are suppressed but the analyze_system output is still there.
3rd_party_keep_alive 8abd8f40 1 OpenVOS Release 17.1.1aa, analyze_system Release 17.1.1aa Current process is 1424, ptep 8BF8BA80, Noah_Davids.CAC as: as: as: as: as: as: as: as: OpenVOS Release 17.1.1aa, analyze_system + Release 17.1.1aa Current process is 1424, ptep 8BF8BA80, Noah_Davids.CAC as: as: as: as: as: as: as: as: OpenVOS Release 17.1.1aa, analyze_system + Release 17.1.1aa Current process is 1424, ptep 8BF8BA80, Noah_Davids.CAC as: as: as: as: as: as: as: as: OpenVOS Release 17.1.1aa, analyze_system + Release 17.1.1aa Current process is 1424, ptep 8BF8BA80, Noah_Davids.CAC as: as: as: as: as: as: as: as: OpenVOS Release 17.1.1aa, analyze_system + Release 17.1.1aa Current process is 1424, ptep 8BF8BA80, Noah_Davids.CAC as: as: as: as: as: as: as: as: C:\Windows\system32>"C:\Program Files\Wireshark\tshark" -i 2 -R "tcp.port == 9999" Capturing on Realtek PCIe GBE Family Controller 4.671048 164.152.77.50 -> 164.152.77.217 TCP 56 49053 > 9999 [PSH, ACK] Seq=1 Ack=1 Win=64240 [TCP CHECKSUM INCORRECT] Len=2 4.890341 164.152.77.217 -> 164.152.77.50 TCP 60 9999 > 49053 [ACK] Seq=1 Ack=3 Win=65535 Len=0 14.847359 164.152.77.217 -> 164.152.77.50 TCP 60 [TCP Keep-Alive] 9999 > 49053 [ACK] Seq=0 Ack=3 Win=128 Len=1 14.847377 164.152.77.50 -> 164.152.77.217 TCP 66 [TCP Keep-Alive ACK] 49053 > 9999 [ACK] Seq=3 Ack=1 Win=64240 Len=0 75.064484 164.152.77.217 -> 164.152.77.50 TCP 60 [TCP Keep-Alive] 9999 > 49053 [ACK] Seq=0 Ack=3 Win=128 Len=1 75.064503 164.152.77.50 -> 164.152.77.217 TCP 66 [TCP Keep-Alive ACK] 49053 > 9999 [ACK] Seq=3 Ack=1 Win=64240 Len=0 135.237528 164.152.77.217 -> 164.152.77.50 TCP 60 [TCP Keep-Alive] 9999 > 49053 [ACK] Seq=0 Ack=3 Win=128 Len=1 135.237546 164.152.77.50 -> 164.152.77.217 TCP 66 [TCP Keep-Alive ACK] 49053 > 9999 [ACK] Seq=3 Ack=1 Win=64240 Len=0 195.412330 164.152.77.217 -> 164.152.77.50 TCP 60 [TCP Keep-Alive] 9999 > 49053 [ACK] Seq=0 Ack=3 Win=128 Len=1 195.412346 164.152.77.50 -> 164.152.77.217 TCP 66 [TCP Keep-Alive ACK] 49053 > 9999 [ACK] Seq=3 Ack=1 Win=64240 Len=0 |
This example shows a few iterations followed by the message that indicates that the socket has been closed.
3rd_party_keep_alive 8abd8f40 1
OpenVOS Release 17.1.1aa, analyze_system Release 17.1.1aa
Current process is 1424, ptep 8BF8BA80, Noah_Davids.CAC
as: as: as: as: as: as: as: as: OpenVOS Release 17.1.1aa, analyze_system
+ Release 17.1.1aa
Current process is 1424, ptep 8BF8BA80, Noah_Davids.CAC
as: as: as: as: as: as: as: as: OpenVOS Release 17.1.1aa, analyze_system
+ Release 17.1.1aa
Current process is 1424, ptep 8BF8BA80, Noah_Davids.CAC
as: as: as: as: as: as: as: as: OpenVOS Release 17.1.1aa, analyze_system
+ Release 17.1.1aa
Current process is 1424, ptep 8BF8BA80, Noah_Davids.CAC
as: as: as: as: as: as: as: as: OpenVOS Release 17.1.1aa, analyze_system
+ Release 17.1.1aa
Current process is 1424, ptep 8BF8BA80, Noah_Davids.CAC
as: as: as: as: as: as: as: as: OpenVOS Release 17.1.1aa, analyze_system
+ Release 17.1.1aa
Current process is 1424, ptep 8BF8BA80, Noah_Davids.CAC
as: as: as: as: as: as: as: as:
socket is not in the ESTABLISHED state it is in 'TS_CLOSED'
exiting . . .
ready 09:57:28
|
This example shows the initial execution, one successful iteration and then the message that indicates that the socket has been closed and then reused so that the IP addresses and/or port numbers do not match. Note that the SLEEP time is now 5 minutes to give me enough time to close the socket and create a new one. I can't choose what the socket's PCB address is but STCP seems to reuse the address space once the socket is closed. This condition is not hard to create.
3rd_party_keep_alive 95aa9440 5
OpenVOS Release 17.0.2au, analyze_system Release 17.0.2au
Current process is 239, ptep 95AA3400, Noah_Davids.CAC
as: as: as: as: as: as: as: as: OpenVOS Release 17.0.2au, analyze_system
+ Release 17.0.2au
Current process is 239, ptep 95AA3400, Noah_Davids.CAC
as: as: as: as: as: as: as: as: OpenVOS Release 17.0.2au, analyze_system
+ Release 17.0.2au
Current process is 239, ptep 95AA3400, Noah_Davids.CAC
as: as: as: as: as: as: as: as:
It appears that the original socket has been closed
A4984D80,9999->A4984D32,51261 has changed to A4984D80,9999->A4984D32,51470
exiting . . .
ready 15:09:31
|
This example shows the 3rd_party_keep_alive macro run from a started process. First there are the commands to create an out file with implicit locking set and then the start process command. A portion of the out file is also shown.
create_file 3rd_party_keep_alive-test_connection.out ready 11:57:16 set_implicit_locking 3rd_party_keep_alive-test_connection.out ready 11:57:29 start_process '3rd_party_keep_alive 95c2c1c0 5' -process_name 3rdpa-l-test_conne +ction -output_path 3rd_party_keep_alive-test_connection.out -privileged ready 11:59:30 display 3rd_party_keep_alive-test_connection.out %phx_vos#m16_mas>SysAdmin>Noah_Davids>3rd_party_keep_alive-test_connection.out Noah_Davids.CAC logged in on %phx_vos#m16 at 12-09-19 11:59:30 mst. 3rd_party_keep_alive 95c2c1c0 5 OpenVOS Release 17.0.2au, analyze_system Release 17.0.2au Current process is 348, ptep 978C0100, Noah_Davids.CAC (3rdpa-l-test_connection) as: as: as: as: as: as: as: as: ready 11:59:32 ready 11:59:32 ready 12:04:32 OpenVOS Release 17.0.2au, analyze_system Release 17.0.2au Current process is 348, ptep 978C0100, Noah_Davids.CAC (3rdpa-l-test_connection) as: as: as: as: as: as: as: as: ready 12:04:35 ready 12:04:35 ready 12:09:35 OpenVOS Release 17.0.2au, analyze_system Release 17.0.2au Current process is 348, ptep 978C0100, Noah_Davids.CAC (3rdpa-l-test_connection) as: as: as: as: as: as: as: as: ready 12:09:37 ready 12:09:38 ready 12:14:38 OpenVOS Release 17.0.2au, analyze_system Release 17.0.2au Current process is 348, ptep 978C0100, Noah_Davids.CAC (3rdpa-l-test_connection) as: as: as: as: as: as: as: as: ready 12:14:40 ready 12:14:40 ready 12:19:40 OpenVOS Release 17.0.2au, analyze_system Release 17.0.2au Current process is 348, ptep 978C0100, Noah_Davids.CAC (3rdpa-l-test_connection) as: as: . . . . |
& 3rd_party_keep_alive begins here & & Version 1.00 12-09-13 & noah.davids@stratus.com & & This command macro uses a socket's PCB address (input PCB argument) to & extract the local and foreign IP addresses, port numbers and sequence & numbers. It then calls the 3rd_part_keep_alive.pm using those values. & Next it sleeps for the requested number of minutes (input SLEEP & argument) and then does it all again. A SLEEP value less than 1 indicates & that the macro should exit without looping. If the VERBOSE value (input & VERBOSE) is > 0 the macro will display the command line being executed & and the command will display its arguments. & & On each iteration the socket state is extracted and if it is not TS_ESTAB & the macro reports an error and exits. There is no point sending a keep & alive segment if the socket is not in an ESTABLISHED state. Also on each & iteration it compares the IP address and port numbers to those obtained on & the first iteration, if they are not the same it assumes that the original & socket has been closed, reports an error and exits. It needs to do this & because it is possible for the socket address space to be reused after the & original socket is closed. & & This software is provided on an "AS IS" basis, WITHOUT ANY WARRANTY OR ANY & SUPPORT OF ANY KIND. The AUTHOR SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES & OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. This disclaimer & applies, despite any verbal representations of any kind provided by the & author or anyone else. & &begin_parameters PCB PCB:string,req SLEEP SLEEP:number,=0 VERBOSE verbose:number,=0 &end_parameters & &set_string FIRST '' &attach_input & & Run dump_onetcb to get the IP addresses and port and sequence numbers and & the socket state. If the PCB is not valid you can get some analyze_system & errors - no big deal. &label again &echo no_input_lines no_command_lines no_macro_lines analyze_system ..attach_default_output (process_dir)>lport match lport; dump_onetcb &PCB& ..detach_default_output ..attach_default_output (process_dir)>fport match fport; dump_onetcb &PCB& ..detach_default_output ..attach_default_output (process_dir)>laddr match 'laddr '; dump_onetcb &PCB& ..detach_default_output ..attach_default_output (process_dir)>faddr match 'faddr '; dump_onetcb &PCB& ..detach_default_output ..attach_default_output (process_dir)>rcvseq match rcvseq; dump_onetcb &PCB& ..detach_default_output ..attach_default_output (process_dir)>sndseq match sndseq; dump_onetcb &PCB& ..detach_default_output ..attach_default_output (process_dir)>state match tcb_state; dump_onetcb &PCB& ..detach_default_output quit & & check the state to make sure the PCB refers to an established socket. I & am appending an X to the state just in case the captured state value is & blank. That way the macro doesn't get a syntax error and fault. It reports & the state is wrong and exists. I start at position 35 which is to the left & of the actual value but the leading spaces will not not. This way the & macro works if for some reason the analyze_system prompt is not quite what & is expected. &set_string STATE (rtrim (substr (contents (process_dir)>state 1) 35)) &if (string &STATE&X) ^= TS_ESTABX &then &do display_line display_line socket is not in the ESTABLISHED state it is in (quote &STATE&) display_line exiting . . . &return &end & & Load up the IP address and port and sequence number variables. &set_string LPORT (substr (contents (process_dir)>lport 1) 35) &set_string FPORT (substr (contents (process_dir)>fport 1) 35) &set_string LADDR (substr (contents (process_dir)>laddr 1) 35) &set_string FADDR (substr (contents (process_dir)>faddr 1) 35) &set_string RCVSEQ (substr (contents (process_dir)>rcvseq 1) 35) &set_string SNDSEQ (substr (contents (process_dir)>sndseq 1) 35) & & The TEMP string holds all the values for easy comparison and to use in an & error message. &set_string TEMP &+ (rtrim &LADDR&),(ltrim &LPORT&)->(rtrim &FADDR&),(ltrim &FPORT&) & & If this is the first iteration, record the TEMP string as FIRST. If it is & not the first iteration compare the TEMP string to FIRST and if they are & not the same report an error. It is possible that the original socket was & closed and a new one created using the same PCB address. If the client is & binding to a fixed port number then all these values will be the same. But & if that is the case you probably want to continue running the keep alives & anyway. &if &FIRST&XXX = 'XXX' &then &set_string FIRST &TEMP& &else &if &FIRST& ^= &TEMP& &then &do display_line display_line It appears that the original socket has been closed display_line &FIRST& has changed to &TEMP& display_line exiting . . . &return &end & & Build the command string to execute the 3rd_party_keep_alive PM &set_string COMMAND 3rd_party_keep_alive.pm -source_address &LADDR& &+ -destination_address &FADDR& -source_port &LPORT& &+ -destination_port &FPORT& -sequence_number &SNDSEQ& &+ -acknowledgment_number &RCVSEQ& -verbose &VERBOSE& &if &VERBOSE& &then &do display_line display_line &COMMAND& &end &COMMAND& & &if &SLEEP& > 0 &then &do sleep -minutes &SLEEP& &goto again &end & & 3rd_party_keep_alive ends here |
/* 3rd_party_keep_alive.c begins here Version 1.00 12-09-13 noah.davids@stratus.com see http://noahdavids.org/self_published/3rd. . . . This software is provided on an "AS IS" basis, WITHOUT ANY WARRANTY OR ANY SUPPORT OF ANY KIND. The AUTHOR SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. This disclaimer applies, despite any verbal representations of any kind provided by the author or anyone else. */ #define _POSIX_C_SOURCE 200112L #define bzero(s, len) memset((char *)(s), 0, len) #include |