Interesting Traces - Your Protocol Analyzer's Expert Analysis may not be

This trace shows how the "expert" analysis of protocol analyzers can get confused. In this case trace 1 shows the contents of the original summary window. In an attempt to make the traces fit better on the screen I've removed some of the fields, like window and length which aren't pertinent to the discussion and compressed some of the white space between the columns. Note the retransmissions in frames 15, 16, and 17. The entire 1806 frame trace has 318 retransmissions - over 17%. The trace was taken in an attempt to understand the cause of the resets and the initial thought after a quick look at the trace was too many retransmissions.

Trace 1

No. Source        Destination   Protocol Info
  1 10.1.1.57     10.1.1.151    TELNET   Telnet Data ...
  2 10.1.1.151    10.1.1.57     TELNET   Telnet Data ...
  3 10.1.1.57     10.1.1.151    TCP      9000 > telnet [ACK] Seq=1 Ack=2
  5 10.1.1.57     10.1.1.151    TELNET   Telnet Data ...
  6 10.1.1.151    10.1.1.57     TELNET   Telnet Data ...
  7 10.1.1.57     10.1.1.151    TCP      9000 > telnet [ACK] Seq=2 Ack=4
  9 10.1.1.57     10.1.1.151    TCP      9000 > telnet [SYN] Seq=3486902455
 11 10.1.1.57     10.1.1.151    TCP      9000 > telnet [RST] Seq=2
 12 10.1.1.57     10.1.1.151    TCP      9000 > telnet [SYN] Seq=3486902455
 13 10.1.1.151    10.1.1.57     TCP      telnet > 9000 [SYN, ACK] Seq=2496060360 Ack=3486902456                        
 14 10.1.1.57     10.1.1.151    TCP      9000 > telnet [ACK] Seq=3486902456 Ack=2496060361
 15 10.1.1.151    10.1.1.57     TELNET   [TCP Retransmission] Telnet Data ... 
 16 10.1.1.151    10.1.1.57     TELNET   [TCP Retransmission] Telnet Data ... 
 17 10.1.1.151    10.1.1.57     TELNET   [TCP Retransmission] Telnet Data ... 
 18 10.1.1.57     10.1.1.151    TCP      9000 > telnet [ACK] Seq=3486902456 Ack=2496060443
 19 10.1.1.57     10.1.1.151    TCP      9000 > telnet [RST] Seq=3486902456

However a closer look shows some anomolies. First there do not appear to be enough retransmissions to cause the resets, second some of the sequence numbers appear to be relative and some do not. The retransmissions are telnet packets so the sequence numbers aren't even displayed.

By configuring the analyzer to not interpret the telnet protocol you can get the sequence numbers for packets 15, 16, 17 (trace 2). Note that the sequence numbers displayed appear to be in 2 ranges.

Trace 2

No. Source        Destination   Protocol Info
  1 10.1.1.57     10.1.1.151    TCP      9000 > telnet [PSH, ACK] Seq=0 Ack=0
  2 10.1.1.151    10.1.1.57     TCP      telnet > 9000 [PSH, ACK] Seq=0 Ack=1
  3 10.1.1.57     10.1.1.151    TCP      9000 > telnet [ACK] Seq=1 Ack=2
  5 10.1.1.57     10.1.1.151    TCP      9000 > telnet [PSH, ACK] Seq=1 Ack=2
  6 10.1.1.151    10.1.1.57     TCP      telnet > 9000 [PSH, ACK] Seq=2 Ack=2
  7 10.1.1.57     10.1.1.151    TCP      9000 > telnet [ACK] Seq=2 Ack=4
  9 10.1.1.57     10.1.1.151    TCP      9000 > telnet [SYN] Seq=3486902455
 11 10.1.1.57     10.1.1.151    TCP      9000 > telnet [RST] Seq=2
 12 10.1.1.57     10.1.1.151    TCP      9000 > telnet [SYN] Seq=3486902455
 13 10.1.1.151    10.1.1.57     TCP      telnet > 9000 [SYN, ACK] Seq=2496060360 Ack=3486902456
 14 10.1.1.57     10.1.1.151    TCP      9000 > telnet [ACK] Seq=3486902456 Ack=2496060361
 15 10.1.1.151    10.1.1.57     TCP      [TCP Retransmission] telnet > 9000 [PSH, ACK] Seq=2496060361 Ack=3486902456
 16 10.1.1.151    10.1.1.57     TCP      [TCP Retransmission] telnet > 9000 [PSH, ACK] Seq=2496060370 Ack=3486902456
 17 10.1.1.151    10.1.1.57     TCP      [TCP Retransmission] telnet > 9000 [PSH, ACK] Seq=2496060419 Ack=3486902456
 18 10.1.1.57     10.1.1.151    TCP      9000 > telnet [ACK] Seq=3486902456 Ack=2496060443                             
 19 10.1.1.57     10.1.1.151    TCP      9000 > telnet [RST] Seq=3486902456

The next step is to configure the analyzer to display the full sequence number not relative numbers (trace 3). When you do that you see that the first sequence number seen from 10.1.1.57 is 1,927,822,363. The sequence number starting in frame 15 is 128,915,428. Since it is smaller the analyzer is flagging it as a retransmission. The fact that the reset in frame 11 has closed the old connection and a new connection has been started in frame 12 doesn't seem to count.

Trace 3

No. Source        Destination   Protocol Info
  1 10.1.1.57     10.1.1.151    TCP      9000 > telnet [PSH, ACK] Seq=891790930 Ack=1927822363
  2 10.1.1.151    10.1.1.57     TCP      telnet > 9000 [PSH, ACK] Seq=1927822363 Ack=891790931
  3 10.1.1.57     10.1.1.151    TCP      9000 > telnet [ACK] Seq=891790931 Ack=1927822365
  5 10.1.1.57     10.1.1.151    TCP      9000 > telnet [PSH, ACK] Seq=891790931 Ack=1927822365
  6 10.1.1.151    10.1.1.57     TCP      telnet > 9000 [PSH, ACK] Seq=1927822365 Ack=891790932
  7 10.1.1.57     10.1.1.151    TCP      9000 > telnet [ACK] Seq=891790932 Ack=1927822367
  9 10.1.1.57     10.1.1.151    TCP      9000 > telnet [SYN] Seq=83726089
 11 10.1.1.57     10.1.1.151    TCP      9000 > telnet [RST] Seq=891790932
 12 10.1.1.57     10.1.1.151    TCP      9000 > telnet [SYN] Seq=83726089
 13 10.1.1.151    10.1.1.57     TCP      telnet > 9000 [SYN, ACK] Seq=128915427 Ack=83726090
 14 10.1.1.57     10.1.1.151    TCP      9000 > telnet [ACK] Seq=83726090 Ack=128915428
 15 10.1.1.151    10.1.1.57     TCP      [TCP Retransmission] telnet > 9000 [PSH, ACK] Seq=128915428 Ack=83726090
 16 10.1.1.151    10.1.1.57     TCP      [TCP Retransmission] telnet > 9000 [PSH, ACK] Seq=128915437 Ack=83726090
 17 10.1.1.151    10.1.1.57     TCP      [TCP Retransmission] telnet > 9000 [PSH, ACK] Seq=128915486 Ack=83726090      
 18 10.1.1.57     10.1.1.151    TCP      9000 > telnet [ACK] Seq=83726090 Ack=128915510
 19 10.1.1.57     10.1.1.151    TCP      9000 > telnet [RST] Seq=83726090

Why is the analyzer confused? The telnet client appears to always use port 9000 and since telnet is port 23 the 4-tuple defining the connection (<IP-1, port-1, IP-2, port-2>) is always the same. It appears to ignore the reset and new connection.

The bottom line is that analysis provided by protocol analyzers is useful but not foolproof; it always pays to analyze the analysis to make sure you understand it and agree with it.

Why was the client resetting the connection? As of this writing we still do not know. It appears to be some interaction with the telnet options sent at the start of the connection but it is still being investigated.

This page was last modified on 07-12-03

Send comments and suggestions
to ndav1@cox.net