Interesting Traces - no FTP banner

Blue Bar separator
The following shows how important it is to always look at all packets and not just filter on what you know is the problem. The following trace is from the Stratus packet_monitor. It shows a client making a connection to an FTP server running on the Stratus system. The connection seems to work, at least the TCP 3-way handshake completed, but the FTP server never sends a banner. Because it never sends a banner the client software doesn’t prompt for a user name so it never sends a user name. After a few minutes the client process kills the connection because it has been idle. The trace does show the Fin packet being received and the server sending a “You could at least say goodbye” message. From this we can conclude that the connection was correctly set up and working so why was the banner not sent.
hh:mm:ss.ttt dir   len proto source             destination         src port  d
+st port  type
16:06:23.706 Rcvd IP   Ver/HL 45, ToS  0, Len   3c, ID 26b2, Flg/Frg    0, TTL
+3e,  Prtl  6
          Cksum  4931, Src c0a8821d, Dst ac101e03
TCP from 192.168.130.29.1068 to 172.16.30.3.ftp
    seq  7bd1bec2, ack     n.a., window 8000, 20. data bytes, flags Syn.
    X/Off 0a, Flags 02, Cksum 51e0,  Urg-> 0000
     offset 0  .  .  .  4  .  .  .   8  .  .  .  C  .  .  .  0...4... 8...C...
      0     2  4  5 64  1  3  3  0   1  1  8  a 8c c2 a1  6  <<B!<
     10     0  0  0  0

16:06:23.712 Xmit IP   Ver/HL 45, ToS  0, Len   28, ID c339, Flg/Frg    0, TTL
+40,  Prtl  6
          Cksum  aabd, Src ac101e03, Dst c0a8821d
TCP from 172.16.30.3.ftp to 192.168.130.29.1068
    seq  3b015a9d, ack 7bd1bec3, window 2ccc, 0. data bytes, flags Syn Ack.
    X/Off 05, Flags 12, Cksum a1b8,  Urg-> 0000
No tcp data.

16:06:23.725 Rcvd IP   Ver/HL 45, ToS  0, Len   28, ID 26b3, Flg/Frg    0, TTL
+3e,  Prtl  6
          Cksum  4944, Src c0a8821d, Dst ac101e03
TCP from 192.168.130.29.1068 to 172.16.30.3.ftp
    seq  7bd1bec3, ack 3b015a9e, window 8000, 0. data bytes, flags Ack.
    X/Off 05, Flags 10, Cksum 4e85,  Urg-> 0000
No tcp data.

16:07:23.728 Rcvd IP   Ver/HL 45, ToS  0, Len   28, ID 26bd, Flg/Frg    0, TTL
+3e,  Prtl  6
          Cksum  493a, Src c0a8821d, Dst ac101e03
TCP from 192.168.130.29.1068 to 172.16.30.3.ftp
    seq  7bd1bec3, ack 3b015a9e, window 8000, 0. data bytes, flags Fin Push Ack
+.
    X/Off 05, Flags 19, Cksum 4e7c,  Urg-> 0000
No tcp data.

16:07:23.729 Xmit IP   Ver/HL 45, ToS  0, Len   28, ID fb2d, Flg/Frg    0, TTL
+40,  Prtl  6
          Cksum  72c9, Src ac101e03, Dst c0a8821d
TCP from 172.16.30.3.ftp to 192.168.130.29.1068
    seq  3b015a9e, ack 7bd1bec4, window 2ccc, 0. data bytes, flags Ack.
    X/Off 05, Flags 10, Cksum a1b8,  Urg-> 0000
No tcp data.

16:07:23.731 Xmit IP   Ver/HL 45, ToS  0, Len   4d, ID fb2f, Flg/Frg    0, TTL
+40,  Prtl  6
          Cksum  72a2, Src ac101e03, Dst c0a8821d
TCP from 172.16.30.3.ftp to 192.168.130.29.1068
    seq  3b015a9e, ack 7bd1bec4, window 2ccc, 37. data bytes, flags Push Ack.
    X/Off 05, Flags 18, Cksum b05e,  Urg-> 0000
     offset 0  .  .  .  4  .  .  .   8  .  .  .  C  .  .  .  0...4... 8...C...
      0    32 32 31 20 59 6f 75 20  63 6f 75 6c 64 20 61 74 * 221 You could at
     10    20 6c 65 61 73 74 20 73  61 79 20 67 6f 6f 64 62 *  least say goodb
     20    79 65 2e  d  a                                   * ye.<<

16:07:23.743 Rcvd IP   Ver/HL 45, ToS  0, Len   28, ID 26be, Flg/Frg    0, TTL
+3e,  Prtl  6
          Cksum  4939, Src c0a8821d, Dst ac101e03
TCP from 192.168.130.29.1068 to 172.16.30.3.ftp
    seq  7bd1bec4, ack     n.a., window n.a., 0. data bytes, flags Rst.
    X/Off 05, Flags 04, Cksum ce90,  Urg-> 0000
No tcp data.
This data was collected by another trace tool on the same subnet as 172.16.20.3. It does not show the exact same packets as the packet_monitor trace above but it does show the same sequence of packets – with 1 very important addition. The tool provided only the raw hex trace, the decode was done manually.
SYN from 192.168.130.29 port 1049 to 172.16.20.3 port 21

XXXXXXXX 820EXXXX XXXX8E60 08004500
003C7772 00003E06 0271C0A8 821DAC10
14030419 0015DBC1 0ED30000 0000A002
80002BE7 00000204 05640103 03000101
080A738E 68450000 0000

Destination		XXXXXXXX820E		target host
Source		        XXXXXXXX8E60		router
Protocol		0800			IP
Ver/Header Len          45
TOS			00
Total Length	        003C			60
ID			7772
Flags/Frag Offset	0000
TTL			3E			62
Protocol		06			TCP
CheckSum		0271
Source IP		C0A8821D		192.168.130.29
Destination IP	        AC101403		172.16.20.3
Source Port		0419			1049
Destination Port	0015			21
Sequence Number	        DBC10ED3
ACK Number		00000000
Header Length	        A			10 * 4 (bytes) = 40 bytes
Reserved		00
Flags			02			SYN
Window		        8000
Checksum		2BE7
Urgent Pointer	        0000
MSS Option 		0204 0564		MSS 1380
NOP Option		01
Window Scale	        030300			0 window scale
NOP Option		01
NOP Option		01
Timestamp Option	080A738E684500000000


SYN+ACK from 172.16.20.3 port 21 to 192.168.130.29 port 1049

XXXXXXXX 8E60XXXX XXXX820E 08004500
00286280 00004006 1577AC10 1403C0A8
821D0015 04194A43 F665DBC1 0ED45012
2CCC7EBF 00000204 05640103

Destination		XXXXXXXX8E60
Source		        XXXXXXXX820E
Protocol		0800			IP
Ver/Header Len	        45
TOS			00
Total Length	        28			40
ID			6280
Flags/Frag Offset       0000
TTL			40			64
Protocol		06			TCP
Checksum		1577
Source IP		AC101403		172.16.20.3
Destination IP          C0A8821D		192.168.130.29
Source Port		0015			21
Destination Port	0419			1049 
Sequence Number	        4A43F665
ACK Number		DBC10ED4
Header Length	        5			5 x 4 (bytes) = 20 bytes
Reserved                0                   
Flags			12			ACK + SYN                            
Window		        2CCC			11468
Checksum		7EBF
Urgent Pointer	        0000
MSS Option		02040564		MSS 1380


ACK from 192.168.130.29 port 1049 to 172.16.20.3 port 21

XXXXXXXX 820EXXXX XXXX8E60 08004500
00287773 00003E06 0284C0A8 821DAC10
14030419 0015DBC1 0ED44A43 F6665010
80002B8C 00000000 00000000

Destination		XXXXXXXX820E
Source		        XXXXXXXX8E60
Protocol		0800
Ver/Header Len	        45
TOS			00
Total Length	        28			40 bytes
ID			7773
Flags/Frag Offset	0000
TTL			3E			62
Protocol		06			TCP
Checksum		0284
Source IP		C0A8821D		192.168.130.29
Destination IP          AC101403		172.16.20.3
Source Port		0419			1049
Destination Port	0015			21
Sequence Number         DBC10ED4
ACK Number		4A43F666
Header Length	        5			5 x 4 (bytes) = 20 (bytes)
Reserved		0                   
Flags			10			ACK
Window		        8000
Checksum		2B8C
Urgent Pointer	        0000
 


ICMP Network unreachable from 200.XXX.XXX.145 to 172.16.20.3 port 21

XXXXXXXX 820EXXXX XXXX8E60 08004500
0038DE10 00003D01 1919C8XX XX91AC10
14030300 B8280000 00004500 00286280
00003D06 1877AC10 1403C0A8 821D0015
04194A43 F665

Destination		XXXXXXXX820E
Source			XXXXXXXX8E60
Protocol         	0800			IP
Ver/Header Len		45
TOS			00
Total Len		0038			56
ID			DE10
Flags/Frag Offset	0000
TTL			3D			61
Protocol		01			ICMP
CheckSum		1919
Source IP		C8XXXX91		200.XXX.XXX.145
Destination IP		AC101403		172.16.1.20.3
Type			03			Destination Unreachable
Code			00			Network Unreachable
Checksum		B828
Unused			00000000
Triggering Packet
Ver/Header Len		45
TOS			00
Total Length		28			40
ID			6280
Flags/Frag Offset 0000
TTL			40			64
Protocol		06			TCP
Checksum		1577
Source IP		AC101403		172.16.20.3
Destination IP    	C0A8821D		192.168.130.29
Source Port		0015			21
Destination Port	0419			1049
Sequence Number		4A43F665
This last packet is from a completely different system on a completely different network but it is replying to the server’s SYN-ACK packet. We know this because the embedded header information in the network unreachable packet includes the ID and sequence number of the SYN+ACK packet. This network unreachable packet caused an error when the server went to send the banner message so it was never sent. How did this happen, well that is not entirely clear. There was some form of load balancing/fault tolerant hardware/software between 172.16.20.3 and 192.168.130.29. Appearently, it was forwarding packets to two different hosts. The router for one of those hosts 200.XXX.XXX.145 did not have its routes configured properly and reported the error. Obviously the server should have tried to retransmit. Both the above issues are beside the point for this example. The problem was caused by a packet that wasn’t an FTP packet and wasn’t even a TCP packet. The initial filtering of just FTP packets caused me to miss the problem entirely.

Blue Bar separator
This page was last modified on 04-06-03
mailbox Send comments and suggestions
to ndav1@cox.net