Interesting Traces - Hung Win2K server/Connection Backlog exceeded

Blue Bar separator


The following traces (taken with Etherpeek 5.0) show what it looks like when connecting to a hung server application. In this case the server is listening on port 2080 and has a listen backlog of 5. The server is hung, i.e. it is no longer calling accept in response to an incomming connection request.

In an attempt to make each connection easier to see I've added a color attribute to each of the connections in the trace. I have also deleted most of the data packets and their ACKs to reduce the size of the trace.

What you see is that the first 5 connections from ports 4679 (packet #1), 4682 (packet #93), 4683 (packet #177), 4684 (packet #261) and 4685 (packet #347) are accepted and the clients start to send data. However, a close look at the acknowlegments shows that the window goes down to 0 which eventually forces the clients to send window probe messages (1 byte of data) which are ACKed with the previous sequence number and a zero window, indicating that the connection is active but the window is still closed. These connections are all in the backlog queue waiting to be accepted by the server. A netstat will show these connections in the established state. Connections from port 4686 (packet #439) and 4687 (packet #455) which are attempted after the backlog queue is filled are reset. 

  1,IP-192.168.1.103,IP-192.168.1.101,26:20.1,TCP,Src= 4679,Dst= 2080,....S.,S=1263115864,L=    0,A=         0,W=64240
  2,IP-192.168.1.101,IP-192.168.1.103,26:23.4,TCP,Src= 2080,Dst= 4679,.A..S.,S=1486105224,L=    0,A=1263115865,W=64240
  3,IP-192.168.1.103,IP-192.168.1.101,26:23.4,TCP,Src= 4679,Dst= 2080,.A....,S=1263115865,L=    0,A=1486105225,W=64240
  4,IP-192.168.1.103,IP-192.168.1.101,26:23.4,TCP,Src= 4679,Dst= 2080,.A....,S=1263115865,L= 1460,A=1486105225,W=64240
  5,IP-192.168.1.103,IP-192.168.1.101,26:23.4,TCP,Src= 4679,Dst= 2080,.A....,S=1263117325,L= 1460,A=1486105225,W=64240
  6,IP-192.168.1.101,IP-192.168.1.103,26:23.4,TCP,Src= 2080,Dst= 4679,.A....,S=1486105225,L=    0,A=1263118785,W=64240
 81,IP-192.168.1.103,IP-192.168.1.101,26:23.4,TCP,Src= 4679,Dst= 2080,.A....,S=1263185781,L= 1460,A=1486105225,W=64240
 82,IP-192.168.1.101,IP-192.168.1.103,26:23.4,TCP,Src= 2080,Dst= 4679,.A....,S=1486105225,L=    0,A=1263187241,W= 1624
 83,IP-192.168.1.103,IP-192.168.1.101,26:23.4,TCP,Src= 4679,Dst= 2080,.A....,S=1263187241,L= 1460,A=1486105225,W=64240
 84,IP-192.168.1.101,IP-192.168.1.103,26:23.4,TCP,Src= 2080,Dst= 4679,.A....,S=1486105225,L=    0,A=1263188701,W=  164
 85,IP-192.168.1.103,IP-192.168.1.101,26:28.4,TCP,Src= 4679,Dst= 2080,.A....,S=1263188701,L=  164,A=1486105225,W=64240
 86,IP-192.168.1.101,IP-192.168.1.103,26:28.5,TCP,Src= 2080,Dst= 4679,.A....,S=1486105225,L=    0,A=1263188865,W=    0
 87,IP-192.168.1.103,IP-192.168.1.101,26:29.3,TCP,Src= 4679,Dst= 2080,.A....,S=1263188865,L=    1,A=1486105225,W=64240
 88,IP-192.168.1.101,IP-192.168.1.103,26:29.3,TCP,Src= 2080,Dst= 4679,.A....,S=1486105225,L=    0,A=1263188865,W=    0
 89,IP-192.168.1.103,IP-192.168.1.101,26:30.8,TCP,Src= 4679,Dst= 2080,.A....,S=1263188865,L=    1,A=1486105225,W=64240
 90,IP-192.168.1.101,IP-192.168.1.103,26:30.8,TCP,Src= 2080,Dst= 4679,.A....,S=1486105225,L=    0,A=1263188865,W=    0
 91,IP-192.168.1.103,IP-192.168.1.101,26:34.1,TCP,Src= 4679,Dst= 2080,.A....,S=1263188865,L=    1,A=1486105225,W=64240
 92,IP-192.168.1.101,IP-192.168.1.103,26:34.1,TCP,Src= 2080,Dst= 4679,.A....,S=1486105225,L=    0,A=1263188865,W=    0
 93,IP-192.168.1.103,IP-192.168.1.101,26:34.9,TCP,Src= 4682,Dst= 2080,....S.,S=1267417142,L=    0,A=         0,W=64240
 94,IP-192.168.1.101,IP-192.168.1.103,26:34.9,TCP,Src= 2080,Dst= 4682,.A..S.,S=1489018948,L=    0,A=1267417143,W=64240
 95,IP-192.168.1.103,IP-192.168.1.101,26:34.9,TCP,Src= 4682,Dst= 2080,.A....,S=1267417143,L=    0,A=1489018949,W=64240
 96,IP-192.168.1.103,IP-192.168.1.101,26:34.9,TCP,Src= 4682,Dst= 2080,.A....,S=1267417143,L= 1460,A=1489018949,W=64240
 97,IP-192.168.1.103,IP-192.168.1.101,26:34.9,TCP,Src= 4682,Dst= 2080,.A....,S=1267418603,L= 1460,A=1489018949,W=64240
 98,IP-192.168.1.101,IP-192.168.1.103,26:34.9,TCP,Src= 2080,Dst= 4682,.A....,S=1489018949,L=    0,A=1267420063,W=64240
173,IP-192.168.1.103,IP-192.168.1.101,26:34.9,TCP,Src= 4682,Dst= 2080,.A....,S=1267487059,L= 1460,A=1489018949,W=64240
174,IP-192.168.1.101,IP-192.168.1.103,26:34.9,TCP,Src= 2080,Dst= 4682,.A....,S=1489018949,L=    0,A=1267488519,W= 1624
175,IP-192.168.1.103,IP-192.168.1.101,26:34.9,TCP,Src= 4682,Dst= 2080,.A....,S=1267488519,L= 1460,A=1489018949,W=64240
176,IP-192.168.1.101,IP-192.168.1.103,26:35.0,TCP,Src= 2080,Dst= 4682,.A....,S=1489018949,L=    0,A=1267489979,W=  164
177,IP-192.168.1.103,IP-192.168.1.101,26:36.8,TCP,Src= 4683,Dst= 2080,....S.,S=1267900827,L=    0,A=         0,W=64240
178,IP-192.168.1.101,IP-192.168.1.103,26:36.8,TCP,Src= 2080,Dst= 4683,.A..S.,S=1489535984,L=    0,A=1267900828,W=64240
179,IP-192.168.1.103,IP-192.168.1.101,26:36.8,TCP,Src= 4683,Dst= 2080,.A....,S=1267900828,L=    0,A=1489535985,W=64240
180,IP-192.168.1.103,IP-192.168.1.101,26:36.8,TCP,Src= 4683,Dst= 2080,.A....,S=1267900828,L= 1460,A=1489535985,W=64240
181,IP-192.168.1.103,IP-192.168.1.101,26:36.8,TCP,Src= 4683,Dst= 2080,.A....,S=1267902288,L= 1460,A=1489535985,W=64240
182,IP-192.168.1.101,IP-192.168.1.103,26:36.8,TCP,Src= 2080,Dst= 4683,.A....,S=1489535985,L=    0,A=1267903748,W=64240
257,IP-192.168.1.103,IP-192.168.1.101,26:36.8,TCP,Src= 4683,Dst= 2080,.A....,S=1267970744,L= 1460,A=1489535985,W=64240
258,IP-192.168.1.101,IP-192.168.1.103,26:36.8,TCP,Src= 2080,Dst= 4683,.A....,S=1489535985,L=    0,A=1267972204,W= 1624
259,IP-192.168.1.103,IP-192.168.1.101,26:36.8,TCP,Src= 4683,Dst= 2080,.A....,S=1267972204,L= 1460,A=1489535985,W=64240
260,IP-192.168.1.101,IP-192.168.1.103,26:37.0,TCP,Src= 2080,Dst= 4683,.A....,S=1489535985,L=    0,A=1267973664,W=  164
261,IP-192.168.1.103,IP-192.168.1.101,26:38.4,TCP,Src= 4684,Dst= 2080,....S.,S=1268324782,L=    0,A=         0,W=64240
262,IP-192.168.1.101,IP-192.168.1.103,26:38.4,TCP,Src= 2080,Dst= 4684,.A..S.,S=1489972190,L=    0,A=1268324783,W=64240
263,IP-192.168.1.103,IP-192.168.1.101,26:38.4,TCP,Src= 4684,Dst= 2080,.A....,S=1268324783,L=    0,A=1489972191,W=64240
264,IP-192.168.1.103,IP-192.168.1.101,26:38.4,TCP,Src= 4684,Dst= 2080,.A....,S=1268324783,L= 1460,A=1489972191,W=64240
265,IP-192.168.1.103,IP-192.168.1.101,26:38.4,TCP,Src= 4684,Dst= 2080,.A....,S=1268326243,L= 1460,A=1489972191,W=64240
266,IP-192.168.1.101,IP-192.168.1.103,26:38.4,TCP,Src= 2080,Dst= 4684,.A....,S=1489972191,L=    0,A=1268327703,W=64240
341,IP-192.168.1.103,IP-192.168.1.101,26:38.4,TCP,Src= 4684,Dst= 2080,.A....,S=1268394699,L= 1460,A=1489972191,W=64240
342,IP-192.168.1.101,IP-192.168.1.103,26:38.4,TCP,Src= 2080,Dst= 4684,.A....,S=1489972191,L=    0,A=1268396159,W= 1624
343,IP-192.168.1.103,IP-192.168.1.101,26:38.4,TCP,Src= 4684,Dst= 2080,.A....,S=1268396159,L= 1460,A=1489972191,W=64240
344,IP-192.168.1.101,IP-192.168.1.103,26:38.6,TCP,Src= 2080,Dst= 4684,.A....,S=1489972191,L=    0,A=1268397619,W=  164
345,IP-192.168.1.103,IP-192.168.1.101,26:39.9,TCP,Src= 4682,Dst= 2080,.A....,S=1267489979,L=  164,A=1489018949,W=64240
346,IP-192.168.1.101,IP-192.168.1.103,26:40.1,TCP,Src= 2080,Dst= 4682,.A....,S=1489018949,L=    0,A=1267490143,W=    0
347,IP-192.168.1.103,IP-192.168.1.101,26:40.2,TCP,Src= 4685,Dst= 2080,....S.,S=1268803676,L=    0,A=         0,W=64240
348,IP-192.168.1.101,IP-192.168.1.103,26:40.2,TCP,Src= 2080,Dst= 4685,.A..S.,S=1490482062,L=    0,A=1268803677,W=64240
349,IP-192.168.1.103,IP-192.168.1.101,26:40.2,TCP,Src= 4685,Dst= 2080,.A....,S=1268803677,L=    0,A=1490482063,W=64240
350,IP-192.168.1.103,IP-192.168.1.101,26:40.2,TCP,Src= 4685,Dst= 2080,.A....,S=1268803677,L= 1460,A=1490482063,W=64240
351,IP-192.168.1.103,IP-192.168.1.101,26:40.2,TCP,Src= 4685,Dst= 2080,.A....,S=1268805137,L= 1460,A=1490482063,W=64240
352,IP-192.168.1.101,IP-192.168.1.103,26:40.2,TCP,Src= 2080,Dst= 4685,.A....,S=1490482063,L=    0,A=1268806597,W=64240
427,IP-192.168.1.103,IP-192.168.1.101,26:40.2,TCP,Src= 4685,Dst= 2080,.A....,S=1268873593,L= 1460,A=1490482063,W=64240
428,IP-192.168.1.101,IP-192.168.1.103,26:40.2,TCP,Src= 2080,Dst= 4685,.A....,S=1490482063,L=    0,A=1268875053,W= 1624
429,IP-192.168.1.103,IP-192.168.1.101,26:40.2,TCP,Src= 4685,Dst= 2080,.A....,S=1268875053,L= 1460,A=1490482063,W=64240
430,IP-192.168.1.101,IP-192.168.1.103,26:40.4,TCP,Src= 2080,Dst= 4685,.A....,S=1490482063,L=    0,A=1268876513,W=  164
433,IP-192.168.1.103,IP-192.168.1.101,26:40.5,TCP,Src= 4682,Dst= 2080,.A....,S=1267490143,L=    1,A=1489018949,W=64240
434,IP-192.168.1.101,IP-192.168.1.103,26:40.5,TCP,Src= 2080,Dst= 4682,.A....,S=1489018949,L=    0,A=1267490143,W=    0
435,IP-192.168.1.103,IP-192.168.1.101,26:41.7,TCP,Src= 4683,Dst= 2080,.A....,S=1267973664,L=  164,A=1489535985,W=64240
436,IP-192.168.1.101,IP-192.168.1.103,26:41.9,TCP,Src= 2080,Dst= 4683,.A....,S=1489535985,L=    0,A=1267973828,W=    0
437,IP-192.168.1.103,IP-192.168.1.101,26:42.0,TCP,Src= 4682,Dst= 2080,.A....,S=1267490143,L=    1,A=1489018949,W=64240
438,IP-192.168.1.101,IP-192.168.1.103,26:42.0,TCP,Src= 2080,Dst= 4682,.A....,S=1489018949,L=    0,A=1267490143,W=    0
439,IP-192.168.1.103,IP-192.168.1.101,26:42.1,TCP,Src= 4686,Dst= 2080,....S.,S=1269285439,L=    0,A=         0,W=64240
440,IP-192.168.1.101,IP-192.168.1.103,26:42.1,TCP,Src= 2080,Dst= 4686,.A.R..,S=         0,L=    0,A=1269285440,W=    0
441,IP-192.168.1.103,IP-192.168.1.101,26:42.5,TCP,Src= 4686,Dst= 2080,....S.,S=1269285439,L=    0,A=         0,W=64240
442,IP-192.168.1.103,IP-192.168.1.101,26:42.5,TCP,Src= 4683,Dst= 2080,.A....,S=1267973828,L=    1,A=1489535985,W=64240
443,IP-192.168.1.101,IP-192.168.1.103,26:42.5,TCP,Src= 2080,Dst= 4686,.A.R..,S=         0,L=    0,A=1269285440,W=    0
444,IP-192.168.1.101,IP-192.168.1.103,26:42.5,TCP,Src= 2080,Dst= 4683,.A....,S=1489535985,L=    0,A=1267973828,W=    0
445,IP-192.168.1.103,IP-192.168.1.101,26:43.1,TCP,Src= 4686,Dst= 2080,....S.,S=1269285439,L=    0,A=         0,W=64240
446,IP-192.168.1.101,IP-192.168.1.103,26:43.1,TCP,Src= 2080,Dst= 4686,.A.R..,S=         0,L=    0,A=1269285440,W=    0
447,IP-192.168.1.103,IP-192.168.1.101,26:43.4,TCP,Src= 4684,Dst= 2080,.A....,S=1268397619,L=  164,A=1489972191,W=64240
448,IP-192.168.1.101,IP-192.168.1.103,26:43.6,TCP,Src= 2080,Dst= 4684,.A....,S=1489972191,L=    0,A=1268397783,W=    0
449,IP-192.168.1.103,IP-192.168.1.101,26:43.9,TCP,Src= 4683,Dst= 2080,.A....,S=1267973828,L=    1,A=1489535985,W=64240
450,IP-192.168.1.101,IP-192.168.1.103,26:43.9,TCP,Src= 2080,Dst= 4683,.A....,S=1489535985,L=    0,A=1267973828,W=    0
451,IP-192.168.1.103,IP-192.168.1.101,26:44.0,TCP,Src= 4684,Dst= 2080,.A....,S=1268397783,L=    1,A=1489972191,W=64240
452,IP-192.168.1.101,IP-192.168.1.103,26:44.0,TCP,Src= 2080,Dst= 4684,.A....,S=1489972191,L=    0,A=1268397783,W=    0
453,IP-192.168.1.103,IP-192.168.1.101,26:44.8,TCP,Src= 4682,Dst= 2080,.A....,S=1267490143,L=    1,A=1489018949,W=64240
454,IP-192.168.1.101,IP-192.168.1.103,26:44.8,TCP,Src= 2080,Dst= 4682,.A....,S=1489018949,L=    0,A=1267490143,W=    0
455,IP-192.168.1.103,IP-192.168.1.101,26:45.2,TCP,Src= 4687,Dst= 2080,....S.,S=1270040598,L=    0,A=         0,W=64240
456,IP-192.168.1.101,IP-192.168.1.103,26:45.2,TCP,Src= 2080,Dst= 4687,.A.R..,S=         0,L=    0,A=1270040599,W=    0
457,IP-192.168.1.103,IP-192.168.1.101,26:45.2,TCP,Src= 4685,Dst= 2080,.A....,S=1268876513,L=  164,A=1490482063,W=64240
458,IP-192.168.1.101,IP-192.168.1.103,26:45.4,TCP,Src= 2080,Dst= 4685,.A....,S=1490482063,L=    0,A=1268876677,W=    0
459,IP-192.168.1.103,IP-192.168.1.101,26:45.5,TCP,Src= 4684,Dst= 2080,.A....,S=1268397783,L=    1,A=1489972191,W=64240
460,IP-192.168.1.101,IP-192.168.1.103,26:45.5,TCP,Src= 2080,Dst= 4684,.A....,S=1489972191,L=    0,A=1268397783,W=    0
461,IP-192.168.1.103,IP-192.168.1.101,26:45.7,TCP,Src= 4687,Dst= 2080,....S.,S=1270040598,L=    0,A=         0,W=64240
462,IP-192.168.1.101,IP-192.168.1.103,26:45.7,TCP,Src= 2080,Dst= 4687,.A.R..,S=         0,L=    0,A=1270040599,W=    0
463,IP-192.168.1.103,IP-192.168.1.101,26:46.0,TCP,Src= 4685,Dst= 2080,.A....,S=1268876677,L=    1,A=1490482063,W=64240
464,IP-192.168.1.101,IP-192.168.1.103,26:46.0,TCP,Src= 2080,Dst= 4685,.A....,S=1490482063,L=    0,A=1268876677,W=    0
465,IP-192.168.1.103,IP-192.168.1.101,26:46.1,TCP,Src= 4687,Dst= 2080,....S.,S=1270040598,L=    0,A=         0,W=64240
466,IP-192.168.1.101,IP-192.168.1.103,26:46.1,TCP,Src= 2080,Dst= 4687,.A.R..,S=         0,L=    0,A=1270040599,W=    0
467,IP-192.168.1.103,IP-192.168.1.101,26:46.8,TCP,Src= 4683,Dst= 2080,.A....,S=1267973828,L=    1,A=1489535985,W=64240
468,IP-192.168.1.101,IP-192.168.1.103,26:46.8,TCP,Src= 2080,Dst= 4683,.A....,S=1489535985,L=    0,A=1267973828,W=    0
469,IP-192.168.1.103,IP-192.168.1.101,26:47.4,TCP,Src= 4685,Dst= 2080,.A....,S=1268876677,L=    1,A=1490482063,W=64240
470,IP-192.168.1.101,IP-192.168.1.103,26:47.4,TCP,Src= 2080,Dst= 4685,.A....,S=1490482063,L=    0,A=1268876677,W=    0
471,IP-192.168.1.103,IP-192.168.1.101,26:48.3,TCP,Src= 4684,Dst= 2080,.A....,S=1268397783,L=    1,A=1489972191,W=64240
472,IP-192.168.1.101,IP-192.168.1.103,26:48.3,TCP,Src= 2080,Dst= 4684,.A....,S=1489972191,L=    0,A=1268397783,W=    0
473,IP-192.168.1.103,IP-192.168.1.101,26:50.3,TCP,Src= 4685,Dst= 2080,.A....,S=1268876677,L=    1,A=1490482063,W=64240
474,IP-192.168.1.101,IP-192.168.1.103,26:50.3,TCP,Src= 2080,Dst= 4685,.A....,S=1490482063,L=    0,A=1268876677,W=    0


Blue Bar separator
This page was last modified on 03-06-23
mailbox Send comments and suggestions
to ndav1@cox.net