tshark: The capture file being read can't be written as a "pcapng" file.
Analyzing a large trace files can take time. In figure 1 it took over 2 minutes to count 21 frames in an 800 megabyte file.
$ ls -l *netscaler* -rw-r--r-- 1 noah noah 874545152 May 29 20:43 netscaler.cap $ time tshark -r netscaler.cap -Y "tcp.port == 50340" | wc -l 21 real 2m7.448s user 0m44.728s sys 0m5.114s |
So t makes sense to try to extract only the packets that you are interested in and write them to a new file. But tshark cannot always do this.
$ tshark -r netscaler.cap -Y "tcp.port == 50340" -w port_50340.pcap tshark: The capture file being read can't be written as a "pcapng" file. |
$ tshark -r windows.cap -Y "tcp.port == 443" -w windows.pcap tshark: The capture file being read can't be written as a "pcapng" file. |
Now at this point I have to say that wireshark may be able to "Export Specified Packets", it will work for the NetScaler trace but its grayed out for the Microsoft trace. Also if you cannot run Wireshark for whatever reason (no GUI interface, not enough memory to process the large file), you still have an option.
First you need to do a dump of the packets using the "-x" argument. Note also that I have disabled TCP de-segmentation, if it is enabled then the last segment of a multi-packet segment will contain all of the bytes which is not what we want in this case.
$ tshark -r netscaler.cap -Y "(ip.addr eq 192.168.2.9 and ip.addr eq 10.1.23.4) and \ (tcp.port eq 50340 and tcp.port eq 443)" -o tcp.desegment_tcp_streams:FALSE -x > port_50340_netscaler.cap.txt |
Next you need to remove all the bytes up to the start of the Ethernet header, I must admit there is some hand-waving here. The exact number of bytes before the Ethernet header will vary depending on how the trace file was generated and this approach requires that the number of bytes before the Ethernet header is a constant. In the case of the netscaler trace the constant is 75.
The following command first adds a blank line to the start of the hex dump and then uses egrep to strip the first 4 lines (64 bytes) of each output record by ignoring lines beginning with the offsets 0000, 0010, 0020, and 0030. Then using awk if the line is blank it writes 2 new lines, creating a blank line between the last record and the next record and begins the next record with offset 0000 (no new line). This is why I added a new line at the start of the file, so that awk can add the first 0000 offset. If the line begins with offset 0040 it writes bytes 12 thru 16 (offsets 13 thru 17 because awk is counting the offset as 1). For every other line beginning with an offset it just writes bytes 0 thru 16.Note that all these bytes are written without a new line (the new line is added when a blank line is seen). Finally the new byte stream is piped to text2pcap to create a new file.
$ (echo ""; cat port_50340_netscaler.cap.txt) |\
egrep -v "^0000|^0010|^0020|^0030" | \
awk '{if (NF == 0) printf ("\n\n0000 "); else \
if ($1 == "0040") for (x=13; x<=17; x++) printf ("%s ", $x); \
else for (x=2; x<=17; x++) printf ("%s ", $x)}' | text2pcap - port_50340_netscaler.pcap
Input from: Standard input
Output to: port_50340_netscaler.pcap
Output format: pcap
Wrote packet of 699 bytes.
Wrote packet of 74 bytes.
Wrote packet of 635 bytes.
Wrote packet of 54 bytes.
Wrote packet of 1514 bytes.
Wrote packet of 327 bytes.
Wrote packet of 74 bytes.
Wrote packet of 1003 bytes.
Wrote packet of 54 bytes.
Wrote packet of 107 bytes.
Wrote packet of 54 bytes.
Wrote packet of 107 bytes.
Wrote packet of 74 bytes.
Wrote packet of 107 bytes.
Wrote packet of 54 bytes.
Wrote packet of 54 bytes.
Wrote packet of 74 bytes.
Read 19 potential packets, wrote 17 packets (5361 bytes).
|
The new file is much smaller and displays much more quickly. There is one flaw in this approach. The timestamps associated with the frames is not preserved, you will note that each frame in the new trace is exactly 1 microsecond from the frame before it. So if you are trying to calculate round trip times or dealing with latency issues this approach will not help.
$ ls -l *netscaler*
-rw-r--r-- 1 noah noah 874545152 May 29 20:43 netscaler.cap
-rw-r--r-- 1 noah noah 5361 Jun 1 11:20 port_50340_netscaler.pcap
$ time tshark -r port_50340_netscaler.pcap
1 0.000000 192.168.2.9 -> 10.1.23.4 TLSv1.2 699 Application Data
2 0.000001 10.1.23.4 -> 192.168.2.9 TCP 74 443 -> 50340 [ACK] Seq=2683381195 Ack=2593302460 Win=40313 Len=0
3 0.000002 10.1.23.4 -> 192.168.2.9 TLSv1.2 635 Application Data
4 0.000003 192.168.2.9 -> 10.1.23.4 TCP 54 50340 -> 443 [ACK] Seq=2593302460 Ack=2683381776 Win=20440 Len=0
5 0.000004 192.168.2.9 -> 10.1.23.4 TCP 1514 [TCP segment of a reassembled PDU]
6 0.000005 192.168.2.9 -> 10.1.23.4 TLSv1.2 327 Application Data
7 0.000006 10.1.23.4 -> 192.168.2.9 TCP 74 443 -> 50340 [ACK] Seq=2683381776 Ack=2593304193 Win=39225 Len=0
8 0.000007 10.1.23.4 -> 192.168.2.9 TLSv1.2 1003 Application Data
9 0.000008 192.168.2.9 -> 10.1.23.4 TCP 54 50340 -> 443 [ACK] Seq=2593304193 Ack=2683382725 Win=23360 Len=0
10 0.000009 10.1.23.4 -> 192.168.2.9 TLSv1.2 107 Application Data
11 0.000010 192.168.2.9 -> 10.1.23.4 TCP 54 50340 -> 443 [ACK] Seq=2593304193 Ack=2683382778 Win=23360 Len=0
12 0.000011 192.168.2.9 -> 10.1.23.4 TLSv1.2 107 Encrypted Alert
13 0.000012 10.1.23.4 -> 192.168.2.9 TCP 74 443 -> 50340 [ACK] Seq=2683382778 Ack=2593304246 Win=40905 Len=0
14 0.000013 10.1.23.4 -> 192.168.2.9 TLSv1.2 107 Encrypted Alert
15 0.000014 192.168.2.9 -> 10.1.23.4 TCP 54 50340 -> 443 [FIN, ACK] Seq=2593304246 Ack=2683382778 Win=23360 Len=0
16 0.000015 192.168.2.9 -> 10.1.23.4 TCP 54 50340 -> 443 [ACK] Seq=2593304247 Ack=2683382832 Win=23360 Len=0
17 0.000016 10.1.23.4 -> 192.168.2.9 TCP 74 443 -> 50340 [ACK] Seq=2683382832 Ack=2593304247 Win=40905 Len=0
real 0m0.144s
user 0m0.098s
sys 0m0.034s
|
The Microsoft Message Analyzer trace file was captured from a wireless interface so there is no Ethernet header. However text2pcap allows you specify the type of layer2 header that was captured with the -l argument. For a wireless 802.11 header the argument would be "-l 105". A complete list can be found at www.tcpdump.org/linktypes.html. You will note that the capture is only 1 direction, 192.168.1.15 to 204.79.197.200, this is not an artifact of the process, it is just what I was given. Note also that in this case the number of bytes to strip before the 802.11 link layer is 101 (offsets 0000 thru 0050 and then the first 5 bytes from offset 0060).
$ tshark -r windows.cap -Y "(ip.addr eq 192.168.1.15 and ip.addr eq 204.79.197.200) and \
(tcp.port eq 57812 and tcp.port eq 443)" -o tcp.desegment_tcp_streams:FALSE -x > port_57812_windows.cap.txt
$ (echo ""; cat port_57812_windows.cap.txt) | \
egrep -v "^0000|^0010|^0020|^0030|^0040|^0050" | \
awk '{if (NF == 0) printf ("\n\n0000 "); else \
if ($1 == "0060") for (x=7; x<=17; x++) printf ("%s ", $x); \
else for (x=2; x<=17; x++) printf ("%s ", $x)}' | text2pcap -l 105 - port_57812_windows.pcap
Input from: Standard input
Output to: port_57812_windows.pcap
Output format: pcap
Wrote packet of 100 bytes.
Wrote packet of 88 bytes.
Wrote packet of 581 bytes.
Wrote packet of 88 bytes.
Wrote packet of 139 bytes.
Wrote packet of 175 bytes.
Wrote packet of 88 bytes.
Wrote packet of 126 bytes.
Wrote packet of 88 bytes.
Wrote packet of 88 bytes.
Read 11 potential packets, wrote 10 packets (1745 bytes).
$ tshark -r port_57812_windows.pcap -o tcp.relative_sequence_numbers:TRUE
1 0.000000 192.168.1.15 -> 204.79.197.200 TCP 100 57812 -> 443 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 WS=256 SACK_PERM=1
2 0.000001 192.168.1.15 -> 204.79.197.200 TCP 88 57812 -> 443 [ACK] Seq=1 Ack=1 Win=262144 Len=0
3 0.000002 192.168.1.15 -> 204.79.197.200 TLSv1.2 581 Client Hello
4 0.000003 192.168.1.15 -> 204.79.197.200 TCP 88 57812 -> 443 [ACK] Seq=494 Ack=151 Win=261888 Len=0
5 0.000004 192.168.1.15 -> 204.79.197.200 TLSv1.2 139 Change Cipher Spec, Encrypted Handshake Message
6 0.000005 192.168.1.15 -> 204.79.197.200 TLSv1.2 175 Application Data
7 0.000006 192.168.1.15 -> 204.79.197.200 TCP 88 57812 -> 443 [ACK] Seq=632 Ack=220 Win=261888 Len=0
8 0.000007 192.168.1.15 -> 204.79.197.200 TLSv1.2 126 Application Data
9 0.000008 192.168.1.15 -> 204.79.197.200 TCP 88 57812 -> 443 [ACK] Seq=670 Ack=258 Win=261632 Len=0
10 0.000009 192.168.1.15 -> 204.79.197.200 TCP 88 57812 -> 443 [RST, ACK] Seq=670 Ack=258 Win=0 Len=0
|
Send comments and suggestions