Noah Davids - Publications List

Blue Bar separator

Packet Trace Analysis Scripts on GitHub

check-subnet.sh - Test whether an IPv4 address is part of a subnet or notDecember 2019 
dns-time.sh - Create a table of DNS server query response times and list of unanswered queriesOctober 2017 
throughput-per-sec.sh - Calculate throughput per second of a specific stream at resolutions of 1, 1/10, 1/100, and 1/1000 of a second. results are suitable for graphingOctober 2017 
failed-connection-attempts.sh - Find TCP connection attempts that have a failed. There are 7 failure scenariosOctober 2017
December 2017 Updated with the 7th scenario
 
packet-matcher.sh - Extracts byte strings from a TCP stream in a template trace and looks for the strings in a target trace. The goal is to find a match TCP stream in the target trace fileSeptembetr 2017 
packet-matcher-faster.sh - Compares IP ID and absolute TCP sequence and ACK numbers between two traces to match up TCP segments where the IP addresses and or TCP have been changed (i.e. NAT)September 2017 
time-summary.sh - Finds all files in the current directory and any sub directories and displays then start and end times in sorted orderSeptember 2017 
build-filter.sh - Builds a tshark filter by ANDing or ORing the values in a list with a tshark variableSeptember 2017 
fix-pcap.sh - Removes a partial packet at the end of a packet trace fileSeptember 2017 
find-ips.sh - Uses egrep to list all strings in a file that match an IPv4 address format and then sort -u to get a unique list. Really just a one-liner by this way I do not have to remember (or type) the egrep string. Its useful with build-filter.sh to create a filter to display all the IPs listed in say a log fileAugust 2017 
stream_throughput.sh - Calculate the throughput of all TCP streams in a trace fileJuly 2017 
local-drops.sh.htmllocal-drops.sh - For each retransmitted TCP segment determine if the segment is seen more than onceJuly 2017 
split-pcap.py - Reads X.pcap and creates a set of X.pcap_IP1-Port1_IP2-Port2_split.pcap files, one for each TCP four-tuple. Reads only pcap files not pcapng. Requires Python and the scapy moduleJuly 2017 
bytes-in-flight.sh - Calculate the bytes in flight after each ACKJuly 2017 
percent-retransmissions.sh - For every connection in the trace file calculate the percentage of retransmissions for every source IP address as retransmissions / not-retransmitted source segments. segments must contain data, i.e.will not identify retransmitted SYNs or FINs without dataJune 2017 
find-reset-connections.sh - Find TCP connections that have been reset without being closedApril 2017 
average.sh - Average a value returned by tsharkApril 2017 
start-packet-tracing.sh - Runs tcpdump in the background with 10 files of 100 Meg eachApril 2017 
unterminated-connections.sh - Find TCP connections that have not been closed or resetApril 2017 

Self published non-Stratus related articles

tshark: The capture file being read can't be written as a "pcapng" file.June 2019
How can I capture the packet headers but not the data?December 2017
A shell script to measure the effects of retransmissionsSeptember 2017
How to graph concurrent connections in a packet trace file using tshark and gunplotJune 2016
A Perl script to calculate the delta time between timestamps at the start of a lineMay 2016
A shell script to send a ping containing the current date/time as textSeptember 2015
A Native Windows tool to capture packets (no downloading necessary)April 2015
A Better Tshark follow stream - A shell and perl script combination to make a better follow stream than the default "-z follow,..." that tshark has.March 2015
split-pcap.pl - A Perl script to separate the TCP flows in a pcap fileFebruary 2015
March 2016 Updated to handle VLAN tags following the Ethernet header
Calculating TCP throughput and why you cannot rely on WiresharkNovember 2014
Why can't I turn Auto-Negotiation off on my Gigabit and 10 Gigabit links?October 2013
A VBS script to stop a dumpcap (Wireshark) trace when a packet is seen (or not seen) in a tracefileMay 2012
A Perl script to calculate the delta values between 2 or more sets of numbers contained in a fileNovember 2010
March 2016 Added the separator argument
A Perl script to send a labeling frame for inclusion into a protocol analyzer trace file
May 2010
A JavaScript based script to generate passwordsMarch 2010
A VBS script to periodically collect network statistics for baselining and trouble shooting Windows systemsMay 2009
Secure Web Browsing When You are Away From HomeAugust 2008
The Limitations of the Ethernet CRC and TCP/IP checksums for error detectionJune 2008
September 2015 Updated the link to "Performance of Checksums and CRCs over Real Data
November 2017 Updated to explicitedly state that the TCP data is covered by the TCP checksum
Perl script to generate passwordsNovember 2007
The 7 tests of highly reliable server applicationsMarch 2007
VBS script to ping a host and execute a command if the ping failsNovember 2006
Perl script that uses TCP to "ping" a host.October 2006
VBS script to list all files in a directory tree by size.September 2006
Password Generation for MS Windows.September 2006
Using Dummynet to simulate a WAN over a LAN.January 2006
Instructions for using tcpdump to get a useful network protocol trace.November 2005
tping is a ping replacement for Microsoft Windows that uses TCP connections instead of ICMP. This is a ZIP file containing some documetation and the EXE file.May 2005
Instructions for using Microsoft Windows 2000/2003 server Network Monitor. This is a ZIP of a DOC file. The ZIP file 2.5 megSeptember 2004
Default TTL values for various OSes and protocolsAugust 2003
October 2008 updated to remove link to http://www.pmg.com/tip_archive/03_12.htm which no longer seems to be available
January 2009 updated with MAC OS X and Netgear FVG318
January 2011 updated with a DELL, 2 Extreme switches, Red Hat 4 update 8 and Ubuntu 10.04 LTS
November 2011 updated with Android 3.2.1
April 2016 Added serveral new OSes and versions
June 2018 Added Tandem Guardian

Self published Stratus related articles

Command macro to list all processes which have at least 1 STCP device clone locked. Run this when the system reports that you have exceeded your clone limitJanuary 2013
Command macro and command that together can be used to send a TCP segment mimicking a keep-alive segment from the specified connection every N minutes. The goal is to fool stateful middleware appliances into thinking that the connection is active. This is used when the actual application cannot send its own keep-alive messagesSeptember 2012
Command macro to display process data from a set of processesJuly 2010
A macro to dump the TCB and socket meters for all sockets associated with a process (or processes) This keys off of a process name or number
December 2009
A macro to dump the stcp metering data and/or the TCB structure for a socket or sockets based on local or remote IP addresses or port numbers or state This keys off IP address, port number or TCB address
October 2009
A scanner to find all hosts on a local subnet (uses ARP)
March 2009
Monitor netstat output filtering out uninteresting lines
January 2009
Listing processes attached to an STCP TCP socket
November 2008
Anonymizing VOS
October 2008
Feb 2009 updated to include information on the network time protocol daemon
Default port numbers used by Stratus supported server and client applications
October 2008
this is an update of the July 2007 Customer eNewsletter
How to identify and correct Ethernet duplex mismatch problemsAugust 2008
C program that allows you to log into a remote system via "telnet" and execute commands in a script. Allows execution within a command macroApril 2008
Perl script to match lines from filesAril 2008
May 2008 updated to allow output to be directed to a file
October 2009 updated to all output from a command to be matched
Managing a large output file by creating multiple little output filesAril 2008
Setting up Stratus STCP SSH to use public key authenticationMarch 2008
Listing the STCP IP connections (sockets) associated with a processFebruary 2008
Mapping TCP ports back to a VOS, ftServer, or ftLinux processDecember 2007
Sept 2008 updated to include a procedure for VOS STCP UDP ports
Beacon packet to assist in protocol trace synchrontizationNovember 2007
Updated June 2008 to include source code for MS Windows console application
Updated July 2008 to include a link to an MS Windows EXE file
Perl script to take the output from packet_monitor and reformat it for processing into a pcap file.October 2007
March 2008 updated to handle packet_mionitor release from VOS 14 and earlier
Command macro to loop through interface list doing a netstat -interface.August 2007
Command macros to display the names of users who are connected to a module via telnetd or sshd, the devices they are connected to and the IP address and port number they are connecting from.February 2007
Command macro that lists the vterm parameters needed by telnet_msd for a set of vterms defined by a star nameAugust 2006
Command macro that runs netstat and filters the output for specific stringsJuly 2006
Updated August 2008 to include a second match string
Command Macro that sets up an output file and runs packet_monitor in a started process sending its output to that file instead of the terminal screen. Also sets up useful arguments and filtersJuly 2006
October 2007 updated to include all_headers control argument
April 2008 updated to include ouput_path and process_name arguments for cycle_output_files
Command Macro to gather STCP interface and Ethernet adapter statisticsApril 2006
May 2007 updated
STCP tool that allows you to ping (sort of) when a firewall is blocking pingsMay 2004
Writing a server started by STCP's POSIX version of inetdSeptember 2003
Eliminating Underrun errors on the ftServerSeptember 2003
The TCP Keepalive feature as implemented under STCPAugust 2003
May 2007 Updated to describe stcp-2349 and stcp-2367 and the use of list_stcp_params and set_stcp_param
Dead Gateway Detection on VOSJuly 2003
Dead Gateway Detection on ftServerJuly 2003
Tools to help monitor STCP and TCP_OS networks and devicesJune 2003
The ARP cache (for all 6 TCP stacks)
 
 
December 2002
December 2, 2005 Updated to include ARP cache sizes and a little on Linux
December 7, 2005 Updated with a some information on Red Hat
How to map TCP ports back to a VOS or ftServer processDecember 2002
This article has been updated by Mapping TCP ports back to a VOS, ftServer, or ftLinux process.html

Stratus OpenVOS Blogs

All of the OpenVOS blogs can be found
here


Inconsistent success using public keys2013-06-03
Remote Command Execution2013-04-24
Update to "Changes in the STCP accept code may affect applications"2013-04-15
How to access the graphical administrative interface of devices on the maintenance network2013-03-11
Clone Wars2013-01-25
Telnet versus SSH2013-01-11
Host versus Hosts2012-11-18
The Importance of -12012-10-29
Third Party Keep-Alives2012-09-20
STCP's Duplicate IP Address Detection2012-08-21
packet_monitor and the TCP options supported in OpenVOS 17.12012-07-06
Securing the VOS Telnet Daemon: Technical Webinar2012-05-25
TCP Baclog Capture2012-05-18
Taking the Scenic Route2012-04-14
VOS OSL Server Processes, How many do I really need? Technical Webinar2012-04-02
How active is that connection in the window?2012-02-19
OpenVOS Network Application Performance Technical Webinar2012-01-19
SSH keys: DSA versus RSA2011-12-22
Maximum Transmission Unit (MTU) versus Maximum Segment Size (MSS)2011-12-01
A (very) simple log server for VOS2011-11-17
You don't exist go away2011-10-07
Issues with newly registered users and SSH2011-09-09
Sharing the Load - Multiple Processes Listening on the Same Port Number2011-08-01
The amazing appearing/disappearing host routes2011-07-07
Changes in the STCP accept code may affect applications2011-06-26
Host name resolution, a whole new paradigm 2011-05-30
Whats in a hostname2011-04-11
Do you have enough OSL server processes runing2011-03-23
Flushing the STCP ARP cache2011-02-28
Pardon me but do you have the time?2011-02-09
Embedding passwords in a program is not very secure2011-01-24
Don't forget to handle a graceful close, even if they can't happen2011-01-07
What is your IP address?2010-12-08
SSH and Passwords, Oh My!2010-12-03
IT Myths - TCP guarantees delivery of your data2010-11-22
Application event notification via E-mail2010-11-01
Network traces and data security2010-10-24
Telnetd and remote (TCP connected) printers2010-10-01
Note that on September 29, 2010 the blog engine that Stratus uses was changed. The blogs
were moved over during the previous months. The result is that my dates no longer match
the dates shown on the blog page.
Line termination problems when using SFTP2010-09-09
Testing network connectivity - alternatives to ping2010-08-31
When an employee leaves the company deleting log-in credentials is not enough2010-08-18
TCP is a stream of bytes not messages2010-07-18
SDLMUX Revisited2010-07-08
Accessing the ftStorage array's GUI interface from your workstation 2010-06-29
Services file not just for server processes2010-06-04
Monitoring network adapter status2010-05-25
Interop report - April 27th 2010 Las Vegas NV2010-05-04
Unblocking calls to the recv function without using non-blocking mode2010-04-28
Why didn't my bandwidth upgrade speed up my file copies2010-04-08
Explaining the traceroute command2010-03-30
Automating file transfers with SFTP2010-03-08
Automating file transfers with FTP macros2010-02-22
Five tips to help speed problem resolution2010-02-12
Using SNMP to monitor your connection to your switch2010-01-25
Do you know where your TCP connections really end?2009-11-23
Application Performance Problems and Latency2009-11-16
Dealing with Daylight Savings Time2009-10-21
Are you forwarding packets?2009-10-08
packet_monitor mirror ports and taps Oh My!2009-09-24
Pings can be Dangerous2009-08-30
Is your pre-production network testing effective?2009-08-18
Can you improve fault tolerance with multiple IP interfaces on the same subnet?2009-07-30
An easy way to improve TCP throughput across subnets2009-07-07
Things to consider on a Multihomed OpenVOS Module2009-06-19
Do You Know Your Network Neighbors2009-06-16
Network Related Performance Problems? Check for low level Ethernet errors first2009-06-05
Getting the most out of packet_monitor2009-05-29
Are these processes really needed2009-05-15
Telnet can't Lve with it can't live without it2009-05-07
"SSH 2" versus "OpenSSL and OpenSSH release 2"2009-04-30
How to Reserve a Port Number for Your Application2009-04-23
When Sockets Go Bad2009-04-16
Whither TCP statistics2009-04-08
A host based firewall for VOS2009-04-02
SSH tunneling2009-03-23
Test Monday (or: Are Your Network Connections Really Fault Tolerant?)2009-03-16

Stratus Customer eNewsletter/Stratus Partner eNewsletter articles

Default port numbers used by Stratus supported server and client applications?
July 2007
An update of this article can be found here
How many sockets can STCP really support and how do you determine the number currently in use?
December 2006
An updated version of socket_count.cm can be found here
Demystifying STREAMS memoryVol. 12, October 2006
MAC address assignment of ftServer\AE V Series and Continuum\AE network interface adaptersVol. 11, May 2006
MAC Address Assignment of ftServer System Network Interface AdaptersVol. 10, January 2006
FTP Issues and SolutionsVol. 9, October 2005
Updated 06-06-05 to include information about added support for FTP client passive mode in SFTP
Updated 06-07-18 to include sftp-248 bug fix information for release 14.7.2ah
Realistic Expectations for your Gigabit NetworkVol. 8, July 2005
VOS File Transfer OptionsVol. 7, April 2005
Assuring a Seamless Ethernet Failover with Proper Switch ConfigurationVol. 6, December 2004
Updated 05-08-30 to include information regarding link issues between the U713 and certain Cisco 6500 blades
Updated 05-11-02 to include information about gbe-88 and what releases contain the fixes to gbe-76 and gbe-88
Updated 05-11-07 to include information about sdlmux-129 and multiple switch topologies
Understanding STCP Send and Receive WindowsVol. 5, October 2004
Updated 07-01-11 to reflect changes made in newer releases
Updated 08-05-07 to reflect stcp-1447 and stcp-2387 fixes
Tuning Your MSSVol. 4, June 2004

Stratasphere newsletter articles

The Stratasphere newsletter was suppose to come out quarterly but its schedule was less regular. These documents are just the way that I sent them to the Stratasphere editor. Unfortunately my formating varies from file to file. Someday I'll clean things up and make them consistent -- but not today. All these files are Microsoft Word DOC files.

Selecting TCP Port Numbers for Fun, Profit and ReliabilitySpring 2002
updated 03-10-21 to reflect changes in STCP made since publication
updated 03-11-13 to include FTX 3.3 information
updated 05-07-29 to include differences between STCP's TCP and UDP ports
updated 05-12-21 to include Linux
updated 08-06-27 converted to HTML format and added STCP UDP changes
TCP keep-alive NOT! (STCP keep-alive has changed significantly since this document was written, an updated description of STCP keepalive can be found here)Winter 2001
Some thoughts on security for the VOS Operating SystemWinter 2001
Connection issues with STCPSummer 2001
The performance impact of packet lossSpring 2001
Balancing the load, multiple TCP application serversSummer 2000
DLMUX - fault tolerance for VOS Ethernet DevicesSpring 2000
Why won't my TCP based server startWinter 2000
The secrets of inetdWinter 1999/2000
Check the low level Ethernet statisticsWinter 1999/2000
Help, I'm stuck in FIN_WAIT_2 and I can't get outWinter 1998 (updated 05-07-05 to include STCP information)
Multi-homing is simpleFall 1998
Full duplex Ethernet - faster Ethernet for a priceSummer 1998
FTX's TCP/IP EnvironmentSpring 1998
Host name resolution under TCP_OSWinter 1997
SQE isn't trickyFall 1997
Terminals are trickySummer 1997
Subnets are TrickySpring 1996


Magazine articles on Programming, Networking, and User Environments

About half of these articles where never put on the web, for the ones that were the links I have are no longer valid (as of May 5, 2013). I have scanned in the copies of the articles that I have. I imagine that I am breaking copyright and if asked I will remove the scans; until then here they are.

Netstat Statistics Comparison with Perl original link
A link to the scanned pages is here
NaSPA Technical Support February 2004
Dangerous ARPs
A link to the scanned pages is here
Sys Admin 3/2003
TCP Programming Gotchas original link
A link to the scanned pages is here
Windows Developer Oct 2002
TCP - Either Fast or Efficient
A link to the scanned pages is here
Sys Admin 5/2001
Fault Tolerance In Networking - A Walk-Through original link
A link to the scanned pages is here
Network Magazine E-zine article September 26, 2000
A field Guide to Loopback plugs original link
A link to the scanned pages is here
Data Communications E-zine feature Article April 1, 2000
Analyze This: Six Mistakes To Avoid When Tracing Network Problems original link
A link to the scanned pages is here
Data Communications E-zine feature Article Feb 2000
Network Administration for Systems Administrators original link
A link to the scanned pages is here
Data Communications E-zine feature Article Jan 2000
The Mailman May Deliver But There Are No Guarantees With TCP original link
A link to the scanned pages is here
Data Communications E-zine tutorial Dec 1999
TCP Connection States - A clue to Network Health original link
A link to the scanned pages is here
Sys Admin 7/99
IP Forwarding
A link to the scanned pages is here
Sys Admin 1/99
When packets collide (significance of Ethernet collisions)
A link to the scanned pages is here
Sys Admin 6/98
Interpreting netstat Statistics
A link to the scanned pages is here
Sys Admin 5/97
10 Principles for Supportable Application Design
A link to the scanned pages is here
PC Techniques Oct/Nov 95
A Drag and Drop Shell
A link to the scanned pages is here
Windows/DOS Developers Journal 12/92
How People Use Large Vocabulary Systems
There is no scan; It's not quite 100 pages and really isn't all that interesting
Masters Thesis, Arizona State University (May 1988)
Experiences with an Interactive Electronic Meeting Facility
(as of May 5th, 2013 the link is still good)
Phoenix Conference on Computers and Communications 1983 (March 1983)


Product Reviews

Lab Test: Redundant TransceiversData Communications E-zine feature Article July 5, 2000
Watson, come here (Watson voice recognition API)Windows Tech Journal 10/97
Personal Digital Assistants: Part 2IEEE Computer 11/96
Personal Digital Assistants: Part 1IEEE Computer 9/96
Neatness Counts (PC-Lint 7.0)Windows Tech Journal 9/96
V.34 Boca Office Communicator Version 1.0IEEE Computer 2/96
If You've Got It, Flaunt It (demo programs)Windows Tech Journal 7/95
Atypical mice and related devicesIEEE Computer 6/95
Voice recognition for WindowsIEEE Computer 3/94
The Truth About TCP/IP (Winsock APIs)Windows Tech Journal 1/94
Nautilus (CD ROM based magazine)Windows User 3/93
Cursors for Windows 3.1 (mouse cursor replacements)IEEE Computer 1/93
Smooth Transition (IPC between DOS and Windows)Windows Tech Journal 10/92
Liana (C++ like language)IEEE Computer 4/92
Windows programming environmentsIEEE Computer 1/92
A Free Form Database Package for WindowsIEEE Software 9/91
A perfect 10 (Super Base 4)IEEE Computer 12/90
A Window onto your data (Q+E and Windows Filer)IEEE Computer 9/90
Mitsubishi scannerIEEE Computer 6/90
Keeping in touch through Windows (comm programs)IEEE Computer 2/90
Talking computers (voice recognition)IEEE Computer 10/89
Print SpoolerIEEE Computer 9/89
Can machines help improve your writing skillsIEEE Computer 7/89
A Turbo family portraitIEEE Computer 2/89

Miscellaneous

Wireless LAN at JFK AirportJanuary 2018
Pain ScaleMay 2016
MRDS/Linus With O. Friesen and R. BrinegarRelational Database Systems, Analysis and Comparison; Schmidt, J. W. and Brodie, M. L. editors (January 1983).
Seismic Network Analysis and Processing System (SNAPS): System Design and User's Guide for Hypocenter Determination With A. TarrU.S. Department of the Interior, Geological Survey, Open File Report 77-268 (June 1977)


Blue Bar separator
This page was last modified on 19-12-01
mailbox Send comments and suggestions
to noah@noahdavids.org