This page is an update of the How many sockets can STCP really support and how do you determine the number currently in use? Stratus eNewsletter article published in December 2006. It updates both the text and the macro. It also converts the article from a PDF file to HTML. It replaces a page that just had the updated macro and referred you to the PDF file for the explanation.




How many sockets can STCP really support and how do you determine the number currently in use?

Blue Bar separator

There is a fair amount of confusion regarding how many TCP sockets STCP can really support and how to count them. This article addresses that confusion. This is for VOS users running some version of VOS 14.7, 15.1 or 15.2, 16x or 17x. The socket_count macro at the end of the article will help you determine what your current limits are and how close you are to hitting them.

The number of TCP sockets that can be created at any one time is based on several different limits and on which of these limits you hit first.

The first limit is the size of the tc_entry array. There are 16130 entries in this array and once they are all in use no more TCP sockets can be created. Until STCP changes the way that it searches its internal data structures it is not feasible to increase the size of this array, so 16130 is the absolute maximum number of TCP sockets that you can have.

The second limit is the stcp.mN clone limit. This is the stcp device that is created in the devices.tin file. In release 14.6.1 this limit was raised from 8120 to 16000, if you upgraded from a pre-14.6.1 version of VOS you may still be using a limit of 8120 or even 5120 which was the clone limit STCP originally (VOS release 14.1) shipped with. Almost every time a TCP socket is created, a clone of this device is also created. Once you have reached the clone limit, then with the exceptions noted in the next paragraph no more TCP sockets can be created – even if there are free entries in the tc_entry array.

Not every socket uses a stcp_mN clone device. When the telnetd process accepts a connection it first creates a clone but then after some processing it frees the clone. Outgoing (there are no incoming) x25_cpc connections to the NIO also do not use a stcp.mN clone. The same is true of the sockets used in incoming and outgoing OSL connections. But NOT the OSL listening sockets. All these TCP sockets still use an entry in the tcb_entry array but they do not use a stcp.mN clone. What this means is that typically the number of used tc_entries will be greater than the number of stcp.mN device clones.

If you have a clone limit below 16000 you can increase it by

  1. edit the devices.tin file to change the clone limit
  2. create a new devices.table file
  3. move the devices.table file to >system
  4. reboot
Actually, all you have to do is stop the STCP stack, unload the STCP driver and do a “configure_devices –flush”. Then load the driver back up and restart the STCP stack. In my opinion it is easier to reboot, especially on a VOS release 15 or later system.

Starting in release 17.1 you can use the update_device_info command to increase the clone_limit for the current boot. However unless you change the devices.tin and create a new devices.table file the clone_limit will revert to the old value after a reboot.

Note that Stratus recommends that you increase the clone limit only high enough to meet you current needs plus expected growth. There have been multiple instances of bugs in applications causing loops that allocate sockets until they hit the stcp_mN clone limit and exhaust streams memory trying to use the sockets. Setting the clone limit to a lower value reduces the chances that streams memory will be exhausted.

The next limit is the total number of devices that the system is configured to support. You can see this by displaying the value of max_local_devices$ in analyze_system. The value is a short and the default is 4000x or 16384. The current number of devices is stored in the variable configured_local_devices$. Remember that when the system boots every entry in the devices table for the module will count toward the maximum. If, when a socket is created a stcp.mN clone is also created, then configured_local_devices$ is incremented by 1. The same holds true for x25_cpc and telnetd connections which use their own clone devices. When the clone is freed the value is decremented by 1. If the number of devices defined when the system is booted is X you will not be able to create more than 16384 – X sockets.

You can raise the maximum number of devices just by patching the max_local_devices$ with the analyze_system request set_word. You can do this at any point after the system is booted (see figure 1).

analyze_system -request_line 'set_word max_local_devices$ 4e20' -quit           
VOS Release 14.7.2am, analyze_system Release 14.7.2am
Current process is 4055, ptep C34C84C0, Noah_Davids.CAC
addr from to
C094250C 4000 4E20
Figure 1 – Increasing maximum number of devices supported on a module

The final limitation is the amount of memory available to the STREAMS subsystem. When a socket is created some amount of memory is needed. The amount used is added to the amount used by the Streams subsystem. If that amount exceeds certain thresholds you will not be able to create the socket. There are three thresholds LO, MED, HI. Socket creation checks both the HI and MED thresholds so if your Streams memory usage exceeds the MED threshold you will not be able to create a socket.

You can increase the amount of memory available to the STREAMS subsystem by changing the sys_denominator parameters with the analyze_system request set_streams_param. The default value is 8 which corresponds to 1/8 of system memory. Changing the value to 4 will double the available amount of streams memory. Like max_local_devices$, you can make this change any time after the system is booted.

analyze_system -request_line 'set_streams_param sys_denominator 4' -quit        
VOS Release 14.7.2am, analyze_system Release 14.7.2am
Current process is 4055, ptep C34C84C0, Noah_Davids.CAC
Changing sys max heap denomin (sys_denominator)
from 8 to 4
Figure 2 – Increasing the amount of memory available to the STREAMS subsystem

The following macro, socket_count, displays the currently used number of entries in the tc_entry array, the number of stcp.mN clone devices and just calls netstat and counts the number of TCP sockets displayed. It also displays the number of devices and the current streams memory usage as a percentage of the LO, MED and HI thresholds. The LO threshold because if you have exceeded the LO threshold you will have problems moving data through STREAMS even if you can still create sockets.

socket_count                                                                    
OpenVOS Release 17.0.1aj, analyze_system Release 17.0.1aj
Current process is 126, ptep 8DC4F000, Noah_Davids.CAC
as:  as:  as:  as:  as:  as:

The number of tc_entries at 09-05-14 14:06:58 mst is 81 out of 16130
The stcp.m16 clone count at 09-05-14 14:06:58 mst is 52 out of 5120
TCP sockets from netstat at 09-05-14 14:06:58 mst is 79
The number of devices at   09-05-14 14:06:58 mst is 128 out of 16384
0.567% of LO Streams memory threshold in use as of 09-05-14 14:06:58 mst
0.52% of MED Streams memory threshold in use as of 09-05-14 14:06:58 mst
0.5% of HI Streams memory threshold in use as of 09-05-14 14:06:58 mst
ready  14:06:58
Figure 3 – Example run of socket_count.cm

Some of you may be asking why there is a date-time stamp on each line of output. The reason is that if you monitor your system by periodically executing this command and dumping all the output into a single file you can extract a set of counters, for example, the tc_entry count with the "display -match tc_entry" command and have the time stamps available.

d x.cm                                                                          

%phx_vos#m16_mas>SysAdmin>Noah_Davids>x.cm  09-05-14 15:15:49 mst

&label again
socket_count
sleep -minutes 5
&goto again

ready  15:15:49
d x.out -match tc_entries

%phx_vos#m16_mas>SysAdmin>Noah_Davids>x.out  09-05-14 15:16:03 mst

The number of tc_entries at 09-05-14 14:38:18 mst is 81 out of 16130
The number of tc_entries at 09-05-14 14:43:18 mst is 81 out of 16130
The number of tc_entries at 09-05-14 14:48:19 mst is 81 out of 16130
The number of tc_entries at 09-05-14 14:53:19 mst is 81 out of 16130
The number of tc_entries at 09-05-14 14:58:19 mst is 81 out of 16130
The number of tc_entries at 09-05-14 15:03:20 mst is 81 out of 16130
The number of tc_entries at 09-05-14 15:08:20 mst is 81 out of 16130
The number of tc_entries at 09-05-14 15:13:21 mst is 81 out of 16130

ready  15:16:03
Figure 4 – Example of a continuous monitor macro and display of specific counter

socket_count.cm

& socket_count begins here
&
& socket_count.cm
& version 2.0 06-11-01
&         2.1 08-02-07 changed the way tc_entries and netstat lines are
&                      counted to fix a problem caused when the process
&                      directory is a link
&         2.2 10-11-26 added disclaimer
&         2.3 13-01-17 added changes needed to correctly determine the
&                      maximum tc_entries value on 17.1
& Noah Davids Stratus CAC noah.davids@stratus.com
&
& This software is provided on an "AS IS" basis, WITHOUT ANY WARRANTY OR ANY
& SUPPORT OF ANY KIND. The AUTHOR SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES
& OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE.  This disclaimer
& applies, despite any verbal representations of any kind provided by the
& author or anyone else.
&
&attach_input
&set START (calc (length (system_name)) + 3)
&set_string STCP_DEVICE stcp.(substr (current_module) &START&)
&
&
&set RELEASE171 (index (module_info system_release) 'Release 17.1')
&
analyze_system
..attach_default_output (process_dir)>tc_entries
match tcb; dump_tcentry
..detach_default_output
&if &RELEASE171& > 0 &then &do
..attach_default_output (process_dir)>ntcpcon
match max_tcp; dump_tcp_global
&end
&else &do
..attach_default_output (process_dir)>ntcpcon
d ntcpcon
&end
..detach_default_output
..attach_default_output (process_dir)>clones
match clone_count; dump_dvt -name &STCP_DEVICE&
match clone_limit; dump_dvt -name &STCP_DEVICE&
..detach_default_output
..attach_default_output (process_dir)>devices
d configured_local_devices$
d max_local_devices$
..detach_default_output
..attach_default_output (process_dir)>pmm
match 'pmm_limit' ; dump_stream -stm_msg
match 'hq_pool[0]->pmm_total_size' ; dump_stream -stm_msg
match 'hq_pool[0]->pmm_allocated_size' ; dump_stream -stm_msg
..detach_default_output
quit
attach_default_output (process_dir)>netstat
>system>stcp>command_library>netstat -numeric -all_sockets -protocol tcp
detach_default_output
&
&if (exists (process_dir)>tc_entries2)
&then delete_file (process_dir)>tc_entries2 -no_ask
create_file (process_dir)>tc_entries2 -organization relative -record_size 256
copy_file (process_dir)>tc_entries (process_dir)>tc_entries2 -truncate
&set records (file_info (process_dir)>tc_entries2 last_record)
&
display_line
display_line
&if &RELEASE171& > 0 &then &do
&set NTCPCON (substr (contents (process_dir)>ntcpcon 1) 40 6)
&end     
&else &do
&set NTCPCON (calc (substr (contents (process_dir)>ntcpcon 1) 19 8)x)
&end
display_line The number of tc_entries at &+
(file_info (process_dir)>tc_entries date_used) is (calc &records& - 2) &+
out of &NTCPCON&
&
&set CLONE_COUNT (substr (contents (process_dir)>clones 1) 25)
&set CLONE_LIMIT (substr (contents (process_dir)>clones 2) 25)
display_line The &STCP_DEVICE& clone count at &+
(file_info (process_dir)>clones date_used) is &CLONE_COUNT& &+
out of &CLONE_LIMIT&
&
&if (exists (process_dir)>netstat2)
&then delete_file (process_dir)>netstat2 -no_ask
create_file (process_dir)>netstat2 -organization relative -record_size 256
copy_file (process_dir)>netstat (process_dir)>netstat2 -truncate
&set records (file_info (process_dir)>netstat2 last_record)
&if &records& = 0
&then display_line No info from netstat, probably no sockets available
&else display_line TCP sockets from netstat at &+
(file_info (process_dir)>netstat date_used) is (calc &records& - 2)
&        
&set CONFIG_DEVICES (calc (substr (contents (process_dir)>devices 1) 19 4)x)
&set MAX_DEVICES (calc (substr (contents (process_dir)>devices 2) 19 4)x)
display_line The number of devices at ' ' &+
(file_info (process_dir)>devices date_used) is &CONFIG_DEVICES& &+
out of &MAX_DEVICES&
&
&set ALLOC_MEM (substr (contents (process_dir)>pmm 6) 46)
&set TOTAL_MEM (substr (contents (process_dir)>pmm 5) 46)
&set LO_MEM (substr (contents (process_dir)>pmm 2) 41)
&set MED_MEM (substr (contents (process_dir)>pmm 3) 41)
&set HI_MEM (substr (contents (process_dir)>pmm 4) 41)
&set LO_PER (substr (calc &ALLOC_MEM& / &LO_MEM& * 100) 1 5)
&set MED_PER (substr (calc &ALLOC_MEM& / &MED_MEM& * 100) 1 5)
&set HI_PER (substr (calc &ALLOC_MEM& / &HI_MEM& * 100) 1 5)
display_line &LO_PER&% of LO Streams memory &+
threshold in use as of (file_info (process_dir)>pmm date_used)
display_line &MED_PER&% of MED Streams memory &+
threshold in use as of (file_info (process_dir)>pmm date_used)
display_line &HI_PER&% of HI Streams memory &+
threshold in use as of (file_info (process_dir)>pmm date_used)
&
& socket_count ends here




This page is an update of the How many sockets can STCP really support and how do you determine the number currently in use? Stratus eNewsletter article published in December 2006. It updates both the text and the macro. It also converts the article from a PDF file to HTML. It replaces a page that just had the updated macro and referred you to the PDF file for the explanation.


Blue Bar separator
This page was last modified on 13-01-24
mailbox Send comments and suggestions
to noah.davids@stratus.com