SUMMARY Re: allocating >128M of memory

From: Ken Miller <ken_at_phy.ucsf.edu>
Date: Wed, 7 Aug 1996 22:20:09 -0700

The quick summary seems to be in this message:

David G. Loone <dloone_at_atnf.csiro.au> writes:
-> There is a limit on the amount of virtual memory a process can
-> use. You can do one of three things:
->
-> 1.
-> Rebuld your kernel with a larger value of "dfldsiz".
->
-> 2.
-> Use the "limit" command in csh, tcsh, etc to increase this value online
-> (up to the "maxdsiz" kernel parameter).
[NOTE: in bash or ksh this is "ulimit", see next messages below -- KM]
-> 3.
-> Use the "getrlimit" and "setrlimit" system calls in your software to
-> achieve #2.

A couple of additions:
4. modify /etc/sysconfigtab, preferentially by using
   /sbin/sysconfigdb, possibly by direct editing;
5. Other relevant commands: stanza; sysconfig and doconfig (for
   rebuilding kernel)

Following is lots more details on each of these in the form of the raw
original messages, all of which are very helpful and each of which
seems to contain at least one piece of unique information.

Thanks so much to everyone!

Ken Miller
ken_at_phy.ucsf.edu

"Sure the world breeds monsters, but kindness grows just as wild ..."
        -- Mary Karr, The Liars' Club

---------------------------------------------------------------------

Phil Lawrence <philip_at_uvo.dec.com> writes:

-> The problem could possibly be related to shell limits:
->
-> See if the following makes a difference:
->
-> > ulimit
-> cputime unlimited
-> filesize unlimited
-> datasize 131072 kbytes << here is a 128MB limit
-> stacksize 2048 kbytes
-> coredumpsize unlimited
-> memory 25288 kbytes
-> descriptors 4096 files
-> addressspace 1048576 kbytes
-> >ulimit -d unlimited
-> >ulimit -d
-> datasize 1048576 kbytes << now set to maxdsiz as defined in kernel

---------------------------------------------------------------------

bouchard_l_at_decus.fr (Louis Bouchard - Bouygues Telecom) writes:

-> Try changing your shell's limits (ulimit -a in ksh, limit in csh I
-> think). On my station I get :
->
-> # ulimit -a
-> time(seconds) unlimited
-> file(blocks) unlimited
-> data(kbytes) 131072
-> stack(kbytes) 2048
-> memory(kbytes) 89920
-> coredump(blocks) 0
-> nofiles(descriptors) 4096
-> vmemory(kbytes) 1048576
-> root_at_bt1wd012> bc -l
-> 131072/1024
-> 128.00000000000000000000
-> quit
->
-> which shows that the data limit is 128 Mb. If you do a "ulimit -a
-> unlimited" you will get as much as the parameters of sysconfigtab and the
-> kernel configuration files will allow which is :
->
-> sysconfigtab :
-> per-proc-stack-size = 2097152
-> max-per-proc-stack-size = 33554432
-> per-proc-data-size = 134217728
-> max-per-proc-data-size = 1073741824
->
-> Kernel config :
-> dfldsiz 134217728
-> maxdsiz 1073741824
-> dflssiz 2097152
-> maxssiz 33554432
->
-> If you modify your sysconfigtab and the value gets bigger than
-> its equivalent in the kernel config. file, you won't get more than
-> what the kernel allow. Here is an example :
->
-> # ulimit -a unlimited
-> # ulimit -a
-> time(seconds) unlimited
-> file(blocks) unlimited
-> data(kbytes) 1048576
-> stack(kbytes) 2048
-> memory(kbytes) 89920
-> coredump(blocks) 0
-> nofiles(descriptors) 4096
-> vmemory(kbytes) 1048576
-> root_at_bt1wd012> bc -l
-> 1048576/1024
-> 1024.00000000000000000000
-> quit
->
-> When we look at per-proc-data-size, it is at 134217728 which
-> is 128 Mb. The value of max-per-proc-data-size is 1073741824 which
-> gives 1024 Mb. This is what we find when we do the ulimit's.

---------------------------------------------------------------------

Dave Norton <norton_at_nrlmry.navy.mil> writes:

-> Try looking to see what the system limits are on allocating memory.
-> You can do this by:
-> /sbin/sysconfig -q proc
->
-> You should see two strings -
-> per-proc-stack-size
-> max-per-proc-stack-size
->
-> If the max-per-proc-stack-size is bigger the what you can alloc (if it
-> is 256M or bigger) and the per-proc-stack-size is smaller then 256M, then
-> you can solve the problem by just issuing (in csh) unlimit stacksize
->
-> entering: limit
-> will tell you what the default allocations are (per-proc), issuing a unlimit
-> will raise them to max-per-proc.
->
-> If max-per-proc is too small, you can change it in yor configuration file
-> and build and install a new kernel, or you can put a definition in a file
-> that I think is in /etc. but don't remember (/etc/sysconfig?) and reboot.
-> I always just rebuild the kernel.
->
-> If max-per-proc-stacksize is big enough and limit says your process has
-> enough space, then something else is wrong.
->
-> (In which case I'm out of guesses!)

---------------------------------------------------------------------

David Warren <warren_at_atmos.washington.edu> writes:

-> edit /etc/sysconfigtab e.g.
-> proc:
-> max-per-proc-stack-size = 524288000
-> max-per-proc-data-size = 524288000
-> max-per-proc-address-space = 524288000
-> per-proc-stack-size = 4194304
-> per-proc-data-size = 209715200
->
-> reboot and you will be able to have 524288000 bytes.

---------------------------------------------------------------------

Alan Rollow <alan_at_nabeth.cxo.dec.com> writes:

-> For each type of memory, text, data and stack, there are two
-> limits. One is the absolute per-process limit set on a system
-> wide limit. The configuration file parameters that control
-> these are maxtsiz, maxdsiz and maxssiz. In recent versions
-> these may be settable via sysconfigtab. The other limits is
-> the default value. The parameters for the defaults are dfltsiz,
-> dfldsiz and dflssiz.

-> In the C-shell these limits can be changed using the limit
-> built-in. Other shells may have their own command for
-> changing them. The typical shipping default for the data
-> size is 128 MB, but the system maximum may be higher. Root
-> can typically use the unlimit built-in.

-> Since this is probably the data size limit, experiment with
-> limit to see if the system datasize is higher than the default.
-> If so, just use limit to raise for the desired processes. If
-> not check the system tuning guide or chapter of the system
-> management manual that describes how to change such parameters.
-> At a minimum it will require a reboot. At most a kernel rebuild
-> and reboot.

---------------------------------------------------------------------

Martin Gosejacob <gosejac_at_rto.dec.com> writes:

-> Hmmm looks like you are hitting the default data size limit.
->
-> log in as root and check:
->
-> sysconfig -q proc
->
-> and have a look at the kernel configured limits:
->
-> per-proc-data-size
-> max-per-proc-data-size
->
-> when I remember correctly the default value is set to 128 MB.
-> Now check the limits in your shell; e.g. in csh type 'limits'
-> (different syntax in sh and ksh). This will spit out a list
-> of limitations for the shell and all its children. You can easily
-> set the shell datasize limit to the system wide limit by using
-> 'limit datasize unlimited'. This will set the shell's datasize limit
-> to whatever is configured as 'max-per-proc-data-size'. If you need to
-> change the kernel parameters please use /etc/sysconfigtab.

---------------------------------------------------------------------

Fabio Bossi - ISIS CEC Ispra <fabio.bossi_at_jrc.it> writes:

-> Process memory size limits are defined by certain system parameters
-> contained into: /usr/sys/conf/<your system name> file and they are:
->
-> dfldsiz=default data segment size limit (generally 128M);
-> maxdsiz=maximum " " " " (in my case approx. 1Gbyte);
-> dflssiz=default stack size limit;
-> maxssiz=maximum " " ".
->
-> If you change one of these you --> must re-genrate the kernel
-> (doconfig -c <your system name>).
->
-> Because of the 1st parameter you are not able to address more than
-> 128M; using the C-shellcommand (or the system call setrlimit) you
-> can increase this value up to "maxdsiz", i.e.
->
-> limit datasize 256M (in C-shell);
->
-> or you can change this parameter in the kernel config file. The
-> same applies to the others parm.s.

---------------------------------------------------------------------

"Patrick O'Brien" <pobrien_at_cfa.harvard.edu> writes:

-> use sysconfigdb(8) and stanza(4) to change
-> max-per-proc-address-space
-> max-per-proc-data-size

---------------------------------------------------------------------

The above messages include I think all of the main points. Others who
answered with information that overlaps the above include
Schuhl, Robert" <robert.schuhl_at_ald-vt.de>
rioux_at_ip6480nl.ce.utexas.edu (Tom Rioux)
Eric Pettersen <pett_at_cgl.ucsf.EDU>
I think one or two others that I deleted, sorry.
Received on Thu Aug 08 1996 - 07:36:43 NZST

This archive was generated by hypermail 2.4.0 : Wed Nov 08 2023 - 11:53:46 NZDT