Thanks to all of the folks that responded to my original query:
>We are running an application on a 4100 with 4.0D that is consuming a
>tremendous amount of memory. Each invocation of the program results in 4
>other processes being created. The 5 together have a resident size of about
>75Mb and we need to have at least 30 running at any one time. Need less to
>say we have to find a way to get the footprint down on these apps since we
>certainly cant scale this way. This is all homegrown code, and it is all
>built with shared object libs. What we dont know is what exactly is in
these
>executables that is making them so big and is there a way to verify that
the
>shared libs are really being shared. It doesn't appear to be the case
>according to our programmers.
>
>Are there any commands or tools that will give us visibility into what
these
>monstrous applications are made of and how they are being run or that they
>are indeed sharing code?
We are working with the developers using the supplied information to help
them whittle down the size of their images. I got a lot of good tips and
ideas which I will share here.
>From Dr Blinn:
Having the large resident size is not necessarily a problem provided that
most
of what's resident is shared code. But if it's mostly private pages (e.g.,
a
lot of transient data on the stack, lots of space allocated in the heap,
even
compiled code that either modifies its text section), then you have a
problem.
I don't have a lot of personal experience with the relevant tools, but there
are such tools. But if it's all home-grown code and the applications really
are built with shared libraries, then you need to be looking at the source
to
understand what it's doing with data structures -- in memory private pages
is
the real scaling problem. The code in the shared libraries is going to all
be
shared -- trust me on that. And the code in the application programs also
is
going to be shared among multiple instances of execution of the same image,
as
long as it's running from the same file in the file system.
One good measure of the number of "private" pages in use on the system is
the
amount of swap consumed, provided you're using "eager" swap allocation.
When
any private page is allocated in memory, a page in swap is reserved for it
as
soon as it is allocated, when you use "eager" swap. Shared pages (that have
a
home in the file system) are NEVER allocated reserved space in swap. So the
total reserved space (not necessarily the in use space) from "swapon -s" is
a good indication.
The "scaling cost" for each subsequent instance of the application set can
be
determined more or less accurately from this data, empirically. The real
cost
is private pages that have to be in the working set (RSS) for the
application
to run. The shared pages may well be resident as well, and count in the
RSS,
but they aren't a "real" cost since there's really only one copy of each
page.
You can also use tools like "vmstat -P" to examine physical memory
consumption.
>From Susan Rodriguez (susrod_at_hbsi.com)
I know from talking to some of my developers that executables can be
compiled in two ways, either independent of the libraries they need, in
which case they go out and find them on the system when they need to run, or
with all the library info as part of the executable.
>From Joe Mario (mario_at_zk3.dec.com)
You will get text sharing if the shared objects are built at
a "reserved" address in the so_locations file.
The system libraries have their reserved addresses, the
user libraries need to have their reserved addresses or else
all your shared libs will conflict and you'll get no sharing.
You can set the _RLD_ARGS variable to "-debug map" to see
if the shared libraries are getting loaded at the
same address (to see if you are getting sharing).
See the loader manpage for more information on _RLD_ARGS.
>From Christian Monch (cristian_at_tm.informatik.uni-frankfurt.de)
Try too set then environment variable _RLD_ARGS to a reasonable value like
"-trace" (consult `man loader' for more information). The loader will
then print a list of symbols it resolves during load together with their
origin. This should give you a first impression of the dynamically loaded
parts of your programm.
In general I would suggest to verify the memory footprint of the processes
at the moment the fork of their children, maybe the parent process already
grew to big (I am not sure how DUNIX supports memory over commitment).
In case that the state information of the different processes
is nearly the same, you may consider to use threads instead of processes,
which, with SMP, will be distributed between different CPUs.
>From David DeWolfe (sxdjd_at_java.sois.alaska.edu)
A few simple things you can do:
odump -Dl your_exe_name
Will show you what shared libraries the executable requires. Setting the
following environment variables will show you how/where the shared objects
are being resolved by the loader when they're executed (or loaded I should
say):
_RLD_ARGS=-trace (or) _RLD_ARGS=-v
See the loader man page for specifics.
>From alan_at_nabeth.cxo.dec.com
The size(1) command is a good start stop. If the text or
data parts of the program are particularly large, that is
where you need to concentrate your effort. If the BSS is
large, that should come from zero filled pages, which will
make it large at run-times as the pages are modified, but
until then the pages should be shared. Data pages should
be shared too if they're left unmodified.
The file(1) command will also offer a clue how the program
is linked.
Thanks to all
John
Received on Tue May 25 1999 - 20:37:50 NZST