I got some very good answers, and 1 truely authoritative answer.
My question: (basically reaching the limits to the number of arguments
allowable - how to find what the limit is...)
> I've got a program which needs to provide a comparison value for
> 95+94+...+1 data sets, and each comparison value comes out of another
> program as a single file. Thus I'm creating large numbers of files as
> output and running it under ksh.
>
> The process: I provide a file of file names (DNA sequences). My
> home-written program reads the file of sequences, and then creates a shell
> script which calls other programs to compare each sequence against every
> other sequence. Each comparison creates a new output file (and this cannot
> be changed - I have no source for the comparison program.)
>
> It's clear that 'ls' alone has no problem with just listing all the files,
> but as soon as I use a wildcard to pick and choose among the output, I get
> error messages about too many files.
>
> So: does anyone know what the limits are? How many files? (it seems to be
> larger than 2900 and smaller that 8100?) I can rewrite the part of the
> analysis I do in order to create subdirectories for each X files, if I know
> what value I should use for X.
Best (and most authoritative answer (from David Korn...))
>You didn't say what OS the Alpha was running? Are you running
>NT or UNIX?
[ I'm using Unix... Tru64 4.0F ]
>The limit is not imposed by the shell, it is imposed by the
>exec() family of system calls. Thus, you should be able to do
> echo *
>but not
> /bin/echo *
>
>Anyway
> getconf ARG_MAX
>should tell you the total number of bytes that exec can use for arguments
>and environment combined.
Which on my system returns:
gsaf.unmc.edu> getconf ARG_MAX
38912
gsaf.unmc.edu>
( Aside - what an odd number.... 38912 bytes?? )
>
>The xargs command can be used to break the argument list into pieces
>and process for a group of arguments.
xargs(1) xargs(1)
NAME
xargs - Constructs argument lists and runs commands
SYNOPSIS
xargs [-e[eofstr]] [-E eofstr] [-i[replstr]] [-I replstr] [-l[number]] [-L
number] [-n[number]] [-ptrx] [-slength] [CommandString] [argument...]
The xargs command constructs a command line by combining a command string,
containing a command and its options or arguments, with additional argu-
ments read from standard input.
Thanks to folks for answering my questions. Several others suggested xargs
also, which I'd heard of, but never used before.
Contributors:
David Korn <dgk_at_research.att.com>
"Raymond Donovan" <rad_at_homemail.com>
Phil Farrell <farrell_at_pangea.Stanford.EDU>
Cherie Willoughby <willough_at_lucent.com>
Hans Ranke <Hans.Ranke_at_ei.tum.de>
Stan Horwitz <stan_at_astro.ocis.temple.edu>
Chad
Chad Price
Systems Manager, Genetic Sequence Analysis Facility
University of Nebraska Medical Center
986495 Nebraska Medical Center
Omaha, NE 68506-6495
cprice_at_molbio.unmc.edu
(402) 559-9527
(402) 559-4077 (FAX)
Received on Thu Nov 04 1999 - 20:54:12 NZDT