Admins;
A/S 4100, DUnix 4.0B, Java JDK 1.1.4
Problem Description
Our java server application uses a ServerSocket to accept connections from
many clients. Each client request spawns a thread that uses 2 sockets.
When the work is done the client issues a socket close() command. After
the java server runs for a few days (2-3), it crashes with the error "Too
many open files", and has to be manually resolved. Of course this always
happens on the weekend !!!
Output from "ulimit -a" indicates "nofiles(descriptors)" is set to 4096,
the system MAX without making a call from a program to set it higher.
We verified the program reaches and executes the socket close() commands.
However, using the UNIX lsof command, we're able to verify that the sockets
are still considered open by UNIX even though java has presumably closed
them (we can watch the open file list grow to +/- 4096, about 5MB of log
files).
We've checked the Sun/Java website and found many similar problems
reported, with resolutions like : "increase the number of file descriptors
size"
"not enough information to evaluate the bug"
"no workaround"
"workaround - can't think of one yet"
"I have the same problem - It's a serious BUG"
"I'll alert the media"
The problem platform was mostly Solaris on x86, with various JDK versions.
Linux running "green" threads doesn't seem to have the problem. One user
switched from "native" to "green" threads on Solaris which seemed to
resolve the problem.
Our Work-Around
We wrote a ksh script, scheduled via cron, that uses "lsof" output to check
the number of open files that are tied to the java server application.
When the number exceeds 4000, we kill the application PID and restart the
process. It's not pretty, but it prevents the application from being shut
down.
Problem Questions
1.) Does anyone have an idea how to solve this open file problem, or what
else to look for ?
2.) How can I tell if my system is running "native" (posix ???) threads,
or "green" threads ?
3.) Can I change the type of threads that I use ? And, if so, at the O/S
or application level ?
All hints, comments, etc. would be much appreciated.
George Guethlein
Lead Unix Systems Admin
Giant Food, Inc.
Landover, MD USA
Received on Mon Apr 26 1999 - 12:59:13 NZST