Tomcat keeps dying. Need help examining logfile [Archive]

Click to See Complete Forum and Search --> : Tomcat keeps dying. Need help examining logfile

sparkles43

04-03-2006, 02:17 PM

My tomcat service keeps dying. I cannot figure out why this is happening but need help decyphering the logfiles

I think these are the pertenant errors but can someone tell me what they mean?

SEVERE: Error decoding request
java.io.IOException
at org.apache.jk.common.JkInputStream.receive(JkInput Stream.java:252)
at org.apache.jk.common.HandlerRequest.decodeRequest( HandlerRequest.java:525)
at org.apache.jk.common.HandlerRequest.invoke(Handler Request.java:363)
at org.apache.jk.common.ChannelSocket.invoke(ChannelS ocket.java:748)
at org.apache.jk.common.ChannelSocket.processConnecti on(ChannelSocket.java:678)
at org.apache.jk.common.SocketConnection.runIt(Channe lSocket.java:871)
at org.apache.tomcat.util.threads.ThreadPool$ControlR unnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)

Apr 2, 2006 10:46:27 PM org.apache.tomcat.util.threads.ThreadPool logFull
SEVERE: All threads (200) are currently busy, waiting. Increase maxThreads (200) or check the servlet status

AND

SEVERE: Error decoding request
java.io.IOException
at org.apache.jk.common.JkInputStream.receive(JkInput Stream.java:252)
at org.apache.jk.common.HandlerRequest.decodeRequest( HandlerRequest.java:525)
at org.apache.jk.common.HandlerRequest.invoke(Handler Request.java:363)
at org.apache.jk.common.ChannelSocket.invoke(ChannelS ocket.java:748)
at org.apache.jk.common.ChannelSocket.processConnecti on(ChannelSocket.java:678)
at org.apache.jk.common.SocketConnection.runIt(Channe lSocket.java:871)
at org.apache.tomcat.util.threads.ThreadPool$ControlR unnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)

Apr 2, 2006 10:46:27 PM org.apache.tomcat.util.threads.ThreadPool logFull
SEVERE: All threads (200) are currently busy, waiting. Increase maxThreads (200) or check the servlet status

Help! :confused:

Choozo

04-03-2006, 03:03 PM

Apr 2, 2006 10:46:27 PM org.apache.tomcat.util.threads.ThreadPool logFull
SEVERE: All threads (200) are currently busy, waiting. Increase maxThreads (200) or check the servlet statusThere you have it. Increase maxThreads on your Tomcat server properties page.

voidinit

04-03-2006, 07:36 PM

Yup. Find the server.xml file in your tomcat install. Look for a section in the file that looks like this:

<Connector port="8080" address="${jboss.bind.address}"
maxThreads="250" strategy="ms" maxHttpHeaderSize="8192"
emptySessionPath="true"
enableLookups="false" redirectPort="8443" acceptCount="100"
connectionTimeout="20000" disableUploadTimeout="true"/>

There might be multiple sections, some commented out or whatever. Basically, match the port you are connecting to to the section. Then up the maxThreads property.

However, keep in mind that running the thread pool dry can actually be caused by three major things.

1.) Raw traffic. If the server is taking a lot of requests per second, it can easily overflow the boundaries of the thread pool and run out of threads. Upping the maxThreads threshold can help fix this problem.

2.) Slow servlets. Even if the server isn't taking a whole lot of requests per second, if the servlets take a long time to execute, then several more requests might be taken while the first request is still running. Basically, requests are coming in faster than they are finishing up and at some point you'll run out of threads to work with. This can be fixed by upping maxThreads to a degree, but optimizing the performance of the servlets is probably a better idea.

3.) Servlets aren't exiting properly. A servlets do{Get,Post,Head,etc.} method must return before that thread will be released to the pool. If they are not, then eventually you will run out of threads no matter what size you set maxThreads to. The only way to fix this is to fix the code. Here's a little example of a servlet that appears to exit, but really doesn't.

protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException{
int i;
PrintWriter out = resp.getWriter();
out.println("<HTML><HEAD><TITLE>Foo</TITLE></HEAD><BODY>Foo!</BODY></HTML>");
out.flush();
out.close();
while(true){
i++;
}
}

To the client web browser, the servlet has finished and returned all the content, so the content is displayed (because the output stream has been flushed/closed). However, the servlet will continue to run, detached from the client, forever and never free it's resources (threads) back into the pool.

The same can be said for slow serlvets that don't appear to be slow. For example, if the servlet writes out a response flushes/closes the stream then does some more work that takes a long time it appears the servlet has exited when in fact it's still running for a long time.

sparkles43

04-04-2006, 11:17 AM

So is my error showing the problem is between the web and app tier or the app and DB tier? I am also getting this message.
Apr 3, 2006 9:07:30 PM org.apache.catalina.core.ApplicationContext log
INFO: org.apache.webapp.balancer.BalancerFilter: init(): ruleChain: [org.apache.webapp.balancer.RuleChain: [org.apache.webapp.balancer.rules.URLStringMatchRul e: Target string: News / Redirect URL: http://www.cnn.com], [org.apache.webapp.balancer.rules.RequestParameterR ule: Target param name: paramName / Target param value: paramValue / Redirect URL: http://www.yahoo.com], [org.apache.webapp.balancer.rules.AcceptEverythingR ule: Redirect URL: http://jakarta.apache.org]]

Thanks