Dropwizard 0.7 service hangs while writing output

I have a dropwizard 0.7.0 service.

  • jetty 9.0.7
  • jersey 1.18.1

Occasionally (1 in 5000 requets) the service will spend 60 seconds writing it's response. AppDynamics is showing that 60 seconds are being spent inside com.sun.jersey.spi.container.servlet.WebComponent$Writer:write:300. After 60 seconds we get the following exception:

org.eclipse.jetty.io.EofException.Closed at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:171) at org.eclipse.jetty.servlets.gzip.AbstractCompressedStream.write(AbstractCompressedStream.java:226) at com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300) at com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135) at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:1862) at com.fasterxml.jackson.core.json.UTF8JsonGenerator.close(UTF8JsonGenerator.java:1087) at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:649) at com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:715) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at io.dropwizard.jetty.NonblockingServletHolder.handle(NonblockingServletHolder.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1515) at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83) at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:348) at io.dropwizard.jetty.BiDiGzipFilter.doFilter(BiDiGzipFilter.java:127) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1486) at io.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:29) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1486) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:519) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1097) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:448) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1031) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:173) at io.dropwizard.jetty.RoutingHandler.handle(RoutingHandler.java:51) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:92) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:162) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:446) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:271) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:246) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:358) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:532) at java.lang.Thread.run(Thread.java:744)

We also see the following in the logs that shows that the connection is idle for at least 30 seconds:

WARN [2015-05-14 09:54:30,511] org.eclipse.jetty.server.HttpChannel: commit failed ! java.util.concurrent.TimeoutException: Idle timeout expired: 30001/30000 ms ! at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:153) ~[report-configuration-service-] ! at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50) ~[report-configuration-service-] ! at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_45] ! at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_45] ! at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) ~[na:1.7.0_45] ! at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) ~[na:1.7.0_45] ! at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_45] ! at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_45] ! at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]

What could cause such behaviour?

A couple of other observations:

  • The errors do not correspond with peaks in load.
  • This error happens on multiple endpoints; there is no definite link between these endpoints, but they are some of the endpoints returning larger entities.
  • The resource method is succeeding, i.e. the entity is fetched from the database successfully along with any required processing and authorization.
Category:java Views:5 Time:2018-11-30

Related post

  • Git hangs while writing objects 2011-07-31

    I'm trying to git push -u origin master And it just hangs at Writing objects: 99% (219/220), 12.65 MiB | 97 KiB/s The 12.65 part shifts around. When I exit the process and run it again, it resumes at 99% but never finishes, same as before. It's never

  • PsExec hang while being executed from a very simple c# or c++ gui program compiled as "windows application" 2011-04-13

    I am experiencing PsExec hang while being executed from a very simple c# or c++ gui program compiled as "windows application" (not as "console application"). Under section C) below I have pasted the code to reproduce the problem and under section D)

  • Performance issues in FileStream.Write while writing bytes decoded using AsciiEncoding.GetBytes and Convert.FromBase64String 2009-05-15

    I am facing a performance problem while using the FileStream.Write function. I have a console application that i use to read a Base64 string from a file (~ size is 400 KB) using a StreamReader object. I convert this string to a byte array using Conve

  • Facing an issue with recv() and send() winsock api. Recv() hangs while receving the last packet 2009-08-22

    I am facing an issue with recv() and send() winsock api. Recv() hangs while receving the last packet. Problem Description:- System A's app is writing data over a non-blocking socket and system B's app is receiving data over a blocking socket in chunk

  • Is there any problem using the '@' while writing the code 2009-10-23

    Possible Duplicate: Reference - What does this symbol mean in PHP? Is there any problem using the '@' while writing the code? and what is the mechanism behind it? --------------Solutions------------- It suppresses error reporting so I guess it makes

  • Using Mono Compiler/Creating a new one while still outputting a Windows EXE? 2010-03-02

    This is just a thought for now, but I wonder how hard it would be to either use the Mono Compiler or write a new one while still outputting a Windows EXE? The reason for this thought is that I see people "abusing" the C# language creatively to get so

  • WCF Service hangs on the 14th call 2010-03-10

    I'm having a problem where the WCF service hangs after 13-14 asynchronous process calls from the client. This occurs all the time. The client is a mobile JavaFX app. There is no specific error outputted in the server as well as in client. Someone sug

  • JAVA - Apache POI OutOfMemoryError while writing Excel File 2010-07-06

    I am writing an Excel File using Apache POI. I want to write in it all the data of myResultSet which has the fieldnames(columns) stored in the String[] fieldnames. I have 70000 rows and 27 columns My Code: String xlsFilename = "myXLSX.xlsx"; org.apac

  • Adding special characters while writing into a java file but not visible everywhere 2011-06-29

    I am writing into a java file through FileWriter fstream = new FileWriter("someFile.java"); BufferedWriter out = new BufferedWriter(fstream); out.write(strContents); // Close the output stream out.close(); but after writing I found it appended some s

  • Why does my application hang while updating database records? 2011-08-29

    The following application code hangs while updating database records. It only displays 0 in output screen, and program hangs forever. However there are more than 50,000 records in table. (The program converts ISCII codes to UNICODE and writes them ba

  • Getting NULL value at the end while writing to the file 2011-11-27

    I have problem while writing to the file. I want to write contents of my input file to output file but while writing to the file, I am getting NULL value written at the end of file. What's the reason behind that? My code is: import java.io.BufferedRe

  • Service Hang WdiService Host 2013-03-20

    After several other issues...maybe related maybe not. I now have a list of WDIService Host errors. Below is one: Product WdiServiceHost Problem Service Hang Report Date 1/30/2014 10:26 AM Product WdiServiceHost Problem Problem Signature Problem Enven

  • Is it required to use sessions while writing API in nodejs? 2013-06-07

    I'm using Connect + ExpressJs. I'm confused about the point of sessions and session management while writing RESTfull API. Can anyone explain to me the point of session management here? Edit: Was going through this example on ExpressJs which does ind

  • Windows hanging while gaming 2013-06-24

    Windows Vista started to hang while gaming Tried various suggestions from the Microsoft support site to no avail. Upgraded to Windows 7 - problem persists. Increased problem whereby on start up display does not work. Only solution forced shut down -

  • 1. Service hang on starting hpqddsvc 2.No help and support when logged on as administrator 2013-12-24

    The following error message is generated and sent every time the computer is started and a report is sent to Microsoft supposedly. Event name Service hang Service name hpqddsvc Image name hpqddsvc.dll Image Version Service Type 20 Start t

  • Get an error impossible creatures setup encountered a disk error while writing to the file 2014-01-17

    Original Title: error installing impossible creatures on computer im trying to install impossible creatures on my computer but i keep getting an error saying "impossible creatures setup encountered a disk error while writing to the file" any suggesti

  • My Firefox/chrome/IE hangs while browsing the internet. 2014-04-02

    My Firefox/chrome/IE hangs while browsing the internet. Is this a problem with Win 7?? I have to restart my system every time this happens. It just freezes and nothing works. --------------Solutions------------- Hello Tarang Anthwal, Thanks for posti

  • Use of Service Agents while calling web Service 2009-03-06

    Can any one explain the use of service agent while calling web service? how it fit in project architecture? --------------Solutions------------- In MVC a.k.a. Model View Controller, View -> Controller -> Service Agent Service Agent acts like pr

  • Duplex WCF service hangs on 11th call 2009-07-17

    I've got a duplex WCF service which hangs after the magic 10 proxy instantiations. The specific error on the client is: "System.TimeoutException: This request operation sent to net.tcp://localhost:8080/RoomService/netTcp did not receive a reply withi

  • How do ensure that while writing C++ code itself it will not cause any memory leaks? 2009-08-20

    Running valgrind or purify would be the next steps But while while writing the code itself how do you ensure that it will not cause any memory leaks? You can ensure following things:- 1: Number of new equal to delete 2: Opened File descriptor is clos

  • Any solution for reducing browser compatibility problem while writing css style as a web designer/web developer 2009-09-13

    Any solution for reducing browser compatibility problem while writing css style. --------------Solutions------------- These three are the main points you need to do yourself: Write correct markup - make sure it validates Make sure your markup is in s

  • 'System.OutOfMemoryException' while writing 400.000 records to file 2009-11-28

    I really REALLY hope that someone could help me. I have to pass almost 400.000 records from a database to a textfile Because of this amount the extension of the textfile is just something I made up ...(.you_name_it) I get an error while writing the d

  • how to use "tab space" while writing in text file 2010-04-06

    SimpleDateFormat formatter = new SimpleDateFormat("ddMMyyyy_HHmmSS"); String strCurrDate = formatter.format(new java.util.Date()); String strfileNm = "Cust_Advice_" + strCurrDate + ".txt"; String strFileGenLoc = strFileLocation + "/" + strfileNm; Str

  • JAVA - Out Of Memory Error while writing Excel Cells in jxl 2010-07-02

    I am using JXL to write an excel file of 50000 rows and 30 columns. My code looks like this: for (int j = 0; j < countOfRows; j++) { myWritableSheet.addCell(new Label(0, j, myResultSet.getString(1), myWritableCellFormat)); myWritableSheet.addCell(

  • NSTextView hangs while typing 2010-10-11

    I'm experiencing hangs while typing on an NSTextView. The NSTextView is on a window that I run modally. [NSApp runModalForWindow:[self window]]; I'm not really sure how, but I think it has something to do with the scrollbars. When I click the "enter

Copyright (C) dskims.com, All Rights Reserved.

processed in 0.136 (s). 11 q(s)