WAP problem

[Michael Allan]

Hey Conseo,

> ... The problem is using PreparedStatementCache as the ResultSet
> depends on the statement. (1) Both queries (and clients) work when I
> create a new PreparedStatement for each query. This cannot be solved
> by synchronization if the ResultSet object leaves it, which it does
> here. ...

If you want, you can mark your HarvestCache.get() method as:

  @ThreadRestricted("holds HarvestCache.this.getDatabase()")

The client is then responsible for synchronizing until finished with
the result set.  Here's an example:
http://zelea.com/project/votorola/_/javadoc/votorola/a/count/CountTable.PollView.html#getByIndeces%28%29

Another approach is to pass in a runner that does the client work:
http://zelea.com/project/votorola/_/javadoc/votorola/a/count/CountTable.PollView.html#runVoters%28java.lang.String,%20java.lang.String,%20votorola.a.count.CountNodeW.Runner%29

> ... We actually don't need to synchronize around Database for the
> connection. (2) Using a single connection is maybe an unnecessary
> limitation (in regard to code complexity) when using a pool of
> connections and let them do synchronization.

Right.  Although we have to synchronize all use of a cached prepared
statements, otherwise the driver is supposed to be thread safe.  It
never used to be documented as such, but lately it is.  See the note:
http://zelea.com/project/votorola/_/javadoc/src-html/votorola/g/sql/Database.html#line.15

Currently it's fast enough, so we don't need to optimize.  Later we
could remove the unecessary synchronization.  That would be step 1.

Step 2 might be connection pooling, as you say.  But that won't help
with cached statements, which must *still* be synchronized.  Right?
So we'd have to choose the most effective optimization, caching or
async execution (pooling) or something else, depending on the specific
bottleneck we're trying to clear.  (Optimization should be done in
specific contexts with a profiler attached and the measurements, old
and new, documented in the code.)

Mike


conseo said:
> Hi,
> 
> I have duplicated HarvestWAP in DiffFeedWAP and migrated the DiffFeed to use 
> this interface. Next I have enabled the TalkTrack to show spans. While loading 
> the two WAPs in parallel (Firebug shows them hitting the server in a frame of 
> a few ms), I have encountered the same mentioned closed-result-set-error as 
> yesterday (where they both have queried HarvestWAP. Switching to the DummyFeed 
> fixed the problem for the single query (by the TalkTrack) to HarvestWAP, while 
> the problem came back with the DiffFeed (causing both queries).
> 
> This is the trace from Tomcat:
> 
> 
> WARNING: 
> org.postgresql.util.PSQLException: Dieses ResultSet ist geschlossen. (meaning 
> result set is closed)
>         at 
> org.postgresql.jdbc2.AbstractJdbc2ResultSet.checkClosed(AbstractJdbc2ResultSet.java:2654)
>         at 
> org.postgresql.jdbc2.AbstractJdbc2ResultSet.next(AbstractJdbc2ResultSet.java:1786)
>         at votorola.s.wap.HarvestWAP.respond(HarvestWAP.java:210)
>         at votorola.a.web.wap.Responding.respond(Responding.java:160)
>         at votorola.a.web.wap.WAP.doGet(WAP.java:170)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
>         at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>         at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>         at 
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
>         at 
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
>         at 
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>         at 
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
>         at 
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
>         at 
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
>         at 
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>         at 
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>         at 
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987)
>         at 
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)
>         at 
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:722)
> 
> I get the result set around line 166 in HarvestWAP:
> 
>             tempRs = 
> HarvestCache.init(req().wap().vsRun()).getTable().get(start, end,
>                     tempPoll, tempUser);
> 
> Seems like this renders the *first* result set closed from looking at the 
> FireBug timing. This makes some sense, maybe the code in WAP is not properly 
> synchronized around a connection.
> 
> *solved*
> 
> Although I just wanted to go to bed, I have found the cause. The problem is 
> using PreparedStatementCache as the ResultSet depends on the statement. (1) 
> Both queries (and clients) work when I create a new PreparedStatement for each 
> query. This cannot be solved by synchronization if the ResultSet object leaves 
> it, which it does here. The cache only works as long as all database 
> access is serialized around votorola.g.sql.Database, which we also can do 
> because usually (in the other *Table classes i have seen) we don't return 
> (expose) ResultSet and copy data to our own objects before leaving 
> synchronization and being destoryed by the next call to the cached statement.
> 
> Javadocs are confusing here, on the one hand claiming to reuse statements with 
> PrepareStatement (3) and on the other hand warning you to not reuse a 
> Statement while you still access the ResultSet. (4) I guess you are expected 
> to generally copy the data immediatly and close the Statement.
> 
> We also could synchronize the queries better to the db by using a pool of 
> connections (which then should work in parallel fine), instead of binding each 
> connection to the database to our single Database object (and its single 
> connection). We actually don't need to synchronize around Database for the 
> connection. (2) Using a single connection is maybe an unnecessary limitation 
> (in regard to code complexity) when using a pool of connections and let them 
> do synchronization. 
> 
> No need to do something immediatly though. I am finally off now zzzZZZzzz
> 
> Gn8,
> c
> 
> (1) http://jdbc.postgresql.org/documentation/84/statement.html
> (2) http://jdbc.postgresql.org/documentation/84/thread.html
> (3) http://docs.oracle.com/javase/6/docs/api/java/sql/PreparedStatement.html
> (4) http://docs.oracle.com/javase/6/docs/api/java/sql/Statement.html