Re: [MonetDB-users] hold cursors over commit in JDBC
Let me ask it this way: How can we configure driver so it doesn't fetch all records at one time. for example 100 records per each fetch, So the application server doesn't get OutOfMemoryException. Thank you.
hi guys,
I have a very large table, which I should scan it and generate reports. My code is like this:
Class.forName("nl.cwi.monetdb.jdbc.MonetDriver"); Connection conn = DriverManager.getConnection("jdbc:monetdb://localhost/online", "monetdb", "monetdb");
conn.setAutoCommit(false); conn.setHoldability(ResultSet.HOLD_CURSORS_OVER_COMMIT);
Statement stmt = conn.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY); >> stmt.setFetchSize(10); >> >> ResultSet rs = stmt.executeQuery("SELECT * FROM denorm"); >> >> while (rs.next()) { >> /* process line */ >> } >> >> conn.commit(); >> >> I do not use much ram inside while loop. I get OutOfMemeryException from JVM. >> So I think the driver ignores HOLD_CURSORS_OVER_COMMIT and fetches all records. Is there any solution? HOLD_CURSORS_OVER_COMMIT means it keeps the cursor open, right? So what > else can the driver do but keep them open (including the memory > associated to that)? What would you have expected to happen with this > setting?
On 12-06-2012 00:13:14 -0700, Majid Azimi wrote:
Let me ask it this way:
How can we configure driver so it doesn't fetch all records at one time.
That's the default mode of operation. It fetches some amount, if you think that's still too big, use Connection.setFetchSize().
for example 100 records per each fetch, So the application server doesn't get OutOfMemoryException.
The default is 250, and it uses weak references to get the best. Make sure you use forward-only cursors if you move forward only (the default), as it allows the driver to cleanup as soon as data has been read. Fabian
participants (2)
-
Fabian Groffen
-
Majid Azimi