you execute everything in one very large batch, why not execute (and
clear) the batch at smaller intervals?  You print "10000 inserted", but
you actually only queued up that amount of inserts, not sending anything
to the database.

Yes, and one very large batch works ok. The problem here in creating PreparedStatements.
 
You really want to make a million prepared statements here?  The server
has to store them all...
I guess so, since you're leaking prepared handles like mad

We create separate PreparedStatement for every query in our project.
There are no memory leak on the client side (we close every PreparedStatement).
Why server should store PreparedStatement forever, is it possible to configure server do not store them?
Or it works so by design, and we should cache our PreparedStatements.
-- 
Best regards,
Andrey.