mercredi 6 mai 2015

Multiple crawlers database connection in Java

Let's say I instantiate multiple crawlers for the same URL. They write URLs that are processed to MySQL database. Before processing URL, they check in database if record for that page exists, so it wouldn't process already processed page again.

Here is the catch, there should exist some kind of lock, so that only one of them can read or write from that specific table, if my logic is right. So, I instantiated only one database connection (JDBC) for them to use. Still, I am unsure if this is right thing to do.

So my question is: do statements executed from single database connection run sequentially(are they queued) or does this depend on database engine it's configuration.

Aucun commentaire:

Enregistrer un commentaire