Django handles database connections transparently in almost all cases. It will start a new connection when your request starts up, and commit it at the end of the request lifetime. Other times you need to dive in further and do your own granular transaction management. But for the most part, it's fully automatic.
However, sometimes your use case may require that you close the current database connection and open a new one. While this is possible in Django, it's not well documented.
Why would you want to do this? I my case, I was writing an automation test framework. Some of the automation tests make database calls through the Django ORM to setup records, clean up after the test, etc. Each test is executed in the same process space, via a thread pool. We found that if one of the early tests threw an unrecoverable database error, such as an IntegrityError due to violating a unique constraint, the database connection would be aborted. Subsequent tests that tried to use the database would raise a DatabaseError:
Traceback (most recent call last):
File /home/user/project/app/test.py, line 73, in tearDown
MyModel.objects.all()
File /usr/local/lib/python2.6/dist-packages/django/db/models/query.py, line 444, in delete
collector.collect(del_query)
File /usr/local/lib/python2.6/dist-packages/django/db/models/deletion.py, line 146, in collect
reverse_dependency=reverse_dependency)
File /usr/local/lib/python2.6/dist-packages/django/db/models/deletion.py, line 91, in add
if not objs:
File /usr/local/lib/python2.6/dist-packages/django/db/models/query.py, line 113, in __nonzero__
iter(self).next()
File /usr/local/lib/python2.6/dist-packages/django/db/models/query.py, line 107, in _result_iter
self._fill_cache()
File /usr/local/lib/python2.6/dist-packages/django/db/models/query.py, line 772, in _fill_cache
self._result_cache.append(self._iter.next())
File /usr/local/lib/python2.6/dist-packages/django/db/models/query.py, line 273, in iterator
for row in compiler.results_iter():
File /usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py, line 680, in results_iter
for rows in self.execute_sql(MULTI):
File /usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py, line 735, in execute_sql
cursor.execute(sql, params)
File /usr/local/lib/python2.6/dist-packages/django/db/backends/postgresql_psycopg2/base.py, line 44, in execute
return self.cursor.execute(query, args)
DatabaseError: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
It turns out that it's relatively easy to reset the database connection. We just called the following function at the start of every test. Django is smart enough to re-initialize the connection the next time it's used, assuming that it's disconnected properly.
def reset_database_connection():
from django import db
db.close_connection()
Just what I was looking for. Thanks. In my case I want to close the database connection because my program briefly talks to django in the beginning, and then runs for an hour. I've got a lot of these running and don't want to hog the mysql connections.
ReplyDeleteI have an offline process that runs every night -- it can make ~10k updates when it runs. About once every 10 days, the site stops working and I have to restart PostgreSQL and then Apache. If I do Apache first, the site doesn't come back up.
ReplyDeleteThe only suspisious thing in the logs are a lot of these in the Postgres log:
2012-02-08 17:02:55 UTC LOG: unexpected EOF on client connection
2012-02-08 18:54:05 UTC LOG: could not receive data from client: Connection reset by peer
Anyway, I'm saying all this because I put a manual db.close_connection() at the end of my script and that message doesn't appear anymore. I have a feeling that the site will not have to restarted in 10 days time either.
So thanks!!