Databases and transactions

Database connections are handled with Database objects. Under the hood the handled objects are regular ZODB DB object.

Before you edit any persistent data, you need to open a database connection, and thus you need to set up a database context. Generally you can configure your Database with a zodburi, but if you do not pass give any database information, a temporary in-memory database will be created.

Depending on the cases you might want to use:

Initializing a database context is done by simply creating a Database object.

>>> db = sheraf.Database("zeo://localhost:8000"): 

The database context is now created, but to handle data you need to open a connection to the database.

Database connections

The easiest way to open a connection is to use the connection() context manager:

>>> with db.connection(): 
...     m = MyModel.create()
...     # do other things

The connection() shortcut allows you to not depend on your Database object, by just passing a database name.

>>> with sheraf.connection("default"): 
...     m = MyModel.create()
...     # do other things

There is another shortcut: if you try to open a connection to the default database, you do not need to pass it to the connection() function.’

>>> with sheraf.connection(): 
...     m = MyModel.create()
...     # do other things

In a context with only one database, this generally the method most database connections are done.

You can also use it as a function decorator:

>>> @sheraf.connection()
... def do_thing(): 
...     m = MyModel.create()
...     # do other things
...
>>> do_thing() 

Warning

Note that by default, you cannot open two connections to the same database:

>>> with sheraf.connection(): 
...     with sheraf.connection():
...         m = MyModel.create()
Traceback (most recent call last):
    ...
sheraf.exceptions.ConnectionAlreadyOpened: First connection was <Connection at ...> on ... at line ...

Transactions and commits

A ITransaction is opened each time you open a connection to a database. If you want to validate the modifications you made on your model, you can use the commit argument:

>>> with sheraf.connection(commit=True): 
...     m = MyModel.create()
...     # do other things
...
>>> @sheraf.connection(commit=True)
... def do_thing(): 
...     m = MyModel.create()
...     # do other things
...
>>> do_thing() 

Another option is to use the commit() shortcut:

>>> with sheraf.connection(): 
...     m = MyModel.create()
...     # do other things
...     sheraf.commit()

If you made risky modifications, for instance something with probabilities to raise ConflictError, you might want to make several attempts so you can reread the data, and maybe avoid the conflict at the second try. For this you can use the attempt() function:

>>> def do_thing(): 
...     m = MyModel.create()
...     # do other things
...
>>> sheraf.attempt(do_thing) 

ZConfig file

Instead of passing arguments to Database, you can configure your database connections with a configuration files. It is done through a zodburi zconfig:// URI scheme.

A simple example ZConfig file:

<zodb>
    <mappingstorage>
    </mappingstorage>
</zodb>

If that configuration file is located at /etc/myapp/zodb.conf, use the following uri argument to initialize your object:

>>> sheraf.Database("zconfig:///etc/myapp/zodb.conf") 

A ZConfig file can specify more than one database. Don’t forget to specify database-name in that case to avoid conflict on name. For instance:

<zodb temp1>
    database-name database1
    <mappingstorage>
    </mappingstorage>
</zodb>
<zodb temp2>
    database-name database2
    <mappingstorage>
    </mappingstorage>
</zodb>

In that case, use a URI with a fragment identifier:

>>> db1 = sheraf.Database("zconfig:///etc/myapp/zodb.conf#temp1") 
<Database database1>
>>> db2 = sheraf.Database("zconfig:///etc/myapp/zodb.conf#temp2") 
<Database database2>

If not specified in the conf file or in the arguments passed at the initialization of the object, default zodburi values will be used:

  • database name: unnamed

  • cache size: 5000

  • cache size bytes: 0

Note that arguments passed at the initialization of the object override the conf file.

Modifying the data into the database is done with a context manager:

>>> with sheraf.connection(database_name="database1"): 
...     # currently connected to db1
...     pass

If the database name is not defined, the database_name parameter is optional.

Concurrency

Let us see how sheraf cowboys behave in parallelize contexts.

>>> class Cowboy(sheraf.Model):
...     table = "cowboys"
...     gunskill = sheraf.IntegerAttribute()
...
>>> db = sheraf.Database("zeo://localhost:{}".format(zeo_port))
...
>>> with sheraf.connection(commit=True):
...     george = Cowboy.create()

Threading

The ZODB documentation about concurrency states that database Connection, ITransactionManager and ITransaction are not thread-safe. However ZODB DB objects can be shared between threads.

This means that it is possible to create a Database object once, and then share it on several threads. However each thread should use its own connection context:

>>> import threading
>>> def practice_gun(cowboy_id):
...     # The database is available in children thread, but they
...     # need to open their own connection contexts.
...     with sheraf.connection(commit=True):
...         cowboy = Cowboy.read(cowboy_id)
...         cowboy.gunskill = cowboy.gunskill + 1000
...
>>> practice_session = threading.Thread(target=practice_gun, args=(george.id,))
>>> practice_session.start()
>>> practice_session.join()
...
>>> with sheraf.connection():
...     Cowboy.read(george.id).gunskill
1000

Opening a thread within a connection context will produce various unexpected behaviors.

Multiprocessing

When using multiprocessing, the behavior is a bit different. The Database are not shared between processes.

>>> import multiprocessing
>>> practice_session = multiprocessing.Process(target=practice_gun, args=(george.id,))
>>> practice_session.start()
>>> practice_session.join()
>>> practice_session.exitcode
1

The connection context in the practice_gun function has raised a KeyError exception because in this new process, no database has been defined. Fortunately there is a simple solution to this. The database needs to be redefined in the new process:

>>> def recreate_db_and_practice_gun(cowboy_id):
...     # The database is re-created in the child process
...     db = sheraf.Database("zeo://localhost:{}".format(zeo_port))
...
...     with sheraf.connection(commit=True):
...         cowboy = Cowboy.read(cowboy_id)
...         cowboy.gunskill = cowboy.gunskill + 1000
...     db.close()
...
>>> practice_session = multiprocessing.Process(target=recreate_db_and_practice_gun, args=(george.id,))
>>> practice_session.start()
>>> practice_session.join()
...
>>> with sheraf.connection():
...     Cowboy.read(george.id).gunskill
2000

Note

Remember that FileStorage, MappingStorage and DemoStorage cannot be used by several processes.