devilry.apps.core.deliverystore — DeliveryStore

A DeliveryStore is a place to put the files from deliveries. In more technical terms, it is a place where each file related to a devilry.apps.core.models.FileMeta is stored.

Selecting a DeliveryStore

Devilry on comes with one DeliveryStore ready for production use, FsDeliveryStore. To enable a DeliveryStore, you have to set the DELIVERY_STORE_BACKEND-setting in your settings.py like this:

DELIVERY_STORE_BACKEND = 'devilry.apps.core.deliverystore.FsDeliveryStore'

The FsDeliveryStore also require you to define where on the disk you wish to store your files in the DELIVERY_STORE_ROOT-setting like this:

DELIVERY_STORE_ROOT = '/path/to/root/directory/of/my/deliverystore'

Creating your own DeliveryStore

To create your own DeliveryStore you have to implement DeliveryStoreInterface. A good example is FsDeliveryStore:

class FsDeliveryStore(DeliveryStoreInterface):
    """
    Filesystem-based DeliveryStore suitable for production use.

    It stores files in a filesystem hierarcy with one directory for each
    Delivery, with the delivery-id as name. In each delivery-directory, the
    files are stored by FileMeta id.
    """
    def __init__(self, root=None):
        """
        :param root: The root-directory where files are stored. Defaults to the value of the ``DELIVERY_STORE_ROOT``-setting.
        """
        self.root = root or settings.DELIVERY_STORE_ROOT

    def _get_dirpath(self, delivery_obj):
        return join(self.root, str(delivery_obj.pk))

    def _get_filepath(self, filemeta_obj):
        return join(self._get_dirpath(filemeta_obj.delivery),
                    str(filemeta_obj.pk))

    def read_open(self, filemeta_obj):
        filepath = self._get_filepath(filemeta_obj)
        if not exists(filepath):
            raise FileNotFoundError(filemeta_obj)
        return open(filepath, 'rb')

    def _create_dir(self, filemeta_obj):
        dirpath = self._get_dirpath(filemeta_obj.delivery)
        if not exists(dirpath):
            makedirs(dirpath)

    def write_open(self, filemeta_obj):
        self._create_dir(filemeta_obj)
        return open(self._get_filepath(filemeta_obj), 'wb')

    def remove(self, filemeta_obj):
        filepath = self._get_filepath(filemeta_obj)
        if not exists(filepath):
            raise FileNotFoundError(filemeta_obj)
        remove(filepath)

    def exists(self, filemeta_obj):
        filepath = self._get_filepath(filemeta_obj)
        return exists(filepath)

    def copy(self, filemeta_obj_from, filemeta_obj_to):
        frompath = self._get_filepath(filemeta_obj_from)
        topath = self._get_filepath(filemeta_obj_to)
        self._create_dir(filemeta_obj_to)
        shutil_copy(frompath, topath)

Testing your own DeliveryStore

We provide a mixing-class, devilry.apps.core.testhelpers.DeliveryStoreTestMixin, for you to extend when writing unit-tests for your DeliveryStore. Here is how we test FsDeliveryStore:

class TestFsDeliveryStore(DeliveryStoreTestMixin, TestCase):
    def setUp(self):
        self.root = mkdtemp()
        super(TestFsDeliveryStore, self).setUp()

    def get_storageobj(self):
        return FsDeliveryStore(self.root)

    def tearDown(self):
        rmtree(self.root)
class devilry.apps.core.testhelpers.DeliveryStoreTestMixin

Bases: devilry.apps.core.testhelper.TestHelper

Mixin-class that tests if devilry.core.deliverystore.DeliveryStoreInterface is implemented correctly.

You only need to override get_storageobj(), and maybe setUp() and tearDown(), but make sure you call super(..., self).setUp() if you override it.

You must mixin this class before django.test.TestCase like so:

class TestMyDeliveryStore(DeliveryStoreTestMixin, django.test.TestCase):
    ...
get_storageobj()

Return a object implementing devilry.core.deliverystore.DeliveryStoreInterface

setUp()

Make sure to call this if you override it in subclasses, or the tests will fail.

Setting the DeliveryStore manually - for tests

You might need to set the DeliveryStore manually if you need to handle deliveries in your own tests. Just set devilry.apps.core.FileMeta.deliveryStore like this:

from django.test import TestCase
from devilry.apps.core.models import FileMeta, Delivery
from devilry.apps.core.deliverystore import MemoryDeliveryStore

class MyTest(TestCase):
    def test_something(self):
        FileMeta.deliverystore = MemoryDeliveryStore()
        delivery = Delivery.begin(assignmentgroup, user)
        delivery.add_file('hello.txt', ['hello', 'world'])
        delivery.finish()

API

exception devilry.apps.core.deliverystore.FileNotFoundError(filemeta_obj)

Bases: exceptions.Exception

Exception to be raised when the remove method of a DeliveryStore does not find the given file.

class devilry.apps.core.deliverystore.DeliveryStoreInterface

Bases: object

The interface all deliverystores must implement. All methods raise NotImplementedError.

read_open(filemeta_obj)

Return a file-like object opened for reading.

The returned object must have close() and read() methods as defined by the documentation of the standard python file-class.

Parameters:filemeta_obj – A devilry.core.models.FileMeta-object.
write_open(filemeta_obj)

Return a file-like object opened for writing.

The returned object must have close() and write() methods as defined by the documentation of the standard python file-class.

Parameters:filemeta_obj – A devilry.core.models.FileMeta-object.
remove(filemeta_obj)

Remove the file.

Note that this method is called before the filemeta_obj is removed. This means that the file might be removed, and the removal of the filemeta_obj can still fail. To prevent users from having to manually resolve such cases implementations should check if the file exists, and raise FileNotFoundError if it does not.

The calling function has to check for FileNotFoundError and handle any other error.

Parameters:filemeta_obj – A devilry.core.models.FileMeta-object.
exists(filemeta_obj)

Return True if the file exists, False if not.

Parameters:filemeta_obj – A devilry.core.models.FileMeta-object.
copy(filemeta_obj_from, filemeta_obj_to)

Copy the underlying file-object for filemeta_obj_from into the file-object for filemeta_obj_to.

Defaults to an inefficient implementation using read_open() and meth:.write_open. Should be overridden for backends with some form of native copy-capability.

class devilry.apps.core.deliverystore.FsDeliveryStore(root=None)

Bases: devilry.apps.core.deliverystore.DeliveryStoreInterface

Filesystem-based DeliveryStore suitable for production use.

It stores files in a filesystem hierarcy with one directory for each Delivery, with the delivery-id as name. In each delivery-directory, the files are stored by FileMeta id.

Parameters:root – The root-directory where files are stored. Defaults to the value of the DELIVERY_STORE_ROOT-setting.
class devilry.apps.core.deliverystore.FsHierDeliveryStore(root=None, interval=None)

Bases: devilry.apps.core.deliverystore.FsDeliveryStore

Filesystem-based DeliveryStore suitable for production use with huge amounts of deliveries.

Parameters:
  • root – The root-directory where files are stored. Defaults to the value of the DEVILRY_FSHIERDELIVERYSTORE_ROOT-setting.
  • interval – The interval. Defaults to the value of the DEVILRY_FSHIERDELIVERYSTORE_INTERVAL-setting.
get_path_from_deliveryid(deliveryid)
>>> fs = FsHierDeliveryStore('/stuff/', interval=1000)
>>> fs.get_path_from_deliveryid(deliveryid=2001000)
(2, 1)
>>> fs.get_path_from_deliveryid(deliveryid=1000)
(0, 1)
>>> fs.get_path_from_deliveryid(deliveryid=1005)
(0, 1)
>>> fs.get_path_from_deliveryid(deliveryid=2005)
(0, 2)
>>> fs.get_path_from_deliveryid(deliveryid=0)
(0, 0)
>>> fs.get_path_from_deliveryid(deliveryid=1)
(0, 0)
>>> fs.get_path_from_deliveryid(deliveryid=1000000)
(1, 0)
class devilry.apps.core.deliverystore.MemoryDeliveryStore

Bases: devilry.apps.core.deliverystore.DeliveryStoreInterface

Memory-base DeliveryStore ONLY FOR TESTING.

This is only for testing, and it does not handle parallel access. Suitable for unittesting.