devilry.apps.core.deliverystore — DeliveryStore

Deprecated since version 3.0: This module is deprecated in 3.0 and will be removed in a future release.

A DeliveryStore is a place to put the files from deliveries. In more technical terms, it is a place where each file related to a devilry.apps.core.models.FileMeta is stored.

Selecting a DeliveryStore

Devilry on comes with one DeliveryStore ready for production use, FsDeliveryStore. To enable a DeliveryStore, you have to set the DELIVERY_STORE_BACKEND-setting in your settings.py like this:

DELIVERY_STORE_BACKEND = 'devilry.apps.core.deliverystore.FsDeliveryStore'

The FsDeliveryStore also require you to define where on the disk you wish to store your files in the DELIVERY_STORE_ROOT-setting like this:

DELIVERY_STORE_ROOT = '/path/to/root/directory/of/my/deliverystore'

Creating your own DeliveryStore

To create your own DeliveryStore you have to implement DeliveryStoreInterface. A good example is FsDeliveryStore:

class FsDeliveryStore(DeliveryStoreInterface):
    """
    Filesystem-based DeliveryStore suitable for production use.

    It stores files in a filesystem hierarcy with one directory for each
    Delivery, with the delivery-id as name. In each delivery-directory, the
    files are stored by FileMeta id.
    """
    def __init__(self, root=None):
        """
        :param root: The root-directory where files are stored. Defaults to the value of the ``DELIVERY_STORE_ROOT``-setting.
        """
        self.root = root or settings.DELIVERY_STORE_ROOT

    def _get_dirpath(self, delivery_obj):
        return join(self.root, str(delivery_obj.pk))

    def _get_filepath(self, filemeta_obj):
        return join(self._get_dirpath(filemeta_obj.delivery),
                    str(filemeta_obj.pk))

    def read_open(self, filemeta_obj):
        filepath = self._get_filepath(filemeta_obj)
        if not exists(filepath):
            raise FileNotFoundError(filemeta_obj)
        return open(filepath, 'rb')

    def _create_dir(self, filemeta_obj):
        dirpath = self._get_dirpath(filemeta_obj.delivery)
        if not exists(dirpath):
            makedirs(dirpath)

    def write_open(self, filemeta_obj):
        self._create_dir(filemeta_obj)
        return open(self._get_filepath(filemeta_obj), 'wb')

    def remove(self, filemeta_obj):
        filepath = self._get_filepath(filemeta_obj)
        if not exists(filepath):
            raise FileNotFoundError(filemeta_obj)
        remove(filepath)

    def exists(self, filemeta_obj):
        filepath = self._get_filepath(filemeta_obj)
        return exists(filepath)

    def copy(self, filemeta_obj_from, filemeta_obj_to):
        frompath = self._get_filepath(filemeta_obj_from)
        topath = self._get_filepath(filemeta_obj_to)
        self._create_dir(filemeta_obj_to)
        shutil_copy(frompath, topath)
class devilry.apps.core.testhelpers.DeliveryStoreTestMixin

Mixin-class that tests if devilry.core.deliverystore.DeliveryStoreInterface is implemented correctly.

You only need to override get_storageobj(), and maybe setUp() and tearDown(), but make sure you call super(..., self).setUp() if you override it.

You must mixin this class before django.test.TestCase like so:

class TestMyDeliveryStore(DeliveryStoreTestMixin, django.test.TestCase):
    ...

Setting the DeliveryStore manually - for tests

You might need to set the DeliveryStore manually if you need to handle deliveries in your own tests. Just set devilry.apps.core.FileMeta.deliveryStore like this:

from django.test import TestCase
from devilry.apps.core.models import FileMeta, Delivery
from devilry.apps.core.deliverystore import MemoryDeliveryStore

class MyTest(TestCase):
    def test_something(self):
        FileMeta.deliverystore = MemoryDeliveryStore()
        delivery = Delivery.begin(assignmentgroup, user)
        delivery.add_file('hello.txt', ['hello', 'world'])
        delivery.finish()

API

exception devilry.apps.core.deliverystore.FileNotFoundError(filemeta_obj)

Exception to be raised when the remove method of a DeliveryStore does not find the given file.

class devilry.apps.core.deliverystore.MemFile(initial_value='', newline='\n')
close()

Close the IO object.

Attempting any further operation after the object is closed will raise a ValueError.

This method has no effect if the file is already closed.

class devilry.apps.core.deliverystore.DeliveryStoreInterface

The interface all deliverystores must implement. All methods raise NotImplementedError.

read_open(filemeta_obj)

Return a file-like object opened for reading.

The returned object must have close() and read() methods as defined by the documentation of the standard python file-class.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

write_open(filemeta_obj)

Return a file-like object opened for writing.

The returned object must have close() and write() methods as defined by the documentation of the standard python file-class.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

remove(filemeta_obj)

Remove the file.

Note that this method is called before the filemeta_obj is removed. This means that the file might be removed, and the removal of the filemeta_obj can still fail. To prevent users from having to manually resolve such cases implementations should check if the file exists, and raise FileNotFoundError if it does not.

The calling function has to check for FileNotFoundError and handle any other error.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

exists(filemeta_obj)

Return True if the file exists, False if not.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

copy(filemeta_obj_from, filemeta_obj_to)

Copy the underlying file-object for filemeta_obj_from into the file-object for filemeta_obj_to.

Defaults to an inefficient implementation using read_open() and meth:.write_open. Should be overridden for backends with some form of native copy-capability.

class devilry.apps.core.deliverystore.FsDeliveryStore(root=None)

Filesystem-based DeliveryStore suitable for production use.

It stores files in a filesystem hierarcy with one directory for each Delivery, with the delivery-id as name. In each delivery-directory, the files are stored by FileMeta id.

Parameters:

root – The root-directory where files are stored. Defaults to the value of the DELIVERY_STORE_ROOT-setting.

read_open(filemeta_obj)

Return a file-like object opened for reading.

The returned object must have close() and read() methods as defined by the documentation of the standard python file-class.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

write_open(filemeta_obj)

Return a file-like object opened for writing.

The returned object must have close() and write() methods as defined by the documentation of the standard python file-class.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

remove(filemeta_obj)

Remove the file.

Note that this method is called before the filemeta_obj is removed. This means that the file might be removed, and the removal of the filemeta_obj can still fail. To prevent users from having to manually resolve such cases implementations should check if the file exists, and raise FileNotFoundError if it does not.

The calling function has to check for FileNotFoundError and handle any other error.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

exists(filemeta_obj)

Return True if the file exists, False if not.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

copy(filemeta_obj_from, filemeta_obj_to)

Copy the underlying file-object for filemeta_obj_from into the file-object for filemeta_obj_to.

Defaults to an inefficient implementation using read_open() and meth:.write_open. Should be overridden for backends with some form of native copy-capability.

class devilry.apps.core.deliverystore.FsHierDeliveryStore(root=None, interval=None)

Filesystem-based DeliveryStore suitable for production use with huge amounts of deliveries.

Parameters:
  • root – The root-directory where files are stored. Defaults to the value of the DEVILRY_FSHIERDELIVERYSTORE_ROOT-setting.

  • interval – The interval. Defaults to the value of the DEVILRY_FSHIERDELIVERYSTORE_INTERVAL-setting.

get_path_from_deliveryid(deliveryid)
>>> fs = FsHierDeliveryStore('/stuff/', interval=1000)
>>> fs.get_path_from_deliveryid(deliveryid=2001000)
(2, 1)
>>> fs.get_path_from_deliveryid(deliveryid=1000)
(0, 1)
>>> fs.get_path_from_deliveryid(deliveryid=1005)
(0, 1)
>>> fs.get_path_from_deliveryid(deliveryid=2005)
(0, 2)
>>> fs.get_path_from_deliveryid(deliveryid=0)
(0, 0)
>>> fs.get_path_from_deliveryid(deliveryid=1)
(0, 0)
>>> fs.get_path_from_deliveryid(deliveryid=1000000)
(1, 0)
class devilry.apps.core.deliverystore.MemoryDeliveryStore

Memory-base DeliveryStore ONLY FOR TESTING.

This is only for testing, and it does not handle parallel access. Suitable for unittesting.

read_open(filemeta_obj)

Return a file-like object opened for reading.

The returned object must have close() and read() methods as defined by the documentation of the standard python file-class.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

write_open(filemeta_obj)

Return a file-like object opened for writing.

The returned object must have close() and write() methods as defined by the documentation of the standard python file-class.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

remove(filemeta_obj)

Remove the file.

Note that this method is called before the filemeta_obj is removed. This means that the file might be removed, and the removal of the filemeta_obj can still fail. To prevent users from having to manually resolve such cases implementations should check if the file exists, and raise FileNotFoundError if it does not.

The calling function has to check for FileNotFoundError and handle any other error.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

exists(filemeta_obj)

Return True if the file exists, False if not.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

class devilry.apps.core.deliverystore.DjangoStorageDeliveryStore(root=None, storage_backend=None)

Delivery store backend that uses Django storages.

Initialize the deliverystore with an optional root directory (relative to MEDIA_ROOT) and optionally a custom storage backend.

Parameters:
  • root – The root-directory relative to MEDIA_ROOT where files are stored. Defaults to the value of the DELIVERY_STORE_ROOT-setting.

  • storage_backend – The django storage backend to use. Defaults to django.core.files.storage.default_storage.

read_open(filemeta_obj)

Return a file-like object opened for reading.

The returned object must have close() and read() methods as defined by the documentation of the standard python file-class.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

write_open(filemeta_obj)

Return a file-like object opened for writing.

The returned object must have close() and write() methods as defined by the documentation of the standard python file-class.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

remove(filemeta_obj)

Remove the file.

Note that this method is called before the filemeta_obj is removed. This means that the file might be removed, and the removal of the filemeta_obj can still fail. To prevent users from having to manually resolve such cases implementations should check if the file exists, and raise FileNotFoundError if it does not.

The calling function has to check for FileNotFoundError and handle any other error.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

exists(filemeta_obj)

Return True if the file exists, False if not.

Parameters:

filemeta_obj – A devilry.core.models.FileMeta-object.

copy(filemeta_obj_from, filemeta_obj_to)

Copy the underlying file-object for filemeta_obj_from into the file-object for filemeta_obj_to.

Defaults to an inefficient implementation using read_open() and meth:.write_open. Should be overridden for backends with some form of native copy-capability.