`devilry_compressionutil` — Devilry utils models and backends¶

The devilry_compressionutil handles the creation of compressed archives.

About the devilry compressionutil app¶

The devilry_compressionutil app provides utilities for creating and adding archives using ZipUtil, by providing a backend and a registry for the specialized backends to use.

devilry_compressionutil also provides a meta class used for caching metainfo about a archive.

How to define a backend¶

Add path setting

First off, add a path to the storage location Django settings config. Call this setting DEVILRY_COMPRESSED_ARCHIVES_DIRECTORY

DEVILRY_COMPRESSED_ARCHIVES_DIRECTORY = os.path.join('path', 'to', 'storage', 'location')

Subclass backend

Create a backend for your app that you will use to compress the files by subclassing e.g PythonZipFileBackend in devilry.devilry_compressionutil.backends.backends_base.

from devilry.devilry_compressionutil.backends import backends_base

class YourAppZipBackend(backends_base.PythonZipFileBackend):
    backend_id = 'some_backend_id'

    def __init__(self, **kwargs):
        super(YourAppZipBackend, self).__init__(**kwargs)

Load backend to registry

Create an AppConfig for your app. Load the backend you created to the devilry_compressionutil s registry. Here you can register as many backends as you want, and just fetch it by its backend_id. The registry is a singleton, and to add a backend to it Registry.get_instance().add(..) must be called.

...
from devilry.devilry_compressionutil import backend_registry

class YourAppAppConfig(AppConfig):
    ...

    def ready(self):
        from devilry.your_app import your_backends
        backend_registry.Registry.get_instance().add(your_backends.YourAppZipBackend)

Get backend and add folders and files to it

Add paths and files to the backend.

from devilry.devilry_compressionutil import backend_registry

class HandleZipping:
    ...

    def your_function_that_uses_zip_backend(...):
        backend_class = backend_registry.Registry.get_instance().get('some_backend_id')

        # Path to archive inside the location specified by the storage location defined in
        # settings. ``DEVILRY_COMPRESSED_ARCHIVES_DIRECTORY``
        archive_name = 'archive_name'
        full_archive_path = os.path.join('path', 'to', 'archive', 'archive_name')

        # Instance the backend.
        backend_instance = backend_class(
            archive_path=full_archive_path,
            archive_name=archive_name,
            readmode=False
         )

        # Get files to compress
        file1 = open('examplefile1.txt', 'r')
        file2 = open('examplefile2.txt', 'r')

        # Add file1 inside a folder named folder1
        # Add file2 inside a folder named folder2
        # The archive with content will now look like this
        # archive_name.zip
        #     - folder1
        #         - examplefile1.txt
        #     - folder2
        #         - examplefile2.txt
        backend_instance.add_file(os.path.join('folder1', 'examplefile1.txt'), file1)
        backend_instance.add_file(os.path.join('folder2', 'examplefile2.txt'), file2)

        # Close the backend
        backend_instance.close()

        # Read the archive as a fileobject
        # ``read_archive()`` returns a FileObject of the archive
        # This can for instance be passed to a HttpRequest.
        backend_instance.readmode = True
        fileobj = backend_instance.read_archive()

Datamodel API for caching metainfo about a archive¶

class devilry.devilry_compressionutil.models.GenericMeta(*args, **kwargs)¶

Abstract class that implements usage of GenericForeignKey.

content_type¶: Foreignkey to Djangos ContentType.

content_object_id¶: ID of model to store.

content_object¶: An arbitrary model to store.

class devilry.devilry_compressionutil.models.CompressedArchiveMetaQueryset(model=None, query=None, using=None, hints=None)¶

Manager for class CompressedArchiveMeta.

create_meta(instance, zipfile_backend, user, user_role='')¶

Manager provides a way to create a meta entry for a archive. See CompressedArchiveMeta.

Parameters:

instance – Instance the archive is for.
zipfile_backend – base backend for compression, see PythonZipFileBackend.

delete_compressed_archives_older_than(days=None, seconds=None)¶

Delete compressed archive meta entries older than a specified number of days or seconds.

Parameters:

older_than_days (int) – Delete everything older than this.
older_than_seconds (int) – Delete everything older than this.

delete_compressed_archives_marked_as_deleted()¶: Delete all compressed archives marked as deleted.

class devilry.devilry_compressionutil.models.CompressedArchiveMeta(*args, **kwargs)¶

Metadata about a compressed archive. Name of the archive, path to it and it’s size.

created_by¶: Who created the archive.

CREATED_BY_ROLE_STUDENT = 'student'¶: Use this as value for user_role if the user is commenting as a student.

CREATED_BY_ROLE_EXAMINER = 'examiner'¶: Use this as value for user_role if the user is commenting as an examiner.

CREATED_BY_ROLE_ADMIN = 'admin'¶: Use this as value for user_role if the user is commenting as an admin.

CREATED_BY_ROLE_CHOICES = (('student', 'Student'), ('examiner', 'Examiner'), ('admin', 'Admin'))¶: Choices for the user_role field.

created_by_role¶: What role did the user create the archive with?

created_datetime¶: When the archive was created.

archive_name¶

SomeArchive2000.zip.

Type:: The actual name of the archive, Example.

archive_path¶: Path at storage location of the compressed archive. Example: https://s3-eu-central-1.amazonaws.com/BUCKET/path/to/archive/SomeArchive2000.zip

archive_size¶: Size of the archive in bytes.

backend_id¶: The ID of the backend used. This is the ID attribute backend_id.

deleted_datetime¶: When the entry was marked for deletion.

clean()¶: Hook for doing any extra model-wide validation after clean() has been called on every field by self.clean_fields. Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

exception DoesNotExist¶

exception MultipleObjectsReturned¶

content_object¶: An arbitrary model to store.

content_object_id¶: ID of model to store.

content_type¶: Foreignkey to Djangos ContentType.

Backend base classes¶

class devilry.devilry_compressionutil.backends.backends_base.BaseArchiveBackend(archive_path, archive_name='', readmode=True)¶

Specifies the interface for a backend compression-subclass.

All backends must implement this class.

Parameters:

archive_path – Full path to archive including the archive name.
archive_name – Only the archive name.
readmode – Can be read from, defaults to True.

backend_id = None¶: A unique string ID for the subclasses to use that describes what kind of backend it is.

storage_backend_url_is_secure¶: Check if storage backend can produce secure (private) urls that only the requesting user can gain access to.

classmethod get_storage_directory() → str¶

Get the storage location for archives.

Returns:: Location specified in settings DEVILRY_COMPRESSED_ARCHIVES_DIRECTORY.
Return type:: str

classmethod delete_archive(archive_path: str)¶

Deletes the archive.

Parameters:: full_path (str) – Full path to the stored archive.

open_read_binary() → File¶

Opens archive in read binary mode. Best suited for non-text files like images, videos etc or when serving file for download.

Returns:: file object in read binary mode.
Return type:: file
Raises:: ValueError – If archive is None, or readmode is False.

open_write_binary() → File¶

Opens archive in write binary mode.

Returns:: file object in read binary mode.
Return type:: file
Raises:: ValueError – If archive is None, or readmode is False.

close()¶

Close archive when done with adding files to it.

Raises:: ValueError – If archive is None.

archive_exists() → bool¶

Check if the archive exists in the storage backend.

Returns:: Does the archive exist in the storage backend?
Return type:: bool

archive_size() → int¶

Get size of archive. Uses os.stat.

Returns:: size of archive.
Return type:: int

add_file(path, djangofile: File)¶

Add files to archive.

Parameters:

path (str) – Path to the file inside the Zip-archive.
djangofile – An django.core.files.File object.

read_archive()¶

Should return a object of the underlying compression tool in readmode.

Raises:: NotImplementedError – If not implemented by subclass.

class devilry.devilry_compressionutil.backends.backends_base.StreamZipBackend(save_to_disk=False, chunk_size=None, **kwargs)¶

Parameters:

archive_path – Full path to archive including the archive name.
archive_name – Only the archive name.
readmode – Can be read from, defaults to True.

archive_size()¶

Get size of archive. Uses os.stat.

Returns:: size of archive.
Return type:: int
Raises:: ValueError – If not in readmode.

add_file(path, filelike_obj)¶

Prep a file to be written to the archive on the given path.

Parameters:

path – Path to file inside the archive.
filelike_obj – An object that behaves like a File(read, write..).

read_archive()¶

Get the zipped archive as ZipFile in readmode.

Returns:: The zipped archive.
Return type:: ZipFile

get_archive()¶: Get the filelike object of the archive for compressed files.

close()¶

Must be invoked in order to process the files into an iterable yielding the bytes of the ZIP file.

Readmode is set to True

class devilry.devilry_compressionutil.backends.backends_base.PythonZipFileBackend(**kwargs)¶

Defines a baseclass backend using ZipFile for the Registry.

This class should be subclassed by backend-specific classes(backends for Heroku, S3, etc).

Parameters:

archive_path – Full path to archive including the archive name.
archive_name – Only the archive name.
readmode – Can be read from, defaults to True.

add_file(path, djangofile: File)¶

Add files to archive.

Parameters:

path (str) – Path to the file inside the Zip-archive.
djangofile – An django.core.files.File object.

read_archive()¶

Get the zipped archive as TarFile in readmode.

Returns:: The zipped archive.
Return type:: TarFile

close()¶

Close archive when done with adding files to it.

Raises:: ValueError – If archive is None.

`devilry_compressionutil` — Devilry utils models and backends¶

About the devilry compressionutil app¶

How to define a backend¶

Datamodel API for caching metainfo about a archive¶

Backend base classes¶

Devilry

Navigation

Related Topics

devilry_compressionutil — Devilry utils models and backends¶

About the devilry compressionutil app¶

How to define a backend¶

Datamodel API for caching metainfo about a archive¶

Backend base classes¶

`devilry_compressionutil` — Devilry utils models and backends¶