Skip to content

cmem_client.repositories.files¤

Repository for the file resources of DataIntegration projects.

Provides FilesRepository for uploading, reading, exporting and deleting the files of a project. Items are keyed by the composite key project_id:file_path, so a single repository spans the files of all projects.

Examples:

Upload a local file into a project and read it back:

>>> from pathlib import Path
>>> from cmem_client.client import Client
>>> client = Client.from_env()
>>> client.files.import_item(
...     path=Path("customers.csv"), key="my-project:customers.csv"
... )
>>> client.files.read("my-project:customers.csv")

List the files of a single project and inspect one of them:

>>> for resource in client.files.get_resources("my-project"):
...     print(resource.name)
>>> client.files.delete_item("my-project:customers.csv")

Classes:

FilesDeleteConfig¤

Bases: DeleteConfig

Files Delete Configuration.

Attributes:

  • model_config –

FilesExportConfig¤

Bases: ExportConfig

Files Export Configuration.

Attributes:

  • model_config –

FilesImportConfig¤

Bases: ImportConfig

Files Import Configuration.

Attributes:

  • use_archive_handler (bool) – Defaults to False here, unlike the base class, so a path is imported as a single file instead of being unpacked by the ArchiveHandler.
  • remote_file_url (str | None) – URL to stream the file content from instead of reading it from a local path. If set, the path argument may be omitted.

model_config¤

model_config = ConfigDict(extra='allow', populate_by_name=True)

remote_file_url¤

remote_file_url: str | None = None

use_archive_handler¤

use_archive_handler: bool = False

FilesRepository¤

Bases: PlainListRepository, ImportItemProtocol, DeleteItemProtocol, ExportItemProtocol

Repository for files

Functions:

  • delete_all – Delete all items from the repository
  • delete_item – Delete an item from the repository
  • export_item – Export an item from the repository to a file path.
  • fetch_data – Fetch all file resources from all projects.
  • get_resource_metadata – Retrieve metadata of a single resource
  • get_resource_usage – Retrieve usage of a single resource
  • get_resources – Fetch the list of file resources for a specific project.
  • import_item – Import an exported file to the repository
  • items – Get the items of the repository
  • keys – Get the keys of the repository
  • read – Read the content of a file into memory.
  • values – Get the values of the repository

Attributes:

  • logger (Logger) – Gets the client logger

delete_all¤

delete_all()

Delete all items from the repository

delete_item¤

delete_item(key, skip_if_missing=False, configuration=None)

Delete an item from the repository

Parameters:

  • key (str) – The key of the item to delete
  • skip_if_missing (bool) – If True, it is ignored if the deleted item even exists
  • configuration (DeleteItemConfig) – Optional configuration for deletion

Raises:

export_item¤

export_item(key, path=None, replace=False, configuration=None)

Export an item from the repository to a file path.

Parameters:

  • key (str) – The key identifying the item to export.
  • path (Path | None) – The target file path for export. If None, a path will be generated.
  • replace (bool) – Whether to replace existing files at the target path.
  • configuration (ExportItemConfig_contra | None) – Optional configuration for export behavior.

Returns:

  • Path – The actual path where the item was exported.

Raises:

fetch_data¤

fetch_data()

Fetch all file resources from all projects.

get_resource_metadata¤

get_resource_metadata(resource)

Retrieve metadata of a single resource

get_resource_usage¤

get_resource_usage(resource)

Retrieve usage of a single resource

get_resources¤

get_resources(project_id)

Fetch the list of file resources for a specific project.

import_item¤

import_item(path=None, key=None, on_conflict=ImportConflictPolicy.FAIL, configuration=None)

Import an exported file to the repository

By default, automatically handles zip files, directories, and single files using ImportItem model. Can be disabled by setting use_archive_handler=False in the configuration.

Returns:

  • str – The key of the imported item.

Raises:

  • RepositoryModificationError – If the item already exists and the conflict policy is FAIL, if the import type is not allowed for this repository, if the import request failed, or if the item is not present afterwards.

items¤

items()

Get the items of the repository

keys¤

keys()

Get the keys of the repository

logger¤

logger: logging.Logger

Gets the client logger

read¤

read(key)

Read the content of a file into memory.

Parameters:

  • key (str) – Composite key in format ‘project_id:file_path’

Returns:

  • bytes – The raw content of the file.

Raises:

values¤

values()

Get the values of the repository

Comments