plaid.storage.common.writer =========================== .. py:module:: plaid.storage.common.writer .. autoapi-nested-parse:: Common storage writer utilities. This module provides common utilities for writing dataset metadata, problem definitions, and other auxiliary files to disk or uploading them to Hugging Face Hub. It handles serialization of infos, problem definitions, and dataset tree structures. Functions --------- .. autoapisummary:: plaid.storage.common.writer.save_infos_to_disk plaid.storage.common.writer.save_problem_definitions_to_disk plaid.storage.common.writer.save_metadata_to_disk plaid.storage.common.writer.push_infos_to_hub plaid.storage.common.writer.push_problem_definitions_to_hub plaid.storage.common.writer.push_metadata_to_hub Module Contents --------------- .. py:function:: save_infos_to_disk(path: Union[str, pathlib.Path], infos: dict[str, dict[str, str]]) -> None Save dataset infos as a YAML file to disk. :param path: The directory path where the infos file will be saved. :type path: Union[str, Path] :param infos: Dictionary containing dataset infos. :type infos: dict[str, dict[str, str]] .. py:function:: save_problem_definitions_to_disk(path: Union[str, pathlib.Path], pb_defs: Union[plaid.ProblemDefinition, collections.abc.Iterable[plaid.ProblemDefinition]]) -> None Save ProblemDefinitions to disk. :param path: The directory path for saving. :type path: Union[str, Path] :param pb_defs: The problem definitions to save. :type pb_defs: Union[ProblemDefinition, Iterable[ProblemDefinition]] .. py:function:: save_metadata_to_disk(path: Union[str, pathlib.Path], flat_cst: dict[str, Any], variable_schema: dict[str, Any], constant_schema: dict[str, Any], cgns_types: dict[str, Any]) -> None Save the structure of a dataset tree to disk. This function writes the constant part of the tree and its key mappings to files in the specified directory. The constant part is serialized as a pickle file, while the key mappings are saved in YAML format. :param path: Directory path where the tree structure files will be saved. :type path: Union[str, Path] :param flat_cst: Dictionary containing the constant part of the tree. :type flat_cst: dict :param variable_schema: Dictionary containing the variable schema. :type variable_schema: dict :param constant_schema: Dictionary containing the constant schema. :type constant_schema: dict :param cgns_types: Dictionary containing CGNS types. :type cgns_types: dict :returns: None .. py:function:: push_infos_to_hub(repo_id: str, infos: dict[str, dict[str, str]]) -> None Upload dataset infos to the Hugging Face Hub. Serializes the infos dictionary to YAML and uploads it to the specified repository as infos.yaml. :param repo_id: The repository ID on the Hugging Face Hub. :type repo_id: str :param infos: Dictionary containing dataset infos to upload. :type infos: dict[str, dict[str, str]] :raises ValueError: If the infos dictionary is empty. .. py:function:: push_problem_definitions_to_hub(repo_id: str, pb_defs: Union[plaid.ProblemDefinition, collections.abc.Iterable[plaid.ProblemDefinition]]) -> None Upload ProblemDefinitions to Hugging Face Hub. :param repo_id: The repository ID on the Hugging Face Hub. :type repo_id: str :param pb_defs: The problem definitions to upload. :type pb_defs: Union[ProblemDefinition, Iterable[ProblemDefinition]] .. py:function:: push_metadata_to_hub(repo_id: str, flat_cst: dict[str, Any], variable_schema: dict[str, Any], constant_schema: dict[str, Any], cgns_types: dict[str, Any]) -> None Upload a dataset's tree structure to a Hugging Face dataset repository. This function pushes two components of a dataset tree structure to the specified Hugging Face Hub repository: 1. `flat_cst`: the constant parts of the dataset tree, serialized as a pickle file (`tree_constant_part.pkl`). 2. `variable_schema`: the variable schema, serialized as a YAML file (`variable_schema.yaml`). 3. `constant_schema`: the constant schema, serialized as a YAML file (`constant_schema.yaml`). 4. `cgns_types`: the CGNS types, serialized as a YAML file (`cgns_types.yaml`). :param repo_id: The Hugging Face dataset repository ID where files will be uploaded. :type repo_id: str :param flat_cst: Dictionary containing constant values in the dataset tree. :type flat_cst: dict[str, Any] :param variable_schema: Dictionary containing the variable schema. :type variable_schema: dict[str, Any] :param constant_schema: Dictionary containing the constant schema. :type constant_schema: dict[str, Any] :param cgns_types: Dictionary containing CGNS types. :type cgns_types: dict[str, Any] :returns: None .. note:: - Each upload includes a commit message indicating the filename. - This function is not covered by unit tests (`pragma: no cover`).