esmvalcore.io.intake_esgf#

Access data using intake-esgf.

Note

It is highly recommended that you take a moment to configure intake-esgf before using it with ESMValCore. Make sure to set local_cache to a path where it can store downloaded files and if (some) ESGF data is already available on your system, point esg_dataroot to it. If you are missing certain search results, you may want to choose a different index node for searching the ESGF.

Run the command esmvaltool config copy data-intake-esgf.yml to update your configuration to use this module. This will create a file with the following content in your configuration directory:

Listing 9 Contents of data-intake-esgf.yml#

# Read CMIP and obs4MIPs data from ESGF using intake-esgf.
projects:
  CMIP6:
    data:
      intake-esgf:
        type: "esmvalcore.io.intake_esgf.IntakeESGFDataSource"
        facets:
          activity: "activity_drs"
          dataset: "source_id"
          ensemble: "member_id"
          exp: "experiment_id"
          institute: "institution_id"
          grid: "grid_label"
          mip: "table_id"
          project: "project"
          short_name: "variable_id"
  CMIP5:
    data:
      intake-esgf:
        type: "esmvalcore.io.intake_esgf.IntakeESGFDataSource"
        facets:
          dataset: "model"
          ensemble: "ensemble"
          exp: "experiment"
          frequency: "time_frequency"
          institute: "institute"
          mip: "cmor_table"
          product: "product"
          project: "project"
          short_name: "variable"
        values:
          dataset:
            "ACCESS1-0": "ACCESS1.0"
            "ACCESS1-3": "ACCESS1.3"
            "bcc-csm1-1": "BCC-CSM1.1"
            "bcc-csm1-1-m": "BCC-CSM1.1(m)"
            "CESM1-BGC": "CESM1(BGC)"
            "CESM1-CAM5": "CESM1(CAM5)"
            "CESM1-CAM5-1-FV2": "CESM1(CAM5.1,FV2)"
            "CESM1-FASTCHEM": "CESM1(FASTCHEM)"
            "CESM1-WACCM": "CESM1(WACCM)"
            "CSIRO-Mk3-6-0": "CSIRO-Mk3.6.0"
            "fio-esm": "FIO-ESM"
            "GFDL-CM2p1": "GFDL-CM2.1"
            "inmcm4": "INM-CM4"
            "MRI-AGCM3-2H": "MRI-AGCM3.2H"
            "MRI-AGCM3-2S": "MRI-AGCM3.2S"
  CMIP3:
    data:
      intake-esgf:
        type: "esmvalcore.io.intake_esgf.IntakeESGFDataSource"
        facets:
          dataset: "model"
          ensemble: "ensemble"
          exp: "experiment"
          frequency: "time_frequency"
          project: "project"
          short_name: "variable"
  obs4MIPs:
    data:
      intake-esgf-v2:
        type: "esmvalcore.io.intake_esgf.IntakeESGFDataSource"
        facets:
          dataset: "source_id"
          frequency: "frequency"
          institute: "institution_id"
          project: "project"
          short_name: "variable_id"
      # TODO: Add support for older ODS V1.0 obs4MIPs (CMIP5 style) data to intake-esgf
      # intake-esgf-v1:
      #   type: "esmvalcore.io.intake_esgf.IntakeESGFDataSource"
      #   facets:
      #     dataset: "source_id"
      #     frequency: "time_frequency"
      #     institute: "institute"
      #     project: "project"
      #     short_name: "variable"

Classes:

`IntakeESGFDataSource`(name, project, ...)	Data source that can be used to find data using intake-esgf.
`IntakeESGFDataset`(name, facets, catalog)	A dataset that can be used to load data found using intake-esgf.

class esmvalcore.io.intake_esgf.IntakeESGFDataSource(name: str, project: str, priority: int, facets: dict[str, str], values: dict[str, dict[str, str]] = <factory>)[source]#

Bases: DataSource

Data source that can be used to find data using intake-esgf.

Attributes:

`catalog`	The intake-esgf catalog used to find data.
`debug_info`	A string containing debug information when no data is found.
`facets`	Mapping between the ESMValCore and ESGF facet names.
`name`	A name identifying the data source.
`priority`	The priority of the data source.
`project`	The project that the data source provides data for.
`values`	Mapping between the ESMValCore and ESGF facet values.

Methods:

find_data(**facets)

Find data.

Parameters:

name (str)
project (str)
priority (int)
facets (dict[str, str])
values (dict[str, dict[str, str]])

catalog: ESGFCatalog#: The intake-esgf catalog used to find data.

debug_info: str = ''#: A string containing debug information when no data is found.

facets: dict[str, str]#: Mapping between the ESMValCore and ESGF facet names.

find_data(**facets: FacetValue) → list[IntakeESGFDataset][source]#

Find data.

Parameters:: **facets (FacetValue) – Find data matching these facets.
Returns:: A list of data elements that have been found.
Return type:: list[IntakeESGFDataset]

name: str#: A name identifying the data source.

priority: int#: The priority of the data source. Lower values have priority.

project: str#: The project that the data source provides data for.

values: dict[str, dict[str, str]]#: Mapping between the ESMValCore and ESGF facet values.

class esmvalcore.io.intake_esgf.IntakeESGFDataset(name: str, facets: Facets, catalog: intake_esgf.ESGFCatalog)[source]#

Bases: DataElement

A dataset that can be used to load data found using intake-esgf.

Attributes:

`attributes`	Attributes are key-value pairs describing the data.
`catalog`	The intake-esgf catalog describing this data.
`facets`	Facets are key-value pairs that were used to find this data.
`name`	A unique name identifying the data.

Methods:

`prepare`()	Prepare the data for access.
`to_iris`()	Load the data as Iris cubes.

Parameters:

name (str)
facets (Facets)
catalog (intake_esgf.ESGFCatalog)

property attributes: dict[str, Any]#: Attributes are key-value pairs describing the data.

catalog: intake_esgf.ESGFCatalog#: The intake-esgf catalog describing this data.

facets: Facets#: Facets are key-value pairs that were used to find this data.

name: str#: A unique name identifying the data.

prepare() → None[source]#

Prepare the data for access.

Return type:: None

to_iris() → iris.cube.CubeList[source]#

Load the data as Iris cubes.

Returns:: The loaded data.
Return type:: iris.cube.CubeList

esmvalcore.io.intake_esgf#

This Page