{ "cells": [ { "cell_type": "markdown", "id": "466b891a", "metadata": {}, "source": [ "# Quick Start - Catalog\n", "\n", "IBM Environmental Intelligence Service: Geospatial APIs contains a several petabytes of queryable data. It is therefore necessary to understand, briefly, the storage data model and how the metadata can be retrieved." ] }, { "cell_type": "markdown", "id": "68410503", "metadata": {}, "source": [ "## The Storage Model\n", "\n", "The storage model in Geospatial APIs is illustrated by the following diagram:\n", "\n", "![Queries1](catalogstoragemodel.png)" ] }, { "cell_type": "markdown", "id": "739179a6", "metadata": {}, "source": [ "### Data Sets\n", "The Data Set is the highest tier in the hierarchy. It functions as a collection for Data Layers for which defaults (e.g. attributes, security controls etc) can be applied. In most cases, the Data Set serves as a collection for interrelated data, usually acquired contemporaneously, (e.g. the 'bands' in the data provided by the ESA Sentinel 2 satellite) however, there is nothing that restricts a Data Set from use on more informal terms." ] }, { "cell_type": "markdown", "id": "afffa0de", "metadata": {}, "source": [ "### Data Layers\n", "The Data Layer is the tier of the hierarchy that is directly connected to the storage sub-system. The metadata defined here overwrites the Data Set defined defaults and applies directly to the manner in which the data is stored.\n", "When querying data from Geospatial APIs, this tier is the entry point." ] }, { "cell_type": "markdown", "id": "81de4763", "metadata": {}, "source": [ "In configuring a Data Layer, two important attributes, in particular, are the level and the datatype (as they determine storage size and speed of retrieval at query time):\n", "\n", " * the level is a granularity band range,\n", " * the datatype is the data type that will be applied to the storage.\n", "\n", "The most efficient level and type that can contain the data to be uploaded to a Data Layer should always be used.\n", "\n", "Level:\n", "\n", " * 29 (11.125 cm at equator)\n", " * 28 (22.25 cm at equator)\n", " * 27 (44.5 cm at equator)\n", " * 26 (0.89 m at equator)\n", " * 25 (1.78 m at equator)\n", " * 24 (3.56 m at equator)\n", " * 23 (7.12 m at equator)\n", " * 22 (14.24 m at equator)\n", " * 21 (28.48 m at equator)\n", " * 20 (56.96 m at equator)\n", " * 19 (113.92 m at equator)\n", " * 18 (227.84 m at equator)\n", " * 17 (455.68 m at equator)\n", " * 16 (911.36 m at equator)\n", " * 15 (1.82272 km at equator)\n", " * 14 (3.64544 km at equator)\n", " * 13 (7.29088 km at equator)\n", " * 12 (14.58176 km at equator)\n", " * 11 (29.16352 km at equator)\n", " * 10 (58.32704 km at equator)\n", " * 9 (116.65408 km at equator)\n", " * 8 (233.30816 km at equator)\n", " * 7 (466.61632 km at equator)\n", " * 6 (933.23264 km at equator)\n", " * 5 (1866.46528 km at equator)\n", " * 4 (3732.93056 km at equator)\n", " * 3 (7465.86112 km at equator)\n", " * 2 (14931.72224 km at equator)\n", " * 1 (29863.44448 km at equator)\n", "\n", "Data Types for Raster:\n", " \n", " * bt (Byte)\n", " * sh (Short Integer)\n", " * in (Integer) \n", " * db (Double) \n", " * fl (Float)" ] }, { "cell_type": "markdown", "id": "b0f385a6", "metadata": {}, "source": [ "### Data Layer Dimensions\n", "A Data Layer Dimension gives context to the value recorded for the data layer. For example, weather forecast models typically create a prediction for each day in the future for a number of days. When the weather model is run on a Monday it will generate a predictions for temperature on Tuesday, Wednesday, Thursday, Friday and Saturday. When the forecast is run on the Tuesday the predications are made for Wednesday, Thursday, Friday etc. The days for which predictions are generated are the data layer dimensions. \n", "\n", "Imagine a temperature prediction data layer which records the temperate against a spatial and temporal key:\n", "\n", "```\n", "lat/long/timestamp = temperature\n", "```\n", "\n", "This only stores one temperature value per key. Each time we run the weather model we generate 5 temperature predictions. One for each of the 5 future days. How can we store them? The answer is that the key is extended with a dimension:\n", "\n", "```\n", "lat/long/timestamp/dimension = temperature\n", "```\n", "\n", "Now each place and time can also have 5 predictions associated with it. \n", "\n", "In Geospatial APIs we typically arrange that data so that:\n", "```\n", "timestamp = the date/time that the value is valid\n", "dimenion = the predicted day number (horizon) from the model run\n", "```\n", "Using this scheme is it straightforward to compare different model predictions for the same date and time. " ] }, { "cell_type": "markdown", "id": "522bdab8", "metadata": {}, "source": [ "## Searching the Catalog\n", "### Search by Word" ] }, { "cell_type": "markdown", "id": "90035d0c", "metadata": {}, "source": [ "In order to search the whole catalog for a term, you can use the catalog sub-module and the method `search`:" ] }, { "cell_type": "markdown", "id": "1e265bb4", "metadata": {}, "source": [ "```bash\n", "catalog.search()\n", "```" ] }, { "cell_type": "markdown", "id": "431dc69d", "metadata": {}, "source": [ "In the example below, the catalog is searched for all datasets and datalayers that contain references to `Sentinel`. The information returned is a reduced version of the metadata held by the catalog that can be used to determine the prima facie usefulness of the returned datasets and datalayers to the user." ] }, { "cell_type": "code", "execution_count": 1, "id": "a2cbb332", "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2023-08-15 09:05:47 - paw - INFO - The client authentication method is assumed to be OAuth2.\n", "2023-08-15 09:05:47 - paw - INFO - Legacy Environment is False\n", "2023-08-15 09:05:49 - paw - INFO - Authentication success.\n", "2023-08-15 09:05:49 - paw - INFO - HOST: https://api.ibm.com/geospatial/run/na/core/v3\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
dataset_iddata_layer_iddata_layer_namedata_layer_description_shortdata_layer_description_longdata_layer_leveldata_layer_typedata_layer_unitdata_set_iddata_set_namedata_set_description_shortdata_set_description_long
017651648Band 11 (SWIR 1612 nm)Central wavelength 1613.7/1610.4 nm, bandwidth...22Raster176High res imagery (ESA Sentinel 2) (TOA)This dataset contains layers from the Level-1C...Images from the European Space Agency Sentinel...
117650364hollsteinA cloud mask as defined in a paper by Hollstei...A cloud mask as defined in a paper by Hollstei...23Raster176High res imagery (ESA Sentinel 2) (TOA)This dataset contains layers from the Level-1C...Images from the European Space Agency Sentinel...
217649358Band 4 (red)Central wavelength 664.5/665.0 nm, bandwidth 3...23Raster176High res imagery (ESA Sentinel 2) (TOA)This dataset contains layers from the Level-1C...Images from the European Space Agency Sentinel...
317649359Band 8 (NIR)Central wavelength 835.1/833.0 nm, bandwidth 1...23Raster176High res imagery (ESA Sentinel 2) (TOA)This dataset contains layers from the Level-1C...Images from the European Space Agency Sentinel...
417650096Band 10 (SWIR 1370 nm)Central wavelength 1373.5/1376.9 nm, bandwidth...20Raster176High res imagery (ESA Sentinel 2) (TOA)This dataset contains layers from the Level-1C...Images from the European Space Agency Sentinel...
517651649Band 12 (SWIR 2202 nm)Central wavelength 2202.4/2185.7 nm, bandwidth...22Raster176High res imagery (ESA Sentinel 2) (TOA)This dataset contains layers from the Level-1C...Images from the European Space Agency Sentinel...
617749464Normalized difference vegetation indexA measure of the amount of vegetation at the p...NDVI is generally calculated as (NIR - VIR) / ...23Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
717749683Band 6 (vegetation red edge)Central wavelength 740.2/739.1 nm, bandwidth 1...22Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
817749680Band 2 (blue)Central wavelength 496.6/492.1 nm, bandwidth 9...23Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
917749689Water vaporAtmospheric water vapor content derived from b...\"Water vapour retrieval over land is performed...22Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
1017749360Band 4 (red)Central wavelength 664.5/665.0 nm, bandwidth 3...23Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
1117749361Band 8 (NIR)Central wavelength 835.1/833.0 nm, bandwidth 1...23Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
1217750250Cloud probability mapA 20m mask indicating the calculated probabili...22Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
1317749686Band 11 (SWIR 1610 nm)Central wavelength 1613.7/1610.4 nm, bandwidth...22Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
1417749687Band 12 (SWIR 2200 nm)Central wavelength 2202.4/2185.7 nm, bandwidth...22Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
1517749690Band 1 (coastal aerosol)Central wavelength 443.9/442.3 nm, bandwidth 2...20Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
1617749688Aerosol optical thickness\"AOT describes attenuation of sunlight by a co...For Sentinel 2 level 2A products, the \"aerosol...22Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
1717749691Band 9 (water vapor)Central wavelength 945.0/943.2 nm, bandwidth 2...20Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
1817749685Band 8a (narrow IR)Central wavelength 864.8/864.0 nm, bandwidth 3...22Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
1917749684Band 7 (vegetation red edge)Central wavelength 782.5/779.7 nm, bandwidth 2...22Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
2017749681Band 3 (green)Central wavelength 560.0/559.0 nm, bandwidth 4...23Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
2117749682Band 5 (vegetation red edge)Central wavelength 703.9/703.8 nm, bandwidth 1...22Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
2217749362Scene classificationPixel-by-pixel classification in image of 4 ty...The different labels and classifications are: ...22Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
2333550254VH polarizationSynthetic Aperture Radar with VH Partial polar...The data is preprocessed following correction ...23Raster335Satellite based radar (ESA Sentinel 1)Sentinel-1 is an imaging radar mission providi...Sentinel-1 provides dual polarization capabili...
2433550253VV polarizationSynthetic Aperture Radar with VV Partial polar...The data is preprocessed following correction ...23Raster335Satellite based radar (ESA Sentinel 1)Sentinel-1 is an imaging radar mission providi...Sentinel-1 provides dual polarization capabili...
\n", "
" ], "text/plain": [ " dataset_id data_layer_id data_layer_name \\\n", "0 176 51648 Band 11 (SWIR 1612 nm) \n", "1 176 50364 hollstein \n", "2 176 49358 Band 4 (red) \n", "3 176 49359 Band 8 (NIR) \n", "4 176 50096 Band 10 (SWIR 1370 nm) \n", "5 176 51649 Band 12 (SWIR 2202 nm) \n", "6 177 49464 Normalized difference vegetation index \n", "7 177 49683 Band 6 (vegetation red edge) \n", "8 177 49680 Band 2 (blue) \n", "9 177 49689 Water vapor \n", "10 177 49360 Band 4 (red) \n", "11 177 49361 Band 8 (NIR) \n", "12 177 50250 Cloud probability map \n", "13 177 49686 Band 11 (SWIR 1610 nm) \n", "14 177 49687 Band 12 (SWIR 2200 nm) \n", "15 177 49690 Band 1 (coastal aerosol) \n", "16 177 49688 Aerosol optical thickness \n", "17 177 49691 Band 9 (water vapor) \n", "18 177 49685 Band 8a (narrow IR) \n", "19 177 49684 Band 7 (vegetation red edge) \n", "20 177 49681 Band 3 (green) \n", "21 177 49682 Band 5 (vegetation red edge) \n", "22 177 49362 Scene classification \n", "23 335 50254 VH polarization \n", "24 335 50253 VV polarization \n", "\n", " data_layer_description_short \\\n", "0 Central wavelength 1613.7/1610.4 nm, bandwidth... \n", "1 A cloud mask as defined in a paper by Hollstei... \n", "2 Central wavelength 664.5/665.0 nm, bandwidth 3... \n", "3 Central wavelength 835.1/833.0 nm, bandwidth 1... \n", "4 Central wavelength 1373.5/1376.9 nm, bandwidth... \n", "5 Central wavelength 2202.4/2185.7 nm, bandwidth... \n", "6 A measure of the amount of vegetation at the p... \n", "7 Central wavelength 740.2/739.1 nm, bandwidth 1... \n", "8 Central wavelength 496.6/492.1 nm, bandwidth 9... \n", "9 Atmospheric water vapor content derived from b... \n", "10 Central wavelength 664.5/665.0 nm, bandwidth 3... \n", "11 Central wavelength 835.1/833.0 nm, bandwidth 1... \n", "12 A 20m mask indicating the calculated probabili... \n", "13 Central wavelength 1613.7/1610.4 nm, bandwidth... \n", "14 Central wavelength 2202.4/2185.7 nm, bandwidth... \n", "15 Central wavelength 443.9/442.3 nm, bandwidth 2... \n", "16 \"AOT describes attenuation of sunlight by a co... \n", "17 Central wavelength 945.0/943.2 nm, bandwidth 2... \n", "18 Central wavelength 864.8/864.0 nm, bandwidth 3... \n", "19 Central wavelength 782.5/779.7 nm, bandwidth 2... \n", "20 Central wavelength 560.0/559.0 nm, bandwidth 4... \n", "21 Central wavelength 703.9/703.8 nm, bandwidth 1... \n", "22 Pixel-by-pixel classification in image of 4 ty... \n", "23 Synthetic Aperture Radar with VH Partial polar... \n", "24 Synthetic Aperture Radar with VV Partial polar... \n", "\n", " data_layer_description_long data_layer_level \\\n", "0 22 \n", "1 A cloud mask as defined in a paper by Hollstei... 23 \n", "2 23 \n", "3 23 \n", "4 20 \n", "5 22 \n", "6 NDVI is generally calculated as (NIR - VIR) / ... 23 \n", "7 22 \n", "8 23 \n", "9 \"Water vapour retrieval over land is performed... 22 \n", "10 23 \n", "11 23 \n", "12 22 \n", "13 22 \n", "14 22 \n", "15 20 \n", "16 For Sentinel 2 level 2A products, the \"aerosol... 22 \n", "17 20 \n", "18 22 \n", "19 22 \n", "20 23 \n", "21 22 \n", "22 The different labels and classifications are: ... 22 \n", "23 The data is preprocessed following correction ... 23 \n", "24 The data is preprocessed following correction ... 23 \n", "\n", " data_layer_type data_layer_unit data_set_id \\\n", "0 Raster 176 \n", "1 Raster 176 \n", "2 Raster 176 \n", "3 Raster 176 \n", "4 Raster 176 \n", "5 Raster 176 \n", "6 Raster 177 \n", "7 Raster 177 \n", "8 Raster 177 \n", "9 Raster 177 \n", "10 Raster 177 \n", "11 Raster 177 \n", "12 Raster 177 \n", "13 Raster 177 \n", "14 Raster 177 \n", "15 Raster 177 \n", "16 Raster 177 \n", "17 Raster 177 \n", "18 Raster 177 \n", "19 Raster 177 \n", "20 Raster 177 \n", "21 Raster 177 \n", "22 Raster 177 \n", "23 Raster 335 \n", "24 Raster 335 \n", "\n", " data_set_name \\\n", "0 High res imagery (ESA Sentinel 2) (TOA) \n", "1 High res imagery (ESA Sentinel 2) (TOA) \n", "2 High res imagery (ESA Sentinel 2) (TOA) \n", "3 High res imagery (ESA Sentinel 2) (TOA) \n", "4 High res imagery (ESA Sentinel 2) (TOA) \n", "5 High res imagery (ESA Sentinel 2) (TOA) \n", "6 High res imagery (ESA Sentinel 2) \n", "7 High res imagery (ESA Sentinel 2) \n", "8 High res imagery (ESA Sentinel 2) \n", "9 High res imagery (ESA Sentinel 2) \n", "10 High res imagery (ESA Sentinel 2) \n", "11 High res imagery (ESA Sentinel 2) \n", "12 High res imagery (ESA Sentinel 2) \n", "13 High res imagery (ESA Sentinel 2) \n", "14 High res imagery (ESA Sentinel 2) \n", "15 High res imagery (ESA Sentinel 2) \n", "16 High res imagery (ESA Sentinel 2) \n", "17 High res imagery (ESA Sentinel 2) \n", "18 High res imagery (ESA Sentinel 2) \n", "19 High res imagery (ESA Sentinel 2) \n", "20 High res imagery (ESA Sentinel 2) \n", "21 High res imagery (ESA Sentinel 2) \n", "22 High res imagery (ESA Sentinel 2) \n", "23 Satellite based radar (ESA Sentinel 1) \n", "24 Satellite based radar (ESA Sentinel 1) \n", "\n", " data_set_description_short \\\n", "0 This dataset contains layers from the Level-1C... \n", "1 This dataset contains layers from the Level-1C... \n", "2 This dataset contains layers from the Level-1C... \n", "3 This dataset contains layers from the Level-1C... \n", "4 This dataset contains layers from the Level-1C... \n", "5 This dataset contains layers from the Level-1C... \n", "6 Images from the European Space Agency Sentinel... \n", "7 Images from the European Space Agency Sentinel... \n", "8 Images from the European Space Agency Sentinel... \n", "9 Images from the European Space Agency Sentinel... \n", "10 Images from the European Space Agency Sentinel... \n", "11 Images from the European Space Agency Sentinel... \n", "12 Images from the European Space Agency Sentinel... \n", "13 Images from the European Space Agency Sentinel... \n", "14 Images from the European Space Agency Sentinel... \n", "15 Images from the European Space Agency Sentinel... \n", "16 Images from the European Space Agency Sentinel... \n", "17 Images from the European Space Agency Sentinel... \n", "18 Images from the European Space Agency Sentinel... \n", "19 Images from the European Space Agency Sentinel... \n", "20 Images from the European Space Agency Sentinel... \n", "21 Images from the European Space Agency Sentinel... \n", "22 Images from the European Space Agency Sentinel... \n", "23 Sentinel-1 is an imaging radar mission providi... \n", "24 Sentinel-1 is an imaging radar mission providi... \n", "\n", " data_set_description_long \n", "0 Images from the European Space Agency Sentinel... \n", "1 Images from the European Space Agency Sentinel... \n", "2 Images from the European Space Agency Sentinel... \n", "3 Images from the European Space Agency Sentinel... \n", "4 Images from the European Space Agency Sentinel... \n", "5 Images from the European Space Agency Sentinel... \n", "6 Sentinel-2 is a set of two satellites in polar... \n", "7 Sentinel-2 is a set of two satellites in polar... \n", "8 Sentinel-2 is a set of two satellites in polar... \n", "9 Sentinel-2 is a set of two satellites in polar... \n", "10 Sentinel-2 is a set of two satellites in polar... \n", "11 Sentinel-2 is a set of two satellites in polar... \n", "12 Sentinel-2 is a set of two satellites in polar... \n", "13 Sentinel-2 is a set of two satellites in polar... \n", "14 Sentinel-2 is a set of two satellites in polar... \n", "15 Sentinel-2 is a set of two satellites in polar... \n", "16 Sentinel-2 is a set of two satellites in polar... \n", "17 Sentinel-2 is a set of two satellites in polar... \n", "18 Sentinel-2 is a set of two satellites in polar... \n", "19 Sentinel-2 is a set of two satellites in polar... \n", "20 Sentinel-2 is a set of two satellites in polar... \n", "21 Sentinel-2 is a set of two satellites in polar... \n", "22 Sentinel-2 is a set of two satellites in polar... \n", "23 Sentinel-1 provides dual polarization capabili... \n", "24 Sentinel-1 provides dual polarization capabili... " ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import os\n", "import ibmpairs.authentication as authentication\n", "import ibmpairs.client as client\n", "import ibmpairs.catalog as catalog\n", "\n", "EI_API_KEY = os.environ.get('EI_API_KEY')\n", "EI_TENANT_ID = os.environ.get('EI_TENANT_ID')\n", "EI_ORG_ID = os.environ.get('EI_ORG_ID')\n", "\n", "ei_client = client.get_client(api_key = EI_API_KEY,\n", " tenant_id = EI_TENANT_ID,\n", " org_id = EI_ORG_ID)\n", "\n", "search_by_word = catalog.search(\"Sentinel\")\n", "search_by_word" ] }, { "cell_type": "markdown", "id": "0ab779c4", "metadata": {}, "source": [ "The full metadata for the dataset or datalayer can then be retrieved by subsequent methods, e.g.:" ] }, { "cell_type": "code", "execution_count": 2, "id": "1d74d676", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"color_table\": {\n", " \"colors\": \"153A91,84F588,FFF787,FF7C3B,FF1921\",\n", " \"id\": \"4\",\n", " \"name\": \"Spectral\"\n", " },\n", " \"created_at\": \"1593733829000\",\n", " \"crs\": \"\",\n", " \"data_layer_response\": {},\n", " \"dataset_id\": \"177\",\n", " \"datatype\": \"sh\",\n", " \"description_long\": \"NDVI is generally calculated as (NIR - VIR) / (NIR + VIR). For Sentinel 2, this translates to (band 8 - band 4) / (band 8 + band 4).\",\n", " \"description_short\": \"A measure of the amount of vegetation at the pixel.\",\n", " \"id\": \"49464\",\n", " \"interpolation\": \"bilinear\",\n", " \"latitude_max\": 90.0,\n", " \"latitude_min\": -90.0,\n", " \"level\": 23,\n", " \"longitude_max\": 180.0,\n", " \"longitude_min\": -180.0,\n", " \"name\": \"Normalized difference vegetation index\",\n", " \"name_alternate\": \"Normalized difference vegetation index\",\n", " \"permanence\": true,\n", " \"properties\": {},\n", " \"rating\": 1.0,\n", " \"spatial_coverage\": {\n", " \"country\": [\n", " \"Belgium\",\n", " \"Bolivia\",\n", " \"France\",\n", " \"Germany\",\n", " \"India\",\n", " \"Indonesia\",\n", " \"Luxembourg\",\n", " \"Netherlands\",\n", " \"Switzerland\",\n", " \"Thailand\",\n", " \"United Kingdom\",\n", " \"United States of America\"\n", " ]\n", " },\n", " \"temporal_max\": \"1665604800000\",\n", " \"temporal_min\": \"1438387200000\",\n", " \"type\": \"R\",\n", " \"units\": \"[-1, 1]\",\n", " \"updated_at\": \"1665648134000\"\n", "}\n" ] } ], "source": [ "dl = catalog.get_data_layer(id = \"49464\")\n", "print(dl)" ] }, { "cell_type": "markdown", "id": "1df6034d", "metadata": {}, "source": [ "### Search by ID" ] }, { "cell_type": "markdown", "id": "b6174f06", "metadata": {}, "source": [ "In the event that you already know the Geospatial APIs ID for a data set or data layer the search method can also be used to retrieve the limited metadata returned by the `search` method:" ] }, { "cell_type": "code", "execution_count": 3, "id": "b11a607a", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
dataset_iddata_layer_iddata_layer_namedata_layer_description_shortdata_layer_description_longdata_layer_leveldata_layer_typedata_layer_unitdata_set_iddata_set_namedata_set_description_shortdata_set_description_long
017749464Normalized difference vegetation indexA measure of the amount of vegetation at the p...NDVI is generally calculated as (NIR - VIR) / ...23Raster177High res imagery (ESA Sentinel 2)Images from the European Space Agency Sentinel...Sentinel-2 is a set of two satellites in polar...
\n", "
" ], "text/plain": [ " dataset_id data_layer_id data_layer_name \\\n", "0 177 49464 Normalized difference vegetation index \n", "\n", " data_layer_description_short \\\n", "0 A measure of the amount of vegetation at the p... \n", "\n", " data_layer_description_long data_layer_level \\\n", "0 NDVI is generally calculated as (NIR - VIR) / ... 23 \n", "\n", " data_layer_type data_layer_unit data_set_id \\\n", "0 Raster 177 \n", "\n", " data_set_name \\\n", "0 High res imagery (ESA Sentinel 2) \n", "\n", " data_set_description_short \\\n", "0 Images from the European Space Agency Sentinel... \n", "\n", " data_set_description_long \n", "0 Sentinel-2 is a set of two satellites in polar... " ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "search_by_id = catalog.search(\"49464\")\n", "search_by_id" ] }, { "cell_type": "markdown", "id": "3b5995aa", "metadata": {}, "source": [ "## Retrieve Metadata\n", "A series of helper methods can be used to retrieve the metadata concerning Data Sets, Data Layers and Data Layer Dimensions. \n", "The results for these catalog methods can be displayed (returned in a truncated form as a dataframe):" ] }, { "cell_type": "markdown", "id": "87f84072", "metadata": {}, "source": [ "```bash\n", "ds_list = catalog.get_data_sets()\n", "ds_list.display()\n", "```" ] }, { "cell_type": "markdown", "id": "429a182d", "metadata": {}, "source": [ "or printed (which returns a string representation of an object):" ] }, { "cell_type": "markdown", "id": "83ac47fa", "metadata": {}, "source": [ "```bash\n", "ds = catalog.get_data_set(id = )\n", "print(ds)\n", "```" ] }, { "cell_type": "markdown", "id": "5cb239bb", "metadata": {}, "source": [ "Where applicable, the methods also allow for return of lists of embedded objects (e.g. Data Layers per Data Set):" ] }, { "cell_type": "markdown", "id": "d8d64d5b", "metadata": {}, "source": [ "```bash\n", "catalog.get_data_layers(data_set_id = )\n", "```" ] }, { "cell_type": "markdown", "id": "d492765e", "metadata": {}, "source": [ "### Get a List of Data Sets\n", "In order to return all data sets available to a user, you can execute the `get_data_sets` method:" ] }, { "cell_type": "code", "execution_count": 4, "id": "42f5820e", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
idnamedescription_shortdescription_long
057415-46 day ECMWF weather forecast (ML post-proc...ML Post-processed temperature and precipitatio...ML Post-processed temperature and precipitatio...
15751-15 day ECMWF weather forecast (ML post-proce...ML Post-processed temperature and precipitatio...ML Post-processed temperature and precipitatio...
2306Atmospheric weather (ERA5)A global reanalysis data set produced by ECMWF...ERA5 is the direct successor to the ERA Interi...
363High resolution aerial imagery (USDA NAIP)High resolution (<1m) aerial imagery from the ...National Agriculture Imagery Program (NAIP) ac...
4369Buoy Data Wave SummaryPrecise wave conditions around bouys belonging...Local measurements of wave attributes and thei...
...............
93379US healthcare infrastructureDataset includes data layers with relevant inf...NaN
94392US mobility (Descartes Labs)Mobility data derived from mobile devices.Note that the exact source of the data -- i.e....
9593Soil properties USAThe USA soil property data is derived from the...It contains information about soil as collecte...
96332Ocean wave forecast (ECMWF ENS)15-day ahead forecast of ocean conditions.The ECMWF Ensemble Prediction System (EPS) cre...
97284Wildfire risk potentialWildfire Hazard Potential can help to inform e...Wildfire Hazard Potential* for the conterminou...
\n", "

98 rows × 4 columns

\n", "
" ], "text/plain": [ " id name \\\n", "0 574 15-46 day ECMWF weather forecast (ML post-proc... \n", "1 575 1-15 day ECMWF weather forecast (ML post-proce... \n", "2 306 Atmospheric weather (ERA5) \n", "3 63 High resolution aerial imagery (USDA NAIP) \n", "4 369 Buoy Data Wave Summary \n", ".. ... ... \n", "93 379 US healthcare infrastructure \n", "94 392 US mobility (Descartes Labs) \n", "95 93 Soil properties USA \n", "96 332 Ocean wave forecast (ECMWF ENS) \n", "97 284 Wildfire risk potential \n", "\n", " description_short \\\n", "0 ML Post-processed temperature and precipitatio... \n", "1 ML Post-processed temperature and precipitatio... \n", "2 A global reanalysis data set produced by ECMWF... \n", "3 High resolution (<1m) aerial imagery from the ... \n", "4 Precise wave conditions around bouys belonging... \n", ".. ... \n", "93 Dataset includes data layers with relevant inf... \n", "94 Mobility data derived from mobile devices. \n", "95 The USA soil property data is derived from the... \n", "96 15-day ahead forecast of ocean conditions. \n", "97 Wildfire Hazard Potential can help to inform e... \n", "\n", " description_long \n", "0 ML Post-processed temperature and precipitatio... \n", "1 ML Post-processed temperature and precipitatio... \n", "2 ERA5 is the direct successor to the ERA Interi... \n", "3 National Agriculture Imagery Program (NAIP) ac... \n", "4 Local measurements of wave attributes and thei... \n", ".. ... \n", "93 NaN \n", "94 Note that the exact source of the data -- i.e.... \n", "95 It contains information about soil as collecte... \n", "96 The ECMWF Ensemble Prediction System (EPS) cre... \n", "97 Wildfire Hazard Potential* for the conterminou... \n", "\n", "[98 rows x 4 columns]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds_list = catalog.get_data_sets()\n", "ds_list.display()" ] }, { "cell_type": "markdown", "id": "bd6ce7f3", "metadata": {}, "source": [ "### Get a Data Set\n", "In order to return all metadata about a Data Set, the `get_data_set` method can be used with a provided Data Set ID:" ] }, { "cell_type": "code", "execution_count": 5, "id": "24837922", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"category\": {\n", " \"id\": 1,\n", " \"name\": \"Satellite\"\n", " },\n", " \"created_at\": \"1593733829000\",\n", " \"crs\": \"\",\n", " \"data_set_response\": {},\n", " \"data_source_attribution\": \"Source: European Space Agency - ESA; Contains modified Copernicus Sentinel data [2018 and Ongoing]\",\n", " \"data_source_description\": \"Level-2A is generated by the Payload Data Ground Segment using the Sen2Cor processor. Level-2A products are made available to users via the Copernicus Open Access Hub: https://scihub.copernicus.eu/dhus/#/home\",\n", " \"data_source_name\": \"European Space Agency Sentinel-2\",\n", " \"description_long\": \"Sentinel-2 is a set of two satellites in polar orbit 180 degrees apart. It monitors land surface and coastal waters every 5 days at the equator and more frequently at mid-latitudes. The coverage is between latitudes 56\\u00b0 south and 84\\u00b0 north. Images are in 13 spectral bands at various ground resolutions: 4 bands at 10 m, 6 at 20 m and 3 at 60 m; the orbital swath is 290 km wide. Level 2A (L2A) images are 100x100 km ortho-rectified and spatially registered on a global reference system; they are corrected for the atmosphere so they represent ground conditions. Currently Geospatial APIs ingests Bands 4 (red), 8 (NIR) and SCL (Scene Classification). An NDVI layer, called \\\"NDVI sh\\\", is calculated from Bands 4 and 8. Tiles are ingested on request. Currently there is some coverage for tiles in USA, Brazil, India and the Netherlands for selected days in 2018 and 2019. Timestamps in this dataset are rounded down to 0:00 UTC from the Satellite's sensing time.\",\n", " \"description_short\": \"Images from the European Space Agency Sentinel 2 satellite pair which view land surface regions in 13 spectral bands every 5 days or faster.\",\n", " \"id\": \"177\",\n", " \"key\": \"esa-sentinel-2-l2a\",\n", " \"lag_horizon\": \"0 years 0 mons 0 days 0 hours 0 mins 432000.0 secs\",\n", " \"lag_horizon_description\": \"Data is available within 24 hours.\",\n", " \"latitude_max\": 90.0,\n", " \"latitude_min\": -90.0,\n", " \"level\": 23,\n", " \"longitude_max\": 180.0,\n", " \"longitude_min\": -180.0,\n", " \"max_layers\": 30,\n", " \"name\": \"High res imagery (ESA Sentinel 2)\",\n", " \"name_alternate\": \"ESA Sentinel 2 Level-2A\",\n", " \"offering_status\": \"Commercial Use\",\n", " \"permanence\": true,\n", " \"properties\": {\n", " \"application\": [\n", " \"Agriculture\"\n", " ],\n", " \"domain\": [\n", " \"Land surface\"\n", " ],\n", " \"sector\": [\n", " \"Animals/livestock\",\n", " \"Transportation/infrastructure\",\n", " \"Vegetation/crops\"\n", " ],\n", " \"source\": [\n", " \"Satellite\",\n", " \"Survey\"\n", " ],\n", " \"type\": [\n", " \"Data product\"\n", " ]\n", " },\n", " \"rating\": 4.0,\n", " \"spatial_coverage\": {\n", " \"country\": [\n", " \"Belgium\",\n", " \"Bolivia\",\n", " \"France\",\n", " \"Germany\",\n", " \"India\",\n", " \"Indonesia\",\n", " \"Luxembourg\",\n", " \"Netherlands\",\n", " \"Switzerland\",\n", " \"Thailand\",\n", " \"United Kingdom\",\n", " \"United States of America\"\n", " ]\n", " },\n", " \"status\": \"Active\",\n", " \"temporal_max\": \"1665604800000\",\n", " \"temporal_min\": \"1438387200000\",\n", " \"temporal_resolution\": \"0 years 0 mons 0 days 0 hours 0 mins 432000.0 secs\",\n", " \"temporal_resolution_description\": \"5 days at the equator, less at mid latitudes.\",\n", " \"update_interval_description\": \"Uploads are run daily. However, note the temporal resolution.\",\n", " \"update_interval_max\": \"0 years 0 mons 0 days 0 hours 0 mins 86400.0 secs\",\n", " \"updated_at\": \"1665645277000\"\n", "}\n" ] } ], "source": [ "ds = catalog.get_data_set(id = \"177\")\n", "print(ds)" ] }, { "cell_type": "markdown", "id": "6a22c0f7", "metadata": {}, "source": [ "### Get a List of Data Layers per Data Set\n", "As discussed above in §. The Storage Model, Data Layers belong to Data Sets. The `get_data_layers` method can be used to return a list of all Data Layers in a specific Data Set by providing the Data Set ID:" ] }, { "cell_type": "code", "execution_count": 6, "id": "769792bb", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
dataset_ididnamedescription_shortdescription_longleveltypeunit
017749464Normalized difference vegetation indexA measure of the amount of vegetation at the p...NDVI is generally calculated as (NIR - VIR) / ...NaNnanNaN
117749683Band 6 (vegetation red edge)Central wavelength 740.2/739.1 nm, bandwidth 1...NaNNaNnanNaN
217749680Band 2 (blue)Central wavelength 496.6/492.1 nm, bandwidth 9...NaNNaNnanNaN
317749689Water vaporAtmospheric water vapor content derived from b...\"Water vapour retrieval over land is performed...NaNnanNaN
417749360Band 4 (red)Central wavelength 664.5/665.0 nm, bandwidth 3...NaNNaNnanNaN
517749361Band 8 (NIR)Central wavelength 835.1/833.0 nm, bandwidth 1...NaNNaNnanNaN
617750250Cloud probability mapA 20m mask indicating the calculated probabili...NaNNaNnanNaN
717749686Band 11 (SWIR 1610 nm)Central wavelength 1613.7/1610.4 nm, bandwidth...NaNNaNnanNaN
817749687Band 12 (SWIR 2200 nm)Central wavelength 2202.4/2185.7 nm, bandwidth...NaNNaNnanNaN
917749690Band 1 (coastal aerosol)Central wavelength 443.9/442.3 nm, bandwidth 2...NaNNaNnanNaN
1017749688Aerosol optical thickness\"AOT describes attenuation of sunlight by a co...For Sentinel 2 level 2A products, the \"aerosol...NaNnanNaN
1117749691Band 9 (water vapor)Central wavelength 945.0/943.2 nm, bandwidth 2...NaNNaNnanNaN
1217749685Band 8a (narrow IR)Central wavelength 864.8/864.0 nm, bandwidth 3...NaNNaNnanNaN
1317749684Band 7 (vegetation red edge)Central wavelength 782.5/779.7 nm, bandwidth 2...NaNNaNnanNaN
1417749681Band 3 (green)Central wavelength 560.0/559.0 nm, bandwidth 4...NaNNaNnanNaN
1517749682Band 5 (vegetation red edge)Central wavelength 703.9/703.8 nm, bandwidth 1...NaNNaNnanNaN
1617749362Scene classificationPixel-by-pixel classification in image of 4 ty...The different labels and classifications are: ...NaNnanNaN
\n", "
" ], "text/plain": [ " dataset_id id name \\\n", "0 177 49464 Normalized difference vegetation index \n", "1 177 49683 Band 6 (vegetation red edge) \n", "2 177 49680 Band 2 (blue) \n", "3 177 49689 Water vapor \n", "4 177 49360 Band 4 (red) \n", "5 177 49361 Band 8 (NIR) \n", "6 177 50250 Cloud probability map \n", "7 177 49686 Band 11 (SWIR 1610 nm) \n", "8 177 49687 Band 12 (SWIR 2200 nm) \n", "9 177 49690 Band 1 (coastal aerosol) \n", "10 177 49688 Aerosol optical thickness \n", "11 177 49691 Band 9 (water vapor) \n", "12 177 49685 Band 8a (narrow IR) \n", "13 177 49684 Band 7 (vegetation red edge) \n", "14 177 49681 Band 3 (green) \n", "15 177 49682 Band 5 (vegetation red edge) \n", "16 177 49362 Scene classification \n", "\n", " description_short \\\n", "0 A measure of the amount of vegetation at the p... \n", "1 Central wavelength 740.2/739.1 nm, bandwidth 1... \n", "2 Central wavelength 496.6/492.1 nm, bandwidth 9... \n", "3 Atmospheric water vapor content derived from b... \n", "4 Central wavelength 664.5/665.0 nm, bandwidth 3... \n", "5 Central wavelength 835.1/833.0 nm, bandwidth 1... \n", "6 A 20m mask indicating the calculated probabili... \n", "7 Central wavelength 1613.7/1610.4 nm, bandwidth... \n", "8 Central wavelength 2202.4/2185.7 nm, bandwidth... \n", "9 Central wavelength 443.9/442.3 nm, bandwidth 2... \n", "10 \"AOT describes attenuation of sunlight by a co... \n", "11 Central wavelength 945.0/943.2 nm, bandwidth 2... \n", "12 Central wavelength 864.8/864.0 nm, bandwidth 3... \n", "13 Central wavelength 782.5/779.7 nm, bandwidth 2... \n", "14 Central wavelength 560.0/559.0 nm, bandwidth 4... \n", "15 Central wavelength 703.9/703.8 nm, bandwidth 1... \n", "16 Pixel-by-pixel classification in image of 4 ty... \n", "\n", " description_long level type unit \n", "0 NDVI is generally calculated as (NIR - VIR) / ... NaN nan NaN \n", "1 NaN NaN nan NaN \n", "2 NaN NaN nan NaN \n", "3 \"Water vapour retrieval over land is performed... NaN nan NaN \n", "4 NaN NaN nan NaN \n", "5 NaN NaN nan NaN \n", "6 NaN NaN nan NaN \n", "7 NaN NaN nan NaN \n", "8 NaN NaN nan NaN \n", "9 NaN NaN nan NaN \n", "10 For Sentinel 2 level 2A products, the \"aerosol... NaN nan NaN \n", "11 NaN NaN nan NaN \n", "12 NaN NaN nan NaN \n", "13 NaN NaN nan NaN \n", "14 NaN NaN nan NaN \n", "15 NaN NaN nan NaN \n", "16 The different labels and classifications are: ... NaN nan NaN " ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dl_list_by_set = catalog.get_data_layers(data_set_id = \"177\")\n", "dl_list_by_set.display()" ] }, { "cell_type": "markdown", "id": "2f6c4a4c", "metadata": {}, "source": [ "### Get a List of Data Layers\n", "In the same way as the `get_data_sets` method can be used to return all Data Sets a user has access to, the `get_data_layers` method can be used to return all Data Layers a user has access to:" ] }, { "cell_type": "code", "execution_count": 7, "id": "078b86d5", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
dataset_ididnamedescription_shortdescription_longleveltypeunit
0555Mid infrared (band 7)MODIS Aqua 16 Day Spectral Image of Band 7 (mi...250m resolution Surface Reflectance Band 7 (21...18RasterNaN
1551Normalized difference vegetation index (NDVI)MODIS Aqua 16 day normalized vegetation indexThe NDVI is calculated from the near-infrared ...18Raster[0-1]
2553Near infrared (band 2)MODIS Aqua 16 Day Spectral Image of Band 2 (Ne...250m resolution Surface Reflectance Band 2 (84...18RasterNaN
3556Composite day of the yearMODIS Aqua 16 Days Composite Day of the YearThis is a day number, from 1 to 36618RasterNaN
4554Blue (band 3)MODIS Aqua 16 Day Spectral Image of Band 3 (blue)250m resolution Surface Reflectance Band 3 (45...18RasterNaN
...........................
89258051586LIDAR Composite DSM 1m 2017Environment Agency LIDAR Composite Digital Sur...NaN26RasterNaN
89358051583LIDAR Composite DTM 10m 2019Environment Agency LIDAR Composite Digital Ter...NaN23RasterNaN
89458151628Risk of Surface Water Flooding DepthSurface water flood depth in integer categorie...NaN25RasterNaN
895102049554Maximum temperatureNaNNaN11RasterNaN
896102049555Minimum temperatureNaNNaN11RasterNaN
\n", "

897 rows × 8 columns

\n", "
" ], "text/plain": [ " dataset_id id name \\\n", "0 5 55 Mid infrared (band 7) \n", "1 5 51 Normalized difference vegetation index (NDVI) \n", "2 5 53 Near infrared (band 2) \n", "3 5 56 Composite day of the year \n", "4 5 54 Blue (band 3) \n", ".. ... ... ... \n", "892 580 51586 LIDAR Composite DSM 1m 2017 \n", "893 580 51583 LIDAR Composite DTM 10m 2019 \n", "894 581 51628 Risk of Surface Water Flooding Depth \n", "895 1020 49554 Maximum temperature \n", "896 1020 49555 Minimum temperature \n", "\n", " description_short \\\n", "0 MODIS Aqua 16 Day Spectral Image of Band 7 (mi... \n", "1 MODIS Aqua 16 day normalized vegetation index \n", "2 MODIS Aqua 16 Day Spectral Image of Band 2 (Ne... \n", "3 MODIS Aqua 16 Days Composite Day of the Year \n", "4 MODIS Aqua 16 Day Spectral Image of Band 3 (blue) \n", ".. ... \n", "892 Environment Agency LIDAR Composite Digital Sur... \n", "893 Environment Agency LIDAR Composite Digital Ter... \n", "894 Surface water flood depth in integer categorie... \n", "895 NaN \n", "896 NaN \n", "\n", " description_long level type unit \n", "0 250m resolution Surface Reflectance Band 7 (21... 18 Raster NaN \n", "1 The NDVI is calculated from the near-infrared ... 18 Raster [0-1] \n", "2 250m resolution Surface Reflectance Band 2 (84... 18 Raster NaN \n", "3 This is a day number, from 1 to 366 18 Raster NaN \n", "4 250m resolution Surface Reflectance Band 3 (45... 18 Raster NaN \n", ".. ... ... ... ... \n", "892 NaN 26 Raster NaN \n", "893 NaN 23 Raster NaN \n", "894 NaN 25 Raster NaN \n", "895 NaN 11 Raster NaN \n", "896 NaN 11 Raster NaN \n", "\n", "[897 rows x 8 columns]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dl_list = catalog.get_data_layers()\n", "dl_list.display()" ] }, { "cell_type": "markdown", "id": "df80a28d", "metadata": {}, "source": [ "### Get a Data Layer\n", "The metadata about a specific Data Layer can be returned by providing the `get_data_layer` method a Data Layer ID:" ] }, { "cell_type": "code", "execution_count": 8, "id": "7d9a9668", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"color_table\": {\n", " \"colors\": \"153A91,84F588,FFF787,FF7C3B,FF1921\",\n", " \"id\": \"4\",\n", " \"name\": \"Spectral\"\n", " },\n", " \"created_at\": \"1593733829000\",\n", " \"crs\": \"\",\n", " \"data_layer_response\": {},\n", " \"dataset_id\": \"177\",\n", " \"datatype\": \"sh\",\n", " \"description_long\": \"NDVI is generally calculated as (NIR - VIR) / (NIR + VIR). For Sentinel 2, this translates to (band 8 - band 4) / (band 8 + band 4).\",\n", " \"description_short\": \"A measure of the amount of vegetation at the pixel.\",\n", " \"id\": \"49464\",\n", " \"interpolation\": \"bilinear\",\n", " \"latitude_max\": 90.0,\n", " \"latitude_min\": -90.0,\n", " \"level\": 23,\n", " \"longitude_max\": 180.0,\n", " \"longitude_min\": -180.0,\n", " \"name\": \"Normalized difference vegetation index\",\n", " \"name_alternate\": \"Normalized difference vegetation index\",\n", " \"permanence\": true,\n", " \"properties\": {},\n", " \"rating\": 1.0,\n", " \"spatial_coverage\": {\n", " \"country\": [\n", " \"Belgium\",\n", " \"Bolivia\",\n", " \"France\",\n", " \"Germany\",\n", " \"India\",\n", " \"Indonesia\",\n", " \"Luxembourg\",\n", " \"Netherlands\",\n", " \"Switzerland\",\n", " \"Thailand\",\n", " \"United Kingdom\",\n", " \"United States of America\"\n", " ]\n", " },\n", " \"temporal_max\": \"1665604800000\",\n", " \"temporal_min\": \"1438387200000\",\n", " \"type\": \"R\",\n", " \"units\": \"[-1, 1]\",\n", " \"updated_at\": \"1665648134000\"\n", "}\n" ] } ], "source": [ "dl = catalog.get_data_layer(id = \"49464\")\n", "print(dl)" ] }, { "cell_type": "markdown", "id": "78d33bef", "metadata": {}, "source": [ "### Get a List of Data Layer Dimensions per Data Layer\n", "To list all Data Layer Dimensions belonging to a Data Layer, the `get_data_layer_dimensions` method is provided a Data Layer ID:" ] }, { "cell_type": "code", "execution_count": 9, "id": "97ae345a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2023-08-15 09:06:08 - paw - INFO - https://api.ibm.com/geospatial/run/na/core/v3/datalayers/49166/datalayer_dimensions\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
idshort_nameidentifierorderfull_nametypeunit
0243issuetimeA1NaNNaNNaN
1244horizonB2NaNNaNNaN
\n", "
" ], "text/plain": [ " id short_name identifier order full_name type unit\n", "0 243 issuetime A 1 NaN NaN NaN\n", "1 244 horizon B 2 NaN NaN NaN" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dlds = catalog.get_data_layer_dimensions(data_layer_id = \"49166\")\n", "dlds.display()" ] }, { "cell_type": "markdown", "id": "ff1ee17b", "metadata": {}, "source": [ "### Get a Data Layer Dimension\n", "To find out more about a Data Layer Dimension, once the Data Layer Dimension ID is known, the `get_data_layer_dimension` method is provided a Data Layer Dimension ID:" ] }, { "cell_type": "code", "execution_count": 10, "id": "8f234088", "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"data_layer_dimension_response\": {},\n", " \"full_name\": \"Issuetime\",\n", " \"id\": \"243\",\n", " \"identifier\": \"A\",\n", " \"order\": 1,\n", " \"short_name\": \"issuetime\",\n", " \"type\": \"integer\",\n", " \"unit\": \"hour\"\n", "}\n" ] } ], "source": [ "dld = catalog.get_data_layer_dimension(id = \"243\")\n", "print(dld)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.0" } }, "nbformat": 4, "nbformat_minor": 5 }