Accessing L1 (raw), L2 (observations and mosaics), and LE3 products through the command line, with the Data Download (sas-dd) service
Once authenticated, this service allow the user to download data products from different levels and data releases. The Search - Observations option allows to download L1 (raw) and L2 (observations and mosaics) using the Euclid web interface, but LE3 products can only be downloaded from the command line. As reference here is the list of possible parameters to ask for the different products, followed below by some examples.
To retrieve Observations use RETRIEVAL_TYPE=OBSERVATION, followed by the next parameters.
Parameter |
Description |
Mandatory |
Possible values |
OBS_ID |
Describes the observation identifier or a list of them (column observation_id) |
YES |
NA |
TYPE |
Describes the type of data products within the following valid values or a list of them |
NO |
|
INSTRUMENT |
Describes the instrument. By default all instruments are considered |
NO |
|
FILTER |
Describes the filter within the following valid values or a list of them.
By default the products for all filters will be downloaded |
NO |
|
RELEASE |
Identifier of the Data Release |
NO |
|
To retrieve Mosaics use RETRIEVAL_TYPE=MOSAIC, followed by the TYPE parameter and those listed in any of the three options.
TYPE: describes the type of data products within the following valid values or a list of them: STK, BKG, FLAG, PSF, GRID_PSF, RMS, and ALL.
Option 1: To retrieve Mosaic data products by Tile Index and Instrument use the following parameters.
TILE_INDEX:: describes the Tile identifier or a list of them.
INSTRUMENT and FILTER: describes the instrument and filter/s.
Option 2: To retrieve Mosaic data products by unique identifier use the following parameter:
MSC_ID: describes the mosaic identifier or a list of them.
Parameter |
Description |
Mandatory |
Possible values |
TYPE |
Describes the type of data products within the following valid values or a list of them |
YES |
|
RELEASE |
Identifier of the Data Release |
NO |
|
TILE_INDEX |
Describes the Tile identifier or a list of them |
YES (Option 1) |
NA |
INSTRUMENT |
Describes the instruments within the following valid values or a list of them |
YES (1) |
|
FILTER |
Describes the filter within the following valid values |
YES (1) |
|
MSC_ID |
Describes the mosaic identifier or a list of them (column mosaic_oid) |
YES (2) |
NA |
To retrieve LE3 use RETRIEVAL_TYPE=LE3, followed by the next parameter.
Parameter |
Description |
Mandatory |
Possible values |
NAME |
Product name to identify the .tar.gz file to download. Both parameter name and values are not case sensitive. |
NO without it all 4 products will be downloaded at once (several GB) |
|
It is possible to download products of any kind knowing their file names or product Ids. See section 4 below for examples.
Parameter |
Description |
Mandatory |
Possible values |
FILE_NAME |
File name/s of the fits file/s to download. If more than one, list them between a comma. |
NO |
NA |
PRODUCT_ID |
Product Id/s of the fits file/s to download. If more than one, list them between a comma. |
NO |
NA |
To retrieve Frame Catalogs use RETRIEVAL_TYPE=FRAME_CATALOG, followed by the next parameters. Note in this case the release parameter is mandatory. See section 5 below for examples.
Parameter |
Description |
Mandatory |
Possible values |
OBS_ID |
Describes the observation identifier or a list of them (column observation_id) |
YES |
NA |
FILTER |
Describes the filter used within the following valid values or a list of them |
NO |
|
RELEASE |
Identifier of the Data Release |
YES |
|
Login to sas-dd
The sas-dd service requires authentication from the user.
Login
curl -k -c cookies.txt -X POST -d username=USERNAME -d password=PASSWORD -L
"https://eas.esac.esa.int/sas-dd/login"
Logout
curl -k -b cookies.txt -X POST -d -L
"https://eas.esac.esa.int/sas-dd/logout"
Observations and Mosaics retrieval examples
The following examples will show the different options to retrieve Euclid data products through the command line. The links can also be copied to a browser once the authentication has been done on it.
Retrieve data products by Observation Id "13", Instrument "VIS" and Type "STK" from release sedm_pvpr01
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?retrieval_type=OBSERVATION&obs_id=13&INSTRUMENT=VIS&type=STK&RELEASE=sedm_pvpr01"
Retrieve data products by Observation Id {"10351", "10354", "10355"}, Instrument "VIS" and Type "STK from release SC_8"
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?RETRIEVAL_TYPE=OBSERVATION&OBS_ID=10351%2C10354%2C10355&INSTRUMENT=VIS&TYPE=STK&RELEASE=SC_8"
The %2C is a way to encode the comma character, but generally the comma can be used directly as well.
Retrieve data products by Observation Id {"10351", "10354", "10355"}, Instruments {"VIR","NIR_H","NIR_J"} and Type "STK" from SC_8
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?RETRIEVAL_TYPE=OBSERVATION&RETRIEVAL_ACCESS=DIRECT&OBS_ID=10351%2C10354%2C10355&INSTRUMENT=VIS%2CNIR_H%2CNIR_J&TYPE=STK&RELEASE=SC_8"
Retrieve aux data products by Observation Id "13", Instrument "VIS" and Type "PSF" from sedm_pvpr01
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?retrieval_type=OBSERVATION&RETRIEVAL_ACCESS=DIRECT&obs_id=13&INSTRUMENT=VIS&type=PSF&RELEASE=sedm_pvpr01"
Retrieve mosaics Mosaic ID "2", Instrument "VIS" from sedm_pvpr01
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?&MSC_ID=2&RELEASE=sedm_pvpr01&RETRIEVAL_TYPE=MOSAIC&INSTRUMENT=VIS"
Retrieve mosaics by: list of Tile Index list {"40002", "40003"} and Instrument "NISP" from SC_8
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?&TILE_INDEX=40003,40002&RELEASE=SC_8&RETRIEVAL_TYPE=MOSAIC&INSTRUMENT=NISP"
Retrieve mosaics by: list of Tile Index "40003", Instrument "NISP" and TYPE "RMS" from SC_8
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?&TILE_INDEX=40003&RELEASE=SC_8&RETRIEVAL_TYPE=MOSAIC&INSTRUMENT=NISP&TYPE=RMS"
LE3 data products retrieval examples
The following examples will show the different options to retrieve LE3 data products. In each subsection you can clic on the product names to access some external documentation.
Retrieve data products for internal products
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?retrieval_type=LE3&name=InternalDP"
Retrieve data products for galaxy clusters products
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?retrieval_type=LE3&name=ClusterDP"
Retrieve data products for weak lensing products
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?retrieval_type=LE3&name=WLDP"
Retrieve data products for galaxy clustering products
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?retrieval_type=LE3&name=GCDP"
Retrieve all data products at once in a zip file
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?retrieval_type=LE3"
Retrieving data products based on file name of product Id
The following examples will show the different options to retrieve any kind of data products if the file name or product Id values are known. Note the parameter RETRIEVAL_TYPE is still required, although the value provided is ignored in case FILE_NAME or PRODUCT_ID are provided. Multiple files can be downloaded at once, as a zip file, by listing them between comma. The parameter RELEASE is mandatory to identify in which data release the file name or product should be searched.
Retrieve one mosaic knowing its file name
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?FILE_NAME=EUC_MER_BGSUB-MOSAIC-NIR-H_TILE100159190-7F8E05_20220919T214307.064518Z_00.00.fits&RETRIEVAL_TYPE=FILE&RELEASE=sedm_pvpr01"
Retrieve two or more data products (listed between comma) of different types, knowing their product Ids
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?PRODUCT_ID=NIR_ProcessField_SelfCal-2.1.1-9-ON_THE_FLY-pcasenov-PLAN-000001-4Q5NYURF-20220915-080534-001-stk_image_xml-0,SIMPV008221026311-9GU5X5Z8-20221026-103547-001-outputSIM-9&RETRIEVAL_TYPE=PRODUCT_ID&RELEASE=sedm_pvpr01"
Retrieve products by file name and product Id at once
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?FILE_NAME=EUC_SIM_NISRGS180-8-1_20220722T094150.427Z_PV023_NISP-S_8_18_0.fits&PRODUCT_ID=MER_ProcessTile_SelfCal_9.0.1-4-ON_THE_FLY-pcasenov-PLAN-000000-G4UOE2QX-20220919-174958-001-all_mosaics-2&RETRIEVAL_TYPE=ID&RELEASE=sedm_pvpr01"
Frame catalog retrieval examples
curl -k -b cookies.txt -X GET
"https://eas.esac.esa.int/sas-dd/data?retrieval_type=FRAME_CATALOG&obs_id=13&FILTER=VIS&RELEASE=sedm_pvpr01"