Sentinel-1 SAR Calibration input data

OTB Usage
1

Dear all,
Thanks for the ongoing support and the new version 8.0 of OTB, I’m looking forward to use it.
I’m experimenting a bit also on performing S-1 SAR calibration, but the default folder structure as provided by ESA seems to be an issue. With RS-2 I dont have any issue calibrating, but with S-1 I run into this error:

2021-03-23 11:11:13 (FATAL) SARCalibration: itk::ERROR: SarDefaultImageMetadataInterface(00000272D5D97790): GetRadiometricCalibrationScale() not implemented in SarDefaultImageMetadataInterface, no captor type found

This gives me the impression it cant find the metadata, while im using the default folder structure. What am I doing wrong?

Thanks in advance,
Thijs

2

EXTRA INFO:

I have read the SARCalibration — Orfeo ToolBox 7.2.0 documentation instructions, but here its just mentioned to use the RSAT_imagery_HH.tif, which doesnt help me a lot when using S1.

I’ve tried to run it with the s1a-iw-grd-vh.tif as input.

3

Hello,

which version of OTB are you using ?

Sentinel 1 products should be opened by providing the .tif file as input. GDAL is also able to open the manifest.safe or the folder directly, but OTB will not read the metadata in this case.

Can you run the application again using debug logs ?

OTB_LOGGER_LEVEL=DEBUG otbcli_SARCalibration -in s1a-iw-grd-vh.tif  -out out.tif

Cédric

4

Dear Cedric,
Thanks for the reply, I ran it on both otb 7.2 and 8.0. The error log with DEBUG on is this one:

2021-03-24 11:42:54 (INFO) SARCalibration: Default RAM limit for OTB is 256 MB
2021-03-24 11:42:54 (INFO) SARCalibration: GDAL maximum cache size is 1619 MB
2021-03-24 11:42:54 (INFO) SARCalibration: OTB will use at most 12 threads
2021-03-24 11:42:54 (DEBUG): Found attached file :  S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED\S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE\annotation/s1a-iw-grd-vh-20200612t042315-20200612t042340-032979-03d1e6-002.xml
2021-03-24 11:42:54 (DEBUG): Found attached file :  S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED\S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE\measurement/s1a-iw-grd-vh-20200612t042315-20200612t042340-032979-03d1e6-002.tiff
2021-03-24 11:42:54 (DEBUG): Found attached file :  S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED\S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE\annotation/s1a-iw-grd-vv-20200612t042315-20200612t042340-032979-03d1e6-001.xml
2021-03-24 11:42:54 (DEBUG): Found attached file :  S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED\S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE\measurement/s1a-iw-grd-vv-20200612t042315-20200612t042340-032979-03d1e6-001.tiff
2021-03-24 11:42:54 (DEBUG): No kwl metadata found in file  S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED\S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE\manifest.safe
2021-03-24 11:42:54 (DEBUG): Recursive evaluation of memory print for ProcessObjectExtractImageFilter (0000020942BF53D0)
2021-03-24 11:42:54 (DEBUG): Recursive evaluation of memory print for ProcessObjectClampImageFilter (00000209429A1980)
2021-03-24 11:42:54 (DEBUG): Recursive evaluation of memory print for ProcessObjectClampImageFilter (000002094231DC80)
2021-03-24 11:42:54 (DEBUG): Recursive evaluation of memory print for ProcessObjectSarRadiometricCalibrationToImageFilter (00000209421ACCF0)
2021-03-24 11:42:54 (DEBUG): Recursive evaluation of memory print for ProcessObjectImageFileReader (000002094025E140)
2021-03-24 11:42:54 (DEBUG): Evaluation of memory print for DataObject Image (0000020940262D60)
2021-03-24 11:42:54 (DEBUG): Evaluation of memory print for DataObject Image (0000020942728230)
2021-03-24 11:42:54 (DEBUG): Evaluation of memory print for DataObject VectorImage (0000020941E77170)
2021-03-24 11:42:54 (DEBUG): Evaluation of memory print for DataObject VectorImage (0000020941E77DA0)
2021-03-24 11:42:54 (DEBUG): Evaluation of memory print for DataObject VectorImage (0000020941E773E0)
2021-03-24 11:42:54 (DEBUG): Evaluation of memory print for DataObject VectorImage (0000020941E773E0)
2021-03-24 11:42:54 (INFO): Estimated memory for full processing: 14534.4MB (avail.: 16000 MB), optimal image partitioning: 1 blocks
2021-03-24 11:42:54 (INFO): File  sar_processed_data\out.tif will be written in 1 blocks of 25443x16639 pixels
Writing  sar_processed_data\out.tif...: 0% [                                                  ]2021-03-24 11:42:55 (DEBUG): GDAL reads [0, 25442]x[0, 16638] x 2 bands of type UInt16 from file  S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED\S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE\manifest.safe
2021-03-24 11:42:57 (DEBUG): GDAL read took 1678 ms
2021-03-24 11:42:57 (DEBUG) SARCalibration: Caught itk::ExceptionObject during application execution:
2021-03-24 11:42:57 (DEBUG) SARCalibration: ..\Modules\Core\Metadata\include\otbSarDefaultImageMetadataInterface.h:65:
itk::ERROR: SarDefaultImageMetadataInterface(000002094233E1F0): GetRadiometricCalibrationScale() not implemented in SarDefaultImageMetadataInterface, no captor type found
2021-03-24 11:42:57 (FATAL) SARCalibration: itk::ERROR: SarDefaultImageMetadataInterface(000002094233E1F0): GetRadiometricCalibrationScale() not implemented in SarDefaultImageMetadataInterface, no captor type found

Does this bring any clues to you?

Thanks for the support!
Thijs

5

Hello,

I don’t understand what’s happening here, the line

2021-03-24 11:42:54 (DEBUG): No kwl metadata found in file  S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED\S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE\manifest.safe

means that metadata parsing has failed.

I downloaded your product and i have no problem preforming the calibration on it (using OTB 7.2.0). Here are the expected logs:

OTB_LOGGER_LEVEL=DEBUG otbcli_SARCalibration -in S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE/measurement/s1a-iw-grd-vv-20200612t042315-20200612t042340-032979-03d1e6-001.tiff  -out /tmp/out.tif
2021-03-25 10:58:07 (INFO) SARCalibration: Default RAM limit for OTB is 256 MB
2021-03-25 10:58:07 (INFO) SARCalibration: GDAL maximum cache size is 794 MB
2021-03-25 10:58:07 (INFO) SARCalibration: OTB will use at most 12 threads
2 calibration files found in /Datas/S1Data/S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE
2021-03-25 10:58:07 (INFO): Loading kwl metadata from official product in file S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE/measurement/s1a-iw-grd-vv-20200612t042315-20200612t042340-032979-03d1e6-001.tiff
2021-03-25 10:58:07 (DEBUG): Recursive evaluation of memory print for ProcessObjectExtractImageFilter (0x1d19360)

[...]

Are you using the original S1 product (I downloaded it form Copernicus hub). Maybe it is corrupted ? This is the content of the product folder in my case:

ls -R
S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE
S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE:
annotation     measurement  S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE-report-20200612T070049.pdf
manifest.safe  preview      support

S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE/annotation:
calibration  s1a-iw-grd-vh-20200612t042315-20200612t042340-032979-03d1e6-002.xml  s1a-iw-grd-vv-20200612t042315-20200612t042340-032979-03d1e6-001.xml

S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE/annotation/calibration:
calibration-s1a-iw-grd-vh-20200612t042315-20200612t042340-032979-03d1e6-002.xml
calibration-s1a-iw-grd-vv-20200612t042315-20200612t042340-032979-03d1e6-001.xml
noise-s1a-iw-grd-vh-20200612t042315-20200612t042340-032979-03d1e6-002.xml
noise-s1a-iw-grd-vv-20200612t042315-20200612t042340-032979-03d1e6-001.xml

S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE/measurement:
s1a-iw-grd-vh-20200612t042315-20200612t042340-032979-03d1e6-002.tiff  s1a-iw-grd-vv-20200612t042315-20200612t042340-032979-03d1e6-001.tiff

S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE/preview:
icons  map-overlay.kml  product-preview.html  quick-look.png

S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE/preview/icons:
logo.png

S1A_IW_GRDH_1SDV_20200612T042315_20200612T042340_032979_03D1E6_B4ED.SAFE/support:
s1-level-1-calibration.xsd  s1-level-1-noise.xsd    s1-level-1-quicklook.xsd  s1-object-types.xsd
s1-level-1-measurement.xsd  s1-level-1-product.xsd  s1-map-overlay.xsd        s1-product-preview.xsd

For the calibration, I think the required files are those in measurement, annotation and annotation/calibration.

Cédric

6

Hey Cedric,
Thanks a lot for your extensive support.

It seems I simply needed to run it on a Linux VM and not my local windows machine. On a linux VM, it does find all the necessary metadata files.

Regards,
Thijs

7

I actually have the same error with a Terrasar X image and I work on linux.
here are my logs:

otbcli_SARCalibration -in TSX-1.SAR.L1B/TDX1_SAR__MGD_RE___SC_S_SRA_20230115T160350_20230115T160419/IMAGEDATA/IMAGE_HH_SRA_wide_005.tif -out Sar_Calibrated_TX01_SAR_WS_MGD_20230115T160350_20230115T160419_NSG_069709_0312.TSX.tif float -ram 16000
SCANNING DIRECTORY
2023-01-24 04:39:39 (INFO) SARCalibration: Default RAM limit for OTB is 256 MB
2023-01-24 04:39:39 (INFO) SARCalibration: GDAL maximum cache size is 733 MB
2023-01-24 04:39:39 (INFO) SARCalibration: OTB will use at most 4 threads
2023-01-24 04:39:41 (INFO): Loading kwl metadata from official product in file /home/worker/procDir/SO_000142571_0015_1/TSX-1.SAR.L1B/TDX1_SAR__MGD_RE___SC_S_SRA_20230115T160350_20230115T160419/IMAGEDATA/IMAGE_HH_SRA_wide_005.tif
2023-01-24 04:39:41 (DEBUG): Recursive evaluation of memory print for ProcessObjectExtractImageFilter (0x3374db0)
2023-01-24 04:39:41 (DEBUG): Recursive evaluation of memory print for ProcessObjectClampImageFilter (0x3412a30)
2023-01-24 04:39:41 (DEBUG): Recursive evaluation of memory print for ProcessObjectClampImageFilter (0x3412690)
2023-01-24 04:39:41 (DEBUG): Recursive evaluation of memory print for ProcessObjectSarRadiometricCalibrationToImageFilter (0x3374270)
2023-01-24 04:39:41 (DEBUG): Recursive evaluation of memory print for ProcessObjectImageFileReader (0x1ad5250)
2023-01-24 04:39:41 (DEBUG): Evaluation of memory print for DataObject Image (0x1ad7390)
2023-01-24 04:39:41 (DEBUG): Evaluation of memory print for DataObject Image (0x3412170)
2023-01-24 04:39:41 (DEBUG): Evaluation of memory print for DataObject VectorImage (0x31962b0)
2023-01-24 04:39:41 (DEBUG): Evaluation of memory print for DataObject VectorImage (0x3218560)
2023-01-24 04:39:41 (DEBUG): Evaluation of memory print for DataObject VectorImage (0x1bacf80)
2023-01-24 04:39:41 (DEBUG): Evaluation of memory print for DataObject VectorImage (0x1bacf80)
2023-01-24 04:39:41 (INFO): Estimated memory for full processing: 6159.95MB (avail.: 16000 MB), optimal image partitioning: 1 blocks
2023-01-24 04:39:41 (INFO): File /home/worker/workDir/outDir/output/Sar_Calibrated_TX01_SAR_WS_MGD_20230115T160350_20230115T160419_NSG_069709_0312.TSX.tif will be written in 1 blocks of 13188x13605 pixels
2023-01-24 04:41:32 (DEBUG): GDAL reads [0, 13187]x[0, 13604] x 1 bands of type UInt16 from file /home/worker/procDir/SO_000142571_0015_1/TSX-1.SAR.L1B/TDX1_SAR__MGD_RE___SC_S_SRA_20230115T160350_20230115T160419/IMAGEDATA/IMAGE_HH_SRA_wide_005.tif
2023-01-24 04:42:29 (DEBUG): GDAL read took 56955 ms
2023-01-24 04:42:30 (DEBUG) SARCalibration: Caught itk::ExceptionObject during application execution:
2023-01-24 04:42:30 (DEBUG) SARCalibration: ../Modules/Core/Metadata/include/otbSarDefaultImageMetadataInterface.h:65:
itk::ERROR: SarDefaultImageMetadataInterface(0x178190e0): GetRadiometricCalibrationScale() not implemented in SarDefaultImageMetadataInterface, no captor type found
2023-01-24 04:42:30 (FATAL) SARCalibration: itk::ERROR: SarDefaultImageMetadataInterface(0x178190e0): GetRadiometricCalibrationScale() not implemented in SarDefaultImageMetadataInterface, no captor type found

is this a problem that would be solved with the new release ( OTB release 7.4.0 ) ?

8

Hi @BeaG

According to the release note it should work better with TerrasarX, you can try with the last 7.4 release which is OTB 7.4.1

Can you make a feedback after your attempt with 7.4.1?

Thank you