OTBTF PatchesExtraction on multiple separated images for a single CNN model

OTB Usage
1

Hello,
I am trying to apply a semantic segmentation using a CNN with images from multiple geographically separated study areas. Following the OTBTF/Keras tutorial, I am using the PatchesExtraction function to extract training, testing and validation patches:

for vec in [vec_train, vec_valid, vec_test]:
    app_extract = pyotb.PatchesExtraction({
        "source1.il": "/data/rgb_img.tif",
        "source1.patchsizex": 64,
        "source1.patchsizey": 64,
        "source1.nodata": 0,
        "source2.il": "/data/labels_img.tif",
        "source2.patchsizex": 64,
        "source2.patchsizey": 64,
        "source2.nodata": 255,
        "vec": vec,
        "field": "id"
    })

However, as far as I understand this function can only be applied to one input image (or multiple sources but from the same area). Since my study areas are quite far apart from each other, I can’t merge the images into a single mosaic before applying the function.
To work around this, I have extracted patches separately from each image using a loop. Now, I would like to combine these patches into a single dataset to train a single CNN model: how can I do this?

I have read this in the OTBTF documentation:

Sampled patches are stored in one single big image that stacks all patches in rows. The motivation is that accessing one unique big file is more efficient than working on thousands of separate very small files stored in the file system, or accessing sparsely located patches in large images. All geospatial metadata is lost in the process.

So the question is: how can I merge these separate patch files into a single dataset for training (and similarly for testing and validation) to have a single CNN model?

Thank you for any advice!
Emilia

2

Hi @emiliapafumi ,

Just create patches images for each pairs.
Then just provide to the otbtf.Dataset the files in the same order for each source:

e.g.

import otbtf

# Create tensorflow dataset
ds = otbtf.DatasetFromPatchesImages(
  filenames_dict={
    "x": ["imgs1.tif", "imgs2.tif"], 
    "y": ["labels1.tif", "labels2.tif"]
  }
).get_tf_dataset()

# Train your model with `ds`
...

Hope that helps,

Rémi