Segmentation Vector smoothing (polygons): topology/geometry problems

Dear all, I performed a raster segmentation from a Spot6 satellite image with OTB. I polygonized/vectorized this raster segmentation in order to get a vector file. To limit the pixelation of this cover, I was thinking of smoothing the layer with the QGIS tool. This works, but the problem is that it generates topology/geometry issues (overlap, holes, etc.). And I can’t afford to have invalid geometries. Could you share some advices? Guidances in order to skip those problems? I’ve attached a screenshot of the smoothed vector segmentation.
Thanks so much.