[Spark] Prevent FetchFailedException when enabling optimize write feature when running remote shuffle service (e.g. celeborn)

[Spark] Prevent FetchFailedException when enabling optimize write feature when running remote shuffle service (e.g. celeborn) #15770