A Holocaust survivor has said he will return his federal order of merit award.