Opposition parties have criticised a former minister who said British people needed a greater work ethic - pointing out his record under Liz Truss.