David Lammy has downplayed calling Donald Trump a "neo-nazi" and insisted he can find common ground with the new president-elect.