A report has outlined how millions of pounds of taxpayer money was spent on an asbestos-riddled former prison that the Conservative government wanted to use to house asylum seekers.