The public should not be "worried" about the cost of government borrowing going up, a cabinet minister has said.