Lords call on Irvine to give up judge's role