Fees attacks weaken aid: Goldsmith