Legislation has 'whittled away' rights of workers