Dozens of ICE agents clashed with protesters using pepper spray to disperse crowds outside a detention centre.