Ruggles helped found the New York Committee of Vigilance, an integrated group focused on protecting runaways and confronting slave catchers, called “blackbirders.” Ruggles is said to personally have helped as many as 600 fugitives, including Frederick Douglass, by sheltering them at his home on Lispenard Street. In his autobiography, Douglass wrote, “I had been in New York but a few days, when Mr. Ruggles sought me out, and very kindly took me to his boarding-house at the corner of Church and Lespenard Streets.”