School No. 5, an African School for Black children, was established by the New York Manumission Society at 161 Duane Street. By 1836, it had 116 students and changed its name to "Colored" School No. 5 in 1838. The school also hosted the Philomathean Society, which advocated for abolition and civil rights, with notable support from figures like Peter Augustus Jay.