who said "I don't care whether slavery is voted up or down"