There's a long history of saying certain people shouldn't be voting. And, unfortunately, the people who are often left out of these conversations are people who are black and brown.