Elie Wiesel, who spent time as a boy in a Nazi concentration camp, once said that the opposite of love is not hate, but indifference. Similarly, Rev. Martin Luther King, Jr. cautioned that "in the end, we will remember not the words of our enemies, but the silence of our friends." He also said that, "there comes a time when silence is betrayal."
There are forms of racism that spring from hatred. But those words from Wiesel and King are reminders that silent indifference is also racist. To apply that to a current controversy, I don't believe that Dr. Seuss hated Africans. But he was indifferent to how they would respond to being portrayed like this in his book, If I Ran the Zoo.