No it is not.
Then Ramayana and Mahabharata aren't scriptures, too.
Smritis are definitely scripture.
And yes, there were dissenters.
And no, not of only some Hindus, the casteism was systematic.
I have reletives in a small town, and my uncle remembers that in every wedding receptions and funerals brahmins ate first and no one else was allowed to eat together with them.
Only after brahmins had finished, people from other castes could begin eating.
This is Bengal, and only 50 years ago and less.