Definition of "Detraction" :

noun: the act of discrediting or detracting from someone's reputation (especially by slander)

"Let it be no detraction from his merits to say he is plainspoken."

noun: a petty disparagement