According to sociologists, why is deviance related to the societal response to a behavior, rather than to the behavior itself?

Some sociologists theorize that deviant behavior is only deviant because it is labeled deviant by society. So the idea is that behaviors labeled deviant are labeled that from societal response and that the behaviors are not inherently labeled deviant.