I would start my argument quoting these verses from the Dhammapada:
It may be well with the evil-doer as long as the evil ripens not. But when it does ripen, then the evil-doer sees (the painful results of) his evil deeds.
It may be ill with the doer of good as long as the good ripens not. But when it does ripen, then the doer of good sees (the pleasant results of) his good deeds.
Think not lightly of evil, saying, "It will not come to me." Drop by drop is the water pot filled. Likewise, the fool, gathering it little by little, fills himself with evil.
Think not lightly of good, saying, "It will not come to me." Drop by drop is the water pot filled. Likewise, the wise man, gathering it little by little, fills himself with good.
By acting moved by sensual desires, we acquire new habits and tendencies or reinforce old ones. A lot those sensual desires (whether they are sexual in nature or not) involve the interaction with other human beings (or sentient beings in general), and most of those interactions require to influence and subjugate others' wills to make them do or help us to reach whatever desire we have. The most extreme examples of this interactions result in labour exploitation, humilliation, death, fights, objectification, etc.
In sum, people become simple means for achieving our goals, and we start to get used to manipulate them to satisfy our whims and cravings. We stop caring if the other doesn't want what we want them to do.
Or even more directly than the above is to see others as obstacles to be eliminated in order to fulfill our wishes.
Of course, not every desire leads to this, but if we not pay attention to our feelings and habits, without noticing, we might find ourselves caught in the net of the results of our own deeds and negligence. If we underestimate the potential effects of indulgence in sensual desires, the pot may get filled without noticing, and others may get wet with our own water as well; others will suffer the effects of our ignorance and lack of mindfulness.