This answer is from the Theravada perspective.

When you feel like you fear or dislike people because of their mental traps and neurotic dramas, you can use this opportunity to cultivate the brahmavihara of compassion (karuna).

The primary purpose of cultivating compassion is to cure this strong aversion that you have of other people. Aversion or hatred (dosa or dvesha) is one of the three poisons that will prevent your progress. Fear is a type of aversion. The secondary purpose is for you to regain a healthy social connection with the people around you.

Why are they the way they are? For e.g. if your grandmother who has senile dementia lashes out at you in anger or doesn't behave like normal people do, would you be fearful or judgemental or contemptuous against her? No. You would be compassionate towards her, because you understand that she has senile dementia.

Similarly, you can generate compassion by trying to understand that other people are suffering and there may be genuine underlying reasons for their suffering and condition. It could be their life situation (e.g. poverty or undergoing divorce) or even mental states (e.g. ignorance, or clouded by anger or other negative emotions) as you have correctly identified.

Instead of playing the role of a victim or potential victim or a contemptuous person or a hateful person, you can become compassionate towards others by recognizing that people who demonstrate neurotic behavior are actually suffering.

By tending to your own renunciation, you may be feeling more calm, but by cultivating compassion, you can create the balance needed in dealing with others. Renunciation and equanimity is how you deal with your own suffering. Meanwhile, compassion is how you deal with others' suffering.

Bhikkhu Bodhi wrote in "The Balanced Way":

Like a bird in flight borne by its two wings, the practice of Dhamma is sustained by two contrasting qualities whose balanced development is essential to straight and steady progress. These two qualities are renunciation and compassion. As a doctrine of renunciation the Dhamma points out that the path to liberation is a personal course of training that centers on the gradual control and mastery of desire, the root cause of suffering. As a teaching of compassion the Dhamma bids us to avoid harming others, to act for their welfare, and to help realize the Buddha's own great resolve to offer the world the way to the Deathless.

Considered in isolation, renunciation and compassion have inverse logics that at times seem to point us in opposite directions. The one steers us to greater solitude aimed at personal purification, the other to increased involvement with others issuing in beneficent action. Yet, despite their differences, renunciation and compassion nurture each other in dynamic interplay throughout the practice of the path, from its elementary steps of moral discipline to its culmination in liberating wisdom. The synthesis of the two, their balanced fusion, is expressed most perfectly in the figure of the Fully Enlightened One, who is at once the embodiment of complete renunciation and of all-embracing compassion.

Both renunciation and compassion share a common root in the encounter with suffering. The one represents our response to suffering confronted in our own individual experience, the other our response to suffering witnessed in the lives of others. Our spontaneous reactions, however, are only the seeds of these higher qualities, not their substance. To acquire the capacity to sustain our practice of Dhamma, renunciation and compassion must be methodically cultivated, and this requires an ongoing process of reflection which transmutes our initial stirrings into full-fledged spiritual virtues.

This answer is from the Theravada perspective.

When you feel like you fear or dislike people because of their mental traps and neurotic dramas, you can use this opportunity to cultivate the brahmavihara of compassion (karuna).

The primary purpose of cultivating compassion is to cure this strong aversion that you have of other people. Aversion or hatred (dosa or dvesha) is one of the three poisons that will prevent your progress. Fear is a type of aversion. The secondary purpose is for you to regain a healthy social connection with the people around you.

Why are they the way they are? For e.g. if your grandmother who has senile dementia lashes out at you in anger or doesn't behave like normal people do, would you be fearful or judgemental or contemptuous against her? No. You would be compassionate towards her, because you understand that she has senile dementia.

Similarly, you can generate compassion by trying to understand that other people are suffering and there may be genuine underlying reasons for their suffering and condition. It could be their life situation (e.g. poverty or undergoing divorce) or even mental states (e.g. ignorance, or clouded by anger or other negative emotions) as you have correctly identified.

Instead of playing the role of a victim or potential victim or a contemptuous person or a hateful person, you can become compassionate towards others by recognizing that people who demonstrate neurotic behavior are actually suffering.

By tending to your own renunciation, you may be feeling more calm, but by cultivating compassion, you can create the balance needed in dealing with others. Renunciation and equanimity is how you deal with your own suffering. Meanwhile, compassion is how you deal with others' suffering.

Ven. Bodhi wrote in "The Balanced Way":

Like a bird in flight borne by its two wings, the practice of Dhamma is sustained by two contrasting qualities whose balanced development is essential to straight and steady progress. These two qualities are renunciation and compassion. As a doctrine of renunciation the Dhamma points out that the path to liberation is a personal course of training that centers on the gradual control and mastery of desire, the root cause of suffering. As a teaching of compassion the Dhamma bids us to avoid harming others, to act for their welfare, and to help realize the Buddha's own great resolve to offer the world the way to the Deathless.

Considered in isolation, renunciation and compassion have inverse logics that at times seem to point us in opposite directions. The one steers us to greater solitude aimed at personal purification, the other to increased involvement with others issuing in beneficent action. Yet, despite their differences, renunciation and compassion nurture each other in dynamic interplay throughout the practice of the path, from its elementary steps of moral discipline to its culmination in liberating wisdom. The synthesis of the two, their balanced fusion, is expressed most perfectly in the figure of the Fully Enlightened One, who is at once the embodiment of complete renunciation and of all-embracing compassion.

Both renunciation and compassion share a common root in the encounter with suffering. The one represents our response to suffering confronted in our own individual experience, the other our response to suffering witnessed in the lives of others. Our spontaneous reactions, however, are only the seeds of these higher qualities, not their substance. To acquire the capacity to sustain our practice of Dhamma, renunciation and compassion must be methodically cultivated, and this requires an ongoing process of reflection which transmutes our initial stirrings into full-fledged spiritual virtues.