What you are experiencing is too much empathy. Many people believe that empathy is a universal virtuous trait that should always be increased, but this is not so.
To be very clear, empathy is distinct from compassion. The two are not the same thing although they are often conflated and misunderstood. The definition of compassion is very simple: the wish for others to be free from suffering. That's it.
Empathy is different from compassion. Empathy is the capacity to put yourself in someone else's shoes and feel what it is they are feeling. To imagine what someone else is feeling and experience some amount of it yourself.
So what is the relationship between compassion and empathy? Well, to some extent you need to be able to recognize someone else's suffering in order to develop the wish for them to be free of it. However, once this recognition is accomplished it does not require you to keep feeling what they are feeling. To suffer with them. When people in the west talk about "compassion fatigue" they are really talking about empathy fatigue.
So in summary, empathy can be both good (allowing us to recognize the suffering of others) and bad (rendering us powerless/overwhelmed or biased towards those we empathize with) depending upon the context. It sounds like you are definitely experiencing the latter and should stop empathizing so much. In order to do so, you first have to become clear on the differences between compassion and empathy. Compassion is a universal good that should always be increased. There is no such thing as compassion fatigue, but there is such a thing as empathy fatigue.
Have a look here for a modern account of how a western psychology recently discovered something that Buddhists have known for centuries ;)
Against Empathy: The Case for Rational Compassion
You can also find an interesting interview with the author here.