If someone building their bad Karma by manipulating against me and if I've do something for revenge then It'd be my bad karma or wouldn't considered as Karma? Or It'd be same as "The Lady and the Ogress - Dhammapada Stories." Not in other, next life but in this life.
If due to some contain you cannot follow through with your action this does not become a potent karma with the potential to give rebirth but the volition forms a mild karma which can give result after rebirth.
Regardless of if you follow through with a action of intend to do some action as retaliatory or not, action and intention will become karma.
this may not be direct answer but more of friendly reminder of 6 things one should know about Karma,
intention is karma
causal of Karma (nidana) which is detection through sense gates. This is important because it is not a common knowledge. Those who have not heard Dhamma would think pleasures or sufferings are caused by self, others, etc.
Vemattata, or level, magnitude, differences of karma. Karma that causes being to be in hell, animal realms, hungry ghost, human, heaven.
Timing, karma in the here and now, later, and even further away.
Ending. Karma ends because detection from sense gates is ended.
path of ending of karma, 8 fold path
And to put things in perspective, Buddha said throughout the Samara, each of us had been a thieve so many times. all of the blood shed from the punishment when caught in all those life time is more than the oceans combined. And the opportunity for one to be born in a human is as rare as a chance of a Buddha to appear.
your karma hypothetical question is impossible to be answered by anyone but Buddha because of the complexity of karma.