The reason to this state is actually not bound to things that's been taken, it's the worthiness (treating it's the best) that you given to the action of politeness. In simple word, asking permission or informing before taking something.
So the core of the buddhism is this. If you try to keep things the way you want (way you want means is the worthiness you have assigned to -> it can be anything like an action, an attitude, a person, a thing, etc..), you are doing a mistake. Because the things in the world are not in our control. Everything happens as a result of a reason.
Of course we can make reasons and control it right? But there are incidents that though we control (make reasons) things won't go the way we want right? If you think about this further the reason for that is,
- First case -> Our actions had some space to impact on the existing
- Second case -> Our actions had no space to change the existing state, because existing state had stronger reasons
So simply, though we think, we can 100% control things, actually we cannot. Without knowing this, we label things as politeness and we expects it from others, it won't happens the way you want it to be. Our foolishness is (not mean to offence) we crazily expect things like that from others and the world. And as a result we get angry. Which is bad for us. Why bad for us -> just have a look on this video which you can get an idea.
That's where the Buddhism comes useful. If we give worthiness to the world and expects things to be the way we want and when we try to control things the way we want, it results in anger which is bad. (Of course there's another side of the same on attraction to the things also doing bad for us, I won't speak about it here, since not relevant to the question)
So let it go, there are things that you can't change because the things are not the way you want, they behave based on their own reasons. So by getting angry by trying to change won't do any good for you but bad. You still can try to change without anger, if there are reasons it might be success.