I like your use of the words 'it seems', which suggests you've gained a particular degree of understanding. It does indeed 'seem' this way, but it is a very convincing mirage; an illusive trick created by the self-reflective nature of the six-sense consciousnesses.
Initially, (prior to stream-entry) one observes there to exist an enduring sense of self - personal space, time and consciousness. The realisation of stream-entry shatters this illusion. What then remains are habitual and episodic instances that Theravada calls 'I am' conceits. These 'I am' conceits are a notoriously stubborn mental fetter.
In Theravada, it is understood that one finds a way to observe the rising and disbanding of these 'I am' conceits using the five aggregate model, but it must be a method of observing whereby one is absent of any preoccupation with the themes and details of those conceits.
By doing the non-doing of this, one comes to understand anicca (impermanence) quite naturally and effortlessly simply because the transient nature of conditionality is a permeating truth. This eventually puts an end to the 'I am' conceit who's job it is to work against the tide of impermanence thus causing a strenuous relationship that we know as this and that. Anatta might then be understood as this and that being no longer defined as neither of those, nor nothing in-between them.