Is it right to say that all Buddhists both do and don't believe in the empirical self?
I think it's more accurate to say that Buddhists ought to avoid having fixed/specific beliefs, or fixed views, about the self: see for example What are examples of identity-view?
And in order I might make sense of your answer, can the latter "not self" be thought of as an absence of any conscious mind from reality itself?
The "not-self" concept is called Anatta.
Buddhism doesn't deny consciousness, but it says that consciousness too is conditioned and impermanent and non-self.
Finally, does that reality include or consist of the Buddha and how can we unenlightened beings confirm that, besides confirming suchness, when it seems we cannot confirm suchness is known?
What "reality" are you talking about?
Still, if you're asking about "reality including the Buddha" then perhaps you want to find out about the doctrines of Buddha nature.
In summary you seem to be saying that the skandhas (including sensation/perception, and consciousness) are "the self", even though (as Wikipedia says),
All of the aggregates are to be seen as empty of self-nature; that is, they arise dependent on causes (hetu) and conditions (paticca). In this scheme, the cause for the arising of consciousness (viññāṇa) is the arising of one of the other aggregates (physical or mental); and, the arising of consciousness in turn gives rise to one or more of the mental (nāma) aggregates. In this way, the chain of causation identified in the aggregate (khandha) model overlaps the chain of conditioning in the Dependent Origination (paticcasamuppāda) model, described more fully below.
I think "have self-nature" means "have independent existence" and "have no self-nature" means "has dependent existence" and therefore temporary and unsatisfactory.
Buddhism suggests that instead of identifying the self with such things ("I am my body! I am my feelings! I am my thoughts!") it's better to not identify yourself in such a way with such things.
This description of right view warns that the following can be called "a thicket of wrong views":
As he attends inappropriately in this way, one of six kinds of view arises in him: The view I have a self arises in him as true & established, or the view I have no self... or the view It is precisely by means of self that I perceive self... or the view It is precisely by means of self that I perceive not-self... or the view It is precisely by means of not-self that I perceive self arises in him as true & established, or else he has a view like this: This very self of mine — the knower that is sensitive here & there to the ripening of good & bad actions — is the self of mine that is constant, everlasting, eternal, not subject to change, and will endure as long as eternity. This is called a thicket of views, a wilderness of views, a contortion of views, a writhing of views, a fetter of views. Bound by a fetter of views, the uninstructed run-of-the-mill person is not freed from birth, aging, & death, from sorrow, lamentation, pain, distress, & despair. He is not freed, I tell you, from suffering & stress.
If I understand they way in which you're using words, I suspect you shouldn't say that there's "an absence of any conscious mind from reality itself" because that would be saying "there are no sentient beings, sentient beings do not exist": which might be self-evidently untrue :-)
I'm not sure about whether all of reality is conscious. Apparently some schools of Buddhism teach that,
Furthermore, and particularly in Tibetan Buddhism and Japanese Buddhism, all beings (including plant life and even inanimate objects or entities considered "spiritual" or "metaphysical" by conventional Western thought) are or may be considered sentient beings.
As I mentioned earlier I think this sort of doctrine is later (i.e. post-Theravada) teachings about 'Buddha-nature'.