The Buddha did explicitly refute the notion of a permanent self in the Ananda Sutta. He says that if he had said that there is a permanent self then that would conflict with the statement that "all phenomena is not self".
"Ananda, if I — being asked by Vacchagotta the wanderer if there is a
self — were to answer that there is a self, that would be conforming
with those brahmans & contemplatives who are exponents of eternalism.
If I — being asked by Vacchagotta the wanderer if there is no self —
were to answer that there is no self, that would be conforming with
those brahmans & contemplatives who are exponents of annihilationism.
If I — being asked by Vacchagotta the wanderer if there is a self —
were to answer that there is a self, would that be in keeping with the
arising of knowledge that all phenomena are not-self?" "No, lord."
"And if I — being asked by Vacchagotta the wanderer if there is no
self — were to answer that there is no self, the bewildered
Vacchagotta would become even more bewildered: 'Does the self I used
to have now not exist?'"
The concept of eternalism says that there is a permanent self and it is undying. The concept of annihilationism says there is no self at all and at death it all ceases.
Buddha's teaching is of the middle way stating there is a self that "emerges", as a result of the dependency on the inter-working of the five aggregates of form, feeling, perception, mental formations and consciousness. Each of these five aggregates also have dependencies. This is what is meant by the Buddha's statement, "all conditioned things are impermanent", in other words, all these things that have dependencies are not permanent.
However, there is no single self that is the charioteer or agent of the five aggregates. Imagine you hurt your hand and say "I hurt myself". But if your hand is chopped off, does your self disappear? No. So, applying this one by one, you realize that your self is neither form, feeling, perception, mental formations or consciousness. But beyond this, there is nothing more to yourself. That is what is meant by the Buddha's statement, "all phenomena is not self".
This is just an overly-generalized summary. More info can be found on the topic of dependent origination.
Now according to the concept of annihilationism, there should be no continuity at all and no rebirth, but there is. If there is no permanent self then what is it that is reborn? The idea is that "information" or chain of dependent processes that never ceases, and keeps reappearing, so long as the desire and attachment to exist does not cease.
This is described here (Milindapanha 3.5.5):
The king asked: "Venerable Nagasena, is it so that one does not
transmigrate and one is reborn?" "Yes, your majesty, one does
not transmigrate and one is reborn." "How, venerable Nagasena, is it
that one does not transmigrate and one is reborn? Give me an analogy."
"Just as, your majesty, if someone kindled one lamp from another, is
it indeed so, your majesty, that the lamp would transmigrate from the
other lamp?" "Certainly not, venerable sir." "Indeed just so, your
majesty, one does not transmigrate and one is reborn." "Give me
another analogy." "Do you remember, your majesty, when you were a boy
learning some verse from a teacher?" "Yes, venerable sir." "Your
majesty, did this verse transmigrate from the teacher?" "Certainly
not, venerable sir." "Indeed just so, your majesty, one does not
transmigrate and one is reborn."