Infinity is surprisingly complicated, so let's look at what the Buddha said:
SN15.14:1.2: “Mendicants, transmigration has no known beginning. … It’s not easy to find a sentient being who in all this long time has not previously been your mother. Why is that? Transmigration has no known beginning. … This is quite enough for you to become disillusioned, dispassionate, and freed regarding all conditions.”
Given this understanding of transmigration leading back to "infinity" (or more precisely, "with no known beginning"), it is natural to ask about all the various Buddhas. So let's see what the Buddha says about the Buddha:
DN1:3.73.1: The Realized One’s body remains, but his attachment to rebirth has been cut off. As long as his body remains he will be seen by gods and humans. But when his body breaks up, after life has ended, gods and humans will see him no more.
In this way, Buddhas appear and disappear, never to be seen again.
Regarding "Buddha-nature", one might perhaps refer to this:
MN121:13.1: Whatever ascetics and brahmins enter and remain in the pure, ultimate, supreme emptiness—whether in the past, future, or present—all of them enter and remain in this same pure, ultimate, supreme emptiness.