It seems to me if had an infinity of past lives we should have
reached wisdom and buddahood at some point.
Indeed.
The Buddha himself did not teach about "past lives". The words commonly translated as "past lives" in the original scriptures are "pubbe nivasa". The word "nivasa" means "homes", "abodes" or even "adherences" (in the related word "nivesa").
A "nivasa" is when one or more of the five aggregates (body, feelings, perception, mental formations & sense consciousness) are attached to as "self".
Therefore, in this life, each mind has clung to aggregates as "self" infinite times. The relevant scripture, properly translated, says:
At Savatthi. Bhikkhus, those ascetics and brahmins who recollect
their manifold past abodes (pubbe nivasa) all recollect the five aggregates subject
to clinging or a certain one among them. What five?
When recollecting thus, bhikkhus: ‘I had such form in the past,’ it
is just form that one recollects. When recollecting: ‘I had such a
feeling in the past,’ it is just feeling that one recollects. When
recollecting: ‘I had such a perception in the past,’ it is just
perception that one recollects. When recollecting: ‘I had such
volitional formations in the past,’ it is just volitional formations
that one recollects. When recollecting: ‘I had such consciousness in
the past,’ it is just consciousness that one recollects.
Therefore, bhikkhus, any kind of form whatsoever … Any kind of
feeling whatsoever … Any kind of perception whatsoever … Any kind of
volitional formations whatsoever … Any kind of consciousness
whatsoever, whether past, future, or present, internal or external,
gross or subtle, inferior or superior, far or near, all consciousness
should be seen as it really is with correct wisdom thus: ‘This is not
mine, this I am not, this is not my self.’
This is called, bhikkhus, a noble disciple who dismantles and does
not build up; who abandons and does not cling; who scatters and does
not amass; who extinguishes and does not kindle.
And what is it that he dismantles and does not build up? He
dismantles form and does not build it up. He dismantles feeling …
perception … volitional formations … consciousness and does not build
it up.
And what is it that he abandons and does not cling to? He abandons
form and does not cling to it. He abandons feeling … perception …
volitional formations … consciousness and does not cling to it.
And what is it that he scatters and does not amass? He scatters form
and does not amass it. He scatters feeling … perception … volitional
formations … consciousness and does not amass it.
And what is it that he extinguishes and does not kindle? He
extinguishes form and does not kindle it. He extinguishes feeling …
perception … volitional formations … consciousness and does not kindle
it.
https://suttacentral.net/sn22.79/en/bodhi
It is clearly noticeable how the answers provided to your question by the common Buddhist are contrary to what is written above. The common Buddhist refers to infinite past lives yet the above says the Noble Disciple "dismantles, does not build up, does not amass, does not kindle".
Other common false translations found in this type of topic are "sentient beings" and "transmigration".
The Pali word falsely translated as "sentient being" is "satta". "Satta" merely means "a being" and, similar to "nivasa", refers merely to an attachment or a view, as explained in SN 23.2 & SN 5.10.
The Pali word falsely translated as "transmigration" is "saṃsāra". "Saṃsāra" merely means "continuance" and, similar to "nivasa", refers merely to the continuance of attachment, as explained in SN 22.99, as follows:
Just as a dog, tied by a leash to a post or stake, keeps running
around and circling around that very post or stake; in the same way,
an uninstructed, run-of-the-mill person — who has no regard for noble
ones, is not well-versed or disciplined in their Dhamma; who has no
regard for people of integrity, is not well-versed or disciplined in
their Dhamma — assumes form to be the self, or the self as possessing
form, or form as in the self, or the self as in form.
He assumes feeling to be the self...
He assumes perception to be the self...
He assumes (mental) fabrications to be the self...
He assumes consciousness to be the self, or the self as possessing
consciousness, or consciousness as in the self, or the self as in
consciousness.
He keeps running around and circling around that very form... that
very feeling... that very perception... those very fabrications...
that very consciousness. He is not set loose from form, not set loose
from feeling... from perception... from fabrications... not set loose
from consciousness. He is not set loose from birth, aging, & death;
from sorrows, lamentations, pains, distresses, & despairs. He is not
set loose, I tell you, from suffering & stress.
Why false translations occur was explained by the renowned Thai Buddhist monk Bhikkhu Buddhadasa, as follows:
People language is used by the ordinary people who don't understand
Dhamma very well and by those worldly people who are so dense that
they are blind to everything but material things. Then, there is the
language which is spoken by those who understand reality (Dhamma),
especially those who know and understand reality in the ultimate
sense. This is another kind of language.
People who are blind to the true reality (Dhamma) can speak only
people language, the conventional language of ordinary people. On the
other hand, people who have genuinely realized the ultimate truth
(Dhamma) can speak either language. They can handle people language
quite well and are also comfortable using Dhamma language, especially
when speaking among those who know reality, who have already realized
the truth (Dhamma). Amongst those with profound understanding, Dhamma
language is used almost exclusively, unfortunately, ordinary people
can't understand a word. Dhamma language is understood only by those
who are in the know. What is more, in Dhamma language it isn't even
necessary to make a sound. For example, a finger is pointed or an
eyebrow raised and the ultimate meaning of reality is understood. So,
please take interest in these two kinds of language--people language
and Dhamma language.
No Religion
More can be read, here: Two Kinds of Language