From the Pali suttas, the first that came to mind is "Assusutta (SN 15.3) -- Tears".
It's translated variously:
By Thanissaro Bhikkhu
From an inconstruable beginning comes transmigration.
By Ven. Sujato
Mendicants, transmigration has no known beginning.
By Piya Tan:
this cycle of life and rebirth is without a knowable beginning
The word in question is anamataggoyaṃ about which, I don't know, there's a blog article here: Anamatagga
Translators render it in the following ways:
• ‘The journeying-on as being without beginning and end’
☸ saṃsāraṃ anamataggato (Norman, Thī.v.496).
• ‘This saṃsāra is without discoverable beginning’
☸ anamataggoyaṃ bhikkhave saṃsāro (Bodhi, S.5.441).
Thus anamatagga potentially means:
- without discoverable beginning
We will now show the problem of these terms, and we will show why we follow PED’s ‘whose beginning and end are alike unthinkable,’ and the commentary’s (ad S.2.178) aparicchinnapubbāparakoṭikoti attho (‘first and last point cannot be determined’).
The translation "without a knowable beginning" is close to my (trained in modern maths) definition/understanding of "infinite" though that may be just a coincidence -- i.e. "infinite" meaning something like "unbounded".
Another reference I think of is SN 44.7 and SN 44.8 (as mentioned in Erik's answer) -- the word translated "eternal" there is sasatta:
Master Gotama, is this right: ‘the world is eternal’?”
“kiṃ nu kho, bho gotama, sassato loko”ti?
See for example Introduction to the Avyakata Samyutta for more about that.
Another sutta, which describes something not literally infinite but pretty large/small, is SN 56.48.
There are also definitions or metaphors of what a kalpa (i.e. an "(aeon)"), is, defined somewhere -- I'm not sure how that's important, though.