The Pali suttas have the following stock phrase about the best time being when you become dispassionate in continuing the lay life:
Now, there is the case where a Tathāgata appears in the world, worthy and rightly self-awakened. He teaches the Dhamma admirable in its
beginning, admirable in its middle, admirable in its end. He proclaims
the holy life both in its particulars and in its essence, entirely
perfect, surpassingly pure.
He [the person discussed above], hearing the Dhamma, gains conviction in the Tathāgata and reflects: 'Household life is confining, a dusty
path. Life gone forth is the open air. It isn't easy, living at
home, to practice the holy life totally perfect, totally pure, a
polished shell. What if I, having shaved off my hair & beard and
putting on the ochre robe, were to go forth from the household life
into homelessness?'
So after some time he abandons his mass of wealth, large or small; leaves his circle of relatives, large or small; shaves off his hair
and beard, puts on the ochre robes, and goes forth from the household
life into homelessness.
MN 38
Some men still delight in sex but become monks because they think jhana will give them a better type of orgasmic pleasure. It is doubtful these men can succeed.