Perhaps it's good to discuss your visit with someone at the temple in advance.
Some temples have a once-a-year event at which infants are presented to the Buddha for the first time; for example:
Hatsumairi (Infant Presentation) Service
Hatsumairi is a ceremony in which ones child is formally presented to the Buddha and to the Sangha for the first time. Contact the Temple Office if you wish your child to participate in the Hatsumairi service. This service is usually held in conjunction with the Buddha Day (Hanamatsuri) service.
Or for example,
Shosanshiki - Infant Presentation
Do you know of any families with a new baby or a new addition to the family in the last year?
Let us know, so they can register to participate in “Shosanshiki” or a Baby presentation to the Buddha. We welcome the new babies to the temple in a special service on Sunday, May 15, 2016 at 11am. If your toddler has never filed their name and missed a previous presentation, it’s never too late. The ceremony is open to any infants and children under three years of age.
If you are interested please contact the temple office at (telephone number).
Or for example this temple has a baby-blessing ceremony three times per year.
This looks relevent to do with Sri Lanka:
Buddhist Ceremonies and Rituals of Sri Lanka (1996)
While this is the method of formal admission of a new entrant into Buddhism, there are also certain ritualistic practices observed when a child is born to Buddhist parents. The baby's first outing would be to a temple. When the baby is fit to be taken out of doors the parents would select an auspicious day or a full-moon day and take the child to the nearest temple. They would first place the child on the floor of the shrine room or in front of a statue of the Buddha for the purpose of obtaining the blessings of the Triple Gem. This is a common sight at the Dalada Maligawa — the Temple of the Sacred Tooth Relic — in Kandy. At the time of the daily religious ceremony (puja) of the temple, one can observe how mothers hand over their babies to an officiating layman (kapuva) inside the shrine room, who in turn keeps it for a few seconds on the floor near the Relic Chamber and hands it back to the mother. The mother accepts the child and gives a small fee to the kapuva for the service rendered. This practice too could be described as a ritual of initiation.
I don't know what "when the baby is fit to be taken out of doors" means.
The above quote claims that, it's "a common sight at the the Temple of the Sacred Tooth Relic" (especially on an auspicious or full-moon day).
Wikipedia's Childbirth in Sri Lanka says,
Preparation for birth
During a first pregnancy, couples often visit temples of special significance in the Buddhist faith. Examples of such temples are Temple of the Tooth in Kandy or the Sacred Bodhi tree at Anuradhapura. At the temple they pray for the pregnancy and ask for an uncomplicated birth and healthy life. They make vows to ensure their wishes are granted. If their wishes are granted, they return to the temple or shrine to make offerings of praise to the Gods who protected their newborn child and mother during pregnancy and birth.
... and ...
Newborn Rites of Passage
The exact time of birth is reported to an astrologer who uses the child’s horoscope to determine the best letters for the child’s name. Nam tebima is the Sinhalese naming ceremony.[39] The baby’s first outing is typically to a Buddhist temple on a full moon day to receive blessings for a prosperous life. Idul kata gema is the ceremony to celebrate a baby’s first solid food. The ceremony, which takes place either in the family home or in a temple, involves placing dishes of rice boiled with milk, traditional sweets, a banana, a book and a piece of jewelry.
An article like Sinhalese birth rituals of yore isn't clear about when the baby goes outside for the first time, but maybe it's "in the fourth month after birth":
The dorata vedima where the child is taken out of the house into the open and exposed to the rising morning sun is another important ritual in traditional Sinhalese society. The practice is evidently the same as the hiru vadana magula or 'Ceremony of the increasing Sun' mentioned in the Saddharma Ratnavaliya which consisted of exposing the child to the sun a few days after birth. The practice bears a striking resemblance to the ancient Hindu custom known as Aditya Darshana or the ceremony of taking the child out to see the sun which took place in the fourth month after birth.
Culture Shock in Sri Lanka suggests this kind of custom is still practiced:
The rituals continue on with a child's first trip outdoors. There is a practice called Dorata where a child is first exposed to the rising morning sun, for example.
