If you struggle to get the citta into samadhi, it is because you struggle to calm the shankaras and this happens because you struggle to track (what happens) in the body, in the mano and in the citta, and this happens because you fail to calm the body, the mano and the citta.
It is very normal to be upset by the little distractions. The most upsetting distraction are sounds, like the motor vehicles, or some frictions of plastic bags.
In any case, the better you are at calming the body, the mano and the citta, the easier it gets to be good at satisampajanna, which is tracking
vedana, sanna and the thoughts
And what is the development of concentration that, when developed & pursued, leads to mindfulness & alertness? There is the case where feelings are known to the monk as they arise, known as they persist, known as they subside. Perceptions are known to him as they arise, known as they persist, known as they subside. Thoughts are known to him as they arise, known as they persist, known as they subside. This is the development of concentration that, when developed & pursued, leads to mindfulness & alertness.
or more generally, non-conceptualizing the kaya, the mano and the citta, the earth element and so on.
But, the better you are at tracking what happens with the kaya, the mano and the citta, the easier it gets to calm them, by memorizing the claim of the buddha that calming the citta is good and feels good and an agitated citta is bad.
The monk Ajahn Brahm is famous and he loves to say that you get good at samatha, meaning calming the kaya, the mano and the citta, by using wisdom power, instead of will power.
Indeed, the little distractions above, like the sounds, are distracting because you force the concentration on whatever object recognized.
As soon as you are upset and tensed in the kaya during meditation, it is because you grasp or hook the objects, instead of ''letting go'' like people say, which means instead of being ''kind'' and not being upset to what happens in the kaya, mano or citta (like Ajahn Brahm says). The use of will power for mediation is what drains your energy.
Meditation does not drain your energy, on the contrary, calming the kaya, mano and citta gives you more energy. The buddha calls that ''the tireless energy''
- “I thought: ‘Suppose, with my teeth clenched and my tongue pressed against the roof of my mouth, I beat down, constrain, and crush mind with mind.’ So, with my teeth clenched and my tongue pressed against the roof of my mouth, I beat down, constrained, and crushed mind with mind. While I did so, sweat ran from my armpits. Just as a strong man might seize a weaker man by the head or shoulders and beat him down, constrain him, and crush him, so too, with my teeth clenched and my tongue pressed against the roof of my mouth, I beat down, constrained, and crushed mind with mind, and sweat ran from my armpits. But although tireless energy was aroused in me and unremitting mindfulness was established, my body was overwrought  and uncalm because I was exhausted by the painful striving. But such painful feeling that arose in me did not invade my mind and remain.
so watch the video of retreats here, especially the question and answer
Also, the easiest way to calm the citta is to give stuff to people who are good at calming the citta, the mano and the kaya, or at least they strive for that.
When you give stuff to these people, strive to know the citta and the kaya and mano and watch how calm you get when the stuff is given to those good people, so that you will see ''for yourself'' the calming of the citta
“Having given this, not seeking his own profit, not with a mind attached [to the reward], not seeking to store up for himself, nor [with the thought], ‘I’ll enjoy this after death,’
“—nor with the thought, ‘Giving is good,’
“—nor with the thought, ‘This was given in the past, done in the past, by my father & grandfather. It would not be right for me to let this old family custom be discontinued,’
“—nor with the thought, ‘I am well-off. These are not well-off. It would not be right for me, being well-off, not to give a gift to those who are not well-off,’ nor with the thought, ‘Just as there were the great sacrifices of the sages of the past—Atthaka, Vamaka, Vamadeva, Vessamitta, Yamataggi, Angirasa, Bharadvaja, Vasettha, Kassapa, & Bhagu—in the same way this will be my distribution of gifts,’
“—nor with the thought, ‘When this gift of mine is given, it makes the mind serene. Gratification & joy arise,’
“—but with the thought, ‘This is an ornament for the mind, a support for the mind’—on the break-up of the body, after death, he reappears in the company of Brahma’s Retinue. Then, having exhausted that action, that power, that status, that sovereignty, he is a non-returner. He does not come back to this world.
"And how is a donation endowed with six factors? There is the case
where there are the three factors of the donor, the three factors of
"And which are the three factors of the donor? There is the case where
the donor, before giving, is glad; while giving, his/her mind is
bright & clear; and after giving is gratified. These are the three
factors of the donor.
"And which are the three factors of the recipients? There is the case
where the recipients are free of passion or are practicing for the
subduing of passion; free of aversion or practicing for the subduing
of aversion; and free of delusion or practicing for the subduing of
delusion. These are the three factors of the recipients.
"Such are the three factors of the donor, the three factors of the
recipients. And this is how a donation is endowed with six factors.