The Buddha reportedly ate a meal of pork at least once in his life. So one pig had to lose its life. The Buddha ate rice pudding so some cow had to eat the grass to produce milk and some weeds must have lost their life to have grown the rice.
It seems that no matter what we do in life some harm is done. Maybe do less harm would be more sensible. The Buddha advocated the middle way between hedonism and asceticism. The Tibetan Buddhists eat meat. So what are we to do?
Do no harm against life for selfish ends. Do no harm against our neighbor to gain advantage. Dedicate your work to all beings.
In your case you could donate some of the crop to food pantries servicing the homeless or needy.
In another sense there is only consciousness. When we weed the garden in meditation we make room for a new consciousness. When we plant seeds in the garden or consciousness, we are planting the divine seed of awakening.
Being paralyzed by fear of doing harm is not the middle way as I see it.
Perhaps the saying should be Clear your intentions of doing harm. Have an intention of giving life.
http://en.wikipedia.org/wiki/Noble_Eightfold_Path
Right intention[edit] Right intention (samyak-saṃkalpa / sammā
sankappa) can also be known as "right thought", "right resolve",
"right conception", "right aspiration" or "the exertion of our own
will to change". In this factor, the practitioner should constantly
aspire to rid themselves of whatever qualities they know to be wrong
and immoral. Correct understanding of right view will help the
practitioner to discern the differences between right intention and
wrong intention. In the Chinese and Pali Canon, it is explained
thus:[19][19][21][31][32]
And what is right resolve? Being resolved on renunciation, on freedom
from ill will, on harmlessness: This is called right resolve.
It means the renunciation of the worldly things and an accordant
greater commitment to the spiritual path; good will; and a commitment
to non-violence, or harmlessness, towards other living beings.