The Buddha himself did not cease having thoughts, although he was fully enlightened. Please see this question for details.
In this answer, there is a quote from the Ayacana Sutta, where it says:
Then, while he (the Buddha) was alone and in seclusion, this line of
thinking arose in his awareness: "This Dhamma that I have attained
is deep, hard to see, hard to realize, peaceful, refined, beyond the
scope of conjecture, subtle, to-be-experienced by the wise. ....."
In the Vitakkasanthana Sutta, we find that unskillful thoughts "imbued with desire, aversion, or delusion" have to be removed.
In the Vina Sutta:
"Monks, in whatever monk or nun there arises desire, passion,
aversion, delusion, or mental resistance with regard to forms
cognizable via the eye, he/she should hold the mind in check.
[Thinking,] 'It's dangerous & dubious, that path, thorny & overgrown,
a miserable path, a devious path, impenetrable. It's a path followed
by people of no integrity, not a path followed by people of integrity.
It's not worthy of you,' he/she should hold the mind in check with
regard to forms cognizable via the eye.
And the same applies to the other senses.
You can find a similar discussion in the Aparihani Sutta where it talks about removing "evil, unskillful qualities such as greed or distress".
There is a slightly different message in the Sacitta Sutta, but also teaching us to remove unskillful thoughts:
"If, on examination, a monk knows, 'I usually remain covetous, with
thoughts of ill will, overcome by sloth & drowsiness, restless,
uncertain, angry, with soiled thoughts, with my body aroused, lazy, or
unconcentrated,' then he should put forth extra desire, effort,
diligence, endeavor, relentlessness, mindfulness, & alertness for the
abandoning of those very same evil, unskillful qualities.
Hence the recommendation here is to remove unskillful thoughts, but not stop thinking altogether.
Of course, if one is meditating and trying to achieve the state of jhana, then in that situation, he has to concentrate his mind and cease thinking. Even this is only for the second jhana and beyond. In the first jhana, there is still directed thought and evaluation. From the Maha-satipatthana Sutta:
"And what is right concentration? There is the case where a monk —
quite withdrawn from sensuality, withdrawn from unskillful (mental) qualities — enters & remains in the first jhana: rapture & pleasure
born from withdrawal, accompanied by directed thought & evaluation.
With the stilling of directed thoughts & evaluations, he enters & remains in the second jhana: rapture & pleasure born of composure,
unification of awareness free from directed thought & evaluation —
So, it is inaccurate to say that the Buddha taught to stop thinking altogether (when you're not meditating). He only taught to remove unskillful thoughts.
Constantly ensuring that you don't have unskillful thoughts, is considered a practice of heedfulness (or appamada). Please see this question for details.