The original Vedas themselves have little that a Buddhist would consider "wisdom", I think, being comprised mainly of battle hymns, sacrificial procedures, and ceremony. They are mentioned often in the Pali texts, e.g.:
Since the rest of the Vedic literature, including the Upanishads, grew up around the time of the Buddha himself, it isn't mentioned as such, but the above quote is actually more applicable to the Upanishadic tradition than the original Vedas.
Though I'm not sure this has anything to do with the Upanishads.
Short answer, yes, the Buddha mentioned the Vedas often, but the word "upanishad" was probably not yet in common use during his lifetime. It appears in the name of one of the Brahmanas (the Jaiminiya Upanishad Brahmana) composed before the Buddha, but probably wasn't used to refer to an entire genre of literature until much later.
As to what he thought about them, here's a passage from the Jatakas that exemplifies his thoughts while still a Bodhisatta (and rhymes, too):
"These Veda studies are the wise man's toils,
The lure which tempts the victims whom he spoils;
A mirage formed to catch the careless eye,
But which the prudent passes safely by.
The Vedas have no hidden power to save
The traitor or the coward or the knave;
The fire, though tended well for long years past,
Leaves his base master without hope at last.
Though all earth's trees in one vast heap were piled
To satisfy the fire's insatiate child,
Still would it crave for more, insatiate still,—
How could a Nāga hope that maw to fill?
Milk ever changes,—thus where milk has been
Butter and curds in natural course are seen;
And the same thirst for change pervades the fire,
Once stirred to life it mounts still higher and higher.
Fire bursts not forth in wood that 's dry or new,
Fire needs an effort ere it leaps to view;
If dry fresh timber of itself could burn,
Spontaneous would each forest blaze in turn.
If he wins merit who to feed the flame
Piles wood and straw, the merit is the same
When cooks light fires or blacksmiths at their trade
Or those who burn the corpses of the dead.
But none, however zealously he prays
Or heaps the fuel round to feed the blaze,
Gains any merit by his mummeries,—
The fire for all its crest of smoke soon dies.
Were Fire the honoured being that you think,
Would it thus dwell with refuse and with stink,
Feeding on carrion with a foul delight,
Where men in horror hasten from the sight?
Some worship as a god the crested flame,
Barbarians give to water that high name;
But both alike have wandered from their road:
Neither is worthy to be called a god.
To worship fire, the common drudge of all,
Senseless and blind and deaf to every call,
And then one's self to live a life of sin,—
How could one dream that this a heaven could win?
These Brahmins all a livelihood require,
And so they tell us Brahma worships fire;
Why should the increate who all things planned
Worship himself the creature of his hand?
Doctrines and rules of their own, absurd and vain,
Our sires imagined wealth and power to gain;
"Brahmins he made for study, for command
He made the Khattiyas; Vessas plough the land;
Suddas he servants made to obey the rest;
Thus from the first went forth his high behest ."
We see these rules enforced before our eyes,
None but the Brahmins offer sacrifice,
None but the Khattiya exercises sway,
The Vessas plough, the Suddas must obey.
These greedy liars propagate deceit,
And fools believe the fictions they repeat;
He who has eyes can see the sickening sight;
Why does not Brahma set his creatures right?
If his wide power no limits can restrain,
Why is his hand so rarely spread to bless?
Why are his creatures all condemned to pain?
Why does he not to all give happiness?
Why do fraud, lies, and ignorance prevail?
Why triumphs falsehood,—truth and justice fail?
I count your Brahma one th’ injust among,
Who made a world in which to shelter wrong.
Those men are counted pure who only kill
Frogs, worms, bees, snakes or insects as they will,—
These are your savage customs which I hate,—
Such as Kamboja hordes might emulate.
If he who kills is counted innocent
And if the victim safe to heaven is sent,
Let Brahmins Brahmins kill—so all were well—
And those who listen to the words they tell.
We see no cattle asking to be slain
That they a new and better life may gain,—
Rather they go unwilling to their death
And in vain struggles yield their latest breath.
To veil the post, the victim and the blow
The Brahmins let their choicest rhetoric flow;
"The post shall as a cow of plenty be
Securing all thy heart's desires to thee";
But if the wood thus round the victim spread
Had been as full of treasure as they said,
As full of silver, gold and gems for us,
With heaven's unknown delights as overplus,
They would have offered for themselves alone
And kept the rich reversion as their own.
These cruel cheats, as ignorant as vile,
Weave their long frauds the simple to beguile,
"Offer thy wealth, cut nails and beard and hair,
And thou shalt have thy bosom's fondest prayer."
The offerer, simple to their hearts' content,
Comes with his purse, they gather round him fast,
Like crows around an owl, on mischief bent,
And leave him bankrupt and stripped bare at last,
The solid coin which he erewhile possessed,
Exchanged for promises which none can test.
Like grasping strangers sent by those who reign
The cultivators' earnings to distrain,
These rob where’er they prowl with evil eye,—
No law condemns them, yet they ought to die.
The priests a shoot of Butea must hold
As part o’ the rite sacred from days of old;
Indra's right arm ’tis called; but were it so,
Would Indra triumph o’er his demon foe?
Indra's own arm can give him better aid,
’Twas no vain sham which made hell's hosts afraid.
"Each mountain-range which now some kingdom guards
Was once a heap in ancient altar-yards,
And pious worshippers with patient hands
Piled up the mound at some great lord's commands."
So Brahmins say,—fie on the idle boast,
Mountains are heaved aloft at other cost;
And the brick mound, search as you may, contains
No veins of iron for tile miner's pains.
A holy seer well known in ancient days,
On the seashore was praying, legend says;
There was he drowned and since this fate befell
The ocean's waves have been undrinkable.
Rivers have drowned their learned men at will
By hundreds and have kept their waters still;
Their streams flow on and never taste the worse,
Why should the sea alone incur the curse?
And the salt-streams which run upon the land
Spring from no curse but own the digger's hand.
At first there were no women and no men;
’Twas mind first brought mankind to light,—and then,
Though they all started equal in the race,
Their various failures made them soon change place;
It was no lack of merit in the past,
But present faults which made them first or last.
A clever low-caste lad would use his wit,
And read the hymns nor find his head-piece split;
The Brahmins made the Vedas to their cost
When others gained the knowledge which they lost.
Thus sentences are made and learned by rote
In metric forms not easily forgot,—
The obscurity but tempts the foolish mind,
They swallow all they're told with impulse blind.
Brahmins are not like violent beasts of prey,
No tigers, lions of the woods are they;
They are to cows and oxen near akin,
Differing outside they are as dull within.
If the victorious king would cease to fight
And live in peace with his friends and follow right,
Conquering those passions which his bosom rend,
What happy lives would all his subjects spend!
The Brahmin's Veda, Khattiya's policy,
Both arbitrary and delusive be,
They blindly grope their way along a road
By some huge inundation overflowed.
In Brahmin's Veda, Khattiya's policy,
One secret meaning we alike can see;
For after all, loss, gain and glory, and shame
Touch the four castes alike, to all the same.
As householders to gain a livelihood
Count all pursuits legitimate and good,
So Brahmins now in our degenerate day
Will gain a livelihood in any way.
The householder is led by love of gain,
Blindly he follows, dragged in pleasure's train,
Trying all trades, deceitful and a fool,
Fallen alas! how far from wisdom's rule."
-- Jat 543 (Cowell, trans)
As a Buddha, he seems to have become a bit more diplomatic, but still has little good to say about worship of fire, brahma, or the Vedas in general.