I'm inclined to agree with Erik.
Here's Ven Yuttadhammo, who is Canadian and in Canada -- Ask A Monk: Avoiding Cold -- saying that you might wear a hat "as a matter of course".
I think it's a matter of learning (or being taught) what the body can tolerate -- physical problems include (depending on how cold it is), hypothermia, chilblains, eventually frostbite.
I used to bicycle to and from work, just over an hour each way, in Toronto, year-round. And they were some of my favourite hours of every day, but a slightly extreme/unusual practice in winter and doing it (subzero and wind and sometimes wet for an hour twice a day every day) did depend on wearing winter gloves and boots.
Can I say, "don't be stupid"?
Every Canadian must learn to live with cold (and to dress appropriately). Winter there even in the south might have a daytime high of -10°C for months on end -- but life goes on, for everyone, you go out to go to school or work or exercise.
One danger is that stupidity (or "confusion") is caused by hypothermia -- i.e. if you do become hypothermic you might be too stupid to realise that and do something about it. Once I spent a week mountain-hiking with a school group in spring, and then we were told to watch out for our "buddies" -- we were told that if they stop being able to talk sense, that if they get angry, then perhaps they're hypothermic and we should act on that.
Another thing to know, I guess, is that "feeling cold" is a kind of a good thing -- where it becomes dangerous is if you lose sensation in a body-part -- usually extremities like the ears or toes.
Ditto if you feel overheated when you're not (not that I have experienced that extreme personally).
If you're interested the first place I feel cold is my ribs -- on either side, left and right, and level with my heart. There's no fat there, under the skin. A typical meditation posture might have your elbows down and upper arms against your sides protecting that area.
I reckon you ought to regulate your core temperature -- for health reasons, that's only sensible and a "middle way" practice -- and that skin temperature (and/or skin being wet) doesn't matter as long as you avoid chilblains and frostbite.