The first time I paddled in Alaska I was a student with a famous outdoor/leadership school. We were camped on an island in Prince William Sound, and before we could really undertake big paddling days, each student had to perform a wet exit, and take part in an assisted rescue as both a rescuer, a rescuee and a rescuee in a double kayak. The water temperature was around 45º and so wetsuits were provided. We were instructed to roll out of the kayak, wet exit, and yell - paddler in the water. I did as Iw as told but when it came time to yell, my brain commanded my body, but almost no sound came out. As hard as I tried I couldn't make my vocal chords produce more than a grunt or a gasp. It turns out this is not unusual. This occurs because of something called the mammalian diving reflex. The Mammalian diving reflex exists in all aquatic mammals - seals, otters, dolphins and it's effects in these mammals helps them swim, and live longer in cold water as well as to dive to great depths. The reflex exists in humans but acts more as a way to keep us alive in a life threatening situation.
It is triggered automatically by a pair of cranial nerves and immediately causes bradycardia or a slowed heart rate. It also causes peripheral vasoconstriction which means blood vessels in your extremities constrict to keep more warm oxygenated blood flowing to your core and your brain. First your fingers and toes, then your hands and feet, and finally your arms and legs. There is oxygen stored in your muscle mass that makes it possible for you to continue using the effected limbs even after blood flow has been cut off. Interestingly seals have significantly much more oxygen stored in their muscles extending the time that they can go without oxygen.
I suspect that the loss of quality speech when immersed in 48º water is due to the vasoconstriction, but importantly there is a way to short circuit the MDR if you are going to practice rescues in cold water. Simply splashing water on your face will trigger it, but thereby you are in control of it, and not the environment - it's not as jarring when it happens if you trigger it first.
Andrew C also mentioned ice cream headache when he goes paddling, this is caused by cold water - or ice cream, or a slushie - touching the roof of his mouth. This causes your brain to think it is being rapidly cooled, and to prevent that it causes massive vasodilation to warm the brain - causing the headache. So Andrew, close your mouth! I am kidding it can probably occur in really cold water just by submerging your head.
So what can you do to keep the head warm? The best option - particularly for whitewater paddlers as they tend to be submerged more than us sea kayakers is a skull cap. It works well under a helmet and works even when wet.
When I am doing a multiday paddle I pack three different pieces of head gear. A ball cap to keep the sun out of my eyes, and to protect my follicley challenged scalp. A wool hat which will insulate when wet or dry, and dries very quickly, I like this on because it is so thin, and warm. And a rain hat.
Also on that first trip to Alaska I debated spending $50.00 on this rain hat. It looked very good, but it was a lot of money for a hat. So I didn't bring it. Someone else on the trip did bring the very same hat, and it worked really well for them, and I could have really used it. I learned a valuable lesson. Don't let an expensive trip get ruined because you had cheap or inappropriate clothing or gear. If you have spent $4000.00 to go paddling in Alaska what difference is another $50 going to make. But the difference of having a good rain hat - in a state that rains all the time - will be huge. I came back from Alaska and immediately bought that hat, and now it is the first thing that goes in a dry bag when I am headed someplace wet.
Gloves I think are even easier. I use a pair of NRS fingerless gloves (that are no longer made!) that I use in one of two occasions. If the backs of my hands are getting sunburned I put them on. If I am planning a long day - 20+ miles - I put them on. This I do to protect the bit of skin between thumb and index finger. I don't get calluses or blisters paddling long days, and if you are it is caused by one thing. Your holding your paddle too tightly. I also have a pair of thick neoprene gloves that I hate. They are all warm and cushy when dry, but when wet or even damp they are impossible to put on, and if you get the first one on, the second is even harder. I generally bring them to loan to people who have forgotten or lost gloves. I am a big fan of pogies. They protect your hands, from wind and water and cold, yet they give you full contact with the paddle shaft. They also allow you to do things with your bare hands that you couldn't do gloved and then quickly return your hands to a warm dry environment. And if you don't need them you can easily slide them to the center of the paddle where they are out of the way, yet handy if you decide you need them.
Three hats, three pairs of gloves. Simple, relatively easy to pack.
But wait, there is one more thing. I am a big fan of wool socks. They are my favorite choice because they insulate when wet, and dry quickly. Also important they don't get as stinky as synthetics do. I generally figure out how many pairs I want for a trip based on the length of the trip. Then I add a pair to that number. This additional pair goes inside my sleeping bag. AND NEVER COMES OUT! This assures that I get to sleep in warm, dry, clean socks. Which while feels nice serves an important purpose. If your feet are wet all day long and don't get a chance to dry out at night you are at risk for getting trench foot. A painful - and all too frequently trip ending - malady. Sleep warm and dry.