A fleece jacket or pullover has been the essential hiking layer for decades. Fleece is robust, keeps you warm even when you get wet, and is generally inexpensive compared to down and other synthetics.