Foundation insulation panels are one good-looking option ... They attach to concrete, stone or block walls with rigid foam ...