Search for Device Manager in the Search box and open it. Click on the arrow to expand Audio Inputs and Outputs. Right-click on the audio device you want to uninstall and select ‘Uninstall device’.

If you ever find yourself swapping between a mix of audio inputs and outputs and get tired of plugging cables all the time, check out [winslomb]’s audio multiplexer with integrated amplifier.