yep you plug your external audio interface via usb to your computer and then the audio go through the usb. The interface drivers will give you less latency than the motherboard sound card.
Then the audio outputs interface via jack cables to jack or jack to rca to your input audio speakers.
You will get a very good sound to listen to your music and of course better results with DAW or Sonic Pi software. You, as you say, will be able to plug microphones or instruments (guitar, bass) or external synth (as Digitone from Elektron, etc.). The price will be higher if you need interface midi ( 5 din midi ouput.)
So towards your budget and your needs, there will be an audio interface for you. ah think about checking the given software, it can be a good point to start with few dollars, euros. as example https://www.steinberg.net/en/products/audio_interfaces/ur_series/start.html comes with Cubase and Focusrite will bring you Ableton live lite, protools lite. Enough to start and discover the MAO