KDE Plasma has a USB connected and disconnected sound, I’m sure may other mainstream DEs do too.

On Windows, if you wanted to confirm the sound was from your computer, device manager usually flickers every time a new device is plugged in. New devices should also be logged in the Event Logs.