You need at least twice the frequency range for sample rate in order to represent the original signal. That's slightly misleading though, that's from the Nyquist-Shannon sampling theory and it's a mathematical fact but that is true for exact numerical samples, once you add in quantization that muddies the water a bit. Taken at the extreme, it's straightforward to see why a 1 bit quantization per sample at 44.1 kHz would not capture a perfect representation of some analog signal even if there's only a 1 kHz frequency component to the signal. If we instead decide to sample at 10 MHz but still one bit quantization, now that 1 kHz frequency component can be much more accurately represented even though we're still using the worst quantization possible. Don't think of quantization like a square wave or a step pattern, think of it as "the signal is closer to here than any other discrete value".
Now in terms of realistic audio encoding, 16 bit at 44.1 kHz is designed to be a faithful representation as far as human hearing is concerned. Can someone with a trained ear potentially tell the difference between that and 24 bit at 192 kHz? In a studio environment it's possible. Most audiophile claims are dubious and a blind A/B test catches them out on most of it but the Nyquist-Shannon sampling theorem does not directly apply to quantized samples, it's about exact samples and with quantization, sampling rate is intertwined somewhat with the quantization depth.