My partner and I rescued Ramón, our first cat ever, from outside a convenience store near our house. He was already an adult by the time we met him. We would always see him outside running from stray dogs (I'm from Mexico, specifically from an area with a lot of stray dogs and cat) and, generally, on alert.
Now, even if we leave our doors open he prefers to stay inside the house with his little brother Vicente, another cat we adopted. We regularly make new toys and play with them.
Vicente has been with us since he was around 1 month old (now 6 months old) so he's way more curious about the outside. We are preparing to start walking them out though I have a feeling we will have to drag Ramón out of the house.
I wouldn't feel bad for confining your cats to your house! They are probably very happy :)