Jem's theory of Boo Radley

Jem Mentioned that Boo Radley had never left his house because he wanted to stay in there. I think that there is a flaw in this theory. The thing is that nobody likes to be locked up in a house 24/7. Naturally, they would go crazy. Besides, for all we know, they are not very nice at him in there. This could definitely be a possibility but not very likely from my point of view.