Resistance across the countryside by the Polish Home Army and Communist partisans was widespread all throughout the Nazi occupation - getting weapons from other people who would likewise love to see some dead Nazis was comparatively easy. Comparatively being the operative word - the uprisings were generally desperate for more weapons, but were not wholly unarmed. In addition, there was the normal routes of black market smuggling, old weapon caches, personal firearms hidden away, and, of course, killing, looting, or stealing from the Nazis themselves.
The largest uprising was done in the Warsaw Ghetto - the Jews there didn't need to 'hide', as the Nazis knew they were there - they only needed to time their uprising at some point before the Nazis tried to ship them off to the nearest death camp. In other places, they hid out in the countryside, in the woods or villages, either amongst friendly partisans or on their own.