In general, no one can possess handguns in NJ without an NJ permit to carry. No one can possess long guns in NJ without a Firearms ID. Exemptions to those two exist for possession in your residence, at a range, transporting between those places... among others.
Now - the place you're renting - can that be considered your residence? There are very specific legal issues around that question - it's not as simple as, "well, yeah... I'm staying there, I'm paying rent... it's my residence". Well, you "rent" a motel room - you stay there - but it's never considered your residence. I will guarantee you any prosecutor looking to convict you of 2nd degree unlawful possession of a handgun (or 3rd degree unlawful possession of a rifle or shotgun), and put you away in state prison for 7 years on the handgun or 3 years on the rifle, will argue (most likely successfully) that since you still own/rent property in another state, have your D/L from that state, auto insurance, etc. - that you had no intention of changing your state of residence, and therefore were just "visiting" - meaning no 2C:39-6 exemption status for you.
So - the answer to your question of "what can I bring" is best answered, "whatever you want, if you want to risk up to 7 years of your life".
But I'm the paranoid one around here - so make of that what you will.