I would seriously any human shooter could tell a difference in accuracy due to material of the barrel. A pencil barrel is not intended to be a tack driver anyway. Where the SS barrels excel are due to ability to smoothly machine a bull target barrel, and for regular contour barrels, their use in high humidity or marine environments, a rifle stored on your boat, trunk of a LEO's car, coastal areas, or a firearm put away for long term storage.'
I think most would agree a chrome-lined barrel will take more abusive use. Many credit the chrome in the chamber for being more slippery and allowing easier extraction with some ammo. (My stainless and chrome-moly barrels have not had any extraction problems, so I don't see what having chrome could improve on.)
It is certainly easier to find chrome-lined barrels in the pencil contour than stainless. If your goal is to build a lightweight corrosion resistant rifle, you might consider buying a chromelined barrel, and having the entire rifle Robar'd or Black T'd.
On the other hand, if you are hoping to gain heavy barrel target accuracy from a pencil barrel by going stainless, I think you will be disappointed. But I also think for practical shooting, a pencil contour barrel will do fine if you are not looking for bench rest accuracy.