I suppose it depends how much the screen door effect bothers you. It will definitely be a noticeable difference.
If you can play with a Google cardboard or similar, try it with different phones with different screen resolutions. It'll be the same as that visually.
Of course it will also depend on how much the vr really matters to you. I personally would definitely get the XL, as the daydream is one of the features I'm most excited about. If it's just something you might play with but isn't a huge draw for you, then definitely just pick the phone that's the most comfortable to use.
