Personally I find buffs very uncomfortable and the same with long sleeved clingy fishing shirts. I only use those when it is rough and very sunny to keep the salt spray off - evaporating salt spray on my skin is even more uncomfortable than actually wearing a buff!
So yes, I tend to use a waterproof sports type sun block, 30+, on face, neck, arms and legs.