I have multiple users using the Popsical. Do all of them have to subscribed to Premium to enjoy unlimited play time?

Nope, as long as the Popsical device is activated/logged in using a Premium account, other users can still pair their mobile devices and still enjoy unlimited play time regardless their account on mobile device.