Is that a problem? For example,. Dust and other stuff getting in and all that, or it's fine because the normal usb port is always open too so side one being open always doesn't really matter.
I leave mine off all the time. Just make sure you avoid liquids and look inside it before plugging stuff in.
If you are afraid of dust building up on the inside of the port, just get a tiny make-up brush or a small paint brush with soft bristles to brush the dust out once in a while. Just to make sure your contact points are clean from any dirt.
If you lose the cover and you want to get a replacement