That glow I think you're referring to is "theater mode" and fortunately it can be disabled on the gear menu. It's incredibly distracting, especially in Firefox it can sometimes be laggy.
I think you mean "ambient mode". Theater mode is like the halfway point between normal-size an fullscreen-size.
As an aside, does anyone know of any good YouTube settings extensions? I usually open YouTube in incognito tabs to avoid the personalization "features", but this means none of my preferences around ambient mode and "stable volume" are saved.