As you see on picture I played to day with transparency for playing video.
We can play video in OverlayMBS (without title bar) and in normal Xojo window. Both with applying the mask and keeping window transparent and background shines through.
The screenshot is with using Xojo window where we let you see the title bar. The version with overlay looks the same except that there is no titlebar.
PS: Example project included with MBS Plugin in AVFoundation examples folder.