I'm thinking about connecting the two bigger bearings with screws, but unlike the lazy suzan bearings I have to drill holes for that.

I had no success with a cobalt drill, but tried a resharpened masonary drill and it worked better then expected.

Ordered a 3mm masonary and will grind it down to 2mm to maybe cut a hole in the outer race from top to bottom without entering the ball groove.