Hello, @CallumLFC29,@joshsmithofficial &@Lewisstory
We're sorry again you are having issues, unfortunately there isn't anything we can do on our end, in our internal testing we were able to have 32 Xbox users in game.
Unfortunately as this issue is only affecting certain users this is going to have to be a case of troubleshooting.
First can you please:
Press the Xbox button
Select Profile & system > Settings > General > Network settings.
Under Current Network Status, the NAT Type field displays your current NAT type.
Make sure your NAT type is saying Open
Next, can you double check the steps I mentioned above, then try having different users to host the lobby, getting everyone to use an ethernet connection and making sure your router has up to date firmware.