Suppressors can compound the effect of barrel heating because they will heat at a different rate than the actual barrel. As you noted three shots will tell you how the rifle is actually doing for any hunting scenario.I have just started with the .300 with a 22” barrel. I am using my Thunderbeast Ultra 5 suppressor with it and I’m hunting the EC with it this month. I had all kinds of trouble and bought like 4 or more different loads. I was frustrated. What I finally figured out is that it gets hot and loses accuracy. Just like you I had poor groups and the occasional flyer. Frustrating!
Then I finally planned to just shoot a three shot group and then go back to work (yes I have two shooting ranges at my ranch so I can shoot anytime). I am wanting to hunt with Barnes 180g but also bought and had good accuracy with Nosler Accubond 180g.
With a cool barrel I am just about putting them in the same hole. 1” group at 100 yards easily. Problem solved! Thank goodness as I leave the 24th.
Best of luck.
Philip