In terms of the US military, it's because the M4 is enough of a jack-of-all-trades to be acceptably useful in a variety of roles. It cuts down on the need for new or extra training, additional equipment, firearms, ammo, accessories. Subguns are gradually getting nudged aside in favor of shorter, more compact rifles, too. The Navy SEALs have been using the Mk 18 (an M4 with a 10.5-in barrel) for years and in assorted modifications. Other branches and militaries use similar weapons for CQB, entry, and clearing.
In terms of law enforcement, subguns are still around but with the rise of body armor, first SWAT personnel and then regular officers have been moving to AR-based patrol rifles. SWAT teams and special LEOs from local to federal levels may use rifles similar to the Mk 18 in the line of duty for the same reasons as the military.
Shotguns in the military haven't really been combat weapons in a few decades. Mostly they're just used for breaching and entry and by security/MP personnel, you'll see them being used with buckshot as combat weapons but they're a lot more limited in usefulness now we don't fight in trenches and quaint French villages anymore. In LEO circles, they're still fairly popular because it's tighter quarters, it'll stop the threat, and they can be used with non/less-lethal shells.
As for sidearms, most soldiers and marines will be armed with one. It's a defensive weapon intended to be used when your primary is out of ammo, you're in tight and need to stop a threat to your life, or when your primary isn't gonna work (like if you have a big bolt action sniper rifle and you need to clear a building). It's not your number one gun like a beat cop might have on a regular day at work.