Microorganism-mediated nitrogen (N) cycling and dynamics are important functions of ecosystems, but such information is limited for high-elevation rivers. This study analyzed the abundance and community structure of anammox and denitrifying bacteria and quantified N loss induced by these bacteria in five high-elevation river sediments (including the Yellow River source region) on the Qinghai-Tibet Plateau using high-throughput sequencing and 15N-labeling techniques. Anammox bacterial diversity was limited in these high-elevation rivers, with Brocadia-like species averagely accounting for 97.0% of anammox community composition. Several denitrifier nirS phylotypes were distributed across these high-elevation rivers with high specificity, and the proportion of each of these phylotypes increased with elevation (p < 0.05). Stochastic processes such as dispersal limitation in denitrifying and anammox bacterial community assemblies were more important than environmental selection. Anammox and denitrification rates in the Yellow River source region were at the low end of their reported ranges in the literature on low-elevation rivers. The high contribution of anammox to nitrogen loss (27% on average) in the Yellow River source region was likely assisted by DNRA (dissimilatory nitrate reduction to ammonium) and nitrification through supplying ammonium and nitrite, respectively, as substrate for the anammox reaction. The contribution of anammox to N2 production was positively related to temperature (p < 0.05). Denitrification rates were more significantly affected by denitrifier community structure than environmental factors. This study suggests that anammox contribution to N loss should not be neglected in high-elevation rivers, and community diversity should be considered when studying denitrification in high-elevation rivers.