Objectives: To investigate determinants of the physician global assessment of disease activity (PhGA) and the influence of the contextual factors on this relationship in patients with early axial spondyloarthritis (axSpA). Method: Five-year data of DESIR, a cohort of early axSpA, were analyzed. Univariable generalized estimating equations (GEE) were used to investigate contributory explanatory effects of various potential determinants of PhGA. Effect modification by contextual factors (age, gender and educational level) was tested and, if significant, models were stratified. Autoregressive GEE models (i.e., models adjusted for PhGA at the previous time point) were used to confirm a longitudinal relationship. Results: A total of 708 patients were included. Higher Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) individual questions, swollen joint count in 28 joints (SJC28), tender joint count in 53 joints, Maastricht Ankylosing Spondylitis Enthesitis Score, C-reactive protein (CRP) and Bath Ankylosing Spondylitis Metrology Index were associated with a higher PhGA. Gender and age were effect modifiers of SJC28; the contributory effect of SJC28 was largest in the younger male stratum (β [95% CI]; 1.07 [0.71-1.43]), and the smallest in the older female stratum (0.13 [0.04-0.22]). Autoregressive GEE models revealed the same determinants as having a longitudinal association with PhGA and the same pattern of effect modification. Conclusions: Patient's subjective symptoms, peripheral arthritis and enthesitis, higher CRP and impaired spinal mobility contribute to explaining PhGA in patients with early axSpA irrespective of gender and age. Intriguingly, physicians consider the presence of swollen joints as more important in males than in females.