Sulphamate and sulphamide derivatives have been largely investigated as carbonic anhydrase inhibitors (CAIs) by means of different experimental techniques. However, the structural determinants responsible for their different binding mode to the enzyme active site were not clearly defined so far. In this paper, we report the X-ray crystal structure of hCA II in complex with a sulphamate inhibitor incorporating a nitroimidazole moiety. The comparison with the structure of hCA II in complex with its sulphamide analogue revealed that the two inhibitors adopt a completely different binding mode within the hCA II active site. Starting from these results, we performed a theoretical study on sulphamate and sulphamide derivatives, demonstrating that electrostatic interactions with residues within the enzyme active site play a key role in determining their binding conformation. These findings open new perspectives in the design of effective CAIs using the sulphamate and sulphamide zinc binding groups as lead compounds.