Antibody molecules are able to recognize any antigen with high affinity and specificity. To get insight into the molecular diversity at the source of this functional diversity, we compiled and analyzed a non-redundant aligned collection of 227 structures of antibody-antigen complexes. Free energy of binding of all the residue side-chains was quantified by computational alanine scanning, allowing the first large-scale quantitative description of antibody paratopes. This demonstrated that as few as 8 residues among 30 key positions are sufficient to explain 80% of the binding free energy in most complexes. At these positions, the residue distribution is not only different from that of other surface residues, but also dependent on the role played by the side chain in the interaction, residues participating in the binding energy being mainly aromatic residues, and Gly or Ser otherwise. To question the generality of these binding characteristics, we isolated an antibody fragment by phage-display using a biased synthetic repertoire with only two diversified complementary determining regions (CDRs) and solved its structure in complex with its antigen. Despite this restricted diversity, the structure demonstrated that all CDRs were involved in the interaction with the antigen and that the rules derived from the natural antibody repertoire apply to this synthetic binder, thus demonstrating the robustness and universality of our results.