Ultra-high-density single-atom catalysts (UHD-SACs) present unique opportunities for harnessing cooperative effects between neighboring metal centers. However, the lack of tools to establish correlations between the density, type, and arrangement of the isolated metal atoms with the support surface properties hinders efforts to engineer advanced material architectures. Here, we precisely describe the metal center organization in various mono- and multimetallic UHD‑SACs based on nitrogen-doped carbon (NC) supports by coupling transmission electron microscopy with tailored machine-learning methods (released as a user-friendly web app) and density functional theory simulations. Our approach quantifies the non-negligible presence of multimers with increasing atom density, characterizes the size and shape of these low‑nuclearity clusters, and identifies surface atom density criteria to ensure isolation. Further, it provides previously inaccessible experimental insights into coordination site arrangements in the NC host, uncovering a repulsive interaction that influences the disordered distribution of metal centers in UHD-SACs. This observation holds in multimetallic systems, where chemically-specific analysis quantifies the degree of intermixing. These fundamental insights into the materials chemistry of single-atom catalysts are crucial for designing catalytic systems with superior reactivity.