Accurate and comprehensive peptide spectrum annotation is a crucial step to interpreting mass spectrometry-based proteomics data. While peak assignment in peptide fragmentation spectra is central to a broad range of proteomics applications, current tools tend to be specialized to a specific task. Here, we present a more comprehensive interactive graphical tool (Annotator), along with the underlying codebase written in Rust (rustyms). Annotator enables unified spectrum annotation for bottom-up, middle-down, top-down, cross-linked, and glycopeptide fragmentation mass spectra from all fragmentation methods, including all ion types: a/b/c, x/y/z, d/v/w, and immonium ions. The Annotator integrates all known post-translational modifications from common databases and additionally allows for the definition of custom fragmentation models and modifications. Modifications allow for diagnostic fragment ions, site-specific neutral losses, and multiple breakage sites for cross-linkers. The underlying library used for the theoretical fragmentation and matching is based on the unified peptidoform notation ProForma 2.0 and is made available as a Rust library with Python bindings. This enables spectrum annotation in an interactive, graphical interface of diverse and complex peptidoforms across the broad range of mass spectrometry-based proteomics applications.