We examine the performance of a variety of different estimators for the mass of galaxy groups, based on their galaxy distribution alone. We draw galaxies from the Sloan Digital Sky Survey for a set of groups and clusters for which hydrostatic mass estimates based on high-quality Chandra X-ray data are available. These are used to calibrate the galaxy-based mass proxies, and to test their performance. Richness, luminosity, galaxy overdensity, rms radius and dynamical mass proxies are all explored. These different mass indicators all have their merits, and we argue that using them in combination can provide protection against being misled by the effects of dynamical disturbance or variations in star formation efficiency. Using them in this way leads us to infer the presence of significant non-statistical scatter in the X-ray based mass estimates we employ. We apply a similar analysis to a set of mock groups derived from applying a semi-analytic galaxy formation code to the Millennium dark matter simulation. The relations between halo mass and the mass proxies differ significantly in some cases from those seen in the observational groups, and we discuss possible reasons for this.