This is the second of a series of papers studying a sample of radio-excess IRAS galaxies. These galaxies have radio emission in excess of that expected due to star formation, but largely fall between the traditional categories of radio-loud and radio-quiet active galaxies. R-band images of the hosts of far-infrared (FIR)-luminous radio-excess galaxies are presented and analyzed. The hosts of the FIR-luminous radio-excess galaxies are luminous galaxies, on average 0.8 mag brighter than M*R. Their optical luminosities and morphologies are similar to comparison samples of radio-loud compact steep-spectrum and gigahertz peaked-spectrum sources and extended radio galaxies. We find a similar fraction of galaxies in our sample (∼70%) with companions or distorted morphologies as in radio-loud comparison samples. This is consistent with radio activity being associated with tidal interaction. The majority (65%) of the FIR-luminous radio-excess galaxies have radio source sizes that are smaller than the optical host by more than an order of magnitude. These compact radio sources may be young precursors to classical radio galaxies or a different population of radio sources, possibly confined by the host interstellar medium. The host galaxy types were determined by analysis of the surface brightness distributions. The elliptical hosts have effective surface brightnesses and radii consistent with known ellipticals but inconsistent with a population of brightest cluster galaxies. Thus, it is unlikely these objects are the precursors of FR I radio galaxies. The disk hosts have smaller sizes and low radio excesses. However, they have a range of radio source sizes, which is not expected if they are radio-``loud'' Seyfert galaxies.