Fully homomorphic encryption (FHE) promises data protection by computation on encrypted data, but demands resource-intensive computation. The most fundamental resource of FHE is modular multiplier, which needs to be evaluated for efficient implementation. In this work, we develop and evaluate ASIC implementations of the modular multiplier at the block-level and at the system-level. We study the efficiency of the multipliers in terms of performance-for-area and performance-for-power. Since these ASICs are used in FHE, we explore these multipliers within this system-level context with on-chip memory and interconnect limits. We explore ASIC implementations of modular multiplications using a state-of-the-art 22nm technology node with constant operand throughput to ensure a fair comparison. The study yields key insights about the performance-for-area efficiency and power efficiency of bit-serial and bit-parallel designs: Bit-parallel designs are more efficient than their bit-serial counterparts. Montgomery multipliers with constrained modulus are the most power-efficient and area-efficient design. Iterative Montgomery multipliers incur minimum peak power for a polynomial multiplication, making them suitable for low-power designs.