In this paper we propose that the degree of optimization achievable in High Level Synthesis (HLS) with module or function-unit (FU) selection is significantly dependent on how the mix of FUs in the HLS library are parameterized. In particular, for the problem of power minimization under a latency constraint, our proposal is that a larger power optimization is possible when: 1) the FUs for each functional type have a wide range of power and delay metrics, and 2) their pair-wise power-delay product ratios are close to 1, say, in the range [0.8, 1.25], than when these criteria are not satisfied. We also briefly show that it is possible to achieve these parameter ranges for arithmetic functions, which form the bulk of FUs in HLS problems, due to: a) the variety of designs that have been proposed for such functions; b) the possibility of combining different designs in a hierarchical manner to yield hybrid designs that satisfy the aforementioned parameter constraints in case the original designs do not. We provide a probabilistic rationale for our aforementioned hypothesis, and further bolster it empirically by constructing different FU libraries that either meet or do not meet the above FU parameter criteria. Using a new state-of-the-art power-driven simulated annealing based algorithm PSA for scheduling, binding and module or FU selection, we consistently find that the power consumption of designs using libraries that meet our criteria are significantly lower than those that do not, including when the former have fewer different power-speed FUs per function type (and thus a smaller solution space) than the latter.