Design space subsetting has been widely used to select configurations that are suitable for the design objective. However, given the growing number of design constraints and objectives (energy, performance, EDP, temperature, user expectations, etc.) selecting best subsets for a single objective no longer satisfies current design practices. Additionally, the increasing design space sizes in emerging systems, and the variety of configurations that can satisfy multiple objectives, makes design space subsetting very challenging. In this paper, using a configurable cache as a case study, we evaluate the impact of design space subsetting for performance, energy, and temperature design constraint collectively. We evaluate the quality of the subsets obtained for one design constraint against the complete design space and against the remaining design objectives (e.g., best energy subsets quality for performance and thermal objectives). Furthermore, we evaluate a design space of 243 configurations, yielding up to $1.4 \times 10^{73}$ subsets. Our results reveal that prior subsetting methods are insufficient to meet current design trends due to the correlation between design objectives. Our results also suggest that large subsets of 10 or more configurations are required to maintain multi-objective optimization results that are within 3\% of the optimal.