One of the promising solutions for energy-efficient CNNs is to break them down into multiple stages that are executed sequentially (MS-CNN). In this paper, we illustrate that unlike deep CNNs, MS-CNNs develop a form of contextual awareness of input data in initial stages. We show this contextual knowledge could be used to dynamically change the structure and connectivity of such networks to reduce their computational complexity, making them a better fit for low-power and real-time systems. We suggest three run-time optimization policies, which are capable of exploring such contextual knowledge, and illustrate how the proposed policies construct a dynamic architecture suitable for a wide range of applications with varied accuracy requirements, resources, and time-budget, without further need for network re-training. Moreover, we propose variable and dynamic bit-length fixed-point conversion to further reduce the memory footprint of the MS-CNNs.