Energy harvesting devices (EHDs) are becoming inevitable because of their ultra-low-power, highly portable, self-sustainable nature. But it is challenging to use these devices as they rely on sporadic and variable sources of ambient energy and are equipped with very small memories. NVM memories such as FRAM used in EHDs suffer from a security vulnerability where the data remains even after the system is powered down. An attacker with physical access to the system can extract sensitive information from the memory; our goal is to find an efficient way to protect the memory from such vulnerabilities. We propose a secure mechanism that includes encryption for protection against attacks such as snooping, splicing, and replay. Moreover, due to the intermittent energy, there is repeated loss of device state that can cause nontermination of the programs executing on these devices. We use block-based differential checkpointing as the state retention technique that achieves termination and guarantees the program's forward progress. Also, our work includes an efficient way of calculating and comparing energy to start the encryption and checkpointing process. We show a 27.81x improvement in the progress of the program, and more than 99% reduction in latency and energy overhead compared to the baseline method.