With the increase in demand for neural networks(NNs) on edge devices, there is a need for cost-effective and hardware-efficient architectures. Activation functions induce non-linearity in the much-needed NN space and hence are essential to formulate real-world scenarios to an artificial model. In this work, three different architectural styles for Softmax and a single architecture each for Tanh and Sigmoid implementations using an improved CORDIC (COordinate-Rotation-DIgital-Computer) algorithm in diverse floating-point data-formats are explored. The work pays dual attention; firstly focusing on different architectural possibilities to implement hardware efficient activation functions, and secondly investigating the impact of utilizing different floating-point data formats on hardware parameters and error metrics. Different data formats are investigated in this work including the state-of-the-art (SOTA) Floating-Point 16 (FP16), Floating-Point 32 (FP32), Brain Floating-Point-BFloat16 (BF16), POSIT and TensorFloat32 (TF32). The area utilization, critical path delay, and power costs are presented for both FPGA (Zynq 7000 Zedboard) and ASIC (Cadence 45 nm library) synthesis. Error metrics are generated by comparing hardware simulated results with actual Floating-Point 64 bit (FP64) values. In conclusion, a benchmarking analysis that covers the best, nominal, and worst data formatted designs for the CORDIC implementation of activation functions was presented.