3D Dynamic simulator such as Gazebo has become a popular substitution for unmanned aerial vehicle (UAV) because of its user-friendly in real-world scenarios. At this point, well-functioning algorithms on the UAV controller are needed for guidance, navigation, and control for autonomous navigation. Deep reinforcement learning (DRL) comes into sight as its famous self-learning characteristic. This goal-orientated algorithm can learn how to attain a complex objective or maximize along a particular dimension over many steps. In this paper, we propose a general framework to incorporate DRL with the UAV simulation environment. The whole system consists of the DRL algorithm for attitude control, packing algorithm on the Robot Operation System (ROS) to connect DRL with PX4 controller, and a Gazebo simulator that emulates the real-world environment. Experimental results demonstrate the effectiveness of the proposed framework.