The rapid growth of Internet of Things have yielded a remarkable increase in the volume of the data generated on client devices. This technological trend coincides with the rise of machine learning applications, which leverage user-generated data for large scale model training. In this context, Federated Learning (FL) has become a popular model for facilitating model training across edge devices in a decentralized fashion. However, the statistical diversity presented in the client data and performance heterogeneity existed among the user mobile device can seriously impact the accuracy of the result model and system performance of FL. This article first illustrates the state-of-the-art FL algorithms and investigates the major issues presented in the FL implementation, and then presents a novel FL algorithm that jointly optimizes both the model performance and implementation efficiency for the FL systems. Specifically, we propose an intelligent FL client selection scheme by leveraging the recent advance of Reinforcement Learning (RL) in solving complex control problems. The proposed solution, termed~\textit{IntelliFL}, can greatly improve both the accuracy performance and system performance of FL under the training environment with heterogeneous client devices.