Physical Unclonable Functions (PUF) are seen as a promising alternative to traditional cryptographic algorithms for secure and lightweight device authentication for the diverse IoT use cases. However, The essential security of PUF is threatened by a kind of machine learning (ML) based modeling attacks which could successfully impersonate the PUF from an extremely large number of challenge and response pairs (CPR). A mechanism to restrict the availability of CRPs has been proposed to protect the PUF from those modeling attacks. To handle the limitation of CRPs from the attack perspective, we propose a transfer learning assisted modeling attack on PUF which could transfer a welltuned model trained with unlimited CRPs to a target PUF who has limited CRPs. Experiment results show that our transfer learning based scheme could achieve the same accuracy level with an average 64% less of CRPs required. Besides, our proposed method is compatible with side-channel information and could also reduce the number of CRPs when applying side-channel based modeling attacks.