With ever-increasing design complexity and stringent time-to-market pressure, automated topology synthesis tools for operational amplifiers are required to produce designs meeting different specifications. This paper proposes TOTAL, a reinforcement learning-based topology optimization method for operational amplifiers. We decompose the circuit topology design as a Markov decision process to solve the high dimensionality of the design space, with the three-stage cascode paradigm fixed to avoid meaningless structures. Therefore, starting from a basic behavior-level topology, an agent modifies the circuit step by step. Specifically, this agent mainly adopts a graph neural network to understand each design state, including specifications and the design history, and a convolutional neural network to modify the current topology. Every completed circuit is then simulated and evaluated by a customized reward function to guide the agent in finding qualified circuits, among which only the optimal one ever recorded is mapped to the transistor level for further evaluation. Experimental results show that the trained agent can not only generate high-performance circuits, but also be reusable by transferring to other specifications as a pre-trained model and achieving competitive results.