Solving continuous minimax optimization is of extensive practical interest, yet notoriously unstable and difficult. This paper introduces the learning to optimize (L2O) methodology to the minimax problems for the first time, and addresses its accompanying unique challenges. We first present Twin L2O, the first dedicated minimax L2O framework consisting of two LSTMs for updating min and max variables, respectively. That decoupled design is found to facilitate learning, particularly when the min and max variables are highly non-symmetric. Empirical experiments on a variety of minimax problems corroborates the effectiveness of Twin-L2O. We then discuss a crucial concern of Twin-L2O, i.e., its inevitably limited generalizability to unseen optimizees, and present two complementary strategies. Our first solution, Enhanced Twin-L2O, is empirically applicable for general minimax problems, by improving L2O training via leveraging curriculum learning. We extensively benchmark our algorithms on popular minimax problems, and compare against state-of-the-art minimax solvers. Our second alternative, called Safeguarded Twin L2O, is a preliminary theoretical exploration stating that under some strong assumptions, it is possible to theoretically establish the convergence of Twin-L2O on optimizing any unseen objective. Our codes and models will be publicly released upon acceptance.