Backpropagation (BP) is foundational in deep learning. However, its inefficiency is partially caused by backward locking, making simultaneous gradient computation across layers difficult and reducing training efficiency. In this talk, I will introduce our recent research on simultaneously computing parameter gradients in different layers through pipelining. This approach improves the training efficiency while preserving testing accuracies comparable to BP-trained models.