What is target network in DQN?

Target network in DQN is a copy of the main neural network used to compute target Q-values during training. It is updated less frequently than the main network (for example, every few thousand steps), which helps stabilize training and prevent divergence. The idea is that a fixed target network provides a more stable target for updating the main network's parameters.

Example of updating the target network:

if step % target_update_freq == 0:
    target_network.load_state_dict(main_network.state_dict())