If we had unlimited resources and speed of convergence didn't matter, which method of GD, SGD, or Mini-Batch SGD would be better to use?

If resources are unlimited and convergence speed is not critical, it is better to use the classic gradient descent (Gradient Descent, GD).

Rationale:

• GD computes the gradient over the entire dataset, ensuring a stable and accurate direction for parameter updates.
• SGD (Stochastic Gradient Descent) and Mini-Batch SGD use random subsets, which speed up iterations but introduce noise into the gradients, potentially slowing convergence.

With infinite resources, full computations at each step can be afforded to avoid stochastic noise and achieve more predictable algorithm behavior.

Therefore, GD is preferable when there are no constraints on computational power and time.