What is Netpoller? Why is it needed and how do goroutines get there and return back?

Netpoller in Go is a mechanism that implements non-blocking input/output at the operating system level, using system calls like epoll (Linux), kqueue (BSD/macOS), or IOCP (Windows). It is needed for efficient management of a large number of network connections without blocking threads.

How it works:

• Goroutines performing input/output operations are registered in the netpoller.
• When a goroutine tries to read or write, if the operation is not ready (for example, no data to read), it is blocked and passed to the netpoller.
• The netpoller monitors input/output events through system calls and notifies the Go scheduler when the operation is ready.
• The scheduler unblocks the goroutine, and it continues execution.

Thus, goroutines do not block system threads but switch based on input/output events, allowing efficient scaling of network applications.

Here's a simplified example:

// Goroutine attempts to read from a network connection
n, err := conn.Read(buf)
// If no data, the goroutine blocks and registers in the netpoller
// The netpoller waits for the read readiness event
// When data appears, the goroutine is unblocked and continues work

Inside the Go runtime, this process is automated and transparent to the developer.