How to limit the use of RAM for a process via cgroups in Linux?

To limit the process's RAM usage via cgroups in Linux:

1. Create or use an existing cgroup in the memory subsystem.
2. Set the memory limit by writing the desired value to the memory.limit_in_bytes file.
3. Add the process PID to the cgroup.procs file of this cgroup.

Example:

# Create a cgroup directory (if using cgroup v1)
sudo mkdir /sys/fs/cgroup/memory/mygroup

# Set the memory limit, e.g., 500MB
echo $((500*1024*1024)) | sudo tee /sys/fs/cgroup/memory/mygroup/memory.limit_in_bytes

# Add a process with PID 1234 to this cgroup
echo 1234 | sudo tee /sys/fs/cgroup/memory/mygroup/cgroup.procs

For cgroup v2, commands and paths may differ, for example:

# Create a cgroup
sudo mkdir /sys/fs/cgroup/mygroup

# Set the memory limit
echo $((500*1024*1024)) | sudo tee /sys/fs/cgroup/mygroup/memory.max

# Add a process
echo 1234 | sudo tee /sys/fs/cgroup/mygroup/cgroup.procs

Important: working with cgroups requires root privileges or appropriate permissions. Also, note that exceeding the memory limit may result in the process being killed by the kernel (OOM killer).