How to get the last 100 lines from a very large log file containing millions of records?

To get the last 100 lines from a very large log file without reading the entire file, you can use the tail command in Unix-like systems:

tail -n 100 /path/to/large_log_file.log

The tail command efficiently reads the file from the end, allowing you to quickly obtain the last lines without loading the entire file into memory.

If you need to implement this programmatically, for example in Python, you can open the file in binary mode and read from the end in blocks until you have the desired number of lines. But for most tasks, tail is sufficient.

Here's a simplified example in Python:

from collections import deque

def tail(filename, n=100):
    with open(filename, 'r') as f:
        return deque(f, n)

last_lines = tail('/path/to/large_log_file.log', 100)
for line in last_lines:
    print(line, end='')