What methods and tools do you use to collect data from web pages through parsing?

Tools and approaches commonly used in Python for web scraping from web pages include:

• requests — for sending HTTP requests and receiving the HTML code of the page.
• BeautifulSoup — for parsing HTML and extracting desired elements by tags, classes, id, etc.
• lxml — a faster alternative to BeautifulSoup for parsing XML/HTML.
• Selenium — for parsing dynamically loaded pages using a browser, especially when content is generated by JavaScript.

Example of simple parsing:

import requests
from bs4 import BeautifulSoup

url = 'https://example.com'
response = requests.get(url)

if response.status_code == 200:
    soup = BeautifulSoup(response.text, 'html.parser')
    titles = soup.find_all('h2')  # for example, article titles
    for title in titles:
        print(title.text)

For more complex cases where content loads dynamically, tools like Selenium or headless browsers (e.g., with Playwright) are used.

It is also important to follow the website's rules (robots.txt) and avoid overloading the server with frequent requests.