Can you explain what web scraping is and what it is used for?

Web scraping is the process of automatically extracting data from websites. It is commonly used to gather information that is available on pages but not provided in a convenient format for processing (such as tables, lists, texts).

Applications of web scraping:

• Collecting prices and product features for competitor monitoring.
• Aggregating news or articles from various sources.
• Analyzing reviews and comments.
• Gathering data for research and analytics.

Example in Python using the BeautifulSoup library:

import requests
from bs4 import BeautifulSoup

url = 'https://example.com'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

# Extract all h2 headers
for h2 in soup.find_all('h2'):
    print(h2.text)