How can custom errors be checked during testing…

Custom error checking involves analyzing their occurrence, content, and behavior in various scenarios.

Stages of testing custom errors:

Requirements analysis. Understanding which custom errors are expected, under what conditions they should occur, and what information should be contained.
Test design. Creating test scenarios that deliberately trigger custom errors. These may include:
- Entering incorrect data.
- Attempting forbidden operations.
- Simulating failures of external systems.
- Load testing to identify performance-related errors.
Executing tests. Running the developed scenarios and recording the actual system behavior.
Verification. Checking the compliance of actual behavior with expectations based on the following criteria:
- Correct occurrence: The error appeared exactly at the right moment and under the specified conditions.
- Error message content: The message text meets requirements, containing necessary information (e.g., error code, problem description, recommendations).
- Localization: Verifying the correctness of error message translations if the application is multilingual.
- Formatting: The visual presentation of the error (text format, color, placement) matches the design or requirements.
- Logging: Ensuring error information is recorded in system logs.
- Handling: The system's behavior after an error occurs (e.g., reverting to a previous state, blocking further actions).
- Security: Ensuring the error message does not contain confidential information.
Automation. Automating the detection of custom error occurrence and content.

Examples of automation approaches:

Parsing API responses. Analyzing JSON/XML responses to verify status codes and error content.

import requests

url = "http://your_api/endpoint"
data = {"invalid_field": "value"}

try:
    response = requests.post(url, json=data)
    response.raise_for_status() # Check for HTTP errors (4xx, 5xx)
    # Additional logic to check for custom errors in response body
    if response.status_code == 400 and "error_code" in response.json():
        assert response.json()["error_code"] == "INVALID_INPUT", "Incorrect error code"
        assert "Invalid data provided" in response.json()["message"], "Incorrect error message"
    else:
        assert False, f"Unexpected status code: {response.status_code} or response body"
except requests.exceptions.HTTPError as e:
    print(f"HTTP error occurred: {e}")
except Exception as e:
    print(f"Other error occurred: {e}")

UI element testing. Automating interaction with UI and verifying error messages displayed to the user.

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.junit.Assert;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class CustomErrorTest {

    private WebDriver driver;

    @Before
    public void setUp() {
        // Set the path to the browser driver
        System.setProperty("webdriver.chrome.driver", "path/to/chromedriver");
        driver = new ChromeDriver();
    }

    @Test
    public void testInvalidLoginError() {
        driver.get("http://your_app/login"); // Login page URL

        // Enter incorrect data
        WebElement usernameField = driver.findElement(By.id("username"));
        WebElement passwordField = driver.findElement(By.id("password"));
        WebElement loginButton = driver.findElement(By.id("loginButton"));

        usernameField.sendKeys("invalid_user");
        passwordField.sendKeys("wrong_password");
        loginButton.click();

        // Check for error message appearance
        WebElement errorMessageElement = driver.findElement(By.cssSelector(".error-message")); // Error message element selector
        Assert.assertTrue("Error message is not displayed", errorMessageElement.isDisplayed());
        Assert.assertEquals("Incorrect error message text", "Invalid credentials", errorMessageElement.getText());
    }

    @After
    public void tearDown() {
        if (driver != null) {
            driver.quit();
        }
    }
}

Server/application logs analysis. Using tools to monitor and analyze logs for specific error messages or codes.

It is important to document all found errors, including steps to reproduce, actual and expected behavior.

Use different environments (dev, staging, production) to verify error behavior in various configurations.

How can custom errors be checked during testing?.