How to perform encoding and decoding of strings in Python?

Encoding (encode) converts a string (Unicode) into bytes using a specific encoding (e.g., UTF-8). Decoding (decode) performs the reverse transformation: from bytes to a string.

# Encoding a string into bytes
string_to_encode = "Hello, world!"
encoded_bytes = string_to_encode.encode('utf-8')
# Now encoded_bytes is a sequence of bytes representing the string in UTF-8

# Decoding bytes back into a string
decoded_string = encoded_bytes.decode('utf-8')
# Now decoded_string is again the string "Hello, world!"

# Handling errors during encoding/decoding
# If the string contains characters not supported by the specified encoding, or bytes do not form a valid sequence for the encoding
invalid_byte_sequence = b'\xcd\xa1' # Example of an invalid byte sequence for UTF-8
try:
    invalid_byte_sequence.decode('utf-8', errors='strict') # Default is errors='strict' - raises UnicodeDecodeError
except UnicodeDecodeError as e:
    print(f"Decoding error: {e}")

# Using other error handling modes
# errors='ignore' - ignores incorrect characters/bytes
decoded_ignore = invalid_byte_sequence.decode('utf-8', errors='ignore')
print(f"Decoded with ignore: {decoded_ignore}") # Will print an empty string

# errors='replace' - replaces incorrect characters/bytes with a replacement character (usually '?')
decoded_replace = invalid_byte_sequence.decode('utf-8', errors='replace')
print(f"Decoded with replace: {decoded_replace}") # Will print '' or a similar replacement character

# errors='xmlcharrefreplace' (only for ASCII encoding) - replaces characters with their XML entities
string_with_umlaut = "München"
try:
    string_with_umlaut.encode('ascii', errors='xmlcharrefreplace')
except UnicodeEncodeError as e:
    print(f"Encoding error in ASCII: {e}") # ASCII does not support 'ü'

encoded_xmlcharrefreplace = string_with_umlaut.encode('ascii', errors='xmlcharrefreplace')
print(f"Encoded with XML entities: {encoded_xmlcharrefreplace}") # Will print b'München'

The .encode() and .decode() methods accept optional arguments encoding (default UTF-8 in Python 3, but explicit specification is preferred) and errors.

Common encodings include:

• utf-8
• latin-1
• cp1251

When working with files, it is important to specify the encoding when opening, e.g., open('file.txt', 'r', encoding='utf-8').