How to read a string containing hieroglyphs, Cyrillic, and other characters?

In Golang, strings are represented in UTF-8 by default, which allows working with characters from various alphabets, including hieroglyphs and Cyrillic, without issues. To read such strings, it is sufficient to use standard functions and packages.

Reading from a file:

import (
	"bufio"
	"fmt"
	"os"
)

func readStringFromFile(filename string) error {
	file, err := os.Open(filename)
	if err != nil {
		return fmt.Errorf("error opening file: %w", err)
	}
	defer file.Close()

	scanner := bufio.NewScanner(file)
	for scanner.Scan() {
		line := scanner.Text() // The string is already in UTF-8
		fmt.Println(line)
	}

	if err := scanner.Err(); err != nil {
		return fmt.Errorf("file scanning error: %w", err)
	}
	return nil
}

Reading from standard input:

import (
	"bufio"
	"fmt"
	"os"
)

func readStringFromStdin() {
	reader := bufio.NewReader(os.Stdin)
	fmt.Print("Enter a string: ")
	line, _ := reader.ReadString('\n') // The string is read into a byte slice, then converted to string (UTF-8)
	fmt.Println("You entered:", line)
}

Iterating over characters (runes):

import "fmt"

func iterateString(s string) {
	for i, r := range s { // Range iterates over runes (Unicode code points)
		fmt.Printf("Position: %d, Rune: %c, Code point: %U\n", i, r, r)
	}
}

Main points:

• Strings in Golang are immutable sequences of bytes.
• UTF-8 encoding is used by default, ensuring correct representation of characters from different languages.
• Iterating over a string with range processes each rune (Unicode code point), not each byte.

Additionally:

If the source data is not in UTF-8, encoding conversion may be necessary using standard library packages or third-party libraries, such as golang.org/x/text/encoding.