[]byte to string conversion

Go has an old wiki page, titled “Compiler And Runtime Optimizations”.

The part I like most there is different cases where compiler doesn’t allocate memory for string to []byte conversions:

For a map m of type map[string]T and []byte b, m[string(b)] doesn’t allocate (the temporary string copy of the byte slice isn’t made)

Turned out, since this wiki page was written, more similar optimisations were added to the compiler.

As it’s in Go 1.12+ the following cases are also listed in runtime/string.go:

• Strings comcatenation

For the case "<" + string(b) + ">", where b is []byte no extra copying of b is needed.

• Comparison

if string(b) == "foo" { ··· }

In the code above, b []byte also won’t be copied.

There are still cases where compiler can’t optimise the code for us. In some of those cases it’s fine to do string to bytes conversion using a so called “unsafe trick” (accessing string’s underling data directly, with out copying the data from string to bytes and vice versa). One can find several ways of performing the trick, but none of them seems “the one that must be used”.

After years of episodic discussions, a collegue of mine assembled the list of different conserns and about the proper way of doing it (see “unsafe conversion between string <-> []byte” topic on golang-nuts forum). Thanks to replies from Go team, our most valid way of doing it is following:

// Refer to github.com/fmstephe/unsafeutil

type stringHeader struct {
	data      unsafe.Pointer
	stringLen int
}

type sliceHeader struct {
	data     unsafe.Pointer
	sliceLen int
	sliceCap int
}

func StringToBytes(s string) (b []byte) {
	stringHeader := (*stringHeader)(unsafe.Pointer(&s))
	sliceHeader := (*sliceHeader)(unsafe.Pointer(&b))
	sliceHeader.data = stringHeader.data
	sliceHeader.sliceLen = len(s)
	sliceHeader.sliceCap = len(s)
	return b
}

func BytesToString(b []byte) (s string) {
	sliceHeader := (*sliceHeader)(unsafe.Pointer(&b))
	stringHeader := (*stringHeader)(unsafe.Pointer(&s))
	stringHeader.data = sliceHeader.data
	stringHeader.stringLen = len(b)
	return s
}