[]byte to string conversion

Go has an old wiki page, titled “Compiler And Runtime Optimizations”.

The part I like most there is different cases where compiler doesn’t allocate memory for string to []byte conversions:

For a map m of type map[string]T and []byte b, m[string(b)] doesn’t allocate (the temporary string copy of the byte slice isn’t made)

Turned out, since this wiki page was written, more similar optimisations were added to the compiler.

As it’s in Go 1.12+ the following cases are also listed in runtime/string.go:

For the case "<" + string(b) + ">", where b is []byte no extra copying of b is needed.

if string(b) == "foo" { ··· }

In the code above, b []byte also won’t be copied.

There are still cases where compiler can’t optimise the code for us. In some of those cases it’s fine to do string to bytes conversion using a so called “unsafe trick” (accessing string’s underling data directly, with out copying the data from string to bytes and vice versa). One can find several ways of performing the trick, but none of them seems “the one that must be used”.

After years of episodic discussions, a collegue of mine assembled the list of different conserns and about the proper way of doing it (see “unsafe conversion between string <-> []byte” topic on golang-nuts forum). Thanks to replies from Go team, our most valid way of doing it is following:

// Refer to github.com/fmstephe/unsafeutil

type stringHeader struct {
	data      unsafe.Pointer
	stringLen int

type sliceHeader struct {
	data     unsafe.Pointer
	sliceLen int
	sliceCap int

func StringToBytes(s string) (b []byte) {
	stringHeader := (*stringHeader)(unsafe.Pointer(&s))
	sliceHeader := (*sliceHeader)(unsafe.Pointer(&b))
	sliceHeader.data = stringHeader.data
	sliceHeader.sliceLen = len(s)
	sliceHeader.sliceCap = len(s)
	return b

func BytesToString(b []byte) (s string) {
	sliceHeader := (*sliceHeader)(unsafe.Pointer(&b))
	stringHeader := (*stringHeader)(unsafe.Pointer(&s))
	stringHeader.data = sliceHeader.data
	stringHeader.stringLen = len(b)
	return s

All topics

applearduinoarm64askmeawsberlinbookmarksbuildkitcgocoffeecontinuous profilingCOVID-19designdockerdynamodbe-paperenglishesp8266firefoxgithub actionsGogooglegraphqlhomelabIPv6k3skuberneteslinuxmacosmaterial designmDNSmusicndppdneondatabaseobjective-cpasskeyspostgresqlpprofprofeferandomraspberry pirusttravis civs codewaveshareµ-benchmarks