AVX-512 recording portability¶

The GNU C Library (glibc) uses the AVX-512 family of CPU instructions (if available) to implement some of the standard C library functions. These include:

functions declared in the <string.h> header, that is, memcmp(), memcpy(), strcmp(), strcpy(), and so on;
functions declared in the <wchar.h> header, that is, wcpcpy(), wcslen(), wcslen(), wmemset(), and so on; and
vectorized versions of the functions declared in the <math.h> header.

glibc 2.34 and later use AVX-512 in the <string.h> and <wchar.h> functions on any CPU which supports it. Earlier versions use it as well, but only on non-Intel CPUs; on Intel CPUs they use implementations based on AVX2 instead.

Replaying an Undo recording with UDB 6.12 or later¶

UDB versions 6.12 and later have support for the glibc functions defined in <string.h> and, from version 6.12.1 onwards, <wchar.h>, so that an Undo recording can be replayed even if AVX-512 was used by glibc, so long as the requirements in recording portability are met.

If the recorded program called other glibc functions implemented using AVX-512 such as those declared in <math.h>, or used AVX-512 in some other way, then you may see a message like this:

ERROR: 'example.undo' not loaded because CPU features required for replay are
       missing: avx512f.

       Recording portability is only partially supported for glibc 2.34+ when
       the recording CPU has AVX-512, but the replay machine does not. See
       https://docs.undo.io/GlibcAVX512.html for more information.

If you see this message, then the recording can only be replayed on a CPU which supports AVX-512. There are alternative approaches to allow the replay of recordings generated by machines with AVX-512 support on machines without it:

Avoid the use of AVX-512 instructions when the program is being recorded, by Configuring glibc to avoid use of AVX-512 instructions.
Use a replay service whose pool includes machines with the required AVX-512 features.

Replaying an Undo recording with UDB 6.11 or earlier¶

If you attempt to replay an Undo recording which was made on a machine where glibc used the AVX-512 family of instructions, you may see a message from UDB such as:

ERROR: 'example.undo' not loaded because CPU features required for replay are
       missing: avx512f.

       Recording portability is not supported for glibc 2.34+ when the recording
       CPU has AVX-512. See https://docs.undo.io/GlibcAVX512.html for
       more information.

If you see this message, then the recording can only be replayed on a CPU which supports AVX-512. An alternative approach is to avoid the use of AVX-512 instructions when the program is being recorded, by Configuring glibc to avoid use of AVX-512 instructions.

Configuring glibc to avoid use of AVX-512 instructions¶

When making an Undo recording, glibc can be configured to suppress the use of AVX-512 instructions, so that (as long as there are no other uses of AVX-512 instructions in the program) the recording can be replayed on a CPU which does not support AVX-512, subject to the other recording portability requirements.

Do this by setting the GLIBC_TUNABLES environment variable when the program is recorded. The tunable and the feature names depend on the version of glibc on the recording machine, so check it first with ldd --version and use the matching setting:

glibc version	Setting for `GLIBC_TUNABLES`
2.32 and later	`glibc.cpu.hwcaps=-AVX512F,-AVX512BW,-AVX512VL,-AVX512DQ`
2.29 to 2.31	`glibc.cpu.hwcaps=-AVX512F_Usable,-AVX512DQ_Usable`
2.26 to 2.28	`glibc.tune.hwcaps=-AVX512F_Usable,-AVX512DQ_Usable`

Versions from 2.26 to 2.31, such as those in Red Hat Enterprise Linux 8 and Amazon Linux 2, need the names of the derived features that select the implementation, and the versions up to 2.28 also use the glibc.tune namespace rather than glibc.cpu. The _Usable names do not exist from glibc 2.32 onwards, and the plain feature names have no effect before it. glibc ignores tunables it does not recognize, so a setting for the wrong version has no effect and reports no error.

Versions before 2.26 have no such tunable, so glibc’s use of AVX-512 cannot be suppressed this way.

For example, on glibc 2.32 or later:

export GLIBC_TUNABLES=glibc.cpu.hwcaps=-AVX512F,-AVX512BW,-AVX512VL,-AVX512DQ

or as part of the command line when running the live-record tool:

GLIBC_TUNABLES=glibc.cpu.hwcaps=-AVX512F,-AVX512BW,-AVX512VL,-AVX512DQ live-record my-program