Asynchronous I/O is a method of performing I/O in which the application issues requests to the kernel (for example, to write some data into a file) and at some point later the application receives an event from the kernel to indicate the result of the request.
Debugging programs using asynchronous I/O with UDB¶
To debug a program using asynchronous I/O, start it under UDB in the same way as any other program:
$ udb --args examples/aio /dev/zero Reading symbols from examples/aio... not running> start Temporary breakpoint 1 at 0x136f: file aio.c, line 29. Starting program: examples/aio /dev/zero Temporary breakpoint 1, main (argc=2, argv=0x7fffffffdc08) at aio.c:29 29 if (argc != 2)
UDB inspects the program and automatically loads the asynchronous I/O preload
library if it detects that
libaio is in use. If the program does not link
libaio (because it uses the system calls directly, or because it
loads the library dynamically), asynchronous I/O can be forced on or off using
--async-io command-line option to
udb, or the
UNDO_enable_async_io environment variable. These take the
true— always enable support for asynchronous I/O when launching the program, and refuse to attach to a process that does not have the asynchronous I/O library preloaded.
false— never enable support for asynchronous I/O when launching the program, and always attach to a process regardless of whether it has the asynchronous I/O library preloaded.
Note that if this mode is selected but the program uses asynchronous I/O, then UDB will not be able to correctly record or replay the program. In this case it displays a warning, for example:
$ udb --async-io false --args examples/aio /dev/zero Reading symbols from examples/aio... Asynchronous I/O was disabled by user, but the environment or libraries for the debugged process suggest it may be needed. Continuing anyway.
auto(the default) — automatically detect whether the program links against
libaio.so, and select
Attaching to programs using asynchronous I/O¶
When the Undo Engine attaches to a process that is using asynchronous
I/O, there may be asynchronous I/O operations in flight. In order for the Undo
Engine to correctly record and replay these operations, the process must have
been using the asynchronous I/O preload library. This is a shared library
that can be linked into a program at run-time using the
The correct preload library depends on the architecture for which your program was compiled:
The preload library doesn’t support raw asynchronous I/O system calls (i.e.
performed with native assembly instructions). It is however supported to
use the C library
syscall() function to perform asynchronous I/O
When the Undo Engine attaches to a process that is linked against
but is not using the preload library, it prints the following warning:
$ udb not running> attach 2424312 Have reached start of recorded history. 0x00007fee3c37f000 in __GI___libc_read (fd=0, buf=0x55c3796446b0, nbytes=1024) at ../sysdeps/unix/sysv/linux/read.c:26 26 ../sysdeps/unix/sysv/linux/read.c: No such file or directory. UDB has detected that libaio.so is in use. To debug calls to the Linux asynchronous I/O API ("async I/O" or "aio") you must add UDB's preload library to the LD_PRELOAD environment variable of the program you are debugging. For instance: $ LD_PRELOAD=libundodb_aio_preload_x64.so \ YOUR-PROGRAM [ARGUMENTS] &
In this situation the Undo Engine cannot record or replay the program:
recording 1> continue Continuing. Program received signal SIGTRAP, Trace/breakpoint trap. Forward execution cannot continue from here. because: The target process is performing asynchronous I/O with contexts that are not being tracked by the Undo Engine. Use the Undo asynchronous I/O preload library to avoid this issue. because: Asynchronous I/O (in this case, the "io_getevents" system call) is not supported. syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 38 ../sysdeps/unix/sysv/linux/x86_64/syscall.S: No such file or directory.
The solution is to use the asynchronous I/O preload library:
$ LD_PRELOAD=./libundodb_aio_preload_x64.so examples/aio /dev/zero & pid=2424344: press any key to wait for read of 4096 bytes from fd 3... $ udb not running> attach 2424344 Have reached start of recorded history. 0x00007f1801149000 in __GI___libc_read (fd=0, buf=0x55d27e386710, nbytes=1024) at ../sysdeps/unix/sysv/linux/read.c:26 26 ../sysdeps/unix/sysv/linux/read.c: No such file or directory. recording 1> continue Continuing. Program received signal SIGSTOP, Stopped (signal). The program has exited, but is still being debugged. You can use UDB reverse commands to go backwards; see "help udb" for details. __GI__exit (status=status@entry=0) at ../sysdeps/unix/sysv/linux/_exit.c:31 31 ../sysdeps/unix/sysv/linux/_exit.c: No such file or directory.
Asynchronous I/O in LiveRecorder¶
Currently, LiveRecorder does not support the checks or parameters used by UDB to enable or disable asynchronous I/O. To record a program that uses asynchronous I/O operations with LiveRecorder, the user must specify the preload library in the LiveRecorder environment. For example:
$ LD_PRELOAD=./libundodb_aio_preload_x64.so live-record examples/aio /dev/zero & live-record: Termination recording will be written to aio-2424376-2022-11-04T16-24-01.955.undo live-record: Maximum event log size is 1G pid=2424376: press any key to wait for read of 4096 bytes from fd 7... live-record: Saving to 'aio-2424376-2022-11-04T16-24-01.955.undo'... live-record: Termination recording written to aio-2424376-2022-11-04T16-24-01.955.undo live-record: Detaching...
Attaching to a program with LiveRecorder is a similar to attaching to the program in UDB:
$ LD_PRELOAD=./libundodb_aio_preload_x64.so examples/aio /dev/zero & pid=2424438: press any key to wait for read of 4096 bytes from fd 3... $ live-record --pid 2424438 -o aio.undo & live-record: Maximum event log size is 1G live-record: Saving to 'aio.undo'... live-record: Termination recording written to aio.undo live-record: Detaching... $ udb not running> uload aio.undo The debugged program is at the beginning of recorded history. Start debugging from here or, to proceed towards the end, use: continue - to replay from the beginning ugo end - to jump straight to the end of history 0x00007fd304f99000 in __GI___libc_read (fd=0, buf=0x55d273f74710, nbytes=1024) at ../sysdeps/unix/sysv/linux/read.c:26 26 ../sysdeps/unix/sysv/linux/read.c: No such file or directory.
Buffers with incorrect permissions¶
As part of its operation the preload library accesses asynchronous I/O buffers.
If your program provides asynchronous I/O buffers with incorrect permissions
(for example, providing a
PROT_NONE buffer) then you would normally
receive an error from the kernel (generally
EFAULT). However since
the preload library accesses the asynchronous I/O buffers directly it is likely
to experience a
SIGSEGV in these cases. If you experience a crash
from the preload library it may therefore indicate the buffer permissions are