Using the LiveRecorder API

Depending on your license, LiveRecorder can be provided as a library that can be linked into your application in order to give it the ability to record itself.

At its simplest, LiveRecorder can be used as follows:

#include <stdio.h>
#include <stdlib.h>

#include "undolr.h"

int
main(int argc, char **argv)
{
    if (argc != 2)
    {
        fprintf(stderr, "Usage: %s RECORDING\n", argv[0]);
        return EXIT_FAILURE;
    }

    int e = undolr_save_on_termination(argv[1]);
    if (e)
    {
        perror("undolr_save_on_termination");
        return EXIT_FAILURE;
    }

    undolr_error_t error;
    e = undolr_start(&error);
    if (e)
    {
        perror("undolr_start");
        fprintf(stderr, "%s\n", undolr_error_string(error));
        return EXIT_FAILURE;
    }

    /* ... your code here ... */

    return EXIT_SUCCESS;
}

This program will immediately start recording its execution, and the recording will end when the program terminates, at which point a recording will be saved to the file specified in argv[1]. Here’s an example invocation:

$ ./example recording.undo

Recordings produced by LiveRecorder can then be loaded into UDB.

Integrating LiveRecorder

You can find the LiveRecorder libraries and header file in the undolr subdirectory of a UDB release archive:

undo-<version>/
    ...
    undolr/
        libundolr_pic_x64.a
        libundolr_x64.so
        README
        undolr.h
        ...

When the release archive is installed using the included installer makefile, these files are nested within an additional libundolr subdirectory:

undo-<version>/
    ...
    libundolr/
        ...
        undolr/
            libundolr_pic_x64.a
            libundolr_x64.so
            README
            undolr.h
            ...

Header file

LiveRecorder has just a single header file, undolr.h. Simply include this header file into your application to get access to the undolr_* APIs.

Linking

LiveRecorder is available as both a statically linked library (libundolr_pic_<arch>.a) and a dynamically linked library (libundolr_<arch>.so). There are no extra dependencies, so the build command will be similar to:

$ gcc -g -o simple_lr examples/simple_lr.c -I. libundolr_pic_x64.a

All LiveRecorder libraries are position-independent (‘PIC’).

API functions

Except where documented, functions in the LiveRecorder API return 0 if they complete successfully, and −1 in case of failure, with an appropriate error code stored in errno.

Recording program execution

The simplest case is to start recording at program start, and save the recording when the program exits. To start recording, call undolr_start() and if that succeeds, call undolr_save_on_termination(), passing the name of the recording file.

type undolr_error_t

Reason for failing to start recording.

const char *undolr_error_string(undolr_error_t error)

Return string describing error number.

int undolr_start(undolr_error_t *error)

Start recording the current process.

The current process must not already be being recorded: that is, either undolr_start() is being called for the first time, or else there was a call to undolr_stop() since the most recent call to undolr_start().

error is a pointer to a location to store a reason in case of failure, or NULL not to receive the failure reason.

Note

The caller must check the result to determine if recording started successfully. If the result is −1, the general cause of the failure is available in errno and there may be more specific information in *error.

int undolr_save_on_termination(const char *filename)

Instruct LiveRecorder to save a recording when the current process exits.

The recording is only saved if undolr_start() has been called when the process exits. The two functions may be called in either order.

filename is the name of the recording file.

The instruction is canceled by a call to undolr_save_on_termination_cancel() or undolr_stop().

int undolr_save_on_termination_cancel(void)

Cancel any previous call to undolr_save_on_termination().

Recording part of program execution

To record part of the execution of a program, call undolr_start() at the beginning of the part you want to record, and undolr_save() at the end. After the save completes, you can call undolr_stop() to stop recording, passing NULL for the context argument.

This can be repeated if you want to record another part of the execution: after calling undolr_stop(), you can call undolr_start() to start recording again.

int undolr_save(const char *filename)

Save recorded program history to a named recording file.

The current process must be being recorded: that is, undolr_start() must have been successfully invoked without a subsequent call to undolr_stop().

filename is the name of the recording file.

undolr_save() may be called any number of times until undolr_stop() is called. Each subsequent call to undolr_save() contains later execution history. The recordings are independent of each other, and each can be replayed on its own.

type undolr_recording_context_t

Recording context: that is, a handle to the recorded history of the process up to the point where undolr_stop() was called.

int undolr_stop(undolr_recording_context_t *context)

Stop recording the current process.

context is a pointer to a location to store the recording context, or NULL if the recording context should be immediately discarded.

If not discarded immediately, the recorded history is held in memory until context is passed to undolr_discard().

Example: recording a function call

To record the execution of a single function call, you might write a wrapper like this:

#include <stdlib.h>

#include "undolr.h"

/* Use LiveRecorder to record a call to function(arg) and save it to
 * 'recording'. Return 0 if successful, or -1 if not, with an error code in
 * errno. */
int
record_function_call(void (*function)(void *), void *arg, const char *recording)
{
    int e = undolr_start(NULL);
    if (e) return e;

    function(arg);

    e = undolr_save(recording);
    if (e) return e;

    return undolr_stop(NULL);
}

Recording program execution until the event log is full

When LiveRecorder’s circular :doc:event log <EventLog> fills up with execution history, the event log is “rotated” by discarding older events in order to make space for newer events. As a result, execution history from the start of the recording session may not be saved to the recording file.

To capture the start of execution history instead, even if this means that the end of execution history is not saved, the LiveRecorder API provides a undolr_save_and_stop_on_event_log_full() function which saves a recording when the event log is full.

int undolr_save_and_stop_on_event_log_full(const char *filename)

Instruct LiveRecorder to save execution history to a file, and stop recording, when the event log is full.

filename is the name of the recording file.

Saving a recording in the background

The function undolr_save() runs synchronously: it does not return until the recording has finished saving, and all threads in the calling process are suspended during this time. Recordings can be very large, so this call can take significant time to complete.

In applications that have real-time requirements, waiting for a save to complete may not be acceptable, and so the LiveRecorder API provides a mechanism to save asynchronously while the application continues running. To use this mechanism, call undolr_stop(), passing a pointer to a location to store the recording context, and call undolr_save_async(), passing the recording context, to start the save operation.

Then, depending on how your program’s event loop works, you can either ask repeatedly for the progress of the save operation by calling undolr_poll_saving_progress() or undolr_poll_saving_complete(), or else call undolr_get_select_descriptor() to get a file descriptor to pass to select() or pselect(). Finally, free the memory associated with the recorded history by passing the recording context to undolr_discard().

int undolr_save_async(undolr_recording_context_t context, const char *filename)

Start asynchronously saving recorded program history to a named recording file.

context is the recording context returned by a previous call to undolr_stop().

filename is the name of the recording file.

After this call, the recording context may be passed to undolr_poll_saving_progress() or undolr_poll_saving_complete(), but must not be passed to undolr_discard() or to another call to undolr_save_async() until the save has completed.

Note

After calling undolr_stop() to get a context, it is safe to start recording again with undolr_start() before calling undolr_save_async() to start an asynchronous save of the context. That is, recording can proceed in parallel with saving.

int undolr_poll_saving_progress(undolr_recording_context_t context, int *complete, int *progress, int *result)

Check the status of an asynchronous save operation.

context is the recording context passed to undolr_save_async().

complete points to a location that is updated to zero if the save is still in progress, or non-zero if it is complete (whether successfully or unsuccessfully).

progress points to a location that is updated only if the save is still in progress. It is set to the percentage of completion, rounded down, from 0 to 100 inclusive, or to −1 if progress information is unavailable.

result points to a location that is updated only if the save operation is complete. It is set to 0 if the recording was saved successfully, or to an appropriate error code if it was not.

If the state of the asynchronous save operation cannot be determined, −1 is returned and *complete, *progress and *result are left unchanged.

int undolr_poll_saving_complete(undolr_recording_context_t context, int *complete, int *result)

Check the status of an asynchronous save operation. This is identical to undolr_poll_saving_progress(), except that it does not take the progress argument.

int undolr_get_select_descriptor(undolr_recording_context_t context, int *fd)

Get a selectable file descriptor to detect save state changes.

context is the recording context passed to undolr_save_async().

fd points to a location to store a file descriptor.

When the asynchronous save operation is complete, a byte is written to the file descriptor, allowing it to be selected for read using select() or pselect().

The file descriptor is closed when context is passed to undolr_discard().

int undolr_discard(undolr_recording_context_t context)

Discard recorded program history from memory.

context is the recording context returned by a previous call to undolr_stop().

After calling this, the memory used to maintain the recording state has been freed, and context must not be passed to any other function in this API.

Example: reporting progress of a save

This example polls for progress of an asynchronous save operation in a loop:

#include <stdio.h>
#include <unistd.h>

#include "undolr.h"

/* Stop recording and save the recorded history to 'recording', displaying
 * progress to standard output. Return 0 if successful, or -1 if not, with an
 * error code in errno. */
int
save_displaying_progress(const char *recording)
{
    undolr_recording_context_t context;
    int e = undolr_stop(&context);
    if (e) return e;
    e = undolr_save_async(context, recording);
    if (e) goto fail;
    int reported_progress = -1;
    for (;;)
    {
        int complete, progress, result;
        e = undolr_poll_saving_progress(context, &complete, &progress, &result);
        if (e) goto fail;
        if (complete)
        {
            progress = 100;
        }
        if (progress != reported_progress)
        {
            printf("\rSaving %s...% 3d%%", recording, progress);
            reported_progress = progress;
        }
        if (complete)
        {
            printf("\n");
            break;
        }
        e = usleep(1000);
        if (e) goto fail;
    }
fail:
    (void)undolr_discard(context);
    return e;
}

Example: waiting for a save to complete

This example saves asynchronously and waits for the save to complete by calling poll() on the file descriptor returned by undolr_get_select_descriptor().

#include <poll.h>
#include <stdio.h>
#include <unistd.h>

#include "undolr.h"

/* Stop recording, save the recorded history asynchronously to 'recording', and
 * wait for the save to complete by calling poll(2). Return 0 if successful, or
 * -1 if not, with an error code in errno. */
int
save_and_wait(const char *recording)
{
    undolr_recording_context_t context;
    int e = undolr_stop(&context);
    if (e) return e;
    e = undolr_save_async(context, recording);
    if (e) goto fail;
    struct pollfd pollfd = { .events = POLLIN };
    e = undolr_get_select_descriptor(context, &pollfd.fd);
    if (e) goto fail;
    e = poll(&pollfd, 1, -1);
    if (e < 0) goto fail;
    e = 0;
fail:
    (void)undolr_discard(context);
    return e;
}

Creating a point recording

A point recording is an Undo recording that contains the state of the program at a single point in time. A point recording is similar to an ordinary Linux core dump but with the other benefits of an Undo recording which are not possible with a core dump. For instance if moved to another computer it still contains any external debugging symbol files, and it is possible to evaluate expressions and call functions in the program. This is useful if you don’t need the whole of execution history, or if the computer has insufficient memory for the Undo Engine to record its whole history.

Point recordings can be created with a call to undolr_save_point_recording(), passing the name of the recording file.

int undolr_save_point_recording(const char *filename, undolr_error_t *o_error)

Save the current point in time to a named recording file.

filename is the name of the recording file.

o_error is a pointer to a location to store a reason in case of failure, or NULL not to receive the failure reason.

Note

The caller must check the result to determine if the recording was saved successfully. If the result is −1, the general cause of the failure is available in errno and there may be more specific information in *o_error.

Logging accesses to shared memory

int undolr_shmem_log_filename_set(const char *filename)

Set the shared memory log file.

filename is the name of the shared memory log (which must end with .shmem so that UDB can find it during replay), or NULL to disable logging shared memory.

All accesses to shared memory from the process are recorded in the log, and this can then be queried in UDB using the ublame command. The shared memory must be mapped at the same address in each process using the file.

This function must be called before recording starts or it will fail with EINVAL. Stopping recording with undolr_stop() does not unset the shared memory log, so that after a subsequent call to undolr_start(), shared memory accesses will be logged to the same file for the new recording.

int undolr_shmem_log_filename_get(const char **o_filename)

Get the name of the shared memory log file.

o_filename is a location to store a pointer to the shared memory log filename, or NULL if the shared memory log is not enabled. The pointer remains valid until the next call to undolr_shmem_log_filename_set().

int undolr_shmem_log_size_set(unsigned long max_size)

Set the maximum size of the shared memory log file.

max_size is the new maximum size in bytes (after rounding up to a multiple of the page size), or 0 to reset the maximum size to the default value.

This function must be called before recording starts or it will fail with EINVAL.

int undolr_shmem_log_size_get(unsigned long *o_max_size)

Get the maximum size of the shared memory log file.

o_max_size is a location to store the maximum size of the shared memory log file.

Debugging a Live Recording without symbols

Debug symbols often contain sensitive information which makes it desirable to strip them from your binaries. If an Undo Live Recording is made from an executable without symbols then the debug session you will get from loading the recording will also be lacking symbolic information.

The use of GDB’s symbol-file command can be used to combine a Live Recording without symbols with a special file that does not need to be released to your customers.

Example

Build a LiveRecorder-enabled program test that includes debug symbols:

$ gcc -ggdb undolr/examples/simple_lr.c -Iundolr undolr/libundolr_pic_x64.a -o test

Use objcopy to extract the debug information from test.

$ objcopy --only-keep-debug test test.debug

Strip the debug information from test.

$ strip -s test

Run the stripped test to create a recording of itself.

$ ./test -o example.undo
i=0
i=1
Saving recording to: example.undo
Have created recording: example.undo
i=2
i=3

Loading the recording shows that there are no debug symbols:

$ udb
not running> uload example.undo
0x00005651a4dec46d in ?? ()

The debugged program is at the beginning of recorded history. Start debugging
from here or, to proceed towards the end, use:
    continue - to replay from the beginning
    ugo end  - to jump straight to the end of history
start 1> backtrace
#0  0x00005651a4dec46d in ?? ()
#1  0x00007fd9b9567c8a in __libc_start_call_main (main=main@entry=0x5651a4dec3a9,
    argc=argc@entry=3, argv=argv@entry=0x7ffd13d30318)
    at ../sysdeps/nptl/libc_start_call_main.h:58
#2  0x00007fd9b9567d45 in __libc_start_main_impl (main=0x5651a4dec3a9, argc=3,
    argv=0x7ffd13d30318, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
    stack_end=0x7ffd13d30308) at ../csu/libc-start.c:360
#3  0x00005651a4dec2e1 in ?? ()

Use the symbol-file command to load the separate debug information:

start 1> symbol-file test.debug
start 1> backtrace
#0  0x00005651a4dec46d in main (argc=3, argv=0x7ffd13d30318) at undolr/examples/simple_lr.c:55

Signals

The SIGSTOP signal is used internally by LiveRecorder, so that SIGTSTP should be used instead to stop a process while it is being recorded. See this note for details.

Thread-safety and signal-safety

It is not safe to call functions in this API concurrently, either from two or more threads, or from the same thread with one of calls from a signal handler. It is safe to call functions in this API from signal handlers provided that no other API calls are in progress.

Controlling diagnostic output

To control the diagnostic output from the LiveRecorder API, call undolr_set_output_mode(), passing a value from the undolr_output_mode_t enumeration. Alternatively, the UNDO_lr_output_mode environment variable can be set. All messages output from LiveRecorder start with live-record: and are always sent to stderr.

int undolr_set_output_mode(undolr_output_mode_t mode)

Set the diagnostic output mode to mode.

type undolr_output_mode_t

Mode for diagnostic output from LiveRecorder.

enum undolr_output_mode_NONE

Suppress all output from LiveRecorder. The default output mode.

enum undolr_output_mode_ERROR

Write errors to standard error, but suppress other output.

enum undolr_output_mode_ALL

Write all output to standard error.

UNDO_lr_api_output_mode

Environment variable to set the LiveRecorder output mode: none (suppress all output; the default), error (output errors only), or all (output all diagnostic messages).

Support for other languages

As well as the C API described above there is also a Go language binding for LiveRecorder. This is a wrapper for libundolr. It can be obtained using go get go.undo.io/bindings/undolr. Documentation and examples are included in the package. Recordings of Go programmes are best replayed in the GoLand IDE or Delve.