Asynchronous System Trap

Asynchronous system trap (AST) refers to a mechanism used in several computer operating systems designed by the former Digital Equipment Corporation (DEC) of Maynard, Massachusetts.

Various events within these systems can be optionally signalled back to the user processes via the AST mechanism. These ASTs act like subroutine calls but they are delivered "asynchronously", that is, without any regard to the context of the main thread. Because of this, care must be taken:
* to ensure that any code that is shared between the main thread and the AST must be designed to be reentrant, and
* any data that is shared must be safe against corruption if modified at any time by the AST. Otherwise, the data must be guarded by blocking ASTs during critical sections.

ASTs are most commonly encountered as a result of issuing QIO calls to the kernel. Completion of the I/O can be signalled by the issuance of an AST to the calling process/task. Certain runtime errors could also be signalled using the AST mechanism. Within VMS, Special Kernel-Mode ASTs are used as the standard mechanism for getting access a process context; they are executed at the highest possible per-process priority the next time the scheduler makes that process current, and are used among other things for retrieving process-level information (in response to a $GETJPI "getjob/process information" system call) and for performing process deletion.

The following operating systems implemented ASTs:
* RSX-11 (including all of the variants)
* RSTS/E
* VMS

ASTs are roughly analogous to Unix signals. The important differences are:
* There are no "signal codes" assigned to ASTs: instead of assigning a handler to a signal code and raising that code, the AST is specified directly by its address. This allows any number of ASTs to be pending at once (subject to process quotas).
* ASTs never abort any system call in progress. In fact, it is possible for a process to put itself into a "hibernate" state (with the $HIBER system call) where it does nothing but wait for ASTs to be delivered; when an AST is delivered, the process is temporarily woken up, and after the AST completes, the process goes right back into hibernation again. The only way to get out of this state (apart from process deletion) is to execute the $WAKE system call; this can be done by an AST within that process, or by another suitably-privileged process.

VAX/VMS V4 and later implemented an interesting optimization to the problem of synchronizing between AST-level and non-AST-level code. A system service named $SETAST could be used to disable or enable the delivery of ASTs for the current and all less-privileged "access modes" (the VMS term for ring-based security features). However, if the critical section needing protection from ASTs was only a few instructions long, then the overhead of making the $SETAST calls could far outweigh the time to execute those instructions.

So for user mode only (the least privileged ring, normally used by ordinary user programs), a pair of bit flags was provided at a predefined user-writable memory location (in per-process "P1" space). The meanings of these two flags could be construed as "don't deliver any ASTs" and "ASTs have been disabled". Instead of the usual pair of $SETAST calls, the user-mode code would set the first flag before executing the sequence of instructions during which ASTs need to be blocked, and clear it after the sequence. "Then" (note the ordering here, to avoid race conditions) it would check the second flag to see if it had become set during this time: if so, then ASTs really have become disabled, and $SETAST should be called to re-enable them. In the most common case, no ASTs would have become pending during this time, so there would be no need to call $SETAST at all.

The kernel AST delivery code, for its part, would check the first flag before trying to deliver a user-mode AST; if it was set, then it would directly set the ASTs-disabled bit in the process control block (the same bit that would be set by an explicit $SETAST call from user mode), and also set the second flag, before returning and leaving the AST undelivered.

See also

* Signal (computing)