;;; -*- Mode:gate; Fonts:(HL12 HL12I HL12B CPTFONTB HL12BI HL12B HL12I ) -*- =Node: Introduction =Text: 3INTRODUCTION* The Lisp Machine supports 1multi-processing*; several computations can be executed concurrently by placing each in a separate 1process*. A process is like a processor, simulated by software. Each process has its own program counter, its own stack of function calls and its own special-variable binding environment in which to execute its computation. (This is implemented with stack groups; see 4(STACKGROUPS-0)Stack Groups*.) If all the processes are simply trying to compute, the machine allows them all to run an equal share of the time. This is not a particularly efficient mode of operation since dividing the finite memory and processor power of the machine among several processes certainly cannot increase the available power and in fact wastes some of it in overhead. The typical use for processes is that at any time only one or two are trying to run. The rest are either 1waiting* for some event to occur or 1stopped* and not allowed to compete for resources. A process waits for an event by means of the 2process-wait* primitive, which is given a predicate function which defines the event being waited for. A module of the system called the process scheduler periodically calls that function. If it returns 2nil* the process continues to wait; if it returns 2t* the process is made runnable and its call to 2process-wait* returns, allowing the computation to proceed. A process may be 1active* or 1stopped*. Stopped processes are never allowed to run; they are not considered by the scheduler, and so can never become the current process until they are made active again. The scheduler continually tests the waiting functions of all the active processes, and those which return non-2nil* values are allowed to run. When you first create a process with 2make-process*, it is inactive. The activity of a process is controlled by two sets of Lisp objects associated with it, called its 1run reasons* and its 1arrest reasons*. These sets are implemented as lists. Any kind of object can be in these sets; typically keyword symbols and active objects such as windows and other processes are found. A process is considered active when it has at least one run reason and no arrest reasons. To get a computation to happen in another process, you must first create a process, then say what computation you want to happen in that process. The computation to be executed by a process is specified as an 1initial function* and a list of arguments to that function. When the process starts up it applies the function to the arguments. In some cases the initial function is written so that it never returns, while in other cases it performs a certain computation and then returns, which stops the process. To 1reset* a process means to exit its entire computation nonlocally using 2*unwind-stack* (see 4(FLOWCTL-2)Dynamic Non-Local Exits*). Some processes are temporary and die when reset. The other, permanent functions start their computations over again when reset. Resetting a process clears its waiting condition, so that if it is active it becomes runnable. To 1preset* a function is to set up its initial function (and arguments) and then reset it. This is how you start up a computation in a process. All processes in a Lisp Machine run in the same virtual address space, sharing the same set of Lisp objects. Unlike other systems that have special restricted mechanisms for inter-process communication, the Lisp Machine allows processes to communicate in arbitrary ways through shared Lisp objects. One process can inform another of an event simply by changing the value of a global variable. Buffers containing messages from one process to another can be implemented as lists or arrays. The usual mechanisms of atomic operations, critical sections, and interlocks are provided (see 2store-conditional* (4(PROCESSES-1)Locks*), 2without-interrupts* (4(PROCESSES-1)The Scheduler*), and 2process-lock* (4(PROCESSES-1)Locks*)). A process is a Lisp object, an instance of one of several flavors of process (see 4(FLAVOR-0)Objects, Message Passing, and Flavors*). The remainder of this chapter describes the operations defined on processes, the functions you can apply to a process, and the functions and variables a program running in a process can use to manipulate its process. =Node: 4The Scheduler* =Text: 3THE SCHEDULER* At any time there is a set of 1active processes*; as described above, these are all the processes that are not stopped. Each active process is either currently running, runnable (ready to run), or waiting for some condition to become true. The active processes are managed by a special stack group called the 1scheduler*, which repeatedly examines each active process to determine whether it is waiting or ready to run. The scheduler then selects one process and starts it up. The process chosen by the scheduler becomes the 1current process*, that is, the one process that is running on the machine. The scheduler sets the variable 2current-process* to it. It remains the current process and continues to run until either it decides to wait, or a 1sequence break* occurs. In either case, the scheduler stack group is resumed. It then updates the process's run time meters and chooses a new process to run next. This way, each process that is ready to run gets its share of time in which to execute. Each process has a 1priority* which is a number. Most processes have priority zero. Larger numbers give a process more priority. The scheduler only considers the highest priority runnable processes, so if there is one runnable process with priority 20 then no process with lesser priority can run. The scheduler determines whether a process is runnable by applying the process's 1wait-function* to its 1wait-argument-list*. If the wait-function returns a non-2nil* value, then the process is ready to run; otherwise, it is waiting. A process can wait for some condition to become true by calling 2process-wait* (see 4(PROCESSES-1)The Scheduler*). This function sets the process's wait-function and wait-argument-list as specified by the caller, and resumes the scheduler stack group. A process can also wait for just a moment by calling 2process-allow-schedule* (see 4(PROCESSES-1)The Scheduler*), which resumes the scheduler stack group but leaves the process runnable; it will run again as soon as all other runnable processes have had a chance. A sequence break is a kind of interrupt that is generated by the Lisp system for any of a variety of reasons; when it occurs, the scheduler is resumed. The function 2si:sb-on* (see 4(PROCESSES-1)The Scheduler*) can be used to control when sequence breaks occur. The default is to sequence break once a second. Thus even if a process never waits and is not stopped, it is forced to return control to the scheduler once a second so that any other runnable processes can get their turn. The system does not generate a sequence break when a page fault occurs; thus time spent waiting for a page to come in from the disk is ``charged'' to a process the same as time spent computing, and cannot be used by other processes. It is done this way for the sake of simplicity; this allows the whole implementation of the process system to reside in ordinary virtual memory, so that it does not have to worry specially about paging. The performance penalty is small since Lisp Machines are personal computers, not multiplexed among a large number of processes. Usually only one process at a time is runnable. A process's wait function is free to touch any data structure it likes and to perform any computation it likes. Of course, wait functions should be kept simple, using only a small amount of time and touching only a small number of pages, or system performance will be impacted since the wait function will consume resources even when its process is not running. If a wait function gets an error, the error occurs inside the scheduler. If this enters the debugger, all scheduling comes to a halt until the user proceeds or aborts. Aborting in the debugger inside the scheduler ``blasts'' the current process by giving it a trivial wait function that always returns 2nil*; this prevents recurrence of the same problem. It is best to write wait functions that cannot get errors, by keeping them simple and by arranging for any problems to be detected before the scheduler sees the wait function. 2process-wait* calls the wait function once before giving it to the scheduler, and this often exposes an error before it can interfere with scheduling. Note well that a process's wait function is executed inside the scheduler stack-group, 1not* inside the process. This means that a wait function may not access special variables bound in the process. It is allowed to access global variables. It can access variables bound by a process through the closure mechanism (4(CLOSURES-0)Closures*). If the wait function is defined lexically within the caller of 2process-wait* then it can access local variables through the lexical scoping mechanism. Most commonly any values needed by the wait function are passed to it as arguments. 3current-process* 1Variable* The value of 2current-process* is the process that is currently executing, or 2nil* while the scheduler is running. When the scheduler calls a process's wait-function, it binds 2current-process* to the process so that the wait-function can access its process. 3without-interrupts* 1body...* 1Macro* The 1body* forms are evaluated with 2inhibit-scheduling-flag* bound to 2t*. This is the recommended way to lock out multi-processing over a small critical section of code to prevent timing errors. In other words the body is an 1atomic operation*. The values of the last form in the body are ultimately returned. In this example, 2list* is presumed to be a global variable referred to from two places in the code which different processes will execute. 3(without-interrupts* 3 (push item list))* 3(without-interrupts* 3 (cond ((memq item list)* 3 (setq list (delq item list))* 3 t)* 3 (t nil))) inhibit-scheduling-flag* 1Variable* The value of 2inhibit-scheduling-flag* is normally 2nil*. 2without-interrupts* binds it to 2t*, which prevents process-switching until 2inhibit-scheduling-flag* becomes 2nil* again. It is cleaner to use 2without-interrupts* than to refer directly to this variable. 3process-wait* 1whostate* 1function* &rest 1arguments* This is the primitive for waiting. The current process waits until the application of 1function* to 1arguments* returns non-2nil* (at which time 2process-wait* returns). Note that 1function* is applied in the environment of the scheduler, not the environment of the 2process-wait*, so special bindings in effect when 2process-wait* was called are 1not* be in effect when 1function* is applied. Be careful when using any free references in 1function*. 1whostate* is a string containing a brief description of the reason for waiting. If the who-line at the bottom of the screen is looking at this process, it will show 1whostate*. Example: 3(process-wait "Buffer"* 3 #'(lambda (b) (not (zerop (buffer-n-things b))))* 3 the-buffer) process-sleep* 1interval* Waits for 1interval* sixtieths of a second, and then returns. It uses 2process-wait*. 3sleep* 1seconds* Waits 1seconds* seconds and then returns. 1seconds* need not be an integer. This also uses 2process-wait*. 3process-wait-with-timeout* 1whostate* 1interval* 1function* &rest 1arguments* This is like 2process-wait* except that if 1interval* sixtieths of a second go by and the application of 1function* to 1arguments* is still returning 2nil*, then 2process-wait-with-timeout* returns anyway. The value returned is the value of applying 1function* to 1arguments*; thus, it is non-2nil* if the wait condition actually occurred, 2nil* for a time-out. If 1interval* is 2nil*, there is no timeout, and this function is then equivalent to 2process-wait*. 3with-timeout* 1(interval* 1timeout-forms...)* 1body...* 1Macro* 1body* is executed with a timeout in effect for 1interval* sixtieths of a second. If 1body* finishes before that much time elapses, the values of the last form in 1body* are returned. If after 1interval* has elapsed 1body* has not completed, its execution is terminated with a 2throw* caught by the 2with-timeout* form. Then the 1timeout-forms* are evaluated and the values of the last one of them are returned. For example, 3(with-timeout ((* 60. 60.) (format *query-io* " ... Yes.") t)* 3 (y-or-n-p "Really do it? (Yes after one minute) "))* is a convenient way to ask a question and assume an answer if the user does not respond promptly. This is a good thing to do for queries likely to occur when the user has walked away from the terminal and expects an operation to finish without his attention. 3process-allow-schedule* Resumes the scheduler momentarily; all other processes will get a chance to run before the current process runs again. 3sys:scheduler-stack-group* 1Constant* This is the stack group in which the scheduler executes. 3sys:clock-function-list* 1Variable* This is a list of functions to be called by the scheduler 60 times a second. Each function is passed one argument, the number of 60ths of a second since the last time that the functions on this list were called. These functions implement various system overhead operations such as blinking the blinking cursor on the screen. Note that these functions are called inside the scheduler, just as are the functions of simple processes (see 4(PROCESSES-1)Process Flavors*). The scheduler calls these functions as often as possible, but never more often than 60 times a second. That is, if there are no processes ready to run, the scheduler calls the clock functions 60 times a second, assuming that, all together, they take less than 1/60 second to run. If there are processes continually ready to run, then the scheduler calls the clock functions as often as it can; usually this is once a second, since usually the scheduler gets control only once a second. 3sys:active-processes* 1Variable* This is the scheduler's data-structure. It is a list of lists, where the car of each element is an active process or 2nil* and the cdr is information about that process. 3sys:all-processes* 1Variable* This is a list of all the processes in existence. It is mainly for debugging. 3si:initial-process* 1Constant* This is the process in which the system starts up when it is booted. 3si:sb-on* &optional 1when* Controls what events cause a sequence break, i.e. when rescheduling occurs. The following keywords are names of events which can cause a sequence break. 2:clock* This event happens periodically based on a clock. The default period is one second. See 2sys:%tv-clock-rate*, 4(SUBPRIMITIVES-3)Microcode Meters*. 2:keyboard* Happens when a character is received from the keyboard. 2:chaos* Happens when a packet is received from the Chaosnet, or transmission of a packet to the Chaosnet is completed. Since the keyboard and Chaosnet are heavily buffered, there is no particular advantage to enabling the 2:keyboard* and 2:chaos* events, unless the 2:clock* event is disabled. With no argument, 2si:sb-on* returns a list of keywords for the currently enabled events. With an argument, the set of enabled events is changed. The argument can be a keyword, a list of keywords, 2nil* (which disables sequence breaks entirely since it is the empty list), or a number, which is the internal mask, not documented here. =Node: 4Locks* =Text: 3LOCKS* A 1lock* is a software construct used for synchronization of two processes. A lock is either held by some process, or is free. When a process tries to seize a lock, it waits until the lock is free, and then it becomes the process holding the lock. When it is finished, it unlocks the lock, allowing some other process to seize it. A lock protects some resource or data structure so that only one process at a time can use it. In the Lisp Machine, a lock is a locative pointer to a cell. If the lock is free, the cell contains 2nil*; otherwise it contains the process that holds the lock. The 2process-lock* and 2process-unlock* functions are written in such a way as to guarantee that two processes can never both think that they hold a certain lock; only one process can ever hold a lock at one time. 3process-lock* 1locative* &optional 1(lock-value* 3current-process1)** 1(whostate* 3"Lock"1)** 1timeout* This is used to seize the lock that 1locative* points to. If necessary, 2process-lock* waits until the lock becomes free. When 2process-lock* returns, the lock has been seized. 1lock-value* is the object to store into the cell specified by 1locative*, and 1whostate* is passed on to 2process-wait*. If 1timeout* is non-2nil*, it should be a fixnum representing a time interval in 60ths of a second. If it is necessary to wait more than that long, an error with condition name 2sys:lock-timeout* is signaled. 3process-unlock* 1locative* &optional 1(lock-value* 3current-process1)** This is used to unlock the lock that 1locative* points to. If the lock is free or was locked by some other process, an error is signaled. Otherwise the lock is unlocked. 1lock-value* must have the same value as the 1lock-value* parameter to the matching call to 2process-lock*, or else an error is signaled. 3sys:lock-timeout* (3error*) 1Condition* This condition is signaled when 2process-lock* waits longer than the specified timeout. It is a good idea to use 2unwind-protect* to make sure that you unlock any lock that you seize. For example, if you write 3(unwind-protect* 3 (progn (process-lock lock-3)* 3 (function-1)* 3 (function-2))* 3 (process-unlock lock-3))* then even if 2function-1* or 2function-2* does a 2throw*, 2lock-3* will get unlocked correctly. Particular programs that use locks often define special forms that package this 2unwind-protect* up into a convenient stylistic device. A higher level locking construct is 2with-lock*: 3with-lock* 1(lock* &key 1norecursive)* 1body...* 1Macro* Executes the 1body* with 1lock* locked. 1lock* should actually be an expression whose value would be the status of the lock; it is used inside 2locf* to get a locative pointer with which the locking and unlocking are done. It is OK for one process to lock a lock multiple times, recursively, using 2with-lock*, provided 1norecursive* is not 2nil*. 1norecursive* should be literally 2t* or 2nil*; it is not evaluated. If it is 2t*, this call to 2with-lock* signals an error if the lock is already locked by the running process. A lower level construct which can be used to implement atomic operations, and is used in the implementation of 2process-lock*, is 2store-conditional*. 3store-conditional* 1location* 1oldvalue* 1newvalue* This stores 1newvalue* into 1location* iff 1location* currently contains 1oldvalue*. The value is 2t* iff the cell was changed. If 1location* is a list, the cdr of the list is tested and stored in. This is in accord with the general principle of how to access the contents of a locative properly, and makes 2(store-conditional (locf (cdr x)) ...)* work. An even lower-level construct is the subprimitive 2%store-conditional*, which is like 2store-conditional* with no error checking, but is faster. =Node: 4Process Queues* =Text: 3PROCESS QUEUES* A process queue is a kind of lock which can record several processes which are waiting for the lock and grant them the lock in the order that they requested it. The queue has a fixed size. If the number of processes waiting remains less than that size then they will all get the lock in the order of requests. If too many processes are waiting then the order of requesting is not remembered for the extra ones, but proper interlocking is still maintained. 3si:make-process-queue* 1name* 1size* Makes and returns a process queue object named 1name*, able to record 1size* processes. The count of 1size* includes the process that owns the lock. 3si:process-enqueue* 1process-queue* &optional 1lock-value* 1who-state* Attempts to lock 1process-queue* on behalf of 1lock-value*. If 1lock-value* is 2nil* then the locking is done on behalf of 2current-process*. If the queue is locked, then 1lock-value* or the current process is put on the queue. Then this function waits for that lock value to reach the front of the queue. When it does so, the lock has been granted, and the function returns. The lock is now locked in the name of 1lock-value* or the current process, until 2si:process-dequeue* is used to unlock it. 1who-state* appears in the who line during the wait. It defaults to 2"Lock"*. 3si:process-deqeueue* 1process-queue* &optional 1lock-value* Unlocks 2process-queue*. 1lock-value* (which defaults to the current process) must be the value which now owns the lock on the queue, or an error occurs. The next process or other object on the queue is granted the lock and its call to 2si:process-enqueue* will therefore return. 3si:reset-process-queue* 1process-queue* Unlocks the queue and clears out the list of things waiting to lock it. 3si:process-queue-locker* 1process-queue* Returns the object in whose name the queue is currently locked, or 2nil* if it is not now locked. =Node: 4Creating a Process* =Text: 3CREATING A PROCESS* There are two ways of creating a process. One is to create a permanent process which you will hold on to and manipulate as desired. The other way is to say simply, ``call this function on these arguments in another process, and don't bother waiting for the result.'' In the latter case you never actually use the process itself as an object. 3make-process* 1name* &key 1...* Creates and returns a process named 1name*. The process is not capable of running until it has been reset or preset in order to initialize the state of its computation. Usually you do not need to specify any of the keyword arguments. The following keyword arguments are allowed: 2:simple-p* Specifying 2t* here gives you a simple process (see 4(PROCESSES-1)Process Flavors*). 2:flavor* Specifies the flavor of process to be created. See 4(PROCESSES-1)Process Flavors*, for a list of all the flavors of process supplied by the system. 2:stack-group* Specifies the stack group the process is to use. If this option is not specified, a stack group is created according to the relevant options below. 2:warm-boot-action* What to do with the process when the machine is booted. See . 2:quantum* See . 2:priority* See . 2:run-reasons* Lets you supply an initial run reason. The default is 2nil*. 2:arrest-reasons* Lets you supply an initial arrest reason. The default is 2nil*. 2:sg-area* The area in which to create the stack group. The default is the value of 2default-cons-area*. 2:regular-pdl-area* 2:special-pdl-area* 2:regular-pdl-size* 2:special-pdl-size*These are passed on to 2make-stack-group*, 4(STACKGROUPS-1)Stack Group Functions*. 2:swap-sv-on-call-out* 2:swap-sv-of-sg-that-calls-me* 2:trap-enable*Specify those attributes of the stack group. You don't want to use these. If you specify 2:flavor*, there can be additional options provided by that flavor. The following functions allow you to call a function and have its execution happen asynchronously in another process. This can be used either as a simple way to start up a process that will run ``forever'', or as a way to make something happen without having to wait for it to complete. When the function returns, the process is returned to a pool of free processes for reuse. The only difference between these three functions is in what happens if the machine is booted while the process is still active. Normally the function to be run should not do any I/O to the terminal, as it does not have a window and terminal I/O will cause it to make a notification and wait for user attention. Refer to 4(STACKGROUPS-1)Input/Output in Stack Groups* for a discussion of the issues. 3process-run-function* 1name-or-options* 1function* &rest 1args* Creates a process, presets it to apply 1function* to 1args*, and starts it running. The value returned is the new process. The process is killed if 1function* returns; by default, it is also killed if it is reset. Example: 3(defun background-print (file)* 3 (process-run-function "Print" 'hardcopy-file file))* creates a background process that prints the specified file. 1name-or-options* can be either a string specifying a name for the process or a list of alternating keywords and values that can specify the name and various other parameters. 2:name* This keyword should be followed by a string which specifies the name of the process. The default is 2"Anonymous"*. 2:restart-after-reset* This keyword says what to do to the process if it is reset. 2nil* means the process should be killed; anything else means the process should be restarted. 2nil* is the default. 2:warm-boot-action* What to do with the process when the machine is booted. See . 2:restart-after-boot* This is a simpler way of saying what to do with the process when the machine is booted. If the 2:warm-boot-action* keyword is not supplied or its value is 2nil*, then this keyword's value is used instead. 2nil* means the process should be killed; anything else means the process should be restarted. 2nil* is the default. 2:quantum* See . 2:priority* See . 3process-run-restartable-function* 1name-or-keywords* 1function* &rest 1args* This is the same as 2process-run-function* except that the default is that the process will be restarted if reset or after a warm boot. You can get the same effect by using 2process-run-function* with appropriate keywords.