How to Monitor and Manage Linux Processes

A program loaded into the memory of a Linux computer becomes a process. Processes need to be managed and monitored because they consume system resources like CPU time, memory and disk space. There are also security and safety implications. Monitoring and managing processes is, therefore, an important function of systems administrators.

A process is a running instance of a launched, executable program. It consists of:

• An address space of allocated memory
• Security properties, including ownership credentials and privileges
• One or more execution threads of program code
• The process state

The environment of a process includes:

• Local and global variables
• A current scheduling context
• Allocated system resources, such as file descriptors and network ports

An existing process duplicates its own address space (fork) to create a new (child) process structure. Every new process has a unique process ID (PID) for tracking and for security. The PID and parent process (PPID) are elements of a new process environment. Any process may create its own child process. All processes are descendants of the first system process, which is systemd in the case of the Red Hat Enterprise Linux 7 system.

Through the for routine, a child process inherits security identities, the previous and current file descriptors, port and resource privileges, environment variables and program code. A child process may then execute its own program code. Normally, a parent process sleeps while the child process runs, setting a request (wait) to be signalled when the child process completes. On exiting, the child process closes or discards its resources and environment; the remainder is referred to as a zombie. The parent, which is signalled awake when the child exits, cleans the remaining structure and continues with executing its own program code.

Process states

In a multi-tasking operating system, each CPU (or CPU core) can work on one process at a single point in time. As a process runs, its immediate requirements for CPU time and resource allocation change. Processes are assigned as states, which change with the circumstances. The Linux process states are explained in Table 1.

Listing processes

The ps command is used for listing current processes. The command can provide detailed process information including:

• The user identification (UID), which determines process privileges
• The unique process identification
• The CPU and real-time already expended
• How much memory the process has allocated in various locations
• The location of process STDOUT known as the controlling terminal
• The state of the current process

Important: The Linux version of ps supports three options/formats, including:

• UNIX (POSIX) options, which may be grouped and must be preceded by a dash
• BSD options, which may be grouped and must not be used with a dash
• GNU long options, which are preceded by two dashes

For example, ps –aux is not the same as ps aux.

A common display listing (options aux) displays all processes, with columns that users will be interested in; this includes processes without a controlling terminal. A long listing (options lax) provides more technical details, but may display faster by avoiding the user name lookup. A similar UNIX syntax uses the options –ef to display all processes. The output of some commands is shown in Figures 1, 2 and 3.

By default, ps with no option selects all the processes with the same effective user ID (EUID) as the current user, and is associated with the same terminal where ps was invoked.

• Processes in brackets (usually at the top) are scheduled kernel threads.
• ps displays once. The man page of top (1) can be referred to for a repetitive updated process display.
• ps can be displayed in tree format to view parent/child relationships.
• The default output is unsorted. The display order matches that of the system process table, which reuses table rows as processes die and new ones are created. The output may appear chronological, but this is not guaranteed unless explicitly stated –O, –sort options are used.

Controlling jobs

Jobs and sessions: Job control is a command shell feature allowing a single shell instance to run and manage multiple commands. Without job control, a parent shell forks a child process to run a command, sleeping until the child process exits. When the shell prompt redisplays, the parent shell returns. With job control, commands can be selectively suspended, resumed and run asynchronously, allowing the shell to return for additional commands while the child process can run.

A foreground process is a command running in a terminal window. The terminal device ID (tty) is the process’s controlling terminal. Foreground processes receive keyboard-generated input and signals, and are allowed to read from or write to the terminal (e.g., via stdin and stdout).

A process session is created when a terminal or console first opens (e.g., at login or by invoking a new terminal instance). All processes (e.g., the first command shell, its children and pipelines) initiated from that terminal share the same session ID. Within that session, only one process can be in the foreground at a time.

A background process is started without a controlling terminal because it has no need for terminal interaction. In a ps listing, such processes (e.g., services, daemons and kernel process threads) display a question mark (?) in the TTY column, while background processes that (improperly) attempt to read from or write to the terminal may be suspended.

Running jobs in the background

Any command can be started in the background by appending an ampersand (&) to the command line. The bash shell displays a job number (unique to the session) and the PID of the new child process. The command shell does not wait for the child process and redisplays the shell prompt.

The bash command shell tracks jobs, per session, in a table displayed with the jobs command as shown in Figure 4.

Background jobs can reconnect to the controlling terminal by being brought to the foreground using the fg command with the job ID (%job number).

The example sleep command is now running on the controlling terminal. The command shell is again asleep, waiting for this child process to exit. To resend it to the background, or to send any command in which the trailing ampersand was not originally included, send the keyboard generated suspend request (Ctrl-z) to the process. The suspend request takes effect immediately. The job is placed in the background. The pending output and keyboard typeahead are discarded.

The ps option j displays job information, including the initial command shell of each session. Since the sleep example command is currently suspended, the state flag displayed is T.

To restart the process in the background, use the bg command with the same job ID. The command shell will warn a user who attempts to exit a terminal window (session) with suspended jobs. If the user tries exiting again immediately, the suspended jobs are killed.

Process control using signals

A signal is a software interrupt delivered to a process. Signals report events to an executing program. Events that generate a signal can be an error, or can be external events (like an I/O request or expired timer) or can be generated by explicit requests (e.g., using a signal-sending command or by a keyboard sequence).

Table 2 lists the fundamental signals used by systems administrators for routine process management.

Note: Signal numbers vary on different Linux hardware platforms, but signal names and meanings are standardised. For command use, it is advised to use signal names instead of numbers.

Each signal has a default action, which is usually one of the following:

• Term – Causes a program to terminate (exit) at once.
• Core – Causes a program to save a memory image (core dump), and then terminate.
• Stop – Causes a program to stop executing (suspend) and waits to continue (resume).
• A program can be written for expected event signals by implementing handler routines to ignore, replace or extend a signal’s default action.

Commands for sending signals by explicit request

Users signal their current foreground process by typing a keyboard control sequence to suspend (Ctrl –z), kill (Ctrl-c), or core dump (Ctrl-\) the process. To signal a background process or processes in a different session requires a signal-sending command.

Signals can be specified either by name (-HUP or –SIGHUP) or by number (like -1). Users may kill their own processes but root privileges are required to kill processes owned by others.

• The kill command sends a signal to a process using ID. Despite its name, the kill command can be used for sending any signal, not just those for terminating programs.
• Use the killall command to send a signal to one or more processes with matching selection criteria, such as a command name, processes owned by a specific user or system-wide processes.
• The pkill command, like killall, can signal multiple processes. Pkill uses advanced selection criteria, which can include a combination of the following:

1. Command – Processes with a pattern-matched command name
2. UID – Processes owned by a Linux user account, effective or real
3. GID – Processes owned by a Linux group account, effective or real
4. Parent – Child processes of a specific parent process
5. Terminal – Processes running on a specific controlling terminal