内容简介:I just optimized Oil's runtime by reducing the number of processes that it starts. Surprisingly, you can implement shell features like pipelines and subshells with more than one "process topology".I described these optimizationsThat post feels dense, so le
I just optimized Oil's runtime by reducing the number of processes that it starts. Surprisingly, you can implement shell features like pipelines and subshells with more than one "process topology".
I described these optimizations on Zulip , and I want to write a post called Oil Starts Fewer Processes Than Other Shells .
That post feels dense, so let's first review some background knowledge , with the help of several great drawings from Julia Evans .
Table of Contents
1. The Shell Language and the Unix Kernel
2. The Shell Controls What's in a Process
Interlude: Two Kinds of Processes
3. A Pipe Has Two Ends
What I'd Like to Draw
Appendix: Related Comics
1. The Shell Language and the Unix Kernel
User space and kernel space are key concepts for understanding shell. Why?
- The shell mostly lives in user space , but common constructs like
ls /bin
and$(dirname $x)
require support from the kernel to start processes. - That is, a shell is a process that starts other processes. Unlike programs in say Python or C, almost all shell programs involve many processes. (I'll explain why this is agood thing later.)
- In old operating systems, command shells were built into the kernel. The Unix/Multics design of putting the shell in user space enables Oil. We don't have to modify Linux to use Oil, and it can run on other Unix systems like OS X and FreeBSD.
2. The Shell Controls What's in a Process
Shell has dedicated syntax andbuiltin commands to manipulate its own process state , and thus the inherited state of child processes. You can:
- Control environment vars with
export
. - Change the working directory with
cd
,pushd
, andpopd
. Retrieve it withpwd
or$PWD
. - Read the process ID from the
$$
variable, and the parent PID from$PPID
. - Use the
trap
builtin to register code to run when the process receives a signal (e.g.SIGINT
or Ctrl-C). - Wait for child processes to stop with the
wait
builtin. - Use the shell keyword
time
to ask the kernel how long a process has taken to run. (See the last pane of the first comic.)
In other words, shell is a thin layer over the process abstraction provided by the kernel . Processes used to be thought of as virtual machines , although that term now has a different connotation.
Interlude: Two Kinds of Processes
It's not obvious from the syntax, but there are two different kinds of processes in a shell program:
-
Those that run a different executable, i.e. assembly code that's not in
/bin/sh
or/usr/local/bin/oil
. Examples:$ ls # 1 new process and 1 executable (usually) $ ls | wc -l # 2 new processes and 2 executables (usually)
After calling
fork()
to create a process, the shell also callsexec()
to run code in/bin/ls
or/usr/bin/wc
. Theexec()
system call loads and starts a new "binary image" in the current process. -
Those that run the same executable. For example, the left-hand-side of this pipeline
# at least 1 new process, but no new executables $ { echo a; echo b; } | read x
denotes an independent copy of the shell interpreter, created with the
fork()
system call. Noexec()
call is needed.Is it inefficient to start a process for those two statements? Not really, no. See this related comic: Copy On Write .
The (usually)
qualifiers above are what the next post is about. I optimized the usage of fork()
and exec()
syscalls in Oil. I was surprised to learn that all shells do this to some extent.
3. A Pipe Has Two Ends
I really like the red and blue dots in this drawing. It's an intuitive way of explaining the pipe()
system call, which forces us to understand file descriptors .
I think of file descriptors as "pointers" from user space into the kernel. They can't literally be pointers, because user space and kernel each have their own memory. So instead they're small integers that are offsets into a table in the kernel.
Related Comics:
The next post will discuss shopt -s lastpipe
, abash option that implementszsh-like pipeline behavior.
What I'd Like to Draw
I still want to get more done in 2020 by cutting scope , but I'd like to illustrate these related concepts:
-
There are three processes involved in
ls | wc -l
.- The shell, which calls
pipe()
. Importantly, the file descriptors returned bypipe()
are inherited by children. - Its child
ls
, which uses the red end of the pipe it inherits. - Its child
wc -l
, which uses the blue end.
- The shell, which calls
-
The difference between
fork()
andexec()
. I explained this above, but a drawing would make it clearer. If you know of one, let me know in the comments .- The shell does important things between
fork()
andexec()
, like manipulate file descriptors. - Windows doesn't have the distinction between creating a process and loading executable code into it.
- The shell does important things between
Next
I'll show that Oil starts fewer processes than other shells for snippets like:
date; date (date) date | wc -l echo $(date)
And I'll describe how I measured this withstrace.
Appendix: Related Comics
These comics are also related to Oil, and I may reference them in future posts:
- libc . The C standard library is the only required dependency of a shell.
- Bash Tips . The first point, "no spaces", echoes The Simplest Explanation of Oil .
- The Page Table (again). Processes are implemented with dedicated hardware (an MMU ). That's one thing that makes them powerful and reliable.
For what it's worth, I just bought a printed set of zines, Your Linux Toolbox , which have related but different content. You can also buy e-books on wizardzines.com .
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网
猜你喜欢:本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们。
人工智能产品经理——AI时代PM修炼手册
张竞宇 / 电子工业出版社 / 2018-6 / 59
随着人工智能热潮的兴起,企业对人工智能领域产品经理的人才需求也开始井喷,人工智能产品经理成为顺应时代潮流的重要人力资源。实际上,人工智能确实给现有的产品和服务带来了全方位的升级,这也给产品经理从业人员提出了更高的要求,是关注人工智能产品的产品经理们面临的一次关键转型考验。 《人工智能产品经理——AI时代PM修炼手册》从知识体系、能力模型、沟通技巧等方面帮助大家系统地梳理了人工智能产品经理所必......一起来看看 《人工智能产品经理——AI时代PM修炼手册》 这本书的介绍吧!