Three Comics For Understanding Unix Shell

栏目: IT技术 · 发布时间: 4年前

内容简介:I just optimized Oil's runtime by reducing the number of processes that it starts. Surprisingly, you can implement shell features like pipelines and subshells with more than one "process topology".I described these optimizationsThat post feels dense, so le

I just optimized Oil's runtime by reducing the number of processes that it starts. Surprisingly, you can implement shell features like pipelines and subshells with more than one "process topology".

I described these optimizations on Zulip , and I want to write a post called Oil Starts Fewer Processes Than Other Shells .

That post feels dense, so let's first review some background knowledge , with the help of several great drawings from Julia Evans .

Table of Contents

1. The Shell Language and the Unix Kernel

2. The Shell Controls What's in a Process

Interlude: Two Kinds of Processes

3. A Pipe Has Two Ends

What I'd Like to Draw

Appendix: Related Comics

1. The Shell Language and the Unix Kernel

Three Comics For Understanding Unix Shell

User space and kernel space are key concepts for understanding shell. Why?

  • The shell mostly lives in user space , but common constructs like ls /bin and $(dirname $x) require support from the kernel to start processes.
  • That is, a shell is a process that starts other processes. Unlike programs in say Python or C, almost all shell programs involve many processes. (I'll explain why this is agood thing later.)
  • In old operating systems, command shells were built into the kernel. The Unix/Multics design of putting the shell in user space enables Oil. We don't have to modify Linux to use Oil, and it can run on other Unix systems like OS X and FreeBSD.

2. The Shell Controls What's in a Process

Three Comics For Understanding Unix Shell

Shell has dedicated syntax andbuiltin commands to manipulate its own process state , and thus the inherited state of child processes. You can:

  • Control environment vars with export .
  • Change the working directory with cd , pushd , and popd . Retrieve it with pwd or $PWD .
  • Read the process ID from the $$ variable, and the parent PID from $PPID .
  • Use the trap builtin to register code to run when the process receives a signal (e.g. SIGINT or Ctrl-C).
  • Wait for child processes to stop with the wait builtin.
  • Use the shell keyword time to ask the kernel how long a process has taken to run. (See the last pane of the first comic.)

In other words, shell is a thin layer over the process abstraction provided by the kernel . Processes used to be thought of as virtual machines , although that term now has a different connotation.

Interlude: Two Kinds of Processes

It's not obvious from the syntax, but there are two different kinds of processes in a shell program:

  1. Those that run a different executable, i.e. assembly code that's not in /bin/sh or /usr/local/bin/oil . Examples:

    $ ls          # 1 new process and 1 executable (usually)
    $ ls | wc -l  # 2 new processes and 2 executables (usually)

    After calling fork() to create a process, the shell also calls exec() to run code in /bin/ls or /usr/bin/wc . The exec() system call loads and starts a new "binary image" in the current process.

  2. Those that run the same executable. For example, the left-hand-side of this pipeline

    # at least 1 new process, but no new executables
    $ { echo a; echo b; } | read x

    denotes an independent copy of the shell interpreter, created with the fork() system call. No exec() call is needed.

    Is it inefficient to start a process for those two statements? Not really, no. See this related comic: Copy On Write .

The (usually) qualifiers above are what the next post is about. I optimized the usage of fork() and exec() syscalls in Oil. I was surprised to learn that all shells do this to some extent.

3. A Pipe Has Two Ends

I really like the red and blue dots in this drawing. It's an intuitive way of explaining the pipe() system call, which forces us to understand file descriptors .

I think of file descriptors as "pointers" from user space into the kernel. They can't literally be pointers, because user space and kernel each have their own memory. So instead they're small integers that are offsets into a table in the kernel.

Related Comics:

The next post will discuss shopt -s lastpipe , abash option that implementszsh-like pipeline behavior.

What I'd Like to Draw

I still want to get more done in 2020 by cutting scope , but I'd like to illustrate these related concepts:

  • There are three processes involved in ls | wc -l .

    1. The shell, which calls pipe() . Importantly, the file descriptors returned by pipe() are inherited by children.
    2. Its child ls , which uses the red end of the pipe it inherits.
    3. Its child wc -l , which uses the blue end.
  • The difference between fork() and exec() . I explained this above, but a drawing would make it clearer. If you know of one, let me know in the comments .

    • The shell does important things between fork() and exec() , like manipulate file descriptors.
    • Windows doesn't have the distinction between creating a process and loading executable code into it.

Next

I'll show that Oil starts fewer processes than other shells for snippets like:

date; date
(date)
date | wc -l
echo $(date)

And I'll describe how I measured this withstrace.

Appendix: Related Comics

These comics are also related to Oil, and I may reference them in future posts:

For what it's worth, I just bought a printed set of zines, Your Linux Toolbox , which have related but different content. You can also buy e-books on wizardzines.com .


以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

用户体验的要素

用户体验的要素

Jesse James Garrett / 范晓燕 / 机械工业出版社 / 2007年10月 / 25.00

这不是一本关于“怎样做(How-to)”的书。有很多很多讨论如何建设网站的书,这本不是。 这不是一本关于技术的书。在这里你找不到一行代码。 这不是一本有答案的书。相反,这本书说的是“如何提出正确的问题”。 这本书将告诉你,在你阅读其他书籍的之前,你需要提前了解什么。如果你需要一个大的概念,如果你需要了解用户体验设计师所做出的决策的环境,这本书很适合你。 这本书经过精心设计,......一起来看看 《用户体验的要素》 这本书的介绍吧!

随机密码生成器
随机密码生成器

多种字符组合密码

Markdown 在线编辑器
Markdown 在线编辑器

Markdown 在线编辑器

正则表达式在线测试
正则表达式在线测试

正则表达式在线测试