Three Comics For Understanding Unix Shell

栏目: IT技术 · 发布时间: 4年前

内容简介:I just optimized Oil's runtime by reducing the number of processes that it starts. Surprisingly, you can implement shell features like pipelines and subshells with more than one "process topology".I described these optimizationsThat post feels dense, so le

I just optimized Oil's runtime by reducing the number of processes that it starts. Surprisingly, you can implement shell features like pipelines and subshells with more than one "process topology".

I described these optimizations on Zulip , and I want to write a post called Oil Starts Fewer Processes Than Other Shells .

That post feels dense, so let's first review some background knowledge , with the help of several great drawings from Julia Evans .

Table of Contents

1. The Shell Language and the Unix Kernel

2. The Shell Controls What's in a Process

Interlude: Two Kinds of Processes

3. A Pipe Has Two Ends

What I'd Like to Draw

Appendix: Related Comics

1. The Shell Language and the Unix Kernel

Three Comics For Understanding Unix Shell

User space and kernel space are key concepts for understanding shell. Why?

  • The shell mostly lives in user space , but common constructs like ls /bin and $(dirname $x) require support from the kernel to start processes.
  • That is, a shell is a process that starts other processes. Unlike programs in say Python or C, almost all shell programs involve many processes. (I'll explain why this is agood thing later.)
  • In old operating systems, command shells were built into the kernel. The Unix/Multics design of putting the shell in user space enables Oil. We don't have to modify Linux to use Oil, and it can run on other Unix systems like OS X and FreeBSD.

2. The Shell Controls What's in a Process

Three Comics For Understanding Unix Shell

Shell has dedicated syntax andbuiltin commands to manipulate its own process state , and thus the inherited state of child processes. You can:

  • Control environment vars with export .
  • Change the working directory with cd , pushd , and popd . Retrieve it with pwd or $PWD .
  • Read the process ID from the $$ variable, and the parent PID from $PPID .
  • Use the trap builtin to register code to run when the process receives a signal (e.g. SIGINT or Ctrl-C).
  • Wait for child processes to stop with the wait builtin.
  • Use the shell keyword time to ask the kernel how long a process has taken to run. (See the last pane of the first comic.)

In other words, shell is a thin layer over the process abstraction provided by the kernel . Processes used to be thought of as virtual machines , although that term now has a different connotation.

Interlude: Two Kinds of Processes

It's not obvious from the syntax, but there are two different kinds of processes in a shell program:

  1. Those that run a different executable, i.e. assembly code that's not in /bin/sh or /usr/local/bin/oil . Examples:

    $ ls          # 1 new process and 1 executable (usually)
    $ ls | wc -l  # 2 new processes and 2 executables (usually)

    After calling fork() to create a process, the shell also calls exec() to run code in /bin/ls or /usr/bin/wc . The exec() system call loads and starts a new "binary image" in the current process.

  2. Those that run the same executable. For example, the left-hand-side of this pipeline

    # at least 1 new process, but no new executables
    $ { echo a; echo b; } | read x

    denotes an independent copy of the shell interpreter, created with the fork() system call. No exec() call is needed.

    Is it inefficient to start a process for those two statements? Not really, no. See this related comic: Copy On Write .

The (usually) qualifiers above are what the next post is about. I optimized the usage of fork() and exec() syscalls in Oil. I was surprised to learn that all shells do this to some extent.

3. A Pipe Has Two Ends

I really like the red and blue dots in this drawing. It's an intuitive way of explaining the pipe() system call, which forces us to understand file descriptors .

I think of file descriptors as "pointers" from user space into the kernel. They can't literally be pointers, because user space and kernel each have their own memory. So instead they're small integers that are offsets into a table in the kernel.

Related Comics:

The next post will discuss shopt -s lastpipe , abash option that implementszsh-like pipeline behavior.

What I'd Like to Draw

I still want to get more done in 2020 by cutting scope , but I'd like to illustrate these related concepts:

  • There are three processes involved in ls | wc -l .

    1. The shell, which calls pipe() . Importantly, the file descriptors returned by pipe() are inherited by children.
    2. Its child ls , which uses the red end of the pipe it inherits.
    3. Its child wc -l , which uses the blue end.
  • The difference between fork() and exec() . I explained this above, but a drawing would make it clearer. If you know of one, let me know in the comments .

    • The shell does important things between fork() and exec() , like manipulate file descriptors.
    • Windows doesn't have the distinction between creating a process and loading executable code into it.

Next

I'll show that Oil starts fewer processes than other shells for snippets like:

date; date
(date)
date | wc -l
echo $(date)

And I'll describe how I measured this withstrace.

Appendix: Related Comics

These comics are also related to Oil, and I may reference them in future posts:

For what it's worth, I just bought a printed set of zines, Your Linux Toolbox , which have related but different content. You can also buy e-books on wizardzines.com .


以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

人工智能产品经理——AI时代PM修炼手册

人工智能产品经理——AI时代PM修炼手册

张竞宇 / 电子工业出版社 / 2018-6 / 59

随着人工智能热潮的兴起,企业对人工智能领域产品经理的人才需求也开始井喷,人工智能产品经理成为顺应时代潮流的重要人力资源。实际上,人工智能确实给现有的产品和服务带来了全方位的升级,这也给产品经理从业人员提出了更高的要求,是关注人工智能产品的产品经理们面临的一次关键转型考验。 《人工智能产品经理——AI时代PM修炼手册》从知识体系、能力模型、沟通技巧等方面帮助大家系统地梳理了人工智能产品经理所必......一起来看看 《人工智能产品经理——AI时代PM修炼手册》 这本书的介绍吧!

CSS 压缩/解压工具
CSS 压缩/解压工具

在线压缩/解压 CSS 代码

MD5 加密
MD5 加密

MD5 加密工具

RGB CMYK 转换工具
RGB CMYK 转换工具

RGB CMYK 互转工具