aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorSebastiano Tronto <sebastiano@tronto.net>2022-09-13 16:14:21 +0200
committerSebastiano Tronto <sebastiano@tronto.net>2022-09-13 16:14:21 +0200
commitc56a25130c9335b700252df3979116f4049873f0 (patch)
tree89dbc114e2bbb05d6b88c858244273d8986e4668
parente37c82d7071daa6073e2d0611e9ceb521c9cf204 (diff)
downloadsebastiano.tronto.net-c56a25130c9335b700252df3979116f4049873f0.tar.gz
sebastiano.tronto.net-c56a25130c9335b700252df3979116f4049873f0.zip
Added blog post
-rw-r--r--src/blog/2022-09-13-sh-1/sh-1.md385
-rw-r--r--src/blog/blog.md1
-rw-r--r--src/blog/feed.xml7
3 files changed, 393 insertions, 0 deletions
diff --git a/src/blog/2022-09-13-sh-1/sh-1.md b/src/blog/2022-09-13-sh-1/sh-1.md
new file mode 100644
index 0000000..92b2404
--- /dev/null
+++ b/src/blog/2022-09-13-sh-1/sh-1.md
@@ -0,0 +1,385 @@
1# The man page reading club: sh(1) - part 1: shell grammar
2
3After [last time's short entry](../2022-07-07-shutdown) and a
4relatively long hiatus, we are back in business with a big one!
5
6## A new day
7
8*After a good night of sleep and a cup of whatever people call
9coffee in the post-apocalypse, you turn your computer back on. You
10would like to learn more stuff, but you are unsure where to start
11from. You vaguely remember a `man afterboot` being mentioned
12somewhere, so you start from there.*
13
14```
15DESCRIPTION
16 Starting out
17 This document attempts to list items for the system administrator to
18 check and set up after the installation and first complete boot of the
19 system. The idea is to create a list of items that can be checked off so
20 that you have a warm fuzzy feeling that something obvious has not been
21 missed. A basic knowledge of UNIX is assumed, otherwise type:
22
23 $ help
24```
25
26*You do have some knowledge of UNIX, someone might call it "basic",
27but you believe "scattered" is a more appropriate adjective. In any
28case, a review won't hurt. You type the command*
29
30```
31$ help
32```
33
34*And a manual page shows up. You could have typed `man help` instead
35to get the same result. After skimming throught the introduction,
36you discover something worth digging into.*
37
38```
39 The Unix shell
40 After logging in, some system messages are typically displayed, and then
41 the user is able to enter commands to be processed by the shell program.
42 The shell is a command-line interpreter that reads user input (normally
43 from a terminal) and executes commands. There are many different shells
44 available; OpenBSD ships with csh(1), ksh(1), and sh(1). Each user's
45 shell is indicated by the last field of their corresponding entry in the
46 system password file (/etc/passwd).
47```
48
49*You have a look at `/etc/passwd` and you see that your user's shell
50is `ksh`. So you type `man ksh` and start reading.*
51
52```
53DESCRIPTION
54 ksh is a command interpreter intended for both interactive and shell
55 script use. Its command language is a superset of the sh(1) shell
56 language.
57```
58
59*You are quite rusty on the Math jargon - some of your friends used
60to talk like that in real life, but you never bothered to learn -
61but "superset" sounds like "it is larger than". Is this another
62[`less` vs `more`](../2022-06-08-more) kind of thing, where one
63command is just a simpler version of the other? Let's see what
64`sh(1)` has to say about it*
65
66```
67 This version of sh is actually ksh in disguise.
68```
69
70*Ah-ah! Exactly as you thought. Just like the other time, you prefer
71to go with the simpler version. Enough of this "fun is precious"
72bullshit, you want to learn as soon as possible!*
73
74## sh(1)
75
76*Follow along at [man.openbsd.org](https://man.openbsd.org/OpenBSD-7.1/sh)*
77
78Despite having less features than more complex shells like `ksh`
79or `bash`, the manual page for `sh` is still very long. So we are
80going to split it into two or more parts.
81
82The main sections I intend to cover are BUILTINS, SHELL GRAMMAR and
83COMMANDS. Parts of SPECIAL PARAMETERS and ENVIRONMENT are quoted
84and explained in other sections, so I am probably going to skip
85these too. I think we can skip the invocation options, since we
86are mostly going to run our shell implicitly when logging in or
87when executing a script. Finally, COMMAND HISTORY AND COMMAND LINE
88EDITING is best explained after we cover `vi(1)`, so we'll skip
89that too. This still leaves with a big chunk of the man page to
90discuss.
91
92A technical manual page is not a novel: the content is often laid
93out in an arbitrary order, to make it easier to find what you are
94looking for (e.g. in alphabetic order) and not to make a top-to-bottom
95read entertaining. So I felt like reordering things a bit: not only
96I will cover the sections in a differ order than what you find in
97the manual page, but I will also shuffle the content of each section
98when it make sense to me.
99
100Since I am very much a theoretical, grammar-first kind of person,
101my totally subjective best way to dive into this is starting with
102the grammar section!
103
104## Part 1: shell grammar
105
106After reading the input, either from a file or from the standard
107input, `sh` does the following:
108
1091. It breaks the input into words and operators (special characters).
1102. It expands the text according to the rules in **Expansion** section below.
1113. It splits the text into commands and arguments.
1124. It performs input / output redirection (see the **Redirection** section below).
1135. It runs the commands.
1146. It waits for the commands to complete and collects the exit status.
115
116The next three sub-sections (Redirection, Expansion and Quoting) are found
117in the exact opposite order in the manual page.
118
119### Redirection
120
121Together with *piping*, which we will cover in one of the next episodes,
122redirection is one of the key features of UNIX.
123
124```
125 Redirection is used to open, close, or otherwise manipulate files, using
126 redirection operators in combination with numerical file descriptors. A
127 minimum of ten (0-9) descriptors are supported; by convention standard
128 input is file descriptor 0, standard output file descriptor 1, and
129 standard error file descriptor 2.
130```
131
132If the number `[n]` is not specified, it defaults to either `0`
133(standard input) or `1` (standard output) depending if the angled
134brackets are pointing to the left or to the right.
135
136The main redirectors are `[n]<file`, to read input from `file`
137instead of typing it in manually, and its counterpart `[n]>file`
138to write standard output (or whatever is described by the file
139descriptor `[n]`) to file. For example, if you want to log every
140error message of `command` to `file.log`, you can use
141
142```
143$ command 2>file.log
144```
145
146The `[n]>>file` redirector is similar, but it appends stuff to
147`file` instead of overwriting it. Both `>` and `>>` create the file
148if it does not exist.
149
150There is also `[n]<<`:
151
152```
153[n]<< This form of redirection, called a here document, is used to copy
154 a block of lines to a temporary file until a line matching
155 delimiter is read. When the command is executed, standard input
156 is redirected from the temporary file to file descriptor n, or
157 standard input by default.
158```
159
160For example
161
162```
163$ cat <<BYEBYE
164> one line,
165> another line
166> and so on
167> BYEBYE
168```
169
170Outputs those three lines. It is useful in shell scripts, when you
171want to output a block of text. The variant `[n]<<-` strips out
172`Tab` characters.
173
174Another useful one is `[n]>&fd`, which "merges" the file descriptors
175`[n]` and `fd`. For example, if you want to make your command
176completely silent, you can merge standard output and standard error
177and redirect them both to `/dev/null` with
178
179```
180$ command >&2 >/dev/null
181```
182
183### Expansion
184
185There are essentially five kinds of expansion that the shell performs:
186tilde expansion, parameter expansion, command expansion, arithmetic
187expansion and filename expansion.
188
189**Tilde expansion** is quite straightforward, so let's just quote
190the man page:
191
192```
193 Firstly, tilde expansion occurs on words beginning with the `~'
194 character. Any characters following the tilde, up to the next colon,
195 slash, or blank, are taken as a login name and substituted with that
196 user's home directory, as defined in passwd(5). A tilde by itself is
197 expanded to the contents of the variable HOME. This notation can be used
198 in variable assignments, in the assignment half, immediately after the
199 equals sign or a colon, up to the next slash or colon, if any.
200
201 PATH=~alice:~bob/jobs
202```
203
204**Parameters** can be variable names or special parameters. Variables
205can be assigned with the simple syntax `variable=value` and their
206value can be "accessed" with `$variable`. In case of ambiguity you
207need to enclose the variable name in curly braces `{}`: say you
208want to type the string `subroutines` and you have a variable
209`prefix=sub`. The shell will complain at a `$prefixroutines` about
210there being no variable with such name, so you have to use
211`${prefix}routines`.
212
213The most useful special parameters are:
214
215* Numbers `1`, `2`, `3`... that refer to the *positional parameters*:
216
217```
218 These parameters are set when a shell, shell script, or shell function is
219 invoked. Each argument passed to a shell or shell script is assigned a
220 positional parameter, starting at 1, and assigned sequentially.
221```
222
223* The number `0`, which refers to the name of the shell or of the shell
224 script being executed.
225* The symbols `@` and `*` which expand to all positional parameters
226 at once; they behave differently when enclosed in double quotes:
227 with `"$@"` the parameters are split into fields, with `"$*"` they are not.
228
229There are some useful constructs to expand a parameter in special
230ways. The constructs `${parameter:-[word]}` and `${parameter:=[word]}`
231expand to `[word]` if `parameter` is unset or empty, with the second
232one also assigning the value `[word]` to `parameter` for subsequent
233use. Instead, `${parameter:+[word]}` expands to `[word]` *unless*
234`parameter` is unset or empty, in which case it expands to the empty
235string. In all these cases, if the colon is omitted `[word]` is
236substituted only if `parameter` is unset (not if it is empty).
237
238Another useful one is `${#parameter}`, which expands to the length
239of `parameter`. Finally there are some constructs that can be used
240to remove prefixes or suffixes from the expansion of a parameter:
241
242| Construct | Effect |
243|:---:|:---:|
244| `${parameter%[word]}` | Delete smallest possible suffix matching word |
245| `${parameter%%[word]}` | Delete largest possible suffix matching word |
246| `${parameter#[word]}` | Delete smallest possible prefix matching word |
247| `${parameter##[word]}` | Delete largest possible prefix matching word |
248
249What unfortunately is not explained in the man page of `sh(1)` (but
250can be found in that of `ksh(1)`) is that `[word]` in this case can
251be a *pattern*. See [glob(7)](https://man.openbsd.org/OpenBSD-7.1/glob.7)
252for a description of patterns, which are the same that are used for
253filename expansion (with the exception that slashes and dots are
254treated as normal characters).
255
256For example, using `*` which means "any sequence of zero or more
257characters":
258
259```
260$ x="we can,separate,stuff,with commas"
261$ echo ${x#*,}
262separate,stuff,with commas
263$ echo ${x##*,}
264with commas
265```
266
267Then there is **command expansion**:
268
269```
270 Command expansion has a command executed in a subshell and the results
271 output in its place. The basic format is:
272
273 $(command)
274 or
275 `command`
276
277 The results are subject to field splitting and pathname expansion; no
278 other form of expansion happens. If command is contained within double
279 quotes, field splitting does not happen either.
280```
281
282**Arithmetic expansion** uses the syntax `$((expression))`. An
283`expression` can be a combination of integers (no floating point
284arithmetic in the shell!), parameter names and the usual arithmetic
285operations. I won't copy them here; if you are familiar with C or
286C-like languages, you can use pretty much all the operations you
287are used to, including logic operations (resulting in 0 or 1),
288assignment operations like `+=` and bitwise operations like `~`,
289`&` and `<<`. Even the *ternary if* `expression ? expr1 : expr2`
290is available.
291
292Finally, **filename expansion** uses the aforementioned rules of
293[glob(7)](https://man.openbsd.org/OpenBSD-7.1/glob.7) to expand
294filenames. To sum them up:
295
296* As we have already seen, `*` expands to any sequence of characters.
297* `?` matches any single character.
298* `[..]` matches any character in place of the double dot, or any
299 character *not* listed if the first is an exclamation mark.
300* `[[:class:]]` matches any character of a certain class; for example
301 `class` could be `alnum` for alphanumeric characters or `upper` for
302 uppercase letters.
303* `[x-y]` matches any character in the range between `x` and `y`.
304
305To illustrate what all of this means, check this out (the command `ls` is
306used to list all files in the current directory):
307
308```
309$ ls
310box file3 mbox typescript
311count_args.sh file4 mnt videos
312file1 git music
313file2 mail phone-laptop-swap
314$ echo m*
315mail mbox mnt music
316$ echo m???
317mail mbox
318$ echo file[2-4]
319file2 file3 file4
320```
321
322### Quoting
323
324Sometimes we may want to write some of the special characters
325described above, such as dollar signs, without their special meaning.
326You can do so by *escaping*, or *quoting* them. There are essentially
327three ways to quote a character or a group of characters:
328
329* Backslash:
330
331```
332 A backslash (\) can be used to quote any character except a newline. If
333 a newline follows a backslash the shell removes them both, effectively
334 making the following line part of the current one.
335```
336
337This means that a backslash can also effectively be used to split
338long lines into multiple lines, for example for ease of editing a
339shell script.
340
341* Single quotes:
342
343```
344 A group of characters can be enclosed within single quotes (') to quote
345 every character within the quotes.
346```
347
348* And double quotes:
349
350```
351 A group of characters can be enclosed within double quotes (") to quote
352 every character within the quotes except a backquote (`) or a dollar sign
353 ($), both of which retain their special meaning. A backslash (\) within
354 double quotes retains its special meaning, but only when followed by a
355 backquote, dollar sign, double quote, newline, or another backslash. An
356 at sign (@) within double quotes has a special meaning (see SPECIAL
357 PARAMETERS, below).
358```
359
360Basically the difference between single and double quotes is that
361the former turn literally everything they enclose into simple text,
362while the latter still parse and expand some special characters
363(for example the dollar sign `$` for variables).
364
365As an addition, remember that anything enclosed in single or double
366quotes is considered a single field (word). This was briefly mentioned
367in the Expansion section, but I skipped it. To illustrate what I
368mean, let's write a short script and run it first with some words
369as arguments and then with the same words enclosed in quotes:
370
371```
372$ echo 'echo $#' > count_args.sh
373$ count_args.sh how many words are there
3745
375$ count_args.sh "how many words are there"
3761
377```
378
379## Until next time
380
381This was a very long post, but it made sense to keep all the grammar
382rules together. To finish this manual page we are going to need
383another long post, or two shorter ones.
384
385See you next time!
diff --git a/src/blog/blog.md b/src/blog/blog.md
index 7af22f3..2a02644 100644
--- a/src/blog/blog.md
+++ b/src/blog/blog.md
@@ -2,6 +2,7 @@
2 2
3[RSS Feed](feed.xml) 3[RSS Feed](feed.xml)
4 4
5* 2022-09-13 [The man page reading club: sh(1) - part 1: shell grammar](2022-09-13-sh-1)
5* 2022-09-10 [Long live netbooks!](2022-09-10-netbooks) 6* 2022-09-10 [Long live netbooks!](2022-09-10-netbooks)
6* 2022-09-05 [Pipe man into col -b to get rid of \^H](2022-09-05-man-col) 7* 2022-09-05 [Pipe man into col -b to get rid of \^H](2022-09-05-man-col)
7* 2022-08-14 [How I update my website](2022-08-14-website) 8* 2022-08-14 [How I update my website](2022-08-14-website)
diff --git a/src/blog/feed.xml b/src/blog/feed.xml
index bb05925..3b718d2 100644
--- a/src/blog/feed.xml
+++ b/src/blog/feed.xml
@@ -9,6 +9,13 @@ Thoughts about software, computers and whatever I feel like sharing
9</description> 9</description>
10 10
11<item> 11<item>
12<title>The man page reading club: sh(1) - part 1: shell grammar</title>
13<link>https://sebastiano.tronto.net/blog/2022-09-13-sh-1</link>
14<description>The man page reading club: sh(1) - part 1: shell grammar</description>
15<pubDate>2022-09-13</pubDate>
16</item>
17
18<item>
12<title>Long live netbooks!</title> 19<title>Long live netbooks!</title>
13<link>https://sebastiano.tronto.net/blog/2022-09-10-netbooks</link> 20<link>https://sebastiano.tronto.net/blog/2022-09-10-netbooks</link>
14<description>Long live netbooks!</description> 21<description>Long live netbooks!</description>

Generated with cgit - Back to sebastiano.tronto.net