diff options
| author | Sebastiano Tronto <sebastiano@tronto.net> | 2022-09-13 16:14:21 +0200 |
|---|---|---|
| committer | Sebastiano Tronto <sebastiano@tronto.net> | 2022-09-13 16:14:21 +0200 |
| commit | c56a25130c9335b700252df3979116f4049873f0 (patch) | |
| tree | 89dbc114e2bbb05d6b88c858244273d8986e4668 | |
| parent | e37c82d7071daa6073e2d0611e9ceb521c9cf204 (diff) | |
| download | sebastiano.tronto.net-c56a25130c9335b700252df3979116f4049873f0.tar.gz sebastiano.tronto.net-c56a25130c9335b700252df3979116f4049873f0.zip | |
Added blog post
| -rw-r--r-- | src/blog/2022-09-13-sh-1/sh-1.md | 385 | ||||
| -rw-r--r-- | src/blog/blog.md | 1 | ||||
| -rw-r--r-- | src/blog/feed.xml | 7 |
3 files changed, 393 insertions, 0 deletions
diff --git a/src/blog/2022-09-13-sh-1/sh-1.md b/src/blog/2022-09-13-sh-1/sh-1.md new file mode 100644 index 0000000..92b2404 --- /dev/null +++ b/src/blog/2022-09-13-sh-1/sh-1.md | |||
| @@ -0,0 +1,385 @@ | |||
| 1 | # The man page reading club: sh(1) - part 1: shell grammar | ||
| 2 | |||
| 3 | After [last time's short entry](../2022-07-07-shutdown) and a | ||
| 4 | relatively long hiatus, we are back in business with a big one! | ||
| 5 | |||
| 6 | ## A new day | ||
| 7 | |||
| 8 | *After a good night of sleep and a cup of whatever people call | ||
| 9 | coffee in the post-apocalypse, you turn your computer back on. You | ||
| 10 | would like to learn more stuff, but you are unsure where to start | ||
| 11 | from. You vaguely remember a `man afterboot` being mentioned | ||
| 12 | somewhere, so you start from there.* | ||
| 13 | |||
| 14 | ``` | ||
| 15 | DESCRIPTION | ||
| 16 | Starting out | ||
| 17 | This document attempts to list items for the system administrator to | ||
| 18 | check and set up after the installation and first complete boot of the | ||
| 19 | system. The idea is to create a list of items that can be checked off so | ||
| 20 | that you have a warm fuzzy feeling that something obvious has not been | ||
| 21 | missed. A basic knowledge of UNIX is assumed, otherwise type: | ||
| 22 | |||
| 23 | $ help | ||
| 24 | ``` | ||
| 25 | |||
| 26 | *You do have some knowledge of UNIX, someone might call it "basic", | ||
| 27 | but you believe "scattered" is a more appropriate adjective. In any | ||
| 28 | case, a review won't hurt. You type the command* | ||
| 29 | |||
| 30 | ``` | ||
| 31 | $ help | ||
| 32 | ``` | ||
| 33 | |||
| 34 | *And a manual page shows up. You could have typed `man help` instead | ||
| 35 | to get the same result. After skimming throught the introduction, | ||
| 36 | you discover something worth digging into.* | ||
| 37 | |||
| 38 | ``` | ||
| 39 | The Unix shell | ||
| 40 | After logging in, some system messages are typically displayed, and then | ||
| 41 | the user is able to enter commands to be processed by the shell program. | ||
| 42 | The shell is a command-line interpreter that reads user input (normally | ||
| 43 | from a terminal) and executes commands. There are many different shells | ||
| 44 | available; OpenBSD ships with csh(1), ksh(1), and sh(1). Each user's | ||
| 45 | shell is indicated by the last field of their corresponding entry in the | ||
| 46 | system password file (/etc/passwd). | ||
| 47 | ``` | ||
| 48 | |||
| 49 | *You have a look at `/etc/passwd` and you see that your user's shell | ||
| 50 | is `ksh`. So you type `man ksh` and start reading.* | ||
| 51 | |||
| 52 | ``` | ||
| 53 | DESCRIPTION | ||
| 54 | ksh is a command interpreter intended for both interactive and shell | ||
| 55 | script use. Its command language is a superset of the sh(1) shell | ||
| 56 | language. | ||
| 57 | ``` | ||
| 58 | |||
| 59 | *You are quite rusty on the Math jargon - some of your friends used | ||
| 60 | to talk like that in real life, but you never bothered to learn - | ||
| 61 | but "superset" sounds like "it is larger than". Is this another | ||
| 62 | [`less` vs `more`](../2022-06-08-more) kind of thing, where one | ||
| 63 | command is just a simpler version of the other? Let's see what | ||
| 64 | `sh(1)` has to say about it* | ||
| 65 | |||
| 66 | ``` | ||
| 67 | This version of sh is actually ksh in disguise. | ||
| 68 | ``` | ||
| 69 | |||
| 70 | *Ah-ah! Exactly as you thought. Just like the other time, you prefer | ||
| 71 | to go with the simpler version. Enough of this "fun is precious" | ||
| 72 | bullshit, you want to learn as soon as possible!* | ||
| 73 | |||
| 74 | ## sh(1) | ||
| 75 | |||
| 76 | *Follow along at [man.openbsd.org](https://man.openbsd.org/OpenBSD-7.1/sh)* | ||
| 77 | |||
| 78 | Despite having less features than more complex shells like `ksh` | ||
| 79 | or `bash`, the manual page for `sh` is still very long. So we are | ||
| 80 | going to split it into two or more parts. | ||
| 81 | |||
| 82 | The main sections I intend to cover are BUILTINS, SHELL GRAMMAR and | ||
| 83 | COMMANDS. Parts of SPECIAL PARAMETERS and ENVIRONMENT are quoted | ||
| 84 | and explained in other sections, so I am probably going to skip | ||
| 85 | these too. I think we can skip the invocation options, since we | ||
| 86 | are mostly going to run our shell implicitly when logging in or | ||
| 87 | when executing a script. Finally, COMMAND HISTORY AND COMMAND LINE | ||
| 88 | EDITING is best explained after we cover `vi(1)`, so we'll skip | ||
| 89 | that too. This still leaves with a big chunk of the man page to | ||
| 90 | discuss. | ||
| 91 | |||
| 92 | A technical manual page is not a novel: the content is often laid | ||
| 93 | out in an arbitrary order, to make it easier to find what you are | ||
| 94 | looking for (e.g. in alphabetic order) and not to make a top-to-bottom | ||
| 95 | read entertaining. So I felt like reordering things a bit: not only | ||
| 96 | I will cover the sections in a differ order than what you find in | ||
| 97 | the manual page, but I will also shuffle the content of each section | ||
| 98 | when it make sense to me. | ||
| 99 | |||
| 100 | Since I am very much a theoretical, grammar-first kind of person, | ||
| 101 | my totally subjective best way to dive into this is starting with | ||
| 102 | the grammar section! | ||
| 103 | |||
| 104 | ## Part 1: shell grammar | ||
| 105 | |||
| 106 | After reading the input, either from a file or from the standard | ||
| 107 | input, `sh` does the following: | ||
| 108 | |||
| 109 | 1. It breaks the input into words and operators (special characters). | ||
| 110 | 2. It expands the text according to the rules in **Expansion** section below. | ||
| 111 | 3. It splits the text into commands and arguments. | ||
| 112 | 4. It performs input / output redirection (see the **Redirection** section below). | ||
| 113 | 5. It runs the commands. | ||
| 114 | 6. It waits for the commands to complete and collects the exit status. | ||
| 115 | |||
| 116 | The next three sub-sections (Redirection, Expansion and Quoting) are found | ||
| 117 | in the exact opposite order in the manual page. | ||
| 118 | |||
| 119 | ### Redirection | ||
| 120 | |||
| 121 | Together with *piping*, which we will cover in one of the next episodes, | ||
| 122 | redirection is one of the key features of UNIX. | ||
| 123 | |||
| 124 | ``` | ||
| 125 | Redirection is used to open, close, or otherwise manipulate files, using | ||
| 126 | redirection operators in combination with numerical file descriptors. A | ||
| 127 | minimum of ten (0-9) descriptors are supported; by convention standard | ||
| 128 | input is file descriptor 0, standard output file descriptor 1, and | ||
| 129 | standard error file descriptor 2. | ||
| 130 | ``` | ||
| 131 | |||
| 132 | If the number `[n]` is not specified, it defaults to either `0` | ||
| 133 | (standard input) or `1` (standard output) depending if the angled | ||
| 134 | brackets are pointing to the left or to the right. | ||
| 135 | |||
| 136 | The main redirectors are `[n]<file`, to read input from `file` | ||
| 137 | instead of typing it in manually, and its counterpart `[n]>file` | ||
| 138 | to write standard output (or whatever is described by the file | ||
| 139 | descriptor `[n]`) to file. For example, if you want to log every | ||
| 140 | error message of `command` to `file.log`, you can use | ||
| 141 | |||
| 142 | ``` | ||
| 143 | $ command 2>file.log | ||
| 144 | ``` | ||
| 145 | |||
| 146 | The `[n]>>file` redirector is similar, but it appends stuff to | ||
| 147 | `file` instead of overwriting it. Both `>` and `>>` create the file | ||
| 148 | if it does not exist. | ||
| 149 | |||
| 150 | There is also `[n]<<`: | ||
| 151 | |||
| 152 | ``` | ||
| 153 | [n]<< This form of redirection, called a here document, is used to copy | ||
| 154 | a block of lines to a temporary file until a line matching | ||
| 155 | delimiter is read. When the command is executed, standard input | ||
| 156 | is redirected from the temporary file to file descriptor n, or | ||
| 157 | standard input by default. | ||
| 158 | ``` | ||
| 159 | |||
| 160 | For example | ||
| 161 | |||
| 162 | ``` | ||
| 163 | $ cat <<BYEBYE | ||
| 164 | > one line, | ||
| 165 | > another line | ||
| 166 | > and so on | ||
| 167 | > BYEBYE | ||
| 168 | ``` | ||
| 169 | |||
| 170 | Outputs those three lines. It is useful in shell scripts, when you | ||
| 171 | want to output a block of text. The variant `[n]<<-` strips out | ||
| 172 | `Tab` characters. | ||
| 173 | |||
| 174 | Another useful one is `[n]>&fd`, which "merges" the file descriptors | ||
| 175 | `[n]` and `fd`. For example, if you want to make your command | ||
| 176 | completely silent, you can merge standard output and standard error | ||
| 177 | and redirect them both to `/dev/null` with | ||
| 178 | |||
| 179 | ``` | ||
| 180 | $ command >&2 >/dev/null | ||
| 181 | ``` | ||
| 182 | |||
| 183 | ### Expansion | ||
| 184 | |||
| 185 | There are essentially five kinds of expansion that the shell performs: | ||
| 186 | tilde expansion, parameter expansion, command expansion, arithmetic | ||
| 187 | expansion and filename expansion. | ||
| 188 | |||
| 189 | **Tilde expansion** is quite straightforward, so let's just quote | ||
| 190 | the man page: | ||
| 191 | |||
| 192 | ``` | ||
| 193 | Firstly, tilde expansion occurs on words beginning with the `~' | ||
| 194 | character. Any characters following the tilde, up to the next colon, | ||
| 195 | slash, or blank, are taken as a login name and substituted with that | ||
| 196 | user's home directory, as defined in passwd(5). A tilde by itself is | ||
| 197 | expanded to the contents of the variable HOME. This notation can be used | ||
| 198 | in variable assignments, in the assignment half, immediately after the | ||
| 199 | equals sign or a colon, up to the next slash or colon, if any. | ||
| 200 | |||
| 201 | PATH=~alice:~bob/jobs | ||
| 202 | ``` | ||
| 203 | |||
| 204 | **Parameters** can be variable names or special parameters. Variables | ||
| 205 | can be assigned with the simple syntax `variable=value` and their | ||
| 206 | value can be "accessed" with `$variable`. In case of ambiguity you | ||
| 207 | need to enclose the variable name in curly braces `{}`: say you | ||
| 208 | want to type the string `subroutines` and you have a variable | ||
| 209 | `prefix=sub`. The shell will complain at a `$prefixroutines` about | ||
| 210 | there being no variable with such name, so you have to use | ||
| 211 | `${prefix}routines`. | ||
| 212 | |||
| 213 | The most useful special parameters are: | ||
| 214 | |||
| 215 | * Numbers `1`, `2`, `3`... that refer to the *positional parameters*: | ||
| 216 | |||
| 217 | ``` | ||
| 218 | These parameters are set when a shell, shell script, or shell function is | ||
| 219 | invoked. Each argument passed to a shell or shell script is assigned a | ||
| 220 | positional parameter, starting at 1, and assigned sequentially. | ||
| 221 | ``` | ||
| 222 | |||
| 223 | * The number `0`, which refers to the name of the shell or of the shell | ||
| 224 | script being executed. | ||
| 225 | * The symbols `@` and `*` which expand to all positional parameters | ||
| 226 | at once; they behave differently when enclosed in double quotes: | ||
| 227 | with `"$@"` the parameters are split into fields, with `"$*"` they are not. | ||
| 228 | |||
| 229 | There are some useful constructs to expand a parameter in special | ||
| 230 | ways. The constructs `${parameter:-[word]}` and `${parameter:=[word]}` | ||
| 231 | expand to `[word]` if `parameter` is unset or empty, with the second | ||
| 232 | one also assigning the value `[word]` to `parameter` for subsequent | ||
| 233 | use. Instead, `${parameter:+[word]}` expands to `[word]` *unless* | ||
| 234 | `parameter` is unset or empty, in which case it expands to the empty | ||
| 235 | string. In all these cases, if the colon is omitted `[word]` is | ||
| 236 | substituted only if `parameter` is unset (not if it is empty). | ||
| 237 | |||
| 238 | Another useful one is `${#parameter}`, which expands to the length | ||
| 239 | of `parameter`. Finally there are some constructs that can be used | ||
| 240 | to remove prefixes or suffixes from the expansion of a parameter: | ||
| 241 | |||
| 242 | | Construct | Effect | | ||
| 243 | |:---:|:---:| | ||
| 244 | | `${parameter%[word]}` | Delete smallest possible suffix matching word | | ||
| 245 | | `${parameter%%[word]}` | Delete largest possible suffix matching word | | ||
| 246 | | `${parameter#[word]}` | Delete smallest possible prefix matching word | | ||
| 247 | | `${parameter##[word]}` | Delete largest possible prefix matching word | | ||
| 248 | |||
| 249 | What unfortunately is not explained in the man page of `sh(1)` (but | ||
| 250 | can be found in that of `ksh(1)`) is that `[word]` in this case can | ||
| 251 | be a *pattern*. See [glob(7)](https://man.openbsd.org/OpenBSD-7.1/glob.7) | ||
| 252 | for a description of patterns, which are the same that are used for | ||
| 253 | filename expansion (with the exception that slashes and dots are | ||
| 254 | treated as normal characters). | ||
| 255 | |||
| 256 | For example, using `*` which means "any sequence of zero or more | ||
| 257 | characters": | ||
| 258 | |||
| 259 | ``` | ||
| 260 | $ x="we can,separate,stuff,with commas" | ||
| 261 | $ echo ${x#*,} | ||
| 262 | separate,stuff,with commas | ||
| 263 | $ echo ${x##*,} | ||
| 264 | with commas | ||
| 265 | ``` | ||
| 266 | |||
| 267 | Then there is **command expansion**: | ||
| 268 | |||
| 269 | ``` | ||
| 270 | Command expansion has a command executed in a subshell and the results | ||
| 271 | output in its place. The basic format is: | ||
| 272 | |||
| 273 | $(command) | ||
| 274 | or | ||
| 275 | `command` | ||
| 276 | |||
| 277 | The results are subject to field splitting and pathname expansion; no | ||
| 278 | other form of expansion happens. If command is contained within double | ||
| 279 | quotes, field splitting does not happen either. | ||
| 280 | ``` | ||
| 281 | |||
| 282 | **Arithmetic expansion** uses the syntax `$((expression))`. An | ||
| 283 | `expression` can be a combination of integers (no floating point | ||
| 284 | arithmetic in the shell!), parameter names and the usual arithmetic | ||
| 285 | operations. I won't copy them here; if you are familiar with C or | ||
| 286 | C-like languages, you can use pretty much all the operations you | ||
| 287 | are used to, including logic operations (resulting in 0 or 1), | ||
| 288 | assignment operations like `+=` and bitwise operations like `~`, | ||
| 289 | `&` and `<<`. Even the *ternary if* `expression ? expr1 : expr2` | ||
| 290 | is available. | ||
| 291 | |||
| 292 | Finally, **filename expansion** uses the aforementioned rules of | ||
| 293 | [glob(7)](https://man.openbsd.org/OpenBSD-7.1/glob.7) to expand | ||
| 294 | filenames. To sum them up: | ||
| 295 | |||
| 296 | * As we have already seen, `*` expands to any sequence of characters. | ||
| 297 | * `?` matches any single character. | ||
| 298 | * `[..]` matches any character in place of the double dot, or any | ||
| 299 | character *not* listed if the first is an exclamation mark. | ||
| 300 | * `[[:class:]]` matches any character of a certain class; for example | ||
| 301 | `class` could be `alnum` for alphanumeric characters or `upper` for | ||
| 302 | uppercase letters. | ||
| 303 | * `[x-y]` matches any character in the range between `x` and `y`. | ||
| 304 | |||
| 305 | To illustrate what all of this means, check this out (the command `ls` is | ||
| 306 | used to list all files in the current directory): | ||
| 307 | |||
| 308 | ``` | ||
| 309 | $ ls | ||
| 310 | box file3 mbox typescript | ||
| 311 | count_args.sh file4 mnt videos | ||
| 312 | file1 git music | ||
| 313 | file2 mail phone-laptop-swap | ||
| 314 | $ echo m* | ||
| 315 | mail mbox mnt music | ||
| 316 | $ echo m??? | ||
| 317 | mail mbox | ||
| 318 | $ echo file[2-4] | ||
| 319 | file2 file3 file4 | ||
| 320 | ``` | ||
| 321 | |||
| 322 | ### Quoting | ||
| 323 | |||
| 324 | Sometimes we may want to write some of the special characters | ||
| 325 | described above, such as dollar signs, without their special meaning. | ||
| 326 | You can do so by *escaping*, or *quoting* them. There are essentially | ||
| 327 | three ways to quote a character or a group of characters: | ||
| 328 | |||
| 329 | * Backslash: | ||
| 330 | |||
| 331 | ``` | ||
| 332 | A backslash (\) can be used to quote any character except a newline. If | ||
| 333 | a newline follows a backslash the shell removes them both, effectively | ||
| 334 | making the following line part of the current one. | ||
| 335 | ``` | ||
| 336 | |||
| 337 | This means that a backslash can also effectively be used to split | ||
| 338 | long lines into multiple lines, for example for ease of editing a | ||
| 339 | shell script. | ||
| 340 | |||
| 341 | * Single quotes: | ||
| 342 | |||
| 343 | ``` | ||
| 344 | A group of characters can be enclosed within single quotes (') to quote | ||
| 345 | every character within the quotes. | ||
| 346 | ``` | ||
| 347 | |||
| 348 | * And double quotes: | ||
| 349 | |||
| 350 | ``` | ||
| 351 | A group of characters can be enclosed within double quotes (") to quote | ||
| 352 | every character within the quotes except a backquote (`) or a dollar sign | ||
| 353 | ($), both of which retain their special meaning. A backslash (\) within | ||
| 354 | double quotes retains its special meaning, but only when followed by a | ||
| 355 | backquote, dollar sign, double quote, newline, or another backslash. An | ||
| 356 | at sign (@) within double quotes has a special meaning (see SPECIAL | ||
| 357 | PARAMETERS, below). | ||
| 358 | ``` | ||
| 359 | |||
| 360 | Basically the difference between single and double quotes is that | ||
| 361 | the former turn literally everything they enclose into simple text, | ||
| 362 | while the latter still parse and expand some special characters | ||
| 363 | (for example the dollar sign `$` for variables). | ||
| 364 | |||
| 365 | As an addition, remember that anything enclosed in single or double | ||
| 366 | quotes is considered a single field (word). This was briefly mentioned | ||
| 367 | in the Expansion section, but I skipped it. To illustrate what I | ||
| 368 | mean, let's write a short script and run it first with some words | ||
| 369 | as arguments and then with the same words enclosed in quotes: | ||
| 370 | |||
| 371 | ``` | ||
| 372 | $ echo 'echo $#' > count_args.sh | ||
| 373 | $ count_args.sh how many words are there | ||
| 374 | 5 | ||
| 375 | $ count_args.sh "how many words are there" | ||
| 376 | 1 | ||
| 377 | ``` | ||
| 378 | |||
| 379 | ## Until next time | ||
| 380 | |||
| 381 | This was a very long post, but it made sense to keep all the grammar | ||
| 382 | rules together. To finish this manual page we are going to need | ||
| 383 | another long post, or two shorter ones. | ||
| 384 | |||
| 385 | See you next time! | ||
diff --git a/src/blog/blog.md b/src/blog/blog.md index 7af22f3..2a02644 100644 --- a/src/blog/blog.md +++ b/src/blog/blog.md | |||
| @@ -2,6 +2,7 @@ | |||
| 2 | 2 | ||
| 3 | [RSS Feed](feed.xml) | 3 | [RSS Feed](feed.xml) |
| 4 | 4 | ||
| 5 | * 2022-09-13 [The man page reading club: sh(1) - part 1: shell grammar](2022-09-13-sh-1) | ||
| 5 | * 2022-09-10 [Long live netbooks!](2022-09-10-netbooks) | 6 | * 2022-09-10 [Long live netbooks!](2022-09-10-netbooks) |
| 6 | * 2022-09-05 [Pipe man into col -b to get rid of \^H](2022-09-05-man-col) | 7 | * 2022-09-05 [Pipe man into col -b to get rid of \^H](2022-09-05-man-col) |
| 7 | * 2022-08-14 [How I update my website](2022-08-14-website) | 8 | * 2022-08-14 [How I update my website](2022-08-14-website) |
diff --git a/src/blog/feed.xml b/src/blog/feed.xml index bb05925..3b718d2 100644 --- a/src/blog/feed.xml +++ b/src/blog/feed.xml | |||
| @@ -9,6 +9,13 @@ Thoughts about software, computers and whatever I feel like sharing | |||
| 9 | </description> | 9 | </description> |
| 10 | 10 | ||
| 11 | <item> | 11 | <item> |
| 12 | <title>The man page reading club: sh(1) - part 1: shell grammar</title> | ||
| 13 | <link>https://sebastiano.tronto.net/blog/2022-09-13-sh-1</link> | ||
| 14 | <description>The man page reading club: sh(1) - part 1: shell grammar</description> | ||
| 15 | <pubDate>2022-09-13</pubDate> | ||
| 16 | </item> | ||
| 17 | |||
| 18 | <item> | ||
| 12 | <title>Long live netbooks!</title> | 19 | <title>Long live netbooks!</title> |
| 13 | <link>https://sebastiano.tronto.net/blog/2022-09-10-netbooks</link> | 20 | <link>https://sebastiano.tronto.net/blog/2022-09-10-netbooks</link> |
| 14 | <description>Long live netbooks!</description> | 21 | <description>Long live netbooks!</description> |
