website.md (10718B)
1 # How I update my website 2 3 When I created my website, I decided that I wanted to understand 100% of what 4 I did. In practice, this means that I did not want to use any framework, not 5 even a simple one. Someone might call this *minimalism*, someone else might 6 call it *being a control freak*. I think I like the second one more. 7 8 The principles I follow, which are actually an afterthought after a few months 9 of trial-and-error, are roughly these: 10 11 * **Minimalism**, a.k.a **control-freakism**: As mentioned above, I want to 12 only use tools that I understand completely, 13 at the cost of writing code by hand sometimes. 14 * **Ease of use**: I want to write the bulk of my pages in 15 [Markdown](https://en.wikipedia.org/wiki/Markdown), which is easier to 16 read and edit than html code. 17 * **Reproducibility**: Ideally, building new html pages (from a 18 newly-written Markdown document) and uploading them to my server 19 should be done by issuing simple commands such as `make` and `make deploy`. 20 21 In more practical terms, this boils down to writing a couple of CSS and html 22 files, writing a script that adds a header and a footer to the output of 23 [lowdown](https://kristaps.bsd.lv/lowdown/), and using 24 [rsync](https://en.wikipedia.org/wiki/Rsync) to deploy the files to my server. 25 All of this is available on 26 [my git page](https://git.tronto.net/sebastiano.tronto.net/), but I won't 27 explain every detail of these build scripts here. In particular, my script 28 also builds a [gemini](https://sebastiano.tronto.net/blog/2022-06-04-gemini/) 29 version of my website, which I won't discuss here. 30 31 ## Prerequisites 32 33 My website is hosted on an OpenBSD virtual machine in a remote server. I can 34 access this virtual machine via 35 [SSH](https://en.wikipedia.org/wiki/Secure_Shell), which gives me root 36 access to the operating system. I use rsync for uploading my files to this 37 server. As long as you have an http server that can serve static html 38 files and a way to upload your files to this server, you can easily manage 39 your website in a similar way. I am not going to explain how to do all of this 40 here; if you need help I suggest you have a look at 41 [Roman Zolotarev's website](https://rgz.ee). 42 43 As for my local machine, the one I am actually using to write these blog 44 posts, I just use a Markdown translator, a text editor, rsync 45 and other basic UNIX tools. 46 47 ## Directory structure 48 49 In my working directory 50 there are two main folders with the exact same sub-folder structure: the first 51 one is `src`, which contains all the markdown files I write, plus any other 52 file I need for my pages, such as pictures; the other is `http`, which contains 53 the html pages exactly as they are on my website. The `http` folder is 54 generated from the src folder when I run the `build.sh` script - more 55 on this later! (You won't find the `http` folder on my git page) 56 57 There is one small caveat here: 58 I like my urls to be clean and I want them stripped of the `.html` 59 extension. To do this, I set up my `src` folder so that every 60 subfolder contains at most one `.md` (Markdown) file, which is converted to 61 an `index.html` file in the corresponding subfolder of `http`. 62 In practice, if there are the following files: 63 64 ``` 65 ├── src 66 │ ├── git 67 │ │ └── git-or-any-other-name.md 68 ``` 69 70 The following is generated by `build.sh`: 71 72 ``` 73 ├── http 74 │ ├── git 75 │ │ └── index.html 76 ``` 77 78 So that when one accesses 79 [sebastiano.tronto.net/git](https://sebastiano.tronto.net/git/) 80 the web server automatically serves the `index.html` file. 81 Without this trick the correct URL would have been 82 `sebastiano.tronto.net/git.html` or something, which I don't like. 83 84 The main working directory also contains the 85 [`top.html`](https://git.tronto.net/sebastiano.tronto.net/file/top%2Ehtml%2Ehtml) 86 and 87 [`bottom.html`](https://git.tronto.net/sebastiano.tronto.net/file/bottom%2Ehtml%2Ehtml) 88 files. These files are not uploaded directly to my server - they are not 89 even well-formed html files! - but they are used to build all other 90 html pages. 91 92 ## Building the pages 93 94 The basic idea behind `build.sh` is very simple. If we want to create the 95 html file corresponding to, say, `src/page/file.md`, we just need to create 96 a file called `http/page/index.html` and copy there the contents of 97 `top.html`, followed by the output of `lowdown src/page/file.md`, followed 98 by the contents of `bottom.html`. In shell language: 99 100 ``` 101 cat top.html > http/page/index.html 102 lowdown src/page/file.md >> http/page/index.html 103 cat bottom.html >> http/page/index.html 104 ``` 105 106 Of course you don't have to use lowdown as I do: any Markdown translator 107 works. Indeed, if you want to use a different markup language to write 108 your pages you just need to replace the second line in the code above. 109 110 We would also like to make a small change to the header while 111 we build this page, namely we want the title of the page (the one 112 displayed in your browser's top bar or tab) to match the actual 113 title of the page or blog post. To do this easily I have left a 114 placeholder `TITLE` in `top.html`, that we just need to replace with the 115 actual title of the page. To find out what the title is we just need to 116 get the text following 117 the first `# ` (hash space) in the Markdown file - that is, the first 118 "big title" of the page. We can do this 119 thanks to the classic UNIX tools sed, grep and head: 120 121 ``` 122 sed "s/TITLE/$(grep '^\# ' < src/page/file.md \ 123 | head -n 1 | sed 's/^\# //')/" < top.html > http/page/index.html 124 lowdown src/page/file.md >> http/page/index.html 125 cat bottom.html >> http/page/index.html 126 ``` 127 128 The first two lines might be a bit complicated to work out if you 129 are not familiar with these commands. Let's break them down! 130 131 The main command is `sed "s/TITLE/...stuff.../"` which replaces the 132 first occurrence of the string `TITLE` with that complicated stuff. 133 The end of the second line tells sed to use `top.html` as input and 134 write the output to `http/page/index.html`. The complicated stuff 135 that is going to replace `TITLE` is enclosed in `$()`, which means 136 that it is the result of a command. This command is itself a chain 137 of commands: first we find all lines that start with `# ` with 138 `grep '^\# '` on the correct file (`< src/page/file.html`), then 139 we take the first of these lines (`head -n 1`) and finally we 140 trim the leading `# ` with `sed`. As you can see, the UNIX shell 141 is quite a powerful tool! 142 143 Now we just need to do all of this recursively on the `src` folder. 144 The final result looks something like this: 145 146 ``` 147 #!/bin/sh 148 149 recursivebuild() { 150 local destdir=$(echo $1 | sed 's|^src|http|') 151 mkdir -p "$destdir" 152 for file in $(ls $1); do 153 if [ -d "$1/$file" ]; then 154 155 # Recursively build subdirectories 156 mkdir -p "$destdir/$file" 157 recursivebuild "$1/$file" 158 else 159 extension=$(echo "$file" | sed 's/.*\.//') 160 if [ "$extension" = "md" ]; then 161 162 # Process Markdown files, as above 163 sed "s/TITLE/$(grep '^\# ' < "$1/$file" \ 164 | head -n 1 \ 165 | sed 's/^\# //')/" < top.html \ 166 > "$destdir/index.html" 167 lowdown "$1/$file" >> "$destdir/index.html" 168 cat bottom.html >> "$destdir/index.html" 169 else 170 171 # Copy all other files as they are 172 cp "$1/$file" "$destdir/$file" 173 fi 174 fi 175 done 176 } 177 178 recursivebuild src 179 ``` 180 181 ## Extras: the blog index and RSS feed 182 183 The [blog index page](https://sebastiano.tronto.net/blog/) is also 184 generated by the build script, but the corresponding Markdown file 185 in `src` is not created by hand. Instead, this file is generated by 186 scanning the `src/blog` subfolder. For each post, the date is deduced 187 from the name of the folder containing the markdown file, which always 188 starts with the date itself in the `yyyy-mm-dd` format. 189 190 While we scan the blog directory to create a list of posts, we might as 191 well make an [RSS feed](https://en.wikipedia.org/wiki/RSS) file for the 192 blog. This is a file used by feed reader applications to check if there 193 is any new post. The format is quite simple: check out 194 [mine](https://sebastiano.tronto.net/blog/feed.xml). 195 196 The code to accomplish this looks something like this: 197 198 ``` 199 makeblog() { 200 bf=src/blog/blog.md # Blog index file 201 ff=src/blog/feed.xml # RSS feed file 202 203 printf "# Blog\n\n[RSS Feed](feed.xml)\n\n" > $bf 204 cp feed-top.xml $ff 205 206 for i in $(ls src/blog | sort -r); do 207 if [ -d src/blog/$i ]; then 208 209 # Get basic data of the post (date, title) 210 f="src/blog/$i/*.md" 211 d=$(echo $i | grep -oE '^[0-9]{4}-[0-9]{2}-[0-9]{2}') 212 t=$(head -n 1 $f | sed 's/# //') 213 214 # Add blog post to the list 215 echo "* $d [$t]($i)" >> $bf 216 217 # Create RSS feed item 218 echo "<item>" >> $ff 219 echo "<title>$t</title>" >> $ff 220 echo "<link>https://sebastiano.tronto.net/blog/$i</link>" >> $ff 221 echo "<description>$t</description>" >> $ff 222 echo "<pubDate>$d</pubDate>" >> $ff 223 echo "</item>" >> $ff 224 echo "" >> $ff 225 fi 226 done 227 228 # Close the RSS feed file 229 echo "" >> $ff 230 echo "</channel>" >> $ff 231 echo "</rss>" >> $ff 232 } 233 ``` 234 235 ## Deploying with make 236 237 Updating or adding a page is now very easy: I just need to edit the 238 corresponding Markdown file, run `./build.sh` to build the new html 239 pages and run 240 241 ``` 242 rsync -rv --delete --rsync-path=openrsync http/ \ 243 tronto.net:/var/www/htdocs/sebastiano.tronto.net 244 ``` 245 246 to sync the `http` directory with my server. I need to use the 247 `--rsync-path` option because the `rsync` binary has a different 248 name on my local system (Linux) than on my server (OpenBSD). But apart from 249 this the command is straightforward. 250 251 Of course I don't want to type this lenghty command every time. It is very 252 convenient in this case to write a short Makefile: 253 254 ``` 255 all: clean 256 ./build.sh 257 258 clean: 259 rm -r http 260 mkdir -p http 261 262 deploy: 263 rsync -rv --delete --rsync-path=openrsync http/ \ 264 tronto.net:/var/www/htdocs/sebastiano.tronto.net 265 266 .PHONY: all clean deploy 267 ``` 268 269 So that I just need to run `make` to build and `make deploy` to upload the 270 new files. Watch out: if you want to reproduce this on your system, make 271 sure that the user on your server has sufficient permissions to run 272 that rsync command - in particular you need write permission on the 273 `/var/www/htdocs` folder. 274 275 If you are not familiar with the [make(1)](https://man.openbsd.org/make) 276 syntax, this step is completely optional and you can simply type 277 the full commands every time, or make another small script called 278 `deploy.sh` and run that instead. 279 280 ## Follow-up? 281 282 I am sure my build scripts will keep evolving over time, so at some point I 283 might write a new post about the same topic. I am also probably going to write 284 something about how I generate my [git page](https://git.tronto.net/) using 285 [stagit](https://codemadness.org/stagit.html), if anything just to document 286 my post-receive hooks. So, if you liked this post, stay tuned for more!