website.md (11120B)
1 # How I update my website 2 3 When I created my website, I decided that I wanted to understand 100% of what 4 I did. In practice, this means that I did not want to use any framework, not 5 even a simple one. Someone might call this *minimalism*, someone else might 6 call it *being a control freak*. I think I like the second one more. 7 8 The principles I follow, which are actually an afterthought after a few months 9 of trial-and-error, are roughly these: 10 11 * **Minimalism**, a.k.a **control-freakism**: As mentioned above, I want to 12 only use tools that I understand completely, 13 at the cost of writing code by hand sometimes. 14 * **Ease of use**: I want to write the bulk of my pages in 15 [Markdown](https://en.wikipedia.org/wiki/Markdown), which is easier to 16 read and edit than html code. 17 * **Reproducibility**: Ideally, building new html pages (from a 18 newly-written Markdown document) and uploading them to my server 19 should be done by issuing simple commands such as `make` and `make deploy`. 20 21 In more practical terms, this boils down to writing a couple of CSS and html 22 files, writing a script that adds a header and a footer to the output of 23 [lowdown](https://kristaps.bsd.lv/lowdown/), and using 24 [rsync](https://en.wikipedia.org/wiki/Rsync) to deploy the files to my server. 25 All of this is available on 26 [my git page](https://git.tronto.net/sebastiano.tronto.net/), but I won't 27 explain every detail of these build scripts here. In particular, my script 28 also builds a [gemini](https://sebastiano.tronto.net/blog/2022-06-04-gemini/) 29 version of my website, which I won't discuss here. 30 31 ## Prerequisites 32 33 My website is hosted on an OpenBSD virtual machine in a remote server. I can 34 access this virtual machine via 35 [SSH](https://en.wikipedia.org/wiki/Secure_Shell), which gives me root 36 access to the operating system. I use rsync for uploading my files to this 37 server. As long as you have an http server that can serve static html 38 files and a way to upload your files to this server, you can easily manage 39 your website in a similar way. I am not going to explain how to do all of this 40 here; if you need help I suggest you have a look at 41 [Roman Zolotarev's website](https://rgz.ee) (*Update: as of November 2025, 42 it seems that Roman has removed all content from his website. 43 [His new website](https://romanzolotarev.com/) does not have much content 44 either, although some of the old pages are still accessible if you know 45 the full URL, for exaple 46 [his Markdown cheatsheet](https://romanzolotarev.com/markdown.html) or 47 [his OpenBSD-related pages](https://romanzolotarev.com/openbsd/).*) 48 49 As for my local machine, the one I am actually using to write these blog 50 posts, I just use a Markdown translator, a text editor, rsync 51 and other basic UNIX tools. 52 53 ## Directory structure 54 55 In my working directory 56 there are two main folders with the exact same sub-folder structure: the first 57 one is `src`, which contains all the markdown files I write, plus any other 58 file I need for my pages, such as pictures; the other is `http`, which contains 59 the html pages exactly as they are on my website. The `http` folder is 60 generated from the src folder when I run the `build.sh` script - more 61 on this later! (You won't find the `http` folder on my git page) 62 63 There is one small caveat here: 64 I like my urls to be clean and I want them stripped of the `.html` 65 extension. To do this, I set up my `src` folder so that every 66 subfolder contains at most one `.md` (Markdown) file, which is converted to 67 an `index.html` file in the corresponding subfolder of `http`. 68 In practice, if there are the following files: 69 70 ``` 71 ├── src 72 │ ├── git 73 │ │ └── git-or-any-other-name.md 74 ``` 75 76 The following is generated by `build.sh`: 77 78 ``` 79 ├── http 80 │ ├── git 81 │ │ └── index.html 82 ``` 83 84 So that when one accesses 85 [sebastiano.tronto.net/git](https://sebastiano.tronto.net/git/) 86 the web server automatically serves the `index.html` file. 87 Without this trick the correct URL would have been 88 `sebastiano.tronto.net/git.html` or something, which I don't like. 89 90 The main working directory also contains the 91 [`top.html`](https://git.tronto.net/sebastiano.tronto.net/file/top%2Ehtml%2Ehtml) 92 and 93 [`bottom.html`](https://git.tronto.net/sebastiano.tronto.net/file/bottom%2Ehtml%2Ehtml) 94 files. These files are not uploaded directly to my server - they are not 95 even well-formed html files! - but they are used to build all other 96 html pages. 97 98 ## Building the pages 99 100 The basic idea behind `build.sh` is very simple. If we want to create the 101 html file corresponding to, say, `src/page/file.md`, we just need to create 102 a file called `http/page/index.html` and copy there the contents of 103 `top.html`, followed by the output of `lowdown src/page/file.md`, followed 104 by the contents of `bottom.html`. In shell language: 105 106 ``` 107 cat top.html > http/page/index.html 108 lowdown src/page/file.md >> http/page/index.html 109 cat bottom.html >> http/page/index.html 110 ``` 111 112 Of course you don't have to use lowdown as I do: any Markdown translator 113 works. Indeed, if you want to use a different markup language to write 114 your pages you just need to replace the second line in the code above. 115 116 We would also like to make a small change to the header while 117 we build this page, namely we want the title of the page (the one 118 displayed in your browser's top bar or tab) to match the actual 119 title of the page or blog post. To do this easily I have left a 120 placeholder `TITLE` in `top.html`, that we just need to replace with the 121 actual title of the page. To find out what the title is we just need to 122 get the text following 123 the first `# ` (hash space) in the Markdown file - that is, the first 124 "big title" of the page. We can do this 125 thanks to the classic UNIX tools sed, grep and head: 126 127 ``` 128 sed "s/TITLE/$(grep '^\# ' < src/page/file.md \ 129 | head -n 1 | sed 's/^\# //')/" < top.html > http/page/index.html 130 lowdown src/page/file.md >> http/page/index.html 131 cat bottom.html >> http/page/index.html 132 ``` 133 134 The first two lines might be a bit complicated to work out if you 135 are not familiar with these commands. Let's break them down! 136 137 The main command is `sed "s/TITLE/...stuff.../"` which replaces the 138 first occurrence of the string `TITLE` with that complicated stuff. 139 The end of the second line tells sed to use `top.html` as input and 140 write the output to `http/page/index.html`. The complicated stuff 141 that is going to replace `TITLE` is enclosed in `$()`, which means 142 that it is the result of a command. This command is itself a chain 143 of commands: first we find all lines that start with `# ` with 144 `grep '^\# '` on the correct file (`< src/page/file.html`), then 145 we take the first of these lines (`head -n 1`) and finally we 146 trim the leading `# ` with `sed`. As you can see, the UNIX shell 147 is quite a powerful tool! 148 149 Now we just need to do all of this recursively on the `src` folder. 150 The final result looks something like this: 151 152 ``` 153 #!/bin/sh 154 155 recursivebuild() { 156 local destdir=$(echo $1 | sed 's|^src|http|') 157 mkdir -p "$destdir" 158 for file in $(ls $1); do 159 if [ -d "$1/$file" ]; then 160 161 # Recursively build subdirectories 162 mkdir -p "$destdir/$file" 163 recursivebuild "$1/$file" 164 else 165 extension=$(echo "$file" | sed 's/.*\.//') 166 if [ "$extension" = "md" ]; then 167 168 # Process Markdown files, as above 169 sed "s/TITLE/$(grep '^\# ' < "$1/$file" \ 170 | head -n 1 \ 171 | sed 's/^\# //')/" < top.html \ 172 > "$destdir/index.html" 173 lowdown "$1/$file" >> "$destdir/index.html" 174 cat bottom.html >> "$destdir/index.html" 175 else 176 177 # Copy all other files as they are 178 cp "$1/$file" "$destdir/$file" 179 fi 180 fi 181 done 182 } 183 184 recursivebuild src 185 ``` 186 187 ## Extras: the blog index and RSS feed 188 189 The [blog index page](https://sebastiano.tronto.net/blog/) is also 190 generated by the build script, but the corresponding Markdown file 191 in `src` is not created by hand. Instead, this file is generated by 192 scanning the `src/blog` subfolder. For each post, the date is deduced 193 from the name of the folder containing the markdown file, which always 194 starts with the date itself in the `yyyy-mm-dd` format. 195 196 While we scan the blog directory to create a list of posts, we might as 197 well make an [RSS feed](https://en.wikipedia.org/wiki/RSS) file for the 198 blog. This is a file used by feed reader applications to check if there 199 is any new post. The format is quite simple: check out 200 [mine](https://sebastiano.tronto.net/blog/feed.xml). 201 202 The code to accomplish this looks something like this: 203 204 ``` 205 makeblog() { 206 bf=src/blog/blog.md # Blog index file 207 ff=src/blog/feed.xml # RSS feed file 208 209 printf "# Blog\n\n[RSS Feed](feed.xml)\n\n" > $bf 210 cp feed-top.xml $ff 211 212 for i in $(ls src/blog | sort -r); do 213 if [ -d src/blog/$i ]; then 214 215 # Get basic data of the post (date, title) 216 f="src/blog/$i/*.md" 217 d=$(echo $i | grep -oE '^[0-9]{4}-[0-9]{2}-[0-9]{2}') 218 t=$(head -n 1 $f | sed 's/# //') 219 220 # Add blog post to the list 221 echo "* $d [$t]($i)" >> $bf 222 223 # Create RSS feed item 224 echo "<item>" >> $ff 225 echo "<title>$t</title>" >> $ff 226 echo "<link>https://sebastiano.tronto.net/blog/$i</link>" >> $ff 227 echo "<description>$t</description>" >> $ff 228 echo "<pubDate>$d</pubDate>" >> $ff 229 echo "</item>" >> $ff 230 echo "" >> $ff 231 fi 232 done 233 234 # Close the RSS feed file 235 echo "" >> $ff 236 echo "</channel>" >> $ff 237 echo "</rss>" >> $ff 238 } 239 ``` 240 241 ## Deploying with make 242 243 Updating or adding a page is now very easy: I just need to edit the 244 corresponding Markdown file, run `./build.sh` to build the new html 245 pages and run 246 247 ``` 248 rsync -rv --delete --rsync-path=openrsync http/ \ 249 tronto.net:/var/www/htdocs/sebastiano.tronto.net 250 ``` 251 252 to sync the `http` directory with my server. I need to use the 253 `--rsync-path` option because the `rsync` binary has a different 254 name on my local system (Linux) than on my server (OpenBSD). But apart from 255 this the command is straightforward. 256 257 Of course I don't want to type this lenghty command every time. It is very 258 convenient in this case to write a short Makefile: 259 260 ``` 261 all: clean 262 ./build.sh 263 264 clean: 265 rm -r http 266 mkdir -p http 267 268 deploy: 269 rsync -rv --delete --rsync-path=openrsync http/ \ 270 tronto.net:/var/www/htdocs/sebastiano.tronto.net 271 272 .PHONY: all clean deploy 273 ``` 274 275 So that I just need to run `make` to build and `make deploy` to upload the 276 new files. Watch out: if you want to reproduce this on your system, make 277 sure that the user on your server has sufficient permissions to run 278 that rsync command - in particular you need write permission on the 279 `/var/www/htdocs` folder. 280 281 If you are not familiar with the [make(1)](https://man.openbsd.org/make) 282 syntax, this step is completely optional and you can simply type 283 the full commands every time, or make another small script called 284 `deploy.sh` and run that instead. 285 286 ## Follow-up? 287 288 I am sure my build scripts will keep evolving over time, so at some point I 289 might write a new post about the same topic. I am also probably going to write 290 something about how I generate my [git page](https://git.tronto.net/) using 291 [stagit](https://codemadness.org/stagit.html), if anything just to document 292 my post-receive hooks. So, if you liked this post, stay tuned for more!