path: root/src/blog/2022-08-14-website/website.md

# How I update my website

When I created my website, I decided that I wanted to understand 100% of what
I did. In practice, this means that I did not want to use any framework, not
even a simple one. Someone might call this *minimalism*, someone else might
call it *being a control freak*. I think I like the second one more.

The principles I follow, which are actually an afterthought after a few months
of trial-and-error, are roughly these:

* **Minimalism**, a.k.a **control-freakism**: As mentioned above, I want to
  only use tools that I understand completely,
  at the cost of writing code by hand sometimes.
* **Ease of use**: I want to write the bulk of my pages in
  [Markdown](https://en.wikipedia.org/wiki/Markdown), which is easier to
  read and edit than html code.
* **Reproducibility**: Ideally, building new html pages (from a
  newly-written Markdown document) and uploading them to my server
  should be done by issuing simple commands such as `make` and `make deploy`.

In more practical terms, this boils down to writing a couple of CSS and html
files, writing a script that adds a header and a footer to the output of
[lowdown](https://kristaps.bsd.lv/lowdown/), and using
[rsync](https://en.wikipedia.org/wiki/Rsync) to deploy the files to my server.
All of this is available on
[my git page](https://git.tronto.net/sebastiano.tronto.net), but I won't
explain every detail of these build scripts here. In particular, my script
also builds a [gemini](https://sebastiano.tronto.net/blog/2022-06-04-gemini/)
version of my website, which I won't discuss here.

## Prerequisites

My website is hosted on an OpenBSD virtual machine in a remote server. I can
access this virtual machine via
[SSH](https://en.wikipedia.org/wiki/Secure_Shell), which gives me root
access to the operating system. I use rsync for uploading my files to this
server. As long as you have an http server that can serve static html
files and a way to upload your files to this server, you can easily manage
your website in a similar way. I am not going to explain how to do all of this
here; if you need help I suggest you have a look at
[Roman Zolotarev's website](https://rgz.ee) (*Update: as of November 2025,
it seems that Roman has removed all content from his website.
[His new website](https://romanzolotarev.com/) does not have much content
either, although some of the old pages are still accessible if you know
the full URL, for exaple
[his Markdown cheatsheet](https://romanzolotarev.com/markdown.html) or
[his OpenBSD-related pages](https://romanzolotarev.com/openbsd/).*)

As for my local machine, the one I am actually using to write these blog
posts, I just use a Markdown translator, a text editor, rsync
and other basic UNIX tools.

## Directory structure

In my working directory
there are two main folders with the exact same sub-folder structure: the first
one is `src`, which contains all the markdown files I write, plus any other
file I need for my pages, such as pictures; the other is `http`, which contains
the html pages exactly as they are on my website. The `http` folder is
generated from the src folder when I run the `build.sh` script - more
on this later! (You won't find the `http` folder on my git page)

There is one small caveat here:
I like my urls to be clean and I want them stripped of the `.html`
extension. To do this, I set up my `src` folder so that every
subfolder contains at most one `.md` (Markdown) file, which is converted to
an `index.html` file in the corresponding subfolder of `http`.
In practice, if there are the following files:

```
├── src
│   ├── git
│   │   └── git-or-any-other-name.md
```

The following is generated by `build.sh`:

```
├── http
│   ├── git
│   │   └── index.html
```

So that when one accesses
[sebastiano.tronto.net/git](https://sebastiano.tronto.net/git/)
the web server automatically serves the `index.html` file.
Without this trick the correct URL would have been
`sebastiano.tronto.net/git.html` or something, which I don't like.

The main working directory also contains the 
[`top.html`](https://git.tronto.net/sebastiano.tronto.net/tree/top.html)
and
[`bottom.html`](https://git.tronto.net/sebastiano.tronto.net/tree/bottom.html)
files. These files are not uploaded directly to my server - they are not
even well-formed html files! - but they are used to build all other
html pages.

## Building the pages

The basic idea behind `build.sh` is very simple. If we want to create the
html file corresponding to, say, `src/page/file.md`, we just need to create
a file called `http/page/index.html` and copy there the contents of
`top.html`, followed by the output of `lowdown src/page/file.md`, followed
by the contents of `bottom.html`. In shell language:

```
cat top.html > http/page/index.html
lowdown src/page/file.md >> http/page/index.html
cat bottom.html >> http/page/index.html
```

Of course you don't have to use lowdown as I do: any Markdown translator
works. Indeed, if you want to use a different markup language to write
your pages you just need to replace the second line in the code above.

We would also like to make a small change to the header while
we build this page, namely we want the title of the page (the one
displayed in your browser's top bar or tab) to match the actual
title of the page or blog post. To do this easily I have left a
placeholder `TITLE` in `top.html`, that we just need to replace with the
actual title of the page. To find out what the title is we just need to
get the text following
the first `# ` (hash space) in the Markdown file - that is, the first
"big title" of the page. We can do this
thanks to the classic UNIX tools sed, grep and head:

```
sed "s/TITLE/$(grep '^\# ' < src/page/file.md \
	| head -n 1 | sed 's/^\# //')/" < top.html > http/page/index.html
lowdown src/page/file.md >> http/page/index.html
cat bottom.html >> http/page/index.html
```

The first two lines might be a bit complicated to work out if you
are not familiar with these commands. Let's break them down!

The main command is `sed "s/TITLE/...stuff.../"` which replaces the
first occurrence of the string `TITLE` with that complicated stuff.
The end of the second line tells sed to use `top.html` as input and
write the output to `http/page/index.html`. The complicated stuff
that is going to replace `TITLE` is enclosed in `$()`, which means
that it is the result of a command. This command is itself a chain
of commands: first we find all lines that start with `# ` with
`grep '^\# '` on the correct file (`< src/page/file.html`), then
we take the first of these lines (`head -n 1`) and finally we
trim the leading `# ` with `sed`. As you can see, the UNIX shell
is quite a powerful tool!

Now we just need to do all of this recursively on the `src` folder.
The final result looks something like this:

```
#!/bin/sh

recursivebuild() {
	local destdir=$(echo $1 | sed 's|^src|http|')
	mkdir -p "$destdir"
	for file in $(ls $1); do
		if [ -d "$1/$file" ]; then

			# Recursively build subdirectories
			mkdir -p "$destdir/$file"
			recursivebuild "$1/$file"
		else
			extension=$(echo "$file" | sed 's/.*\.//')
			if [ "$extension" = "md" ]; then

				# Process Markdown files, as above
				sed "s/TITLE/$(grep '^\# ' < "$1/$file" \
					| head -n 1 \
					| sed 's/^\# //')/" < top.html \
					> "$destdir/index.html"
				lowdown "$1/$file" >> "$destdir/index.html"
				cat bottom.html >> "$destdir/index.html"
			else
				
				# Copy all other files as they are
				cp "$1/$file" "$destdir/$file"
			fi
		fi
	done
}

recursivebuild src
```

## Extras: the blog index and RSS feed

The [blog index page](https://sebastiano.tronto.net/blog/) is also
generated by the build script, but the corresponding Markdown file
in `src` is not created by hand. Instead, this file is generated by
scanning the `src/blog` subfolder. For each post, the date is deduced
from the name of the folder containing the markdown file, which always
starts with the date itself in the `yyyy-mm-dd` format.

While we scan the blog directory to create a list of posts, we might as
well make an [RSS feed](https://en.wikipedia.org/wiki/RSS) file for the
blog. This is a file used by feed reader applications to check if there
is any new post. The format is quite simple: check out
[mine](https://sebastiano.tronto.net/blog/feed.xml).

The code to accomplish this looks something like this:

```
makeblog() {
	bf=src/blog/blog.md    # Blog index file
	ff=src/blog/feed.xml   # RSS feed file

	printf "# Blog\n\n[RSS Feed](feed.xml)\n\n" > $bf
	cp feed-top.xml $ff

	for i in $(ls src/blog | sort -r); do
		if [ -d src/blog/$i ]; then

			# Get basic data of the post (date, title)
			f="src/blog/$i/*.md"
			d=$(echo $i | grep -oE '^[0-9]{4}-[0-9]{2}-[0-9]{2}')
			t=$(head -n 1 $f | sed 's/# //')

			# Add blog post to the list
			echo "* $d [$t]($i)" >> $bf

			# Create RSS feed item
			echo "<item>" >> $ff
			echo "<title>$t</title>" >> $ff
			echo "<link>https://sebastiano.tronto.net/blog/$i</link>" >> $ff
			echo "<description>$t</description>" >> $ff
			echo "<pubDate>$d</pubDate>" >> $ff
			echo "</item>" >> $ff
			echo "" >> $ff
		fi
	done

	# Close the RSS feed file
	echo "" >> $ff
	echo "</channel>" >> $ff
	echo "</rss>" >> $ff
}
```

## Deploying with make

Updating or adding a page is now very easy: I just need to edit the
corresponding Markdown file, run `./build.sh` to build the new html
pages and run

```
rsync -rv --delete --rsync-path=openrsync http/ \
	tronto.net:/var/www/htdocs/sebastiano.tronto.net
```

to sync the `http` directory with my server. I need to use the
`--rsync-path` option because the `rsync` binary has a different 
name on my local system (Linux) than on my server (OpenBSD). But apart from
this the command is straightforward.

Of course I don't want to type this lenghty command every time. It is very
convenient in this case to write a short Makefile:

```
all: clean
	./build.sh

clean:
	rm -r http
	mkdir -p http

deploy:
	rsync -rv --delete --rsync-path=openrsync http/ \
		tronto.net:/var/www/htdocs/sebastiano.tronto.net

.PHONY: all clean deploy
```

So that I just need to run `make` to build and `make deploy` to upload the
new files. Watch out: if you want to reproduce this on your system, make
sure that the user on your server has sufficient permissions to run
that rsync command - in particular you need write permission on the
`/var/www/htdocs` folder.

If you are not familiar with the [make(1)](https://man.openbsd.org/make)
syntax, this step is completely optional and you can simply type
the full commands every time, or make another small script called
`deploy.sh` and run that instead.

## Follow-up?

I am sure my build scripts will keep evolving over time, so at some point I 
might write a new post about the same topic. I am also probably going to write
something about how I generate my [git page](https://git.tronto.net/) using
[stagit](https://codemadness.org/stagit.html), if anything just to document
my post-receive hooks. So, if you liked this post, stay tuned for more!