diff options
Diffstat (limited to 'src/blog/2022-11-23-git-host/git-host.md')
| -rw-r--r-- | src/blog/2022-11-23-git-host/git-host.md | 299 |
1 files changed, 299 insertions, 0 deletions
diff --git a/src/blog/2022-11-23-git-host/git-host.md b/src/blog/2022-11-23-git-host/git-host.md new file mode 100644 index 0000000..cc9fc27 --- /dev/null +++ b/src/blog/2022-11-23-git-host/git-host.md | |||
| @@ -0,0 +1,299 @@ | |||
| 1 | # Self-hosted git pages with stagit (featuring ed, the standard editor) | ||
| 2 | |||
| 3 | This is a follow-up to my earlier blog entry | ||
| 4 | [How I update my website](../2022-08-14-website). | ||
| 5 | |||
| 6 | If you work on one or more personal software projects as a hobby, | ||
| 7 | chances are you are using `git`. To publish your project online | ||
| 8 | you may be using a website like GitHub, or perhaps a more | ||
| 9 | open source friendly alternative such as | ||
| 10 | [GitLab](https://about.gitlab.com/) or [sourcehut](https://sourcehut.org/). | ||
| 11 | |||
| 12 | But have you considered hosting your repositories, and serving them | ||
| 13 | via web pages, on your personal server? In this post I am going to show | ||
| 14 | you how I do it, in the usual minimalist and | ||
| 15 | not-using-what-I-do-not-understand style. | ||
| 16 | |||
| 17 | You can see the final result on my [git pages](https://git.tronto.net). | ||
| 18 | The scripts and other files I use to set this up are accessible | ||
| 19 | [here](https://git.tronto.net/git-hooks). | ||
| 20 | |||
| 21 | ## Hosting git repositories on your own server | ||
| 22 | |||
| 23 | This step is quite simple, and you can just follow | ||
| 24 | [Roman Zolotarev's tutorial](https://rgz.ee/git.html) like I did - adapting | ||
| 25 | the first few steps to your OS if you are not running OpenBSD. | ||
| 26 | |||
| 27 | To sum it up, you need to: | ||
| 28 | |||
| 29 | 1. Create a dedicate `git` account on your server (optional, if you know | ||
| 30 | what you are doing). | ||
| 31 | |||
| 32 | 2. Add your public SSH key to the `~/.ssh/authorized_keys` file of your newly | ||
| 33 | created remote account. | ||
| 34 | |||
| 35 | 3. Initialize a git repository on your server with `git init REPOSITORY`. | ||
| 36 | |||
| 37 | 4. Clone it via SSH with `git clone git@SERVER:REPOSITORY`. | ||
| 38 | |||
| 39 | And you are done! If you want to use multiple remotes, for example your private | ||
| 40 | server and GitHub, you can do so by adding a push-only URL with | ||
| 41 | `git remote set-url --add --push origin URL`. But don't trust me on this | ||
| 42 | exact command, I always have to look it up | ||
| 43 | - you should do the same before running it. | ||
| 44 | |||
| 45 | Now your repositories are online, but how can you make them browsable via web? | ||
| 46 | |||
| 47 | ## stagit | ||
| 48 | |||
| 49 | The tool I use to serve my git repositories as static web pages is | ||
| 50 | [stagit](https://codemadness.org/stagit.html). It is very easy to describe what | ||
| 51 | stagit does: running it on a git reporitory produces some directories with | ||
| 52 | a bunch of html files that you can simply move to your www hosting directory. | ||
| 53 | |||
| 54 | After generating the pages you can personalize them by copying your logo, | ||
| 55 | [favicon](https://en.wikipedia.org/wiki/Favicon) or CSS style sheet. You can | ||
| 56 | use `stagit-index` to generate | ||
| 57 | [an index page for your repositories](https://git.tronto.net/). Since everything | ||
| 58 | consists of html files, you can simply edit them to personalize your git pages | ||
| 59 | even further - and below you'll see some examples. | ||
| 60 | |||
| 61 | But you definitely do not want to do this by hand every time you push a commit. | ||
| 62 | Since the pages stagit generates are *static*, they do not update | ||
| 63 | automatically: you'll have to run stagit again every time. You can automate | ||
| 64 | this for example by running stagit periodically with cron, but there is an | ||
| 65 | easier way: | ||
| 66 | [git hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks). | ||
| 67 | |||
| 68 | ## Git hooks | ||
| 69 | |||
| 70 | By saving suitably named executable files inside your project's | ||
| 71 | `.git/hooks` directory you can automate any process you want during | ||
| 72 | certain stages of your git workflow. For example you can use a `pre-commit` | ||
| 73 | script to run commands before you commit changes, or a `pre-push` script | ||
| 74 | to do something before pushing a commit. | ||
| 75 | |||
| 76 | The hooks are divided in client-side and server-side. We are interested in | ||
| 77 | the server-side `post-receive` hook, which is executed on the remote every | ||
| 78 | time a commit is received. This is the last ingredient we need for our setup: | ||
| 79 | a simple `post-receive` hook that runs stagit and copies the files it | ||
| 80 | generates to the appropriate folder will do the trick. | ||
| 81 | |||
| 82 | ## My setup | ||
| 83 | |||
| 84 | Throughout the rest of the post I will use the following variables, to be | ||
| 85 | set at the beginning of the script: | ||
| 86 | |||
| 87 | ``` | ||
| 88 | yourname="Sebastiano Tronto" # Author's name | ||
| 89 | repo="$PWD" # The full path of the repository | ||
| 90 | name="$(basename $repo .git)" # The name of the repository | ||
| 91 | baseurl="https://git.tronto.net" # The base URL for the repository | ||
| 92 | basedir="/var/www/htdocs/git.tronto.net" # The base directory for www files | ||
| 93 | htdir="$basedir/$name" # The www directory for the repository | ||
| 94 | ``` | ||
| 95 | |||
| 96 | ### Basic stagit usage | ||
| 97 | |||
| 98 | The first thing we want to do is to set up some basic information in the | ||
| 99 | `owner` and `url` files that stagit is going to use. The hook is run in the | ||
| 100 | repository's directory, so it is not necessary to specify a full path: | ||
| 101 | |||
| 102 | ``` | ||
| 103 | echo "$yourname" > owner | ||
| 104 | echo "$baseurl/$name" > url | ||
| 105 | ``` | ||
| 106 | |||
| 107 | Next we prepare the target directory by removing old files and creating | ||
| 108 | it if necessary: | ||
| 109 | |||
| 110 | ``` | ||
| 111 | rm -rf "$htdir" | ||
| 112 | mkdir -p "$basedir" | ||
| 113 | ``` | ||
| 114 | |||
| 115 | To make the repository clonable by anyone from the same URL used to | ||
| 116 | view it, we need to copy the whole directory to the | ||
| 117 | www directory we have just created: | ||
| 118 | |||
| 119 | ``` | ||
| 120 | cp -r "$repo" "$htdir" | ||
| 121 | ``` | ||
| 122 | |||
| 123 | And finally we can run stagit: | ||
| 124 | |||
| 125 | ``` | ||
| 126 | cd "$htdir" | ||
| 127 | stagit -l 100 "$repo" | ||
| 128 | ``` | ||
| 129 | |||
| 130 | The `-l` option is used to specify how many commits should be visibile | ||
| 131 | in the log page. | ||
| 132 | |||
| 133 | For some basic personalization we can choose a different default page (index | ||
| 134 | file). I like to have the file list: | ||
| 135 | |||
| 136 | ``` | ||
| 137 | cp "$htdir/files.html" "$htdir/index.html" | ||
| 138 | ``` | ||
| 139 | |||
| 140 | And we can use our css style sheet, logo and icon: | ||
| 141 | |||
| 142 | ``` | ||
| 143 | cp $filesdir/favicon.png ./ | ||
| 144 | cp $filesdir/logo.png ./ | ||
| 145 | cp $filesdir/style.css ./ | ||
| 146 | ``` | ||
| 147 | |||
| 148 | ### Bells and whistles | ||
| 149 | |||
| 150 | I like stagit's simplicity, but there are a couple of things that I want to | ||
| 151 | add or change: | ||
| 152 | |||
| 153 | * I would like every page to show a simple footer at the bottom of | ||
| 154 | each page. | ||
| 155 | * I would like to have a download button so that people who don't use git | ||
| 156 | can still download my files. This makes sense especially for those | ||
| 157 | repos that are mostly documents, such as my | ||
| 158 | [lecture notes](https://git.tronto.net/mathsoftware) or my | ||
| 159 | [FMC tutorial](https://git.tronto.net/fmctutorial). | ||
| 160 | * I would like to convert README.md files to html. | ||
| 161 | |||
| 162 | If I were calling stagit by hand after each `git push`, I | ||
| 163 | could simply make these changes with a text editor. But I want to automate | ||
| 164 | this! How can we edit files in a shell script? | ||
| 165 | |||
| 166 | Enter `ed`, [the standard editor](https://www.gnu.org/fun/jokes/ed-msg.html). | ||
| 167 | `ed` is a [line text editor](https://en.wikipedia.org/wiki/Line_editor) | ||
| 168 | initially released with UNIX Version 1. I am going to talk about it more | ||
| 169 | extensively in the next episode of my | ||
| 170 | [man page reading club](../2022-05-29-man/) | ||
| 171 | series. Without going into detail, `ed` does not show you the text you | ||
| 172 | are editing in a 2-dimensional windows: instead, it offers you a command | ||
| 173 | line prompt that you can use to run editing commands, such as `a` to add | ||
| 174 | text or `p` to print one or more lines of the file. | ||
| 175 | |||
| 176 | This might seems like a totally cumbersome way of editing a file, but | ||
| 177 | there is one nice side-effect: `ed` is completely scriptable. This means | ||
| 178 | that if you know exactly what the file you want to edit looks like, | ||
| 179 | you can write the commands you want to run in advance and feed them to | ||
| 180 | the editor via standard input, instead of typing them interactively. | ||
| 181 | This is exactly what we want to do! | ||
| 182 | |||
| 183 | Going back to my stagit setup, say we have a file `bottom.html` that | ||
| 184 | looks like this: | ||
| 185 | |||
| 186 | ``` | ||
| 187 | <hr class="line"> | ||
| 188 | <footer> <table> | ||
| 189 | <tr> <td class="contact"> | ||
| 190 | Back to <a href="https://sebastiano.tronto.net"> sebastiano.tronto.net </a> | ||
| 191 | </td> | ||
| 192 | <td class="hosted"> | ||
| 193 | Generated with <a href="https://codemadness.org/stagit.html">stagit</a> | ||
| 194 | </td> </tr> | ||
| 195 | </table> </footer> | ||
| 196 | ``` | ||
| 197 | |||
| 198 | and we want to insert its content in the file `file.html`, before | ||
| 199 | the line that contains the closing tag `</body>`. We can use the | ||
| 200 | following one-liner: | ||
| 201 | |||
| 202 | ``` | ||
| 203 | printf '%s\n' "/<\/body>" i "$(cat bottom.html)" . w | ed -s file.html | ||
| 204 | ``` | ||
| 205 | |||
| 206 | Here the `printf` command is used to feed the tokens `/<\/body>`, `i`, | ||
| 207 | `$(cat bottom.html)`, `.` and `w` to `ed`. These are going to | ||
| 208 | be interpreted as: "search for the closing tag `</body>`; | ||
| 209 | insert the following text until you encounter a single dot on a line: | ||
| 210 | [contents of the file `bottom.html`] single dot; save." | ||
| 211 | If this seems obscure, I suggest you read | ||
| 212 | [`ed`'s manual page](https://man.openbsd.org/OpenBSD-7.2/ed), or wait | ||
| 213 | for my next blog post! | ||
| 214 | |||
| 215 | The command for adding the download button is similar, after we | ||
| 216 | generate a zip archive of the repository using `git archive`: | ||
| 217 | |||
| 218 | ``` | ||
| 219 | git archive HEAD -o "$basedir/$name.zip" | ||
| 220 | printf '%s\n' \ | ||
| 221 | "/log\.html\">Log<\/a>" i \ | ||
| 222 | "<a href=\"$baseurl/$name.zip\">Download</a> |" . w \ | ||
| 223 | | ed -s file.html | ||
| 224 | ``` | ||
| 225 | |||
| 226 | Here I am using backlashes to ignore the newline character, so that | ||
| 227 | I can use more lines for readability. | ||
| 228 | |||
| 229 | The two code snippets above have to be run for every html file generated | ||
| 230 | by stagit. To loop over all these files, you can use `find`: | ||
| 231 | |||
| 232 | ``` | ||
| 233 | for f in $(find "$htdir" -name "*.html"); do | ||
| 234 | [stuff...] | ||
| 235 | done | ||
| 236 | ``` | ||
| 237 | |||
| 238 | The command to turn README.md files into a formatted html page is a bit | ||
| 239 | more complicated, but I will try to keep the explanation short, since | ||
| 240 | this post is already quite long. Feel free to send me an email if you | ||
| 241 | have questions! | ||
| 242 | |||
| 243 | To have an idea of what the README.md.html file generated by | ||
| 244 | stagit looks like, you can check out the html of | ||
| 245 | [this page](https://codemadness.org/git/stagit/file/README.html), | ||
| 246 | for example (right click and "View page source" or something similar in | ||
| 247 | most browsers, or `curl [URL]` if you are cool). | ||
| 248 | |||
| 249 | First, since I am using bare git repositories, I need to actually "create" | ||
| 250 | the original README.md file - instead of using its rendered-as-plain-text | ||
| 251 | html version generated by stagit - using `git show`. Then we need to | ||
| 252 | remove from README.md.html all the lines that | ||
| 253 | are part of the code listing, i.e. all those that contain a `class="line"` | ||
| 254 | string. The `ed` command to do this is `g/class=\"line\"/d`. Then we | ||
| 255 | need to remove a couple more lines and finally we can insert the result | ||
| 256 | of the command `lowdown file/README.md`, which converts the markdown | ||
| 257 | file to html, into the correct place. The final result is: | ||
| 258 | |||
| 259 | ``` | ||
| 260 | git show master:README.md > file/README.md | ||
| 261 | printf '%s\n' \ | ||
| 262 | g/class=\"line\"/d \ | ||
| 263 | "/<pre id=\"blob\">" d d i "$(lowdown file/README.md)" . w \ | ||
| 264 | | ed -s file/README.md.html > /dev/null | ||
| 265 | ``` | ||
| 266 | |||
| 267 | ### stagit-index | ||
| 268 | |||
| 269 | Just a quick mention to how I use stagit-index, the command used to | ||
| 270 | generate the index page. | ||
| 271 | The only change I make from the default configuration is to change | ||
| 272 | the links to each repository to point to the file list instead | ||
| 273 | of the log page. stagit-index writes its result to standard output, so | ||
| 274 | I can simply use `sed`: | ||
| 275 | |||
| 276 | ``` | ||
| 277 | stagit-index /home/git/*.git | sed 's|/log\.html||g' > "$basedir/index.html" | ||
| 278 | ``` | ||
| 279 | |||
| 280 | And that's it. Well, I also copy the style files and add a bottom bar, | ||
| 281 | and change the title from a `<span class="desc">` to an `<h1>` element, | ||
| 282 | again using `ed`. If you want to see the details you can check them out | ||
| 283 | [here](https://git.tronto.net/git-hooks/file/post-receive-stagit.html). | ||
| 284 | |||
| 285 | ## Conclusions | ||
| 286 | |||
| 287 | stagit is the perfect minimalist tool to publish your git repository | ||
| 288 | with a simple, static web interface. It requires nothing more than | ||
| 289 | an http server capable of serving html files. Static files are also | ||
| 290 | very simple to customize and tune to your needs. | ||
| 291 | |||
| 292 | I have wanted to make this post for quite some time now, mainly | ||
| 293 | as an excuse to clean up and document my scripts. I finally had some | ||
| 294 | time to work on this - even if scattered around multiple days. | ||
| 295 | |||
| 296 | As always, I have tried but failed to keep my post short - I am too | ||
| 297 | eager to explain everything I know as clearly as possbile! | ||
| 298 | I hope you enjoyed or found it useful. If you have questions or comments, | ||
| 299 | feel free to send me an [email](mailto:sebastiano@tronto.net). | ||
