sebastiano.tronto.net

Source files and build scripts for my personal website
git clone https://git.tronto.net/sebastiano.tronto.net
Download | Log | Files | Refs | README

aoc-download.md (4535B)

      1 # Downloading AoC puzzle inputs with curl
      2 
      3 As in the [past couple of years](../2023-12-25-advent-of-code), soon
      4 I am going to take part in the daily programming challenges of the
      5 [Advent of Code](https://adventofcode.com). This time I decided to take
      6 some time and optimize my working setup a little bit - not that it matters
      7 for someone solving the problems at my speed.
      8 
      9 So let me show you how I automated downloading the inputs for the puzzles
     10 (which require you to login) using [curl](https://curl.se).
     11 
     12 ## The problem
     13 
     14 In case you are not familiar with the Advent of Code, it is a casual
     15 programming challenge that takes place every year in December. Every day
     16 a new problem is released, and you are supposed to write a program that
     17 solves it and submit your answer as a short string, usually a number.
     18 The fastest you solve the problems, the more internet points
     19 you get, and you can brag about it with your colleagues.
     20 
     21 The input data for each problem is given as a simple text file. Not
     22 everyone gets the same, there are a few (hundreds? thousands? not sure)
     23 of different inputs, and each user is assigned one.
     24 
     25 This means that one must log in to be able to download their puzzle input.
     26 To manually download the input file is quite tedious, so it would be nice
     27 if there was a way to automate this. Of course,
     28 [I am](https://github.polettix.it/ETOOBUSY/2022/11/28/aoc-inputs-downloader)
     29 [not](https://packagist.org/packages/colinodell/aoc-downloader)
     30 [the](https://github.com/johlinco/aoc-cli)
     31 [first](https://github.com/Gronner/aoc-downloader)
     32 [to](https://github.com/czyber/aoc-downloader)
     33 [think](https://github.com/martintc/aoc-downloader-rs)
     34 [about](https://github.com/scarvalhojr/aoc-cli)
     35 [doing](https://github.com/colinodell/aoc-downloader)
     36 [this](https://github.com/GreenLightning/advent-of-code-downloader).
     37 
     38 ## Session cookies
     39 
     40 If log in was not required, we could simply download the input file for,
     41 say, day 12 of 2024 and save it to `input.txt` with curl like this:
     42 
     43 ```
     44 $ curl https://adventofcode.com/2024/day/12/input -o input.txt
     45 ```
     46 
     47 But if you run this command, you'll get something like this:
     48 
     49 ```
     50 Puzzle inputs differ by user.  Please log in to get your puzzle input.
     51 ```
     52 
     53 To make curl act as if we were logged in, we need a
     54 [session cookie](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Cookies).
     55 If you log into [adventofcode.com](https://adventofcode.com) on a browser such
     56 as Firefox, you can right-click on the page, select "Inspect" and then navigate
     57 to Storage and then Cookies.
     58 
     59 ![Screenshot of Firefox](./cookie.png)
     60 
     61 The long alphanumerical string under "Value" is your session cookie. Anyone
     62 who gets a hold of it can pretend to be you on this website - by the way,
     63 this is how the famous YouTube channel
     64 [Linus tech tips got hacked](https://www.youtube.com/watch?v=yGXaAWbzl5A)
     65 a couple of years ago.
     66 
     67 Save this cookie somewhere, for example in an
     68 [environment variable](https://en.wikipedia.org/wiki/Environment_variable)
     69 called `AOC_SESSION_COOKIE`. Then run your curl with an extra option, like so:
     70 
     71 ```
     72 $ curl https://adventofcode.com/2024/day/12/input -H "cookie: session=$AOC_SESSION_COOKIE" -o input.txt
     73 ```
     74 
     75 And you are good to go!
     76 
     77 ## The fancy script
     78 
     79 Typing out the full command above may look cooler than downloading the file
     80 by right-clicking "Save page as", but it is just as annoying. So obviously I
     81 saved that command to
     82 [a script](https://git.tronto.net/scripts/file/aoc.html)
     83 that also detects the correct file to download based on the current date
     84 (or on two optional parameters for the year and the day). My script also
     85 opens up an editor and [Lynx](https://en.wikipedia.org/wiki/Lynx_(web_browser))
     86 session to set me up to speed, but that is not as interesting.
     87 
     88 Happy coding!
     89 
     90 ### Update 2025-12-01
     91 
     92 An interesting addition to my script: now it can also download and save
     93 the `<code>` blocks and save them each in a separate file. This can
     94 be useful because these blocks contain example input whose solution is
     95 provided in the problem's page. The relevant code snippet, based on
     96 `curl` and `sed`, is this:
     97 
     98 ```
     99 url="https://adventofcode.com/$year/day/$daynozero"
    100 curl "$url" | sed -n '/<pre><code>/,/<\/code><\/pre>/ p' | (\
    101         i=1
    102         rm -f "code-$i.txt"
    103         while read line; do
    104                 if [ "$line" = "</code></pre>" ]; then
    105                         i=$((i + 1))
    106                         rm -f "code-$i.txt"
    107                 else
    108                         echo "$line" | sed 's/<.*>//g' >> "code-$i.txt"
    109                 fi
    110         done
    111 )
    112 ```
    113