watching for changes

One of the differences between the kind of text-mode/compile interface that I like to use, versus the WYSIWYG-style editors that the rest of the world has been using since forever, is that you generally have to do something or other in order to see whether your changes are having any effect. I've spent some time yesterday and today trying to come up with a mechanism to ease that transition, and I think I like what I have.

There's a solution to this that comes with nikola, which is nikola auto. But that injects javascript in every page so that the pages reload continuously, which causes a lot of unnecessary flickering. I would like for the reload to happen in the browser over there, in a different window than over here where I'm typing, but not for there to be a bunch of unnecessary reloads. The terminal experience with nikola auto is also a little wonky, because it continuously scrolls off of the page. So I've basically reimplemented this feature a couple of different times in brittle little shell scripts. I think that what I have right now is going to last, though.

The solution that I seem to like is a little javascript snippet that I got out of Microsoft Copilot and didn't read very carefully. The live version should be visible here, thanks to the magic of symlinks, but I've got today's version archived and shown below:

watching-for-changes/reload_on_sentinel.js (Source)

(function pollSentinel(interval = 1000) {
  let lastContent = null;

  async function checkForChange() {
    try {
      const response = await fetch('/sentinel', { cache: 'no-store' });
      if (!response.ok) throw new Error('Fetch failed');
      const text = await response.text();

      if (lastContent === null) {
        lastContent = text;
      } else if (text !== lastContent) {
        console.log('Sentinel changed — reloading...');
        location.reload();
      }
    } catch (err) {
      console.error('Error checking sentinel:', err);
    }
  }

  setInterval(checkForChange, interval);
})();

This is just checking periodically for a document at the top of the server, /sentinel, and seeing if its content has changed. If the content has changed, it calls location.reload().

Of course the way that cool people would do this is to use some AJAX channel that stays open, so that the updated information can get sent via push rather than having a whole pile of requests. But the whole point of this blog tool that I'm building is that I don't have to run stuff on the server, so that it's as portable as possible. I won't update the template to include the script on every page; instead I'll just include it on pages where I'm actively working.

To make this actually do things, I needed to create sentinel as part of my build process. I wrote a Makefile that wraps around nikola build, for reasons that are worth , but the heart of the sentinel matter is

output: $(NEWEST)
     nikola build
     date > $@/sentinel
     touch $@

The rest of the Makefile is built to support a continuous target that basically replicates nikola auto:

continuous: all
     while true ; do \
             $(MAKE) -q -s latex output || $(MAKE) newest all; \
             /bin/echo -n -e \\r $$(date +"%F %T.%N :") "" ; \
             sleep 1; \
     done

This has the effect that, when there's nothing to update, the terminal with make continuous running has an updating clock on the final line. If weird stuff is happening, it won't scroll off of the screen. Debugging is precious.

In order to build use make's timestamp-based logic, I find the newest (input) file in the source tree and made the output directory depend on that file. (If that newest file isn't actually important, then nikola build doesn't make any changes.) I had to exclude some output files (in cache and output, and the database that nikola uses to keep track of its state), and some auto-save files that emacs makes.

SOURCE_FILES := $(shell find . -type f | \
                     egrep -v '(cache|output|.doit.db|#)' )
NEWEST       := $(shell ls -t $(SOURCE_FILES) | head -1 )

A separate post talks about why I'm including LaTeX stuff, which is honestly pretty nifty. But it was going to be a learning experience for me to figure out how to incorporate LaTeX into nikola build. On the other hand, I wanted my directory structure for LaTeX illustrations to match the directory structure for the posts they are attached to, in case someday I do get nikola to build them for me. That means that I'm going to have an unknown number of LaTeX projects following an unknown directory structure. So I have written the following:

LATEX_MAKEFILES := $(shell find latex -name Makefile)
LATEX_FOLDERS   := $(dir $(LATEX_MAKEFILES))
LATEX_TEX       := $(shell find $(LATEX_FOLDERS) \
                       -maxdepth 1 -name [^.]\*.tex)
LATEX_PDFS      := $(patsubst %.tex,%.pdf,$(LATEX_TEX))

latex: $(LATEX_PDFS)
     touch $@
%.pdf : %.tex
     $(MAKE) -C $(dir $@)
latex-clean:
     $(foreach folder, $(LATEX_FOLDERS), $(MAKE) -C $(folder) clean;)

This hunts for Makefiles, which can be pretty minimal. Any .tex file in the same directory as a Makefile is assumed to generate a .pdf when make is called without an explicit target. When all of those .pdfs have been made, we update the timestamp on the latex directory, so that make -q will work.

Here's the entire Makefile as of right now, which is possibly different from the live version.