Quick and Dirty Website Change Monitoring
Let’s say, you need to monitor a website for changes and you really don’t have a lot of time to set things up. Also solving the problem with money using services, such as changedetection.io or visualping.io, have failed you, because their accesses are probably filtered out.
I’ve come up with the following scrappy solution. First, I want to get push notifications to my phone. So I installed simplepush on my phone. There are a couple of these services, this was just the first I found and it works well.
I have a couple of Linux servers. So I just logged in to one,
installed ntfy and the Links
text-based web
browser
(probably links2
in your package manager).
Configure ntfy
with your simplepush key:
# ~/.config/ntfy/ntfy.yml
backends:
- simplepush
simplepush:
key: 12345
Afterwards, you can just dump the website to a text file with Links and send a push notification to your phone when something changes:
#!/usr/bin/env bash
# By starting without old.txt, we get a notification when we start the script
rm -f old.txt
# Let's be polite here and not hammer the site.
POLL_FREQ_MIN=15
URL="https://example.com/"
while true; do
touch old.txt
links -dump "$URL" > new.txt
if ! diff -u old.txt new.txt > diff.txt; then
# It's hard to condense the changes (diff.txt) into something readable,
# so we just send the URL to easily click on on the phone.
ntfy send "Check $URL"
fi
mv new.txt old.txt
sleep $(($POLL_FREQ_MIN * 60))
done
This only works for simple websites and there is a lot left to be desired. But it is doable in the 10 minutes of productivity a newborn baby gives you and it works to get appointments at government offices in Spain. 😉
PS. This blog post was written in another 10-minute productivity window.