Commons:Spacemedia/Administration

Web frontend on Toolforge

Workers on Cloud VPS

Setup a new worker

  • Ask for quota extension, see last one. Be careful to also ask for volume quota extension if needed
  • Create worker instance in Horizon:
    • Name: worker-N
    • Source: debian-13.0-trixie
    • Flavor: g4.cores8.ram32.disk20
  • In Horizon, create a volume (if needed) and attach it to the instance (multi-instance attachment of a single volume is not yet possible)
  • Access worker via SSH by following documentation
  • Install requirements
wget -qO - https://packages.adoptium.net/artifactory/api/gpg/key/public | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/adoptium.gpg > /dev/null
echo "deb https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | sudo tee /etc/apt/sources.list.d/adoptium.list
sudo apt update
sudo apt install -y cron temurin-25-jdk webp exiftool
echo " export JAVA_HOME=/usr/lib/jvm/temurin-25-jdk-amd64" | tee ~/.bash_aliases
bash
  • Create project directory:
sudo mkdir -p /data/project/spacemedia /data/project/spacemedia/conf /data/project/spacemedia/files /data/project/spacemedia/logs
sudo chown -R $USER:wikidev /data/project/spacemedia
cd /data/project/spacemedia
ln -s /data/project/spacemedia ~/spacemedia
for script in env.sh ; do wget -q -O $script https://gitlab.wikimedia.org/toolforge-repos/spacemedia/-/raw/main/scripts/$script && chmod u+x $script ; done
git clone https://gitlab.wikimedia.org/toolforge-repos/spacemedia.git --single-branch --branch=main
for script in build.sh list-jobs.sh monitor.sh reset.sh run.sh start.sh stop.sh upgrade.sh watch.sh whats-running.sh ; do ln -s /data/project/spacemedia/spacemedia/app/scripts/$script $script ; done
  • Copy env.sh from another worker and check permission (-rwx------)
  • Build project:
./reset.sh

Control file system usage

  • This disables process accounting to avoid having a huge /var/log/account/pacct:
 sudo accton off
 sudo systemctl stop acct
 sudo systemctl disable acct
 sudo truncate -s 0 /var/log/account/pacct
  • Configure /etc/systemd/journald.conf to be sure it doesn't go berserk:
SystemMaxUse=200M
SystemKeepFree=500M
SystemMaxFileSize=50M
SystemMaxFiles=100
  • Then:
sudo systemctl restart systemd-journald