Implement lossless compression task in OpenRefine transfer

Hi!

I am working on an import of orthophotos of Mecklenburg-Vorpommern. Unfortunately, these files don't have a compression. As the Wikimedia Foundation is working on recompressing already uploaded TIFF files without compression, I would like to ask if there is a possibility to implement a task within the OpenRefine workflow, where the file is caught/downloaded, losslessly compressed (per GDAL), and uploaded to Commons? One file is 400 MiB large (10000×10000 pixels @ 4 8-bit channels). The download list can be found here as reference: Geoportal MV. Thank you :) --PantheraLeo1359531 😺 (talk) 10:49, 14 June 2026 (UTC)

Tracked in Phabricator
Task T427949

Hey @PantheraLeo1359531 👋

Unfortunately, OpenRefine does not such concept of such workflow you're describing. All it can do is work with the data! What I can recommend is to run the compression yourself and upload to Commons; with the file location on disk set in OpenRefine rows instead of the link.

Technically, I can offer a solution in Curator, but I'll be in Berlin for the rest of the month and probably would not have enough free time during that period. Later on, I'd have some free time.

As such, what sort of compression(s) are you looking at? Do you already know which command(s) and/or software you would need/run for a given tiff, say, for this one ? -- DaxServer (talk) 12:36, 21 June 2026 (UTC)

Good evening and thank you for your answer :3

This workflow with downloading would be possible and I did it for some cases. The problem is, that, sometimes the dataset is larger than 1 TB. Reuploading from my PC would take several weeks :(.

Thank you, it has time :3. As TIFFs are presented in GeoTIFFs, they have embedded coordinates that can be used in tools like QGIS. For this, a mindful compression via GDAL is good, like in:

gdal_translate input.tif output_cog.tif \

-of COG \

-co COMPRESS=ZSTD \

-co LEVEL=9

I could upload them per OpenRefine as uncompressed files, then maybe a bot compresses them and if the GeoTIFF attributes are intact, then the uncompressed files could be deleted from the server. But I don't know if there is a better solution ☺️. Kind regards

--PantheraLeo1359531 😺 (talk) 18:36, 21 June 2026 (UTC)

Curator files

Hi, is there a way you can add "| other_fields = {{information field|Academic context|Geospatial street-level imagery from external site.}}" to the information section in Curator files per this request? Thanks ! CubanoBoi (talk) 21:28, 21 June 2026 (UTC)

Nevermind! I see that its already been added. CubanoBoi (talk) 23:16, 21 June 2026 (UTC)