Commons:Machine-readable data/szl
On Wikimedia Commons, a lot of metadata (including license and author) are not machine readable. There is an API module, iiprop=extmetadata which can be used to retrieve some values (example), but as the information is entered as free text into the file description page itself, this is not perfect. The ongoing Structured data on Commons project aims to move the metadata as fully-structured data and will eventually supersede the machine-readable data presented in this page.
In the meantime, and to ease a transition towards more structured data at a future time, Wikimedia Commons use a set of standard templates which have been made machine-readable in some ways, through HTML elements. Some scripts already make use of that. It is worth noting that this data is available for any wiki using Wikimedia Commons, where it can be read from the html of the File: page just as other local data.
Machine readable data
Machine readable data set by infobox templates
These are several standard infobox templates tagging different elements of the template with different tags to allow parsing of the information. Several different styles of tags are used:
- Microformat tags follow industry standards and can be parsed by already existing tools.
- <td> id attributes (identifiers) are custom markings which allow more complete tags, which have to be read by custom tools. Most universal infoboxes have two column structure: column #1 holds name of the field and column #2 holds the value
- Traditionally <td> id attributes were used to tag the name call in the first column in a row. To get the data, you would need to get the contents of the following
<td>cell in the second column. - {{Creator}} and {{Institution}} templates have more complicated structure, so the cells with the actual data are tagged with
attributes using magenta background.
- Traditionally <td> id attributes were used to tag the name call in the first column in a row. To get the data, you would need to get the contents of the following
| Template | Template parameter name | Description | <td> id attribute | Microformat | Comment |
|---|---|---|---|---|---|
| {{Information}} | description | file description | fileinfotpl_desc | hProduct.description. | Often contains multiple languages annotated with {{Lang}}. |
| {{Information}} | date | date the original work was created | fileinfotpl_date | hCalendar vevent.dtstart | Sometimes additionally, or only, contains publication date. These two dates have different meanings for copyright. When used, {{Date context}} can indicate the difference. Microformat added by {{Date}} template |
| {{Information}} | source | file's source | fileinfotpl_src | Often contains entire tables. We have no good way to deal with this source templates yet. Source templates often have references to catalogue IDs, but these are also not machine readable. | |
| {{Information}} | author | file's author | fileinfotpl_aut | This can be author, creator and/or copyright holder and is used mixed. Often contains the {{Creator}} template which is described below. | |
| {{Information}} | permission | zwōla/licyncyjŏ na zbiōr | fileinfotpl_perm | ||
| {{Information}} | other versions | inksze wersyje plikōw | fileinfotpl_ver | ||
| {{Artwork}} | description | ôpis dzieła kōnsztu | fileinfotpl_desc | hProduct.description | |
| {{Artwork}} | date | datōmu utworzyniŏ ôryginalnego dzieła kōnsztu | fileinfotpl_date | hCalendar vevent.dtstart | microformat added by {{Date}} template |
| {{Artwork}} | source | źrōdło zbioru | fileinfotpl_src | ||
| {{Artwork}} | artist | twōrca dzieła kōnsztu | fileinfotpl_aut | "hProduct.fn value" | |
| {{Artwork}} | author | autōr dzieła kōnsztu | fileinfotpl_aut | "hProduct.fn value" | |
| {{Artwork}} | permission | zwōla/licyncyjŏ na zbiōr i dzieło kōnsztu | fileinfotpl_perm | ||
| {{Artwork}} | other versions | inksze wersyje zbioru | fileinfotpl_ver | ||
| {{Artwork}} | title | tytuł dzieła kōnsztu | fileinfotpl_art_title | hProduct.fn | |
| {{Artwork}} | object type | zorta ôbiektu kōnsztu | fileinfotpl_art_object_type | ||
| {{Artwork}} | medium | technika abo strōna sztuki | fileinfotpl_art_medium | ||
| {{Artwork}} | dimensions | fasōngi dzieła kōnsztu | fileinfotpl_art_dimensions | ||
| {{Artwork}} | gallery | instytucyjŏ posiadajōncŏ dzieło kōnsztu | fileinfotpl_art_gallery | ||
| {{Artwork}} | location | lokalizacyjŏ dzieła w instytucyji | fileinfotpl_art_location | hProduct.locality | |
| {{Artwork}} | accession number | numer przijścia dzieła sztuki | fileinfotpl_art_id | hProduct.identifier | |
| {{Artwork}} | object history | historyjŏ ôbiektu dzieła kōnsztu | fileinfotpl_art_object_history | ||
| {{Artwork}} | exhibition history | historyjŏ wystŏwki dzieła kōnsztu | fileinfotpl_art_exhibition_history | ||
| {{Artwork}} | credit line | credit line of the artwork | fileinfotpl_art_credit_line | ||
| {{Artwork}} | inscriptions | szkryfty na dziele kōnsztu | fileinfotpl_art_inscriptions | ||
| {{Artwork}} | notes | zapiski ô dziełach kōnsztu | fileinfotpl_art_notes | ||
| {{Artwork}} | references | przypisy powiōnzane z dziełym kōnsztu | fileinfotpl_art_references | ||
| {{Book}} | Author | autōr ksiōnżki | fileinfotpl_author | ||
| {{Book}} | Editor | redachtōr ksiōnżki | fileinfotpl_book_editor | ||
| {{Book}} | Translator | przekłŏdŏcz ksiōnżki | fileinfotpl_book_translator | ||
| {{Book}} | Illustrator | ilustrator ksiōnżki | fileinfotpl_book_illustrator | ||
| {{Book}} | Title | tytuł ksiōnżki | fileinfotpl_book_title | ||
| {{Book}} | Subtitle | podtytuł ksiōnżki | fileinfotpl_book_subtitle | ||
| {{Book}} | Series title | tytuł seryje ksiōnżki | fileinfotpl_book_series-title | ||
| {{Book}} | Authority file | authority control data | fileinfotpl_book_authority | ||
| {{Book}} | Publisher | wydŏwca ksiōnżki | fileinfotpl_book_publisher | ||
| {{Book}} | Printer | drukeraja ksiōnżki | fileinfotpl_book_printer | ||
| {{Book}} | Year of publication | datōm abo rok publikacyje ksiōnżki | fileinfotpl_date | ||
| {{Book}} | Place of publication | plac abo miasto publikacyje ksiōnżki | fileinfotpl_book_place-of-publication | ||
| {{Book}} | Language | gŏdka ksiōnżki | fileinfotpl_book_language | ||
| {{Book}} | Description | ôpis ksiōnżki | fileinfotpl_desc | ||
| {{Creator}} | Name | Miano twōrcy | creator | vCard.fn | |
| {{Creator}} | Alternative names | Alternatywne miana twōrcy | fileinfotpl_creator_alt-name_value | vCard.nickname | |
| {{Creator}} | Description | Nŏrodność i fach/fachy twōrcy | fileinfotpl_creator_desc_value | vCard.note | |
| {{Creator}} | Date of death | Datōm śmierci twōrcy | fileinfotpl_creator_deathdate_value | ||
| {{Creator}} | Date of birth | Datōm narodzyniŏ twōrcy | fileinfotpl_creator_birthdate_value | vCard.bday | |
| {{Creator}} | Location of birth/death | Lokalizacyjo śmierci twōrcy | fileinfotpl_creator_deathloc_value | ||
| {{Creator}} | Location of birth | Plac narodzyniŏ twōrcy | fileinfotpl_creator_birthloc_value | ||
| {{Creator}} | Work period | Work period of creator | fileinfotpl_creator_work-period_value | ||
| {{Creator}} | Work location | Lokalizacyjo roboty twōrcy | fileinfotpl_creator_work-location_valuev | ||
| {{Creator}} | Image | portret abo symniyńcie #przedstawiać twōrcy | fileinfotpl_creator_image | ||
| {{Creator}} | Authority file | Kōntrola autorytatywna zwiōnzanŏ z twōrcōm | fileinfotpl_creator_authority_value |
| |
| {{FileContentsByBot}} | (roztōmajte) | depends, please confer {{FileContentsByBot}} | (various) | hproduct-by-bot | big data set and still growing, please confer {{FileContentsByBot}} |
| {{Photograph}} | title | tytuł symniyńciŏ | fileinfotpl_art_title | hProduct.fn | |
| {{Photograph}} | description | ôpis symniyńciŏ | fileinfotpl_desc | hProduct.description | |
| {{Photograph}} | original description | ôryginalny ôpis archiwum symniyńciŏ | fileinfotpl_desc | hProduct.description | |
| {{Photograph}} | date | datōm utworzyniŏ ôryginalnego dzieła kōnsztu | fileinfotpl_date | hCalendar vevent.dtstart | microformat added by {{Date}} template |
| {{Photograph}} | medium | technika abo postrzodek fotograficzny | fileinfotpl_art_medium | ||
| {{Photograph}} | dimensions | fasōngi symniyńciŏ | fileinfotpl_art_dimensions | ||
| {{Photograph}} | artist | twōrca symniyńciŏ | fileinfotpl_aut | "hProduct.fn value" | |
| {{Photograph}} | institution | instytucyjŏ posiadajōncŏ dzieło kōnsztu | fileinfotpl_art_gallery | ||
| {{Photograph}} | location | lokalizacyjŏ symniyńciŏ w instytucyji | fileinfotpl_art_location | hProduct.locality | |
| {{Photograph}} | source | źrōdło zbioru | fileinfotpl_src | ||
| {{Photograph}} | permission | zwōla/licyncyjŏ na zbiōr i dzieło kōnsztu | fileinfotpl_perm | ||
| {{Photograph}} | other versions | other versions of the file | fileinfotpl_ver | ||
| {{Photograph}} | accession number | accession number of the photograph | hProduct.identifier | ||
Alternative format for CommonsMetadata
Because the table + id based format proved very hard to add to templates which were not formatted similarly to the Commons information template, CommonsMetadata allows an alternative format, similar to license templates: the whole information template has to be enclosed in a fileinfotpl class and the tag containing the specific information needs to have a fileinfotpl_* class (same names as above, but class, not id).
Machine readable data set by license templates
Introduced in October 2010, using classes <span class="licensetpl_XXX">
licensetpl- An element identifying a license. Wraps the entire license code and should be a SINGLE license, not a multi license.
licensetpl_short- Short name of the license: “Public domain”, “CC BY-SA 3.0”, “CC by 2.0 fr”, etc.
licensetpl_long- Long name of the license: “Public domain”, “Creative Commons Attribution-Share Alike 3.0”,
licensetpl_attr_req- Whether attribution is required. “true” or “false”.
licensetpl_attr- The requested attribution: Free text.
licensetpl_link_req- Whether a link to the license is required for this license. “true” or “false”.
licensetpl_link- The link to the license deed. “www.creativecommons.org/licenses/by-sa/XXX/YYY”
licensetpl_nonfree- “true“ if this is a non-free license (not used on Commons, only on wikis with an EDP)
Multiple licensetpl blocks for the same work might be wrapped in a block using the class licensetpl_wrapper.
Templates setting this information
- Templates setting
licensetplinclude:
{{PD-Layout}}, {{Cc-by-sa-3.0-migrated}}, {{Cc-by-layout}}, {{Cc-by-sa-layout}}, {{Cc-zero}}, {{FAL}}, {{GFDL}}, {{GFDL-1.2}}, {{GPL}} i {{LGPL}}.
Machine readable data set by style formatting templates
Style formatting templates, meant to provide uniform styles to different families of non-license templates, carry machine readable data identifying these families.
| Szablōna | Purpose | class name |
|---|---|---|
| {{Restriction-Layout}} | used by Restriction tags | restrictiontemplate |
| {{FoP-Layout}} | used by freedom of panorama tags | foptemplate |
| {{Partnership-Layout}} | used by Partnership templates | partnershiptemplate |
| {{Source-Layout}} | used by generic Source templates | sourcetemplate |
| {{Created with}} | used by Created with ... templates | createdwithtemplate |
Machine readable data set by non-copyright restriction templates
Templates regarding non-copyright legal restrictions carry these classes to identify specific types of restrictions.
| Template(s) | Purpose | class name |
|---|---|---|
| {{Trademarked}} | Trademarked images | restriction-trademarked |
| {{Copydesign}} | Copyrighted designs | restriction-design |
| {{Communist symbol}} | Communist symbols | restriction-communist |
| {{Italy-MiBAC-disclaimer}} {{Soprintendenza}} | Italian cultural goods | restriction-ita-mibac |
| {{Australian Commonwealth reserve}} | Australian reserves | restriction-aus-reserve |
| {{Personality rights}} {{Romania personality rights}} | Personality rights | restriction-personality |
| {{2257}} | Child Protection and Obscenity Enforcement Act warning (United States) | restriction-2257 |
| {{Costume}} | Costuming | restriction-costume |
| {{Fan art}} | Fan art | restriction-fan-art |
| {{Currency}} | Currency | restriction-currency |
| {{IHL Symbol}} | Symbols restricted by International Humanitarian Law | restriction-ihl |
| {{Nazi symbol}} | Nazi and fascist symbols | restriction-nazi |
| {{Insignia}} | Official insignia | restriction-insignia |
Machine readable data set by specific templates
More machine-readable data are set. Here is a non-exhaustive list:
- {{Personality rights}}
<span class="commons-template-name" style="display:none" id="commons-template-personality-rights">Personality rights</span>- {{Credit line}}
<td id="fileinfotpl_credit" class="fileinfo-paramfield fileinfotpl_credit" style=""></td>
Machine-readable data set by location templates
{{Location}} and similar templates add machine-readable geocodes in the following format: <span class="geo">12.34;24.68</span> (latitude and longitude as floating-point numbers, separated by a semicolon). The coordinates use the en:WGS84 system (same as the GPS and most online maps). See Commons:Geocoding for more details.
Usage
MediaWiki API
(Open in API Sandbox) that returns some useful parameters such as Credit, Artist, LicenseUrl and Copyrighted and is used by Media Viewer, for example.
Scripts using machine-readable data
- MediaWiki:Gadget-Stockphoto.js
- MediaWiki:GallerySlideshow.js
- MediaWiki:Gadget-AddInformation.js
- MediaWiki:FileContentsByBot.js
External tools
Zobŏcz tyż
- Category:Templates generating microformats
- Commons:WikiProject Microformats
- Category:Files with lack of machine-readability
- Experimental, discontinued projects: Commons:API, Commons:Commons API
Defining new machine readable data
- Do NOT use HTML id's, use classes. An ID can only be used once per page and most of these fields can occur multiple times per page. Consider for instance descriptions of derivative works, which can include information about the original and the derivative.
- When possible, wrap the actual data, not some field header. This last method is historically used for all our Information templates, but much harder to support in the long run.
- Wrap data, not the way the data is formatted.
- Expect that formatting is lost when converting to data. Visual dress up is not part of the information.
- Don't wrap multiple units of information inside one field. There is a difference between a publication date and a creation date. Both are dates, but both are different 'data fields'. Also CC BY-SA-4.0-3.0-2.5 is not a license name, those would be 3 licenses with the name CC BY-SA-##.
- Make sure that the data value has one unit, or outputs one consistent unit.
Problems
There are a few things that are currently NOT or badly recognizable. These include:
- Derivative works
- Works included in works. See also Category:FoP_templates
- licenses derivates or works included in works are a mess.
- Author vs. Copyright holder
- usernames vs 'real names'
- Catalogue IDs etc
- VRTS permissions
- Publication date vs creation date
- Donating institutions of materials
- Anything that is NOT using the above structures is not recognizable at all and will require manual cleanup at some point.
- Heirs: {{Heirs-license}}
- Multilicensed CC works, that use {{Cc-by-3.0,2.5,2.0,1.0}}, {{Cc-by-sa-2.5,2.0,1.0}}, {{Cc-by-sa-4.0,3.0,2.5,2.0,1.0}} or {{Cc-by-all}}.
- Non-licensed works: {{Copyrighted free use}}, {{Attribution}} (Problem, how to describe this grant of rights success ?)
- Improvised File description templates like User:Tevaprapas/Information
- Templates denoting the copyright of partials of the work: {{Copyright information}}