About the harvester

Hoplonomy collects facts about hand weapons from public online sources. A program called the harvester makes the requests. This page tells the operators of those sources who is calling and how to reach us.

User agent

Every request carries this user agent:

Hoplonomy-harvester-bot/<version> (+<url>)

<version> is the harvester’s version number. <url> is the address of this page. The user agent contains no personal name and no e-mail address.

Sources

The harvester reads four sources:

• Wikidata, through its query service and its API.
• Wikipedia (English), through its API.
• The Getty Art & Architecture Thesaurus.
• The Metropolitan Museum of Art collection API.

It requests nothing from any other address, except robots.txt on the hosts of these sources. Each source is approved, with its licence and terms recorded, before the first request.

Rates

• One request at a time per source.
• Where a source publishes a rate limit, we stay at or below it. Where it publishes none, we send at most one request per second.
• On HTTP 429 or 503 the harvester waits for the Retry-After time, or 60 seconds when none is given.
• After five failed requests in a row, it stops that source’s run.
• Where a source documents it (the Wikimedia API), the harvester sends maxlag=5.

What it honours

The harvester fetches robots.txt at the start of each run. It never requests a page that robots.txt disallows for Hoplonomy-harvester-bot, or for * when no group names it. For API endpoints, where a source publishes an API policy, it follows that policy instead, and the identifying user agent above.

Contact

The harvester has not sent any request yet. Before its first request, a role mailbox for asking us to slow down, stop or change something will be published here, and the harvester refuses to run until it is. It will be a role mailbox, not a person. There is no contact form.