Readout · 04 · Methodology

    How the meter works.

    Three read-only crawls, one rating rule, and a median. Nothing here needs trust in us.

    01 · Discovery

    Every listing, signature-verified

    Apps announce themselves with kind 31990 handler events (NIP-89). We crawl those from public relays, verify every signature, skip malformed events and strip control characters from names.

    Same-named listings are merged into one record. When one of them was published from the developer's own key, that listing is authoritative; a same-named listing from anyone else never overrides it.

    A seed list from nostrapps.com fills gaps for apps that never published a listing. Listings that look like generated bot fleets are flagged and excluded from the matrix.

    02 · Usage

    Distinct authors, not downloads

    Usage is a NIP-45 COUNT with distinct:author on relay.ditto.pub, keyed by the app's #client tag values over a 30-day window. It counts people who published something from the app, once each.

    One relay's view of public activity has two blind spots, and both are worth stating plainly.

    • Closed relays are invisible. An app whose users publish to private, paid or group relays is under-counted here however busy it is. That kind of interoperability matters as much as the public kind. This measurement simply cannot see it.
    • Untagged apps are unmeasured, not unpopular. NIP-89 says a client SHOULD let users opt out of the client tag, because it tells every relay which software a person uses. Declining it is following the spec, not ignoring it.

    So the meter never prints a zero. A number appears only where the crawl actually saw activity. Everywhere else the column reads , which means we did not measure it, not that nobody uses it.

    Usage affects ordering and review priority. It never affects a rating. An app with no usage figure at all can hold a Flawless cell on every NIP it implements.

    03 · Rating rule

    What a NIP requires of a client

    Every review starts with a read-only pass over the app's source against the current NIP text, then hand-tests the cells that pass flags. The test is always a round trip: publish in the app, open it in a second client. A cell earns Flawless only when the artifact works on both sides. Working only inside the app proves isolation, not interoperability.

    How to rate. Rate an app against what the NIP requires of a client (MUST, and SHOULD where interoperability visibly suffers) plus the features the app actually exposes. Optional methods, kinds, or flows the app doesn't offer are not gaps: a wallet that only implements pay_invoice is still Flawless on NIP-47. Borked means the app fails or crashes on the NIP's basic events, not that the feature is absent. Leave unimplemented NIPs unrated. Always check the current NIP text at nostr-protocol/nips before calling something a violation; NIPs change, and a rating against a stale draft is wrong.

    Where a NIP is silent, so are we. A great deal of real interoperability is a judgement the spec never settles: which relays a kind 1 ought to reach, how far to fall back, how hard to retry. Behaviour a NIP leaves open is recorded in the rater's note, not scored, because there is no text to score it against. Where raters disagree on a judgement the spec does cover, the median across independent keys is the mechanism, which is why every chip carries its rater count.

    Ratings are kind 31986 events (proposed NIP-85 reviews) labelled nip-compatibility and addressed to the app's kind 31990 listing. The same events power NostrHub.

    04 · Tiers and trust

    Four tiers, a median, and a marker for self-ratings

    Each rating carries one of four values. The tier of a cell is the median of every rating in it.

    Flawless

    1.0

    The full round trip works and the result renders and behaves correctly in other clients.

    Incomplete

    0.6

    The common cases work but something is missing: it renders what it receives but cannot compose it, or drops metadata other clients rely on.

    Isolated

    0.3

    It functions only inside the app. Other clients do not see the result, or see it wrong.

    Borked

    0.1

    The app fails or crashes on the NIP’s basic events. Never used for a feature that is simply absent.

    • Median, not mean. One outlier cannot move a cell. The overall tier of an app is the median of its per-NIP medians.
    • Self-ratings are marked. When every rater in a cell is the listing's own publisher, the chip carries a * and a dashed border. It counts, but it is not independent verification.
    • Rater counts are shown. A superscript on each chip and a count per app tell you how many distinct keys stand behind a tier.
    • Current NIP text, always. Reviews check the live spec, not a remembered draft. When a NIP revision shifts a score, it is noted in Updates.
    • Claimed is not verified. A hatched c means markers for the NIP were found in the client's source. Nobody has rated it yet.
    • Real keys, real names. Testing happens on throwaway keys so no community post is touched. Ratings are published under the rater's real key, because credibility is the point.
    • Findings go upstream. Every Borked or Isolated cell gets an issue filed with the app, with the evidence. The developer hears it from us first, and the cell changes when the fix ships.
    • Anyone can rerun it. The crawls, the join and the median are open source. Point them at the same relays and you get the same matrix.

    All crawling is read-only and the crawlers never sign or publish events. Review the code.