Hyperindex

Hyperindex (hi) is a Go AT Protocol AppView server that indexes records and exposes them via GraphQL. Use it to:

• Index all hypercert-related records from the ATProto network in real time
• Query indexed data through a typed GraphQL API
• Backfill historical records from any user or the entire network
• Run your own indexer for full control over data availability and query performance

Built in Go. Source: github.com/gainforest/hyperindex. Tap reference implementation: github.com/bluesky-social.

Hosted production and staging endpoints: Certified Services.

Why indexers & discovery

AT Protocol is federated, so hypercert records are distributed across many PDSs instead of living in a single database. If an app wants to discover records across users and organizations, it needs a way to aggregate that network data into one queryable view.

Indexers handle that job. They consume network events, fetch and parse records by lexicon, normalize them into query-ready storage, and expose APIs for search, filtering, and aggregation.

Hyperindex is the reference indexer used in this ecosystem.

If you want to inspect indexers running across the broader ecosystem, use Hyperscan.

How it works

Hyperindex is Tap-first (recommended). Tap handles ingestion, and Hyperindex consumes Tap events, stores records, and exposes them via GraphQL.

ATProto Relay ──→ Tap ──→ Hyperindex Consumer ──→ Records DB ──→ GraphQL API
                                     │
                               Activity Log ──→ Admin Dashboard

Jetstream mode still exists as a legacy/non-Tap mode, but Tap is the preferred setup.

Quick start

For local development with default settings:

git clone https://github.com/gainforest/hyperindex.git
cd hyperindex
cp .env.example .env
go run ./cmd/hyperindex

Open http://localhost:8080/graphiql/admin to access the admin interface.

Register lexicons

Lexicons define the AT Protocol record types you want to index. You can register them via:

1. Admin GraphQL API at /graphiql/admin
2. Client admin UI at https://<client-url>/lexicons (you must log in with an admin DID)

mutation {
  uploadLexicons(files: [...])  # Upload lexicon JSON files
}

Or place lexicon JSON files in a directory and set the LEXICON_DIR environment variable.

For hypercerts, you would register the org.hypercerts.claim.* lexicons — see Introduction to Lexicons for the full list.

Query via GraphQL

Access your indexed data at /graphql:

# Query records by collection
query {
  records(collection: "org.hypercerts.claim.activity") {
    edges {
      node {
        uri
        did
        value
      }
    }
  }
}


# With typed queries (when lexicon schemas are loaded)
query {
  orgHypercertsClaimActivity(first: 10) {
    edges {
      node {
        uri
        workScope
        startDate
        createdAt
      }
    }
  }
}


# With typed filter queries (title contains "Hypercerts")
query {
  orgHypercertsClaimActivity(
    first: 10
    where: { title: { contains: "Hypercerts" } }
  ) {
    edges {
      node {
        uri
        title
        createdAt
      }
    }
  }
}

Endpoints

Deployment configuration

Use this section when deploying Hyperindex (backend, Tap, and client). It lists the environment variables you should set first for a reliable initial deployment, followed by optional variables for advanced tuning.

Baseline deployment variables

These are the variables you should set first to get a stable deployment running.

Note: some managed platforms (including Railway) may auto-provision a subset of variables.

Hyperindex backend

Tap service

Client (Next.js)

Hyperindex normalizes both NEXT_PUBLIC_HYPERINDEX_URL and HYPERINDEX_URL: it trims surrounding whitespace, removes trailing slashes, and prepends https:// when no scheme is provided.

Optional variables

Set these only when needed.

Backend

• ALLOWED_ORIGINS
• TAP_DISABLE_ACKS

TAP_DISABLE_ACKS is configured on the backend/indexer service. In some deployments, ACK mode (TAP_DISABLE_ACKS=false) can cause repeated websocket disconnect loops (for example: connection reset by peer, close 1006, frequent reconnect backoff). If you see that pattern, set TAP_DISABLE_ACKS=true on the Hyperindex backend to stabilize ingestion first, then investigate Tap resource/config compatibility before re-enabling ACK mode.

Tap

• TAP_FULL_NETWORK
• TAP_FIREHOSE_PARALLELISM
• TAP_RESYNC_PARALLELISM
• TAP_OUTBOX_PARALLELISM
• TAP_MAX_DB_CONNS
• TAP_OUTBOX_CAPACITY
• TAP_NO_REPLAY
• TAP_REPO_FETCH_TIMEOUT

TAP_FULL_NETWORK=true enables full-network tracking and triggers a broad historical backfill across discoverable repos. This can materially increase ingestion load, network requests, and storage use.

Client

• Additional auth/provider-specific settings depending on deployment model

Common pitfalls

• Wrong variable on wrong service

  • TAP_COLLECTION_FILTERS, TAP_SIGNAL_COLLECTION, TAP_FULL_NETWORK belong to Tap
  • TAP_DISABLE_ACKS, TAP_ENABLED, TAP_URL belong to Hyperindex backend

• Client works but admin requests fail

  • HYPERINDEX_URL is missing on the client deployment
  • NEXT_PUBLIC_HYPERINDEX_URL alone is not enough for server-side proxy routes
  • EXTERNAL_BASE_URL does not match the backend's public URL

• admin privileges required while logged in

  • TRUST_PROXY_HEADERS is missing/false
  • Logged-in DID is not present in ADMIN_DIDS

• Trailing slash URL issues

  • NEXT_PUBLIC_CLIENT_URL must not include a trailing slash.
  • Use:
    • https://hyperindex-frontend.example.com ✅
    • https://hyperindex-frontend.example.com/ ❌
  • A trailing slash can cause OAuth client metadata lookup errors (for example: client_metadata not found).

• Healthcheck confusion

  • Backend healthcheck should be /health
  • Frontend usually uses / unless you explicitly add a /health route

Deploy on Railway

1) Deploy Hyperindex backend

1. Create a Railway service from the repository
2. Attach a persistent volume mounted to /data
3. Set healthcheck path to /health
4. Add backend variables from the baseline list above
5. Deploy

Use PORT=8080 for the Hyperindex service. For public URL variables (for example EXTERNAL_BASE_URL, NEXT_PUBLIC_HYPERINDEX_URL, HYPERINDEX_URL), use the Railway-generated HTTPS domain (typically without appending :8080).

2) Deploy Tap

1. Create a Railway service from image: ghcr.io/bluesky-social/indigo/tap:latest (or a pinned tag)
2. Attach a persistent volume mounted to /data
3. Add Tap variables from the baseline list above, but do not manually set TAP_DATABASE_URL on Railway (Railway autoconfigures this for the service)
4. Deploy

See the official ATProto Tap Railway guide: RAILWAY_DEPLOY.md.

3) Connect backend to Tap

Set on backend service:

• TAP_ENABLED=true
• TAP_URL=ws://<tap-private-host>:2480
• TAP_ADMIN_PASSWORD=<same-as-tap-service>

Redeploy backend after updating these values.

4) Deploy client (Next.js)

Deploy client on Railway/Vercel/etc and set:

• NEXT_PUBLIC_HYPERINDEX_URL=<hyperindex-backend-public-url>
• HYPERINDEX_URL=<hyperindex-backend-public-url>
• auth/session vars (NEXT_PUBLIC_CLIENT_URL, COOKIE_SECRET, ATPROTO_JWK_PRIVATE) as needed

Railway-specific DB path notes

For mounted volumes at /data:

Railway

• Backend SQLite: DATABASE_URL=sqlite:/data/hypergoat.db
• Tap SQLite: do not manually set TAP_DATABASE_URL (it is autoconfigured)

Non-Railway/self-managed

• Backend SQLite: DATABASE_URL=sqlite:/data/hypergoat.db
• Tap SQLite: TAP_DATABASE_URL=sqlite:///data/tap.db

Running with Docker

docker compose up --build

Learn more

• GitHub repository — source code, issues, and documentation
• Certified Services — current public indexer endpoints
• Building on Hypercerts — integration patterns for platforms and tools