# Imports API

The Imports API manages background migrations of historical analytics into an Totallytics site. Jobs run asynchronously in monthly chunks.

Base URL: `https://analytics.bitgate.dev`. [Download OpenAPI](/openapi.json) or [read this page as Markdown](/docs/imports.md).

## Access

All import endpoints require a Firebase ID token in `Authorization: Bearer <token>`. Sign in, open [Authentication](/docs/authentication), and use **Copy access token**. Totallytics has no static product API key; SimpleAnalytics credentials are source configuration, not Totallytics authentication.

`GET /api/import-sources` accepts any signed-in account. Every site-specific import endpoint requires the site's owner, **even for public sites**. Authentication is checked before the site lookup.

| Method | Route                                          | Result                                     |
| ------ | ---------------------------------------------- | ------------------------------------------ |
| GET    | `/api/import-sources`                          | Supported sources and configuration fields |
| POST   | `/api/sites/{hostname}/imports`                | Validate, create and queue a job           |
| GET    | `/api/sites/{hostname}/imports`                | Latest 50 jobs for the site                |
| GET    | `/api/sites/{hostname}/imports/{jobId}`        | One job's current state                    |
| POST   | `/api/sites/{hostname}/imports/{jobId}/cancel` | Cancel a queued or running job             |
| POST   | `/api/sites/{hostname}/imports/{jobId}/retry`  | Re-queue a failed or canceled job          |

Responses are JSON with `Cache-Control: no-store`. Product API responses have no cross-origin CORS headers; call from a server or the Totallytics origin. Use the normalized destination hostname returned by the [Sites API](/docs/sites); path lookups do not lowercase it.

TypeScript examples run server-side on Node 20+: save a snippet as `example.mts`, set its environment variables, then run `npx tsx example.mts`. No example automatically retries a POST.

## Available sources

`GET /api/import-sources` returns `{ "sources": [...] }`. Each source has `id`, `label`, `description` and `configFields`. A configuration field has `key`, `label`, `type` (`text`, `password` or `date`), `required`, and optional `placeholder` and `help` strings.

The current source is `simpleanalytics`, labeled **SimpleAnalytics**:

| Configuration key | Type     | Required | Meaning                                                                 |
| ----------------- | -------- | -------- | ----------------------------------------------------------------------- |
| `user_id`         | text     | Yes      | User ID from SimpleAnalytics dashboard → Account → API                  |
| `api_key`         | password | Yes      | SimpleAnalytics API key                                                 |
| `source_hostname` | text     | Yes      | Hostname registered in SimpleAnalytics; may differ from the destination |

## Start an import

`POST /api/sites/{hostname}/imports` requires `source`, `config`, `start` and `end`. Set `source` to `simpleanalytics` and supply all three configuration strings above. Strings are trimmed and truncated to 512 characters; unknown configuration keys are ignored. `source_hostname` is lowercased and must be a hostname containing a dot, without a scheme, port or path.

Dates are inclusive UTC dates in `YYYY-MM-DD` format. `start` must be on or after `2010-01-01`, `start <= end`, and the difference `end - start` cannot exceed 1,826 days. There is no future-end cutoff.

**The planner rounds `start` down to the first day of its month.** A request for `2026-08-15` through `2026-08-20` plans exports from August 1 through August 20. Use a month-boundary start to avoid including earlier days. The returned `range_start` still reflects the requested date. Each month has two chunks: pageviews, then events; the final month ends at the requested `end`.

Before queueing, creation tests the source credentials by requesting a recent pageview export. This can take time. **`201` means queued, not import finished.** Check job progress and actual reports before considering the migration complete.

Read credentials without putting them in shell history. Export `EA_HOSTNAME` for your registered destination, `SA_HOSTNAME` for the source, and `EA_IMPORT_START` / `EA_IMPORT_END` for your chosen dates. These examples create a real job when used with valid credentials.

```bash
read -rsp "Totallytics access token: " TT_TOKEN; printf '\n'
read -rsp "SimpleAnalytics user ID: " SA_USER_ID; printf '\n'
read -rsp "SimpleAnalytics API key: " SA_API_KEY; printf '\n'
export TT_TOKEN SA_USER_ID SA_API_KEY
```

The curl example requires Bash and `jq`; `jq --arg` safely encodes values, including quotes in credentials.

```curl
set -o pipefail
jq -n \
  --arg user_id "${SA_USER_ID:?Set SA_USER_ID}" \
  --arg api_key "${SA_API_KEY:?Set SA_API_KEY}" \
  --arg source_hostname "${SA_HOSTNAME:?Set SA_HOSTNAME}" \
  --arg start "${EA_IMPORT_START:?Set EA_IMPORT_START}" \
  --arg end "${EA_IMPORT_END:?Set EA_IMPORT_END}" \
  '{source:"simpleanalytics",config:{user_id:$user_id,api_key:$api_key,source_hostname:$source_hostname},start:$start,end:$end}' |
  curl --fail-with-body --silent --show-error --max-time 300 \
    -X POST "https://analytics.bitgate.dev/api/sites/${EA_HOSTNAME:?Set EA_HOSTNAME}/imports" \
    -H "Authorization: Bearer ${TT_TOKEN:?Copy an access token}" \
    -H 'Content-Type: application/json' --data-binary @-
```

```typescript
const {
  TT_TOKEN,
  EA_HOSTNAME,
  SA_USER_ID,
  SA_API_KEY,
  SA_HOSTNAME,
  EA_IMPORT_START,
  EA_IMPORT_END,
} = process.env;
if (
  !TT_TOKEN ||
  !EA_HOSTNAME ||
  !SA_USER_ID ||
  !SA_API_KEY ||
  !SA_HOSTNAME ||
  !EA_IMPORT_START ||
  !EA_IMPORT_END
) {
  throw new Error(
    "Set the token, destination, source credentials, hostname and dates",
  );
}

const response = await fetch(
  `https://analytics.bitgate.dev/api/sites/${encodeURIComponent(EA_HOSTNAME)}/imports`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${TT_TOKEN}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      source: "simpleanalytics",
      config: {
        user_id: SA_USER_ID,
        api_key: SA_API_KEY,
        source_hostname: SA_HOSTNAME,
      },
      start: EA_IMPORT_START,
      end: EA_IMPORT_END,
    }),
    signal: AbortSignal.timeout(300_000),
  },
);
if (!response.ok) {
  throw new Error(`Create import ${response.status}: ${await response.text()}`);
}
console.log(await response.json());
```

The response is the job object directly. Save its `id` as `EA_IMPORT_ID` to read progress. At most 20 new jobs per owner can be created in a rolling 24-hour window across all sites and statuses; the limit returns `429`. Only one `queued` or `running` job may exist per site; another creation returns `409`. Retrying the same job is not a new creation.

There is no idempotency key or cross-job deduplication ledger. Reimporting an overlapping period can duplicate records. After a timeout or ambiguous response, inspect the site's jobs before attempting another POST.

## Read progress

`GET /api/sites/{hostname}/imports` returns `{ "imports": [...] }`, ordered by `created_at` descending, with at most 50 entries. There are no pagination, cursor or offset parameters.

`GET /api/sites/{hostname}/imports/{jobId}` returns the job object directly. To list jobs instead, remove `/{jobId}` from the request below. Source discovery uses the same bearer header with `/api/import-sources`.

```curl
curl --fail-with-body --silent --show-error --max-time 20 \
  -H "Authorization: Bearer ${TT_TOKEN:?Copy an access token}" \
  "https://analytics.bitgate.dev/api/sites/${EA_HOSTNAME:?Set EA_HOSTNAME}/imports/${EA_IMPORT_ID:?Set EA_IMPORT_ID}"
```

```typescript
const { TT_TOKEN, EA_HOSTNAME, EA_IMPORT_ID } = process.env;
if (!TT_TOKEN || !EA_HOSTNAME || !EA_IMPORT_ID) {
  throw new Error("Set TT_TOKEN, EA_HOSTNAME and EA_IMPORT_ID");
}
const response = await fetch(
  `https://analytics.bitgate.dev/api/sites/${encodeURIComponent(EA_HOSTNAME)}/imports/${encodeURIComponent(EA_IMPORT_ID)}`,
  {
    headers: { Authorization: `Bearer ${TT_TOKEN}` },
    signal: AbortSignal.timeout(20_000),
  },
);
if (!response.ok) {
  throw new Error(`Import ${response.status}: ${await response.text()}`);
}
console.log(await response.json());
```

## Job object

Create, get, cancel and retry return this object. List entries use the same shape; `owner_uid` and `updated_at` are not returned.

| Field           | Type           | Meaning                                                                                  |
| --------------- | -------------- | ---------------------------------------------------------------------------------------- |
| `id`            | string         | UUID job identifier                                                                      |
| `site`          | string         | Destination Totallytics hostname                                                       |
| `source`        | string         | Source ID, currently `simpleanalytics`                                                   |
| `source_label`  | string         | Source display label, currently `SimpleAnalytics`                                        |
| `status`        | string         | `queued`, `running`, `completed`, `failed` or `canceled`                                 |
| `range_start`   | string         | Requested inclusive start date, `YYYY-MM-DD`                                             |
| `range_end`     | string         | Requested inclusive end date, `YYYY-MM-DD`                                               |
| `chunks_total`  | integer        | Planned pageview and event chunks                                                        |
| `chunks_done`   | integer        | Fully checkpointed chunks                                                                |
| `rows_imported` | number         | Checkpointed inserted records: pageviews, events and derived duration/scroll append rows |
| `rows_skipped`  | number         | Checkpointed source records skipped for another hostname or an event without a name      |
| `error`         | string or null | Latest worker error, up to 500 characters                                                |
| `config`        | object         | Non-password fields: `user_id` and `source_hostname`; never `api_key`                    |
| `created_at`    | string         | ISO date-time of creation                                                                |
| `finished_at`   | string or null | ISO date-time of completion, failure or cancellation; normally null while active         |

Request and returned range dates use `YYYY-MM-DD`, for example `2026-08-01`. Progress updates only after a whole chunk finishes. Partial writes may not appear in counters, and retry duplicates are not reconciled: these counts are neither unique pageviews nor proof of exact or durable storage totals.

A job may have an `error` while still `queued` or `running` as the worker retries. Historic visitor identities are approximated from day, user agent, country and source hostname; report visitor counts need not match SimpleAnalytics exactly.

## Cancel or retry

Both actions accept a POST with no body and return `200` with the job object.

| Action    | Allowed state          | Effect                                                                   |
| --------- | ---------------------- | ------------------------------------------------------------------------ |
| `/cancel` | `queued` or `running`  | Sets status to `canceled`                                                |
| `/retry`  | `failed` or `canceled` | Sets status to `queued`, clears the error and resumes from `chunks_done` |

The examples cancel a job. To retry an eligible job, replace `/cancel` with `/retry` after checking its state.

```curl
curl --fail-with-body --silent --show-error --max-time 20 \
  -X POST \
  -H "Authorization: Bearer ${TT_TOKEN:?Copy an access token}" \
  "https://analytics.bitgate.dev/api/sites/${EA_HOSTNAME:?Set EA_HOSTNAME}/imports/${EA_IMPORT_ID:?Set EA_IMPORT_ID}/cancel"
```

```typescript
const { TT_TOKEN, EA_HOSTNAME, EA_IMPORT_ID } = process.env;
if (!TT_TOKEN || !EA_HOSTNAME || !EA_IMPORT_ID) {
  throw new Error("Set TT_TOKEN, EA_HOSTNAME and EA_IMPORT_ID");
}
const response = await fetch(
  `https://analytics.bitgate.dev/api/sites/${encodeURIComponent(EA_HOSTNAME)}/imports/${encodeURIComponent(EA_IMPORT_ID)}/cancel`,
  {
    method: "POST",
    headers: { Authorization: `Bearer ${TT_TOKEN}` },
    signal: AbortSignal.timeout(20_000),
  },
);
if (!response.ok) {
  throw new Error(`Cancel import ${response.status}: ${await response.text()}`);
}
console.log(await response.json());
```

Cancellation stops later queued work, not an in-flight export or insert, and does not roll back rows. A retry skips checkpointed chunks, but partial writes in the current chunk and ambiguous insert retries can duplicate records. Retrying is **not idempotent or exactly-once**. It does not accept replacement credentials or dates; completed jobs cannot be retried.

Cancel and retry responses use the prior job snapshot with status overrides, so `finished_at` can be stale. Poll the GET endpoint for the persisted state.

## Errors and credentials

Errors use `{ "error": "message" }`.

| Status | Message or condition                                                                                                                      |
| ------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
| 400    | `invalid date range`, `unknown import source`, missing configuration fields, `Source hostname is invalid`, or a provider validation error |
| 401    | `sign in required`: missing, invalid or expired token                                                                                     |
| 403    | `not your site`: authenticated caller is not the site owner                                                                               |
| 404    | `unknown site`, or `unknown import` when the job is absent or belongs to another site                                                     |
| 409    | `an import is already running for this site`, `import is not active` for cancel, or `only failed or canceled imports can be retried`      |
| 429    | `too many imports today, try again tomorrow`: rolling 24-hour creation limit reached                                                      |
| 500    | `internal error`; retry queue failure may leave the job `queued`, and retrying while another job is active can also produce this error    |
| 502    | `import queue unavailable, please try again`: creation could not enqueue and cancels the new job                                          |

Password configuration fields are never echoed in the job's `config`. Completion removes the stored `api_key`; failed and canceled jobs retain it for retry. The non-password fields remain visible. Error text may include upstream response snippets and is not generally secret-redacted.

After an enqueue error, check job state before taking another action. Compare actual data with the [Stats API](/docs/stats); use the [SimpleAnalytics migration guide](/docs/migrate-simple-analytics) to plan the wider cutover.