> ## Documentation Index
> Fetch the complete documentation index at: https://docs.politicalcomms.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Data exports to your S3 bucket

> Scheduled Parquet exports of message, project, opt-out and click data into a bucket you own. Setup, file layout, the four datasets and every column.

Data Exports writes your messaging data as Parquet files into an S3-compatible bucket that you own, on a schedule you choose. It is a dashboard feature under **Admin > Data Exports** and needs the `admin.data_exports.manage` permission. There is no API endpoint for it.

## What you need

* An S3-compatible bucket. AWS S3 works natively. Backblaze B2, Cloudflare R2, Wasabi, DigitalOcean Spaces and MinIO work with their endpoint URL.
* An access key scoped to that bucket. The minimal policy is shown in the destination dialog and needs only `s3:PutObject`, `s3:AbortMultipartUpload` and `s3:ListMultipartUploadParts` on `arn:aws:s3:::YOUR_BUCKET/*`.
* Optionally, a lifecycle rule on the bucket that aborts incomplete multipart uploads after a day. We check for it and warn if it is missing.

The secret key is encrypted with AWS KMS envelope encryption and bound to the destination record. It is never logged and never returned by the API. You can rotate it by editing the destination and revoke it at any time in your own account.

## Setting up a destination

1. Open **Admin > Data Exports** and choose **New destination**.
2. Enter a name, the bucket name, region, access key ID and secret. Add a prefix if you want the files under a folder, and an endpoint URL for a non-AWS provider.
3. Pick a schedule, the datasets to export, and, for parent organizations, which child organizations to include.
4. Save. We write a small probe object under `_validation/` in your bucket. If the write fails, the destination is not created and the error from your provider is shown.

## Schedules and windows

| Schedule | When it runs                  | What it covers                              |
| -------- | ----------------------------- | ------------------------------------------- |
| Hourly   | Five minutes past every hour  | Everything since the previous hourly export |
| Daily    | 08:00 UTC                     | Everything since the previous daily export  |
| Manual   | When you press **Export Now** | A range you choose, up to seven days        |

Every window ends five minutes before the run starts so late carrier receipts are included. Each scheduled run continues from where the previous one ended, so a paused or failed run never leaves a gap; a destination that fell behind catches up seven days per run. The first run after a destination is created covers only the most recent hour or day.

A **Full Export** walks your whole history in seven-day chunks, up to two years back, and can be started once every 30 days per destination. Failed chunks can be retried without restarting the batch.

## File layout

Files land under your prefix in a Hive-style path that Athena, Glue, Snowflake, Databricks, BigQuery and DuckDB discover without configuration:

```
<prefix>/v1/schema_type=message_events/year=2026/month=09/day=26/hour=08/
  <organization_id>_message_events_20260926T080001Z_<job>.part-000.parquet
```

* Parquet with SNAPPY compression. Files rotate every 100,000 rows (`part-000`, `part-001`, ...).
* The `year`, `month`, `day` and `hour` folders are when the export ran, in UTC, not when the rows were created. Filter on the row timestamps for reporting periods.
* A window with no rows writes nothing. The job still appears in the history with **No rows**.
* Rows inside a file are not ordered.
* Timestamps are Parquet `TIMESTAMP_MILLIS` in UTC. Phone numbers are E.164 (`+15555550123`); short-code senders appear as the short code itself (`471016`).
* The `v1` prefix is the schema version. New columns are added at the end; existing columns are never renamed or removed within a version.

## Datasets

### message\_events

One row per outbound and inbound message. A message is exported again in a later window whenever its delivery status changes after it was first exported (for example a delivery receipt that arrives the next day). Keep the row with the greatest `updated_at` per `message_id`.

| Column                | Type                | Meaning                                                                                                                         |
| --------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `message_id`          | string              | Message id                                                                                                                      |
| `external_message_id` | string, nullable    | Carrier message id                                                                                                              |
| `conversation_id`     | string              | Conversation the message belongs to                                                                                             |
| `project_id`          | string, nullable    | Project id                                                                                                                      |
| `project_name`        | string, nullable    | Project name at export time                                                                                                     |
| `project_type`        | string, nullable    | Project type                                                                                                                    |
| `direction`           | string              | `outbound` or `inbound`                                                                                                         |
| `status`              | string              | Delivery status at export time: `unsent`, `queued`, `sending`, `sent`, `delivered`, `failed`, `received` (inbound) or `on_hold` |
| `channel`             | string, nullable    | Channel                                                                                                                         |
| `protocol`            | string, nullable    | `sms`, `mms` or `rcs`                                                                                                           |
| `from_phone_number`   | string              | Sender                                                                                                                          |
| `to_phone_number`     | string              | Recipient                                                                                                                       |
| `message_text`        | string, nullable    | Message body                                                                                                                    |
| `media_url_count`     | int32               | Number of attached media URLs                                                                                                   |
| `send_type`           | string, nullable    | `scheduled` (project send), `manual`, `auto-response`, `test` or `chat`                                                         |
| `error_code`          | string, nullable    | Carrier error code in the display form used by the dashboard (the `40` prefix stripped: `300` for `40300`)                      |
| `error_reason`        | string, nullable    | Human-readable label for `error_code`                                                                                           |
| `sent_at`             | timestamp, nullable | Accepted by the carrier                                                                                                         |
| `delivered_at`        | timestamp, nullable | Delivery receipt time                                                                                                           |
| `failed_at`           | timestamp, nullable | Failure time                                                                                                                    |
| `url_clicked_at`      | timestamp, nullable | First click on a tracked link in this message                                                                                   |
| `url_click_count`     | int32, nullable     | Click count at export time                                                                                                      |
| `contact_id`          | string, nullable    | Contact id, for joins to your own contact data                                                                                  |
| `organization_id`     | string              | Organization that owns the message                                                                                              |
| `created_at`          | timestamp           | When the message row was created                                                                                                |
| `updated_at`          | timestamp, nullable | Last change to the row; use it to keep the latest version of a message                                                          |

### project\_summary

One row per project, written in the window that holds its completion time. Drafts, scheduled and in-progress projects are not exported until they complete.

| Column               | Type                | Meaning                                 |
| -------------------- | ------------------- | --------------------------------------- |
| `project_id`         | string              | Project id                              |
| `campaign_id`        | string, nullable    | Campaign id                             |
| `name`               | string              | Project name                            |
| `type`               | string, nullable    | Project type                            |
| `protocol`           | string, nullable    | `sms`, `mms` or `rcs`                   |
| `channel`            | string, nullable    | Channel                                 |
| `status`             | string, nullable    | Project status                          |
| `total_recipients`   | int32, nullable     | Audience size                           |
| `messages_sent`      | int32, nullable     | Messages sent                           |
| `messages_delivered` | int32, nullable     | Messages delivered                      |
| `messages_failed`    | int32, nullable     | Messages failed                         |
| `scheduled_at`       | timestamp, nullable | Scheduled start                         |
| `sent_at`            | timestamp, nullable | First send                              |
| `completed_at`       | timestamp, nullable | Completion time (the export window key) |
| `organization_id`    | string              | Owning organization                     |
| `brand_id`           | string, nullable    | Brand                                   |
| `created_at`         | timestamp, nullable | Project creation time                   |

### opt\_out\_events

One row per opt-out, in the window that holds its creation.

| Column            | Type                | Meaning                                                                     |
| ----------------- | ------------------- | --------------------------------------------------------------------------- |
| `opt_out_id`      | string              | Opt-out id                                                                  |
| `phone_number`    | string              | Opted-out number                                                            |
| `source`          | string, nullable    | How the opt-out was recorded: `Replied`, `CSV Upload`, `Tagged` or `Manual` |
| `opt_out_message` | string, nullable    | The reply that triggered it, when there was one                             |
| `brand_id`        | string, nullable    | Brand the opt-out list belongs to                                           |
| `organization_id` | string              | Owning organization                                                         |
| `opted_out_at`    | timestamp, nullable | When the person opted out                                                   |
| `created_at`      | timestamp           | When the row was created                                                    |

### click\_events

One row per click on a tracked link, in the window that holds the click.

| Column             | Type              | Meaning                                      |
| ------------------ | ----------------- | -------------------------------------------- |
| `click_id`         | string            | Click id                                     |
| `tracking_link_id` | string            | Tracked link id                              |
| `message_id`       | string            | Message that carried the link                |
| `project_id`       | string            | Project                                      |
| `contact_id`       | string, nullable  | Contact                                      |
| `phone_number`     | string, nullable  | Recipient number                             |
| `short_url`        | string            | The tracked URL as sent                      |
| `destination_url`  | string            | Where it resolved                            |
| `clicked_at`       | timestamp         | Click time                                   |
| `ip_address`       | string, nullable  | Client IP                                    |
| `user_agent`       | string, nullable  | Browser user agent                           |
| `device`           | string, nullable  | Device class                                 |
| `browser`          | string, nullable  | Browser                                      |
| `os`               | string, nullable  | Operating system                             |
| `country_code`     | string, nullable  | Country                                      |
| `region`           | string, nullable  | Region or state                              |
| `city`             | string, nullable  | City                                         |
| `referrer`         | string, nullable  | HTTP referrer                                |
| `utm_source`       | string, nullable  | UTM source on the destination URL            |
| `utm_medium`       | string, nullable  | UTM medium                                   |
| `utm_campaign`     | string, nullable  | UTM campaign                                 |
| `is_unique_click`  | boolean           | First click from this recipient on this link |
| `is_share_click`   | boolean           | Click on a forwarded link                    |
| `is_bot_click`     | boolean, nullable | Classified as a bot or link scanner          |
| `organization_id`  | string            | Owning organization                          |
| `brand_id`         | string, nullable  | Brand                                        |

## Reading the files

* **Athena or Glue**: crawl the `v1/` prefix or create a table with partition projection on `schema_type`, `year`, `month`, `day` and `hour`.
* **Snowflake**: create an external stage on the bucket and load with `COPY INTO` or Snowpipe auto-ingest.
* **Databricks**: point Auto Loader (`cloudFiles`) at the `schema_type=...` prefix.
* **DuckDB**: `SELECT * FROM read_parquet('s3://bucket/prefix/v1/schema_type=message_events/**/*.parquet')`.

For `message_events`, deduplicate on `message_id` keeping the row with the greatest `updated_at`.

## Job history

The **Export jobs** tab lists every run with dataset, status, rows, size and duration. Jobs retry up to three times on transient failures; a job that keeps failing shows the provider's error message. A failed run leaves no partial file behind.