git-remote-s3

Use an S3 bucket as a remote

git clone https://code.rmdd.net/git-remote-s3.git

README.md

# git-remote-s3

A Git remote helper that uses an S3 bucket as a remote.

There were a handful of existing options but this one is mine.

```sh
git remote add origin s3://my-bucket/my-repo
git push origin main
git clone s3://my-bucket/my-repo
```

## Install

Put the `git-remote-s3` binary in a directory on your `PATH`.
Git finds the helper by its name when a remote URL starts with `s3://`.

```sh
nix build .#
cargo install --path .
```

## Configure

The URL format is `s3://<bucket>/<prefix>`. The prefix is necessary.

The helper reads these environment variables:

| Variable                | Use                                                                       |
| ----------------------- | ------------------------------------------------------------------------- |
| `AWS_ACCESS_KEY_ID`     | Access key.                                                               |
| `AWS_SECRET_ACCESS_KEY` | Secret key.                                                               |
| `AWS_SESSION_TOKEN`     | Session token for temporary credentials.                                  |
| `AWS_REGION`            | Region. `AWS_DEFAULT_REGION` is the fallback. The default is `us-east-1`. |
| `AWS_ENDPOINT_URL`      | Endpoint of an S3-compatible store. The helper then uses path-style URLs. |

If the environment has no access key or secret key, the helper gets the credentials from the AWS CLI.
This gives access to profiles, SSO sessions, and roles. Set `AWS_PROFILE` to select a profile.
If the environment also has no region, the helper uses the region of the profile.

This path needs version 2 of the AWS CLI on your `PATH`.
Each Git operation then starts the CLI, which adds a short delay.

## Storage

Each push makes a `git bundle` of the ref and uploads it.
The bucket keeps one bundle for each ref.

| Key                           | Content                                                          |
| ----------------------------- | ---------------------------------------------------------------- |
| `<prefix>/<ref>/<sha>.bundle` | The bundle of the ref at that commit.                            |
| `<prefix>/HEAD`               | The name of the default branch. The first branch push writes it. |
| `<prefix>/<ref>/LOCK#.lock`   | A lock that exists during a push to the ref.                     |

This layout is the same as that of [awslabs/git-remote-s3](https://github.com/awslabs/git-remote-s3).
The two tools can use the same bucket.

To change the default branch, write the new ref name to `<prefix>/HEAD`.

## Limits

- Each push uploads the full history of the ref. Large repositories will push slowly.
- The maximum size of a bundle is 5 GB. The helper does not yet use multipart uploads.
- The lock needs a store that obeys the `If-None-Match` header on `PutObject`. Without it, two pushes at the same time can overwrite each other.
- A push that stops mid-push can leave a lock. Other clients ignore a lock after 60 seconds.
- The helper has no support for Git LFS, `s3+zip://` URLs, or protected branches.

## Develop

```sh
nix develop
just test    # Unit tests
just e2e     # Push, clone, and pull against a local S3 gateway
```

Refs

e5d5cfemain (default)

Commits on main

2026-11-20e5d5cfeRaymond Douglasdocs: tidy
2026-11-2069fce94Raymond Douglaslist: only report on newest bundle; handle 404s
2026-11-206dedd73Raymond Douglascredential fallback
2026-11-208ecc183Raymond Douglasinit

Files on main