README.md
# git-remote-s3
A Git remote helper that uses an S3 bucket as a remote.
There were a handful of existing options but this one is mine.
```sh
git remote add origin s3://my-bucket/my-repo
git push origin main
git clone s3://my-bucket/my-repo
```
## Install
Put the `git-remote-s3` binary in a directory on your `PATH`.
Git finds the helper by its name when a remote URL starts with `s3://`.
```sh
nix build .#
cargo install --path .
```
## Configure
The URL format is `s3://<bucket>/<prefix>`. The prefix is necessary.
The helper reads these environment variables:
| Variable | Use |
| ----------------------- | ------------------------------------------------------------------------- |
| `AWS_ACCESS_KEY_ID` | Access key. |
| `AWS_SECRET_ACCESS_KEY` | Secret key. |
| `AWS_SESSION_TOKEN` | Session token for temporary credentials. |
| `AWS_REGION` | Region. `AWS_DEFAULT_REGION` is the fallback. The default is `us-east-1`. |
| `AWS_ENDPOINT_URL` | Endpoint of an S3-compatible store. The helper then uses path-style URLs. |
If the environment has no access key or secret key, the helper gets the credentials from the AWS CLI.
This gives access to profiles, SSO sessions, and roles. Set `AWS_PROFILE` to select a profile.
If the environment also has no region, the helper uses the region of the profile.
This path needs version 2 of the AWS CLI on your `PATH`.
Each Git operation then starts the CLI, which adds a short delay.
## Storage
Each push makes a `git bundle` of the ref and uploads it.
The bucket keeps one bundle for each ref.
| Key | Content |
| ----------------------------- | ---------------------------------------------------------------- |
| `<prefix>/<ref>/<sha>.bundle` | The bundle of the ref at that commit. |
| `<prefix>/HEAD` | The name of the default branch. The first branch push writes it. |
| `<prefix>/<ref>/LOCK#.lock` | A lock that exists during a push to the ref. |
This layout is the same as that of [awslabs/git-remote-s3](https://github.com/awslabs/git-remote-s3).
The two tools can use the same bucket.
To change the default branch, write the new ref name to `<prefix>/HEAD`.
## Limits
- Each push uploads the full history of the ref. Large repositories will push slowly.
- The maximum size of a bundle is 5 GB. The helper does not yet use multipart uploads.
- The lock needs a store that obeys the `If-None-Match` header on `PutObject`. Without it, two pushes at the same time can overwrite each other.
- A push that stops mid-push can leave a lock. Other clients ignore a lock after 60 seconds.
- The helper has no support for Git LFS, `s3+zip://` URLs, or protected branches.
## Develop
```sh
nix develop
just test # Unit tests
just e2e # Push, clone, and pull against a local S3 gateway
```