Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 ozan
|
33f967e42e | ||
|
ozan
|
4076cf67b7 | ||
|
ozan
|
80a358e8da | ||
|
ozan
|
46bef04a7d | ||
|
ozan
|
6bf4ed8314 | ||
|
ozan
|
501953c636 |
@@ -1,57 +0,0 @@
|
|||||||
# tinqs/ci — Roadmap
|
|
||||||
|
|
||||||
## Done (2026-05-22)
|
|
||||||
|
|
||||||
- [x] Create `tinqs/ci` repo on tinqs.com
|
|
||||||
- [x] Composite actions: checkout, setup-go, setup-node, setup-aws
|
|
||||||
- [x] Wire all 6 studio workflows to `tinqs/ci/*@v1`
|
|
||||||
- [x] Alpine + Debian compatibility in all actions
|
|
||||||
- [x] Runner image Dockerfiles: base, go, node, docker, deploy, godot
|
|
||||||
- [x] Architecture doc: `tinqs/internal/architecture/ci-runner-architecture.md`
|
|
||||||
|
|
||||||
## Next: Runner Images
|
|
||||||
|
|
||||||
Build and push purpose-built images to ECR. Each workflow type gets a lean image.
|
|
||||||
|
|
||||||
| Image | ECR name | Contents | Priority |
|
|
||||||
|-------|----------|----------|----------|
|
|
||||||
| base | tinqs-runner-base | Alpine + git + AWS CLI + SSH | High |
|
|
||||||
| go | tinqs-runner-go | base + Go 1.26 | High |
|
|
||||||
| node | tinqs-runner-node | base + Node 22 + pnpm | High |
|
|
||||||
| docker | tinqs-runner-docker | dind + Go + AWS CLI | High |
|
|
||||||
| deploy | tinqs-runner-deploy | base only | Medium |
|
|
||||||
| godot | tinqs-runner-godot | base + headless Godot 4.6 | Low (future) |
|
|
||||||
|
|
||||||
**Blocker:** No Docker on Forge, Lightsail unreachable. Options:
|
|
||||||
1. AWS CodeBuild project to build images from this repo
|
|
||||||
2. SSH to Lightsail when back online
|
|
||||||
3. Self-build: add a workflow here that builds images on the current runner
|
|
||||||
|
|
||||||
## Next: Label Routing
|
|
||||||
|
|
||||||
Once images exist:
|
|
||||||
1. Update ECS task definitions — one per image
|
|
||||||
2. Register runners with matching labels: `go`, `node`, `docker`, `deploy`
|
|
||||||
3. Update studio workflows: `runs-on: host` → specific labels
|
|
||||||
4. Lambda dispatcher (future): webhook → Fargate task with correct image
|
|
||||||
|
|
||||||
## Future: Lambda Dispatch + Cancel
|
|
||||||
|
|
||||||
Lambda code already exists in `tinqs-ltd/docs/lambda/` (ci-dispatch + ci-exec).
|
|
||||||
Move to `tinqs/studio/deploy/lambda/`, deploy with SAM.
|
|
||||||
|
|
||||||
- DynamoDB table for run tracking (`tinqs-ci-runs`)
|
|
||||||
- Cancel via `ecs:StopTask` on webhook or timeout
|
|
||||||
- EventBridge cron for stale task cleanup
|
|
||||||
- Deploy-only jobs run directly in Lambda (no Fargate needed)
|
|
||||||
|
|
||||||
See: `tinqs/internal/architecture/ci-runner-architecture.md`
|
|
||||||
|
|
||||||
## Future: More Actions
|
|
||||||
|
|
||||||
| Action | Purpose |
|
|
||||||
|--------|---------|
|
|
||||||
| `tinqs/ci/setup-python` | Python + pip/uv for ML pipelines |
|
|
||||||
| `tinqs/ci/deploy-ecs` | ECS update-service wrapper with wait |
|
|
||||||
| `tinqs/ci/deploy-s3` | S3 sync + CloudFront invalidation |
|
|
||||||
| `tinqs/ci/notify` | Post build status to Lobster GChat |
|
|
||||||
@@ -1,57 +1,44 @@
|
|||||||
# tinqs/ci
|
# tinqs/ci
|
||||||
|
|
||||||
Tinqs CI toolchain — composite Gitea Actions and purpose-built runner images.
|
CI toolchain for Tinqs Studio — composite Gitea Actions and a Lambda dispatcher that orchestrates ephemeral EC2 Spot runners.
|
||||||
|
|
||||||
## Actions
|
> ⚠️ **This repo must stay public.** `act_runner` (go-git) clones action repos without auth; every other tinqs repo is private. If this repo goes private, every `uses: tinqs/ci/...` step breaks.
|
||||||
|
|
||||||
| Action | What it does |
|
```
|
||||||
|--------|-------------|
|
Push → Gitea webhook → Lambda (tinqs-ci-dispatch) → EC2 Spot → act_runner → job → self-terminate
|
||||||
| `tinqs/ci/checkout@v1` | Clone a repo from tinqs.com |
|
```
|
||||||
| `tinqs/ci/setup-go@v1` | Install Go (skips if pre-baked) |
|
|
||||||
| `tinqs/ci/setup-node@v1` | Install Node.js + pnpm (skips if pre-baked) |
|
## Using the actions
|
||||||
| `tinqs/ci/setup-aws@v1` | Install AWS CLI + optional ECR login |
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
steps:
|
steps:
|
||||||
- uses: tinqs/ci/checkout@v1
|
- uses: tinqs/ci/checkout@v1
|
||||||
|
with:
|
||||||
|
sparse: 'cmd/tstudio'
|
||||||
- uses: tinqs/ci/setup-go@v1
|
- uses: tinqs/ci/setup-go@v1
|
||||||
- uses: tinqs/ci/setup-aws@v1
|
- uses: tinqs/ci/setup-aws@v1
|
||||||
with:
|
with:
|
||||||
ecr-login: 'true'
|
ecr-login: 'true'
|
||||||
ecr-repo: '149751500842.dkr.ecr.eu-west-1.amazonaws.com/tinqs-git'
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Runner Images
|
| Action | What it does |
|
||||||
|
|--------|-------------|
|
||||||
|
| `tinqs/ci/checkout@v1` | Clone a repo from tinqs.com (sparse, depth, token auth) |
|
||||||
|
| `tinqs/ci/setup-go@v1` | Install Go (skips if pre-baked in AMI) |
|
||||||
|
| `tinqs/ci/setup-node@v1` | Install Node.js + pnpm (skips if pre-baked) |
|
||||||
|
| `tinqs/ci/setup-aws@v1` | Install AWS CLI + optional ECR login |
|
||||||
|
|
||||||
Each workflow type gets a lean, purpose-built runner image. No fat "everything" image.
|
## Layout
|
||||||
|
|
||||||
| Image | `runs-on` | Contents | Used by |
|
- `checkout/`, `setup-go/`, `setup-node/`, `setup-aws/` — composite actions
|
||||||
|-------|-----------|----------|---------|
|
- `orchestrator/dispatch/` — the dispatcher Lambda (`main.go`)
|
||||||
| **base** | — | Alpine + git + AWS CLI + SSH | Foundation for all others |
|
- `images/` — runner image Dockerfiles
|
||||||
| **go** | `go` | base + Go 1.26 | build-tstudio, deploy-proxy |
|
|
||||||
| **node** | `node` | base + Node 22 + pnpm | deploy-docs |
|
|
||||||
| **docker** | `docker` | docker:dind + Go + AWS CLI | build (platform), deploy-bot |
|
|
||||||
| **deploy** | `deploy` | base only (lightest) | deploy-arikigame, release |
|
|
||||||
| **godot** | `godot` | base + headless Godot 4.6 + export templates | ariki-game builds (future) |
|
|
||||||
|
|
||||||
### Build all images
|
## 📖 Full docs → [`wiki/`](wiki/README.md)
|
||||||
|
|
||||||
```bash
|
The team wiki lives in **[`wiki/`](wiki/README.md)** (plain markdown, rendered by Gitea):
|
||||||
cd images
|
|
||||||
./build-all.sh v1
|
|
||||||
```
|
|
||||||
|
|
||||||
Images push to ECR as `tinqs-runner-{name}:v1`.
|
- [Architecture](wiki/Architecture.md) — design, dispatcher, labels, runner lifecycle
|
||||||
|
- [DevOps Reference](wiki/DevOps-Reference.md) — AWS resources, webhook flow, Spot lifecycle, env vars, cost
|
||||||
### How routing works
|
- [Operations](wiki/Operations.md) — deploy the dispatcher, template deploy, rotate tokens, AMI, monitoring, incidents
|
||||||
|
- [Roadmap](wiki/Roadmap.md) — done / next
|
||||||
The Lambda dispatcher reads `runs-on:` from the workflow and starts a Fargate task with the matching image:
|
|
||||||
|
|
||||||
```
|
|
||||||
runs-on: go → tinqs-runner-go (Go builds)
|
|
||||||
runs-on: node → tinqs-runner-node (frontend builds)
|
|
||||||
runs-on: docker → tinqs-runner-docker (Docker image builds)
|
|
||||||
runs-on: deploy → tinqs-runner-deploy (S3/ECS deploys)
|
|
||||||
runs-on: godot → tinqs-runner-godot (game exports)
|
|
||||||
runs-on: host → legacy runner (forge-runner, temporary)
|
|
||||||
```
|
|
||||||
|
|||||||
+24
-3
@@ -1,11 +1,11 @@
|
|||||||
# tinqs/ci/checkout — Tinqs Studio CI
|
# tinqs/ci/checkout — Tinqs Studio CI
|
||||||
# Clones a repo from tinqs.com (self-hosted Gitea) using plain git.
|
# Clones a repo from tinqs.com (self-hosted Gitea) using plain git.
|
||||||
# Supports depth control, branch/tag selection, and token auth for private repos.
|
# Supports depth control, branch/tag selection, token auth, and sparse checkout.
|
||||||
# Composite action — runs directly on the host, no Node.js runtime needed.
|
# Composite action — runs directly on the host, no Node.js runtime needed.
|
||||||
# Author: Ozan + Claude Code — 2026-05-22
|
# Author: Ozan + Claude Code — 2026-05-22
|
||||||
|
|
||||||
name: 'Tinqs Checkout'
|
name: 'Tinqs Checkout'
|
||||||
description: 'Clone a Gitea repository (replaces actions/checkout)'
|
description: 'Clone a Gitea repository'
|
||||||
|
|
||||||
inputs:
|
inputs:
|
||||||
repository:
|
repository:
|
||||||
@@ -23,6 +23,9 @@ inputs:
|
|||||||
token:
|
token:
|
||||||
description: 'Gitea access token (for private repos)'
|
description: 'Gitea access token (for private repos)'
|
||||||
default: ''
|
default: ''
|
||||||
|
sparse:
|
||||||
|
description: 'Space-separated paths for sparse checkout (empty = full clone)'
|
||||||
|
default: ''
|
||||||
|
|
||||||
runs:
|
runs:
|
||||||
using: 'composite'
|
using: 'composite'
|
||||||
@@ -33,6 +36,7 @@ runs:
|
|||||||
DEPTH="${{ inputs.depth }}"
|
DEPTH="${{ inputs.depth }}"
|
||||||
TARGET="${{ inputs.path }}"
|
TARGET="${{ inputs.path }}"
|
||||||
TOKEN="${{ inputs.token }}"
|
TOKEN="${{ inputs.token }}"
|
||||||
|
SPARSE="${{ inputs.sparse }}"
|
||||||
|
|
||||||
if [ "$DEPTH" = "0" ]; then
|
if [ "$DEPTH" = "0" ]; then
|
||||||
DEPTH_FLAG=""
|
DEPTH_FLAG=""
|
||||||
@@ -46,11 +50,28 @@ runs:
|
|||||||
URL="https://tinqs.com/${REPO}.git"
|
URL="https://tinqs.com/${REPO}.git"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
if [ -n "$SPARSE" ]; then
|
||||||
|
# Sparse checkout — only fetch specified paths
|
||||||
|
mkdir -p "$TARGET" && cd "$TARGET"
|
||||||
|
git init
|
||||||
|
git remote add origin "$URL"
|
||||||
|
git config core.sparseCheckout true
|
||||||
|
for P in $SPARSE; do
|
||||||
|
echo "$P" >> .git/info/sparse-checkout
|
||||||
|
done
|
||||||
|
# Also always include go.mod/go.sum for Go builds
|
||||||
|
echo "go.mod" >> .git/info/sparse-checkout
|
||||||
|
echo "go.sum" >> .git/info/sparse-checkout
|
||||||
|
git fetch $DEPTH_FLAG origin "$REF"
|
||||||
|
git checkout FETCH_HEAD
|
||||||
|
echo "Sparse checkout ${REPO}@${REF} ($(echo $SPARSE | wc -w) paths)"
|
||||||
|
else
|
||||||
|
# Full clone
|
||||||
if [ "$TARGET" = "." ]; then
|
if [ "$TARGET" = "." ]; then
|
||||||
git clone $DEPTH_FLAG --branch "$REF" "$URL" .
|
git clone $DEPTH_FLAG --branch "$REF" "$URL" .
|
||||||
else
|
else
|
||||||
git clone $DEPTH_FLAG --branch "$REF" "$URL" "$TARGET"
|
git clone $DEPTH_FLAG --branch "$REF" "$URL" "$TARGET"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "Checked out ${REPO}@${REF}"
|
echo "Checked out ${REPO}@${REF}"
|
||||||
|
fi
|
||||||
shell: bash
|
shell: bash
|
||||||
|
|||||||
@@ -1,45 +0,0 @@
|
|||||||
.PHONY: build build-dispatch build-exec tidy test-local deploy deploy-guided clean
|
|
||||||
|
|
||||||
REGION ?= eu-west-1
|
|
||||||
|
|
||||||
build: build-dispatch build-exec
|
|
||||||
|
|
||||||
build-dispatch:
|
|
||||||
cd dispatch && GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -tags lambda.norpc -o bootstrap .
|
|
||||||
cd dispatch && zip -j function.zip bootstrap && rm bootstrap
|
|
||||||
|
|
||||||
build-exec:
|
|
||||||
cd exec && GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -tags lambda.norpc -o bootstrap .
|
|
||||||
cd exec && zip -j function.zip bootstrap && rm bootstrap
|
|
||||||
|
|
||||||
tidy:
|
|
||||||
cd dispatch && go mod tidy
|
|
||||||
cd exec && go mod tidy
|
|
||||||
|
|
||||||
test-local:
|
|
||||||
cd deploy && sam local invoke DispatchFunction \
|
|
||||||
--template-file template.yaml \
|
|
||||||
--event ../test-event.json \
|
|
||||||
--region $(REGION)
|
|
||||||
|
|
||||||
deploy:
|
|
||||||
cd deploy && sam build --template-file template.yaml && \
|
|
||||||
sam deploy \
|
|
||||||
--template-file .aws-sam/build/template.yaml \
|
|
||||||
--stack-name tinqs-ci \
|
|
||||||
--region $(REGION) \
|
|
||||||
--capabilities CAPABILITY_IAM \
|
|
||||||
--parameter-overrides \
|
|
||||||
GiteaToken=$${GITEA_TOKEN} \
|
|
||||||
Subnets=$${SUBNETS} \
|
|
||||||
SecurityGroup=$${SECURITY_GROUP} \
|
|
||||||
--resolve-s3 \
|
|
||||||
--no-confirm-changeset
|
|
||||||
|
|
||||||
deploy-guided:
|
|
||||||
cd deploy && sam build --template-file template.yaml && \
|
|
||||||
sam deploy --guided --template-file .aws-sam/build/template.yaml --stack-name tinqs-ci
|
|
||||||
|
|
||||||
clean:
|
|
||||||
rm -f dispatch/bootstrap dispatch/function.zip
|
|
||||||
rm -f exec/bootstrap exec/function.zip
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
# tinqs/ci orchestrator
|
|
||||||
|
|
||||||
Lambda-based CI dispatcher for Tinqs Studio. Receives Gitea webhooks and routes jobs to the right execution environment.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
Gitea push webhook
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
API Gateway POST /webhook
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
ci-dispatch Lambda
|
|
||||||
│
|
|
||||||
├── runs-on: go/node/docker/godot
|
|
||||||
│ → Start Fargate task with matching image
|
|
||||||
│ → Track in DynamoDB for cancel
|
|
||||||
│
|
|
||||||
├── runs-on: deploy
|
|
||||||
│ → Invoke ci-exec Lambda directly
|
|
||||||
│
|
|
||||||
└── runs-on: host
|
|
||||||
→ Skip (handled by registered runner)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Deploy
|
|
||||||
|
|
||||||
Requires: AWS SAM CLI, AWS credentials, Gitea token.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# First time (interactive)
|
|
||||||
GITEA_TOKEN=xxx make deploy-guided
|
|
||||||
|
|
||||||
# Subsequent deploys
|
|
||||||
GITEA_TOKEN=xxx SUBNETS=subnet-abc,subnet-def SECURITY_GROUP=sg-xxx make deploy
|
|
||||||
```
|
|
||||||
|
|
||||||
After deploy, configure the webhook URL as a Gitea **system webhook**:
|
|
||||||
- URL: `https://<api-id>.execute-api.eu-west-1.amazonaws.com/prod/webhook`
|
|
||||||
- Method: POST
|
|
||||||
- Content type: application/json
|
|
||||||
- Events: Push, Workflow Job (for cancel)
|
|
||||||
|
|
||||||
## Cancel support
|
|
||||||
|
|
||||||
When a user cancels a job in the Gitea UI, the `workflow_job` webhook fires with `action: cancelled`. The dispatcher looks up the Fargate task ARN in DynamoDB and calls `ecs:StopTask`.
|
|
||||||
|
|
||||||
A cleanup cron (every 5 min) also kills Fargate tasks that have been running longer than 30 minutes.
|
|
||||||
|
|
||||||
## Local testing
|
|
||||||
|
|
||||||
```bash
|
|
||||||
make build
|
|
||||||
make test-local # requires SAM CLI + Docker
|
|
||||||
```
|
|
||||||
@@ -1,177 +0,0 @@
|
|||||||
AWSTemplateFormatVersion: '2010-09-09'
|
|
||||||
Transform: AWS::Serverless-2016-10-31
|
|
||||||
Description: Tinqs CI Orchestrator — Lambda dispatch + EC2 Spot runners
|
|
||||||
|
|
||||||
Parameters:
|
|
||||||
GiteaURL:
|
|
||||||
Type: String
|
|
||||||
Default: https://tinqs.com
|
|
||||||
GiteaToken:
|
|
||||||
Type: String
|
|
||||||
NoEcho: true
|
|
||||||
RunnerAMI:
|
|
||||||
Type: AWS::EC2::Image::Id
|
|
||||||
Description: Pre-baked AMI with Go, Node, Docker, AWS CLI, act_runner
|
|
||||||
Subnet:
|
|
||||||
Type: AWS::EC2::Subnet::Id
|
|
||||||
Description: Public subnet for spot instances
|
|
||||||
SecurityGroup:
|
|
||||||
Type: AWS::EC2::SecurityGroup::Id
|
|
||||||
Description: Security group for spot instances
|
|
||||||
InstanceProfileArn:
|
|
||||||
Type: String
|
|
||||||
Description: IAM instance profile ARN for runners
|
|
||||||
|
|
||||||
Globals:
|
|
||||||
Function:
|
|
||||||
Runtime: provided.al2023
|
|
||||||
Architectures: [x86_64]
|
|
||||||
Timeout: 30
|
|
||||||
MemorySize: 128
|
|
||||||
|
|
||||||
Resources:
|
|
||||||
|
|
||||||
# --- API Gateway ---
|
|
||||||
WebhookApi:
|
|
||||||
Type: AWS::Serverless::Api
|
|
||||||
Properties:
|
|
||||||
Name: tinqs-ci-webhook
|
|
||||||
StageName: prod
|
|
||||||
|
|
||||||
# --- DynamoDB ---
|
|
||||||
RunsTable:
|
|
||||||
Type: AWS::DynamoDB::Table
|
|
||||||
Properties:
|
|
||||||
TableName: tinqs-ci-runs
|
|
||||||
BillingMode: PAY_PER_REQUEST
|
|
||||||
AttributeDefinitions:
|
|
||||||
- AttributeName: repo
|
|
||||||
AttributeType: S
|
|
||||||
- AttributeName: run_id
|
|
||||||
AttributeType: S
|
|
||||||
KeySchema:
|
|
||||||
- AttributeName: repo
|
|
||||||
KeyType: HASH
|
|
||||||
- AttributeName: run_id
|
|
||||||
KeyType: RANGE
|
|
||||||
TimeToLiveSpecification:
|
|
||||||
AttributeName: ttl
|
|
||||||
Enabled: true
|
|
||||||
|
|
||||||
# --- Dispatcher Lambda ---
|
|
||||||
DispatchFunction:
|
|
||||||
Type: AWS::Serverless::Function
|
|
||||||
Properties:
|
|
||||||
FunctionName: tinqs-ci-dispatch
|
|
||||||
Handler: bootstrap
|
|
||||||
CodeUri: ../dispatch/
|
|
||||||
Description: Receives webhook, starts Spot instances or invokes Lambda executor
|
|
||||||
Timeout: 60
|
|
||||||
MemorySize: 256
|
|
||||||
Environment:
|
|
||||||
Variables:
|
|
||||||
GITEA_URL: !Ref GiteaURL
|
|
||||||
GITEA_TOKEN: !Ref GiteaToken
|
|
||||||
EXECUTOR_FUNCTION_NAME: !Ref ExecFunction
|
|
||||||
RUNNER_AMI: !Ref RunnerAMI
|
|
||||||
SUBNET: !Ref Subnet
|
|
||||||
SECURITY_GROUP: !Ref SecurityGroup
|
|
||||||
DDB_TABLE: !Ref RunsTable
|
|
||||||
INSTANCE_PROFILE: !Ref InstanceProfileArn
|
|
||||||
Policies:
|
|
||||||
- LambdaInvokePolicy:
|
|
||||||
FunctionName: !Ref ExecFunction
|
|
||||||
- DynamoDBCrudPolicy:
|
|
||||||
TableName: !Ref RunsTable
|
|
||||||
- Version: '2012-10-17'
|
|
||||||
Statement:
|
|
||||||
- Effect: Allow
|
|
||||||
Action:
|
|
||||||
- ec2:RunInstances
|
|
||||||
- ec2:TerminateInstances
|
|
||||||
- ec2:DescribeInstances
|
|
||||||
- ec2:CreateTags
|
|
||||||
Resource: '*'
|
|
||||||
- Effect: Allow
|
|
||||||
Action: iam:PassRole
|
|
||||||
Resource: !Ref InstanceProfileArn
|
|
||||||
Events:
|
|
||||||
Webhook:
|
|
||||||
Type: Api
|
|
||||||
Properties:
|
|
||||||
RestApiId: !Ref WebhookApi
|
|
||||||
Path: /webhook
|
|
||||||
Method: POST
|
|
||||||
|
|
||||||
# --- Executor Lambda (deploy-only jobs) ---
|
|
||||||
ExecFunction:
|
|
||||||
Type: AWS::Serverless::Function
|
|
||||||
Properties:
|
|
||||||
FunctionName: tinqs-ci-exec
|
|
||||||
Handler: bootstrap
|
|
||||||
CodeUri: ../exec/
|
|
||||||
Description: Executes deploy-only workflow steps directly in Lambda
|
|
||||||
Timeout: 900
|
|
||||||
MemorySize: 2048
|
|
||||||
EphemeralStorage:
|
|
||||||
Size: 5120
|
|
||||||
Environment:
|
|
||||||
Variables:
|
|
||||||
GITEA_URL: !Ref GiteaURL
|
|
||||||
GITEA_TOKEN: !Ref GiteaToken
|
|
||||||
Policies:
|
|
||||||
- S3CrudPolicy:
|
|
||||||
BucketName: tinqs-cli-releases
|
|
||||||
- S3CrudPolicy:
|
|
||||||
BucketName: arikigame.com
|
|
||||||
- S3CrudPolicy:
|
|
||||||
BucketName: docs.tinqs.com
|
|
||||||
- Version: '2012-10-17'
|
|
||||||
Statement:
|
|
||||||
- Effect: Allow
|
|
||||||
Action:
|
|
||||||
- ecs:UpdateService
|
|
||||||
- ecs:DescribeServices
|
|
||||||
Resource: !Sub 'arn:aws:ecs:${AWS::Region}:${AWS::AccountId}:service/tinqs-git/*'
|
|
||||||
- Effect: Allow
|
|
||||||
Action:
|
|
||||||
- cloudfront:CreateInvalidation
|
|
||||||
Resource: '*'
|
|
||||||
|
|
||||||
# --- Cleanup Cron (every 5 min) ---
|
|
||||||
CleanupRule:
|
|
||||||
Type: AWS::Events::Rule
|
|
||||||
Properties:
|
|
||||||
Name: tinqs-ci-cleanup
|
|
||||||
ScheduleExpression: 'rate(5 minutes)'
|
|
||||||
State: ENABLED
|
|
||||||
Targets:
|
|
||||||
- Id: cleanup
|
|
||||||
Arn: !GetAtt DispatchFunction.Arn
|
|
||||||
Input: '{"body":"{\"action\":\"cleanup\"}","headers":{}}'
|
|
||||||
|
|
||||||
CleanupPermission:
|
|
||||||
Type: AWS::Lambda::Permission
|
|
||||||
Properties:
|
|
||||||
FunctionName: !Ref DispatchFunction
|
|
||||||
Action: lambda:InvokeFunction
|
|
||||||
Principal: events.amazonaws.com
|
|
||||||
SourceArn: !GetAtt CleanupRule.Arn
|
|
||||||
|
|
||||||
# --- Log Group ---
|
|
||||||
CILogGroup:
|
|
||||||
Type: AWS::Logs::LogGroup
|
|
||||||
Properties:
|
|
||||||
LogGroupName: /tinqs/ci
|
|
||||||
RetentionInDays: 14
|
|
||||||
|
|
||||||
Outputs:
|
|
||||||
WebhookURL:
|
|
||||||
Description: Configure as Gitea system webhook
|
|
||||||
Value: !Sub 'https://${WebhookApi}.execute-api.${AWS::Region}.amazonaws.com/prod/webhook'
|
|
||||||
DispatchArn:
|
|
||||||
Value: !GetAtt DispatchFunction.Arn
|
|
||||||
ExecArn:
|
|
||||||
Value: !GetAtt ExecFunction.Arn
|
|
||||||
RunsTable:
|
|
||||||
Value: !Ref RunsTable
|
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
/bootstrap
|
||||||
|
/function.zip
|
||||||
|
/dispatch.exe
|
||||||
@@ -4,7 +4,7 @@ go 1.23
|
|||||||
|
|
||||||
require (
|
require (
|
||||||
github.com/aws/aws-lambda-go v1.47.0
|
github.com/aws/aws-lambda-go v1.47.0
|
||||||
github.com/aws/aws-sdk-go-v2 v1.32.5
|
github.com/aws/aws-sdk-go-v2 v1.32.6
|
||||||
github.com/aws/aws-sdk-go-v2/config v1.28.0
|
github.com/aws/aws-sdk-go-v2/config v1.28.0
|
||||||
github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue v1.15.12
|
github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue v1.15.12
|
||||||
github.com/aws/aws-sdk-go-v2/service/dynamodb v1.38.0
|
github.com/aws/aws-sdk-go-v2/service/dynamodb v1.38.0
|
||||||
@@ -12,3 +12,21 @@ require (
|
|||||||
github.com/aws/aws-sdk-go-v2/service/lambda v1.69.0
|
github.com/aws/aws-sdk-go-v2/service/lambda v1.69.0
|
||||||
gopkg.in/yaml.v3 v3.0.1
|
gopkg.in/yaml.v3 v3.0.1
|
||||||
)
|
)
|
||||||
|
|
||||||
|
require (
|
||||||
|
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.7 // indirect
|
||||||
|
github.com/aws/aws-sdk-go-v2/credentials v1.17.41 // indirect
|
||||||
|
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.17 // indirect
|
||||||
|
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.25 // indirect
|
||||||
|
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.25 // indirect
|
||||||
|
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/dynamodbstreams v1.24.2 // indirect
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1 // indirect
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.10.6 // indirect
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.6 // indirect
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/sso v1.24.2 // indirect
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.2 // indirect
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/sts v1.32.2 // indirect
|
||||||
|
github.com/aws/smithy-go v1.22.1 // indirect
|
||||||
|
github.com/jmespath/go-jmespath v0.4.0 // indirect
|
||||||
|
)
|
||||||
|
|||||||
@@ -0,0 +1,60 @@
|
|||||||
|
github.com/aws/aws-lambda-go v1.47.0 h1:0H8s0vumYx/YKs4sE7YM0ktwL2eWse+kfopsRI1sXVI=
|
||||||
|
github.com/aws/aws-lambda-go v1.47.0/go.mod h1:dpMpZgvWx5vuQJfBt0zqBha60q7Dd7RfgJv23DymV8A=
|
||||||
|
github.com/aws/aws-sdk-go-v2 v1.32.6 h1:7BokKRgRPuGmKkFMhEg/jSul+tB9VvXhcViILtfG8b4=
|
||||||
|
github.com/aws/aws-sdk-go-v2 v1.32.6/go.mod h1:P5WJBrYqqbWVaOxgH0X/FYYD47/nooaPOZPlQdmiN2U=
|
||||||
|
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.7 h1:lL7IfaFzngfx0ZwUGOZdsFFnQ5uLvR0hWqqhyE7Q9M8=
|
||||||
|
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.7/go.mod h1:QraP0UcVlQJsmHfioCrveWOC1nbiWUl3ej08h4mXWoc=
|
||||||
|
github.com/aws/aws-sdk-go-v2/config v1.28.0 h1:FosVYWcqEtWNxHn8gB/Vs6jOlNwSoyOCA/g/sxyySOQ=
|
||||||
|
github.com/aws/aws-sdk-go-v2/config v1.28.0/go.mod h1:pYhbtvg1siOOg8h5an77rXle9tVG8T+BWLWAo7cOukc=
|
||||||
|
github.com/aws/aws-sdk-go-v2/credentials v1.17.41 h1:7gXo+Axmp+R4Z+AK8YFQO0ZV3L0gizGINCOWxSLY9W8=
|
||||||
|
github.com/aws/aws-sdk-go-v2/credentials v1.17.41/go.mod h1:u4Eb8d3394YLubphT4jLEwN1rLNq2wFOlT6OuxFwPzU=
|
||||||
|
github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue v1.15.12 h1:zYf8E8zaqolHA5nQ+VmX2r3wc4K6xw5i6xKvvMjZBL0=
|
||||||
|
github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue v1.15.12/go.mod h1:vYGIVLASk19Gb0FGwAcwES+qQF/aekD7m2G/X6mBOdQ=
|
||||||
|
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.17 h1:TMH3f/SCAWdNtXXVPPu5D6wrr4G5hI1rAxbcocKfC7Q=
|
||||||
|
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.17/go.mod h1:1ZRXLdTpzdJb9fwTMXiLipENRxkGMTn1sfKexGllQCw=
|
||||||
|
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.25 h1:s/fF4+yDQDoElYhfIVvSNyeCydfbuTKzhxSXDXCPasU=
|
||||||
|
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.25/go.mod h1:IgPfDv5jqFIzQSNbUEMoitNooSMXjRSDkhXv8jiROvU=
|
||||||
|
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.25 h1:ZntTCl5EsYnhN/IygQEUugpdwbhdkom9uHcbCftiGgA=
|
||||||
|
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.25/go.mod h1:DBdPrgeocww+CSl1C8cEV8PN1mHMBhuCDLpXezyvWkE=
|
||||||
|
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 h1:VaRN3TlFdd6KxX1x3ILT5ynH6HvKgqdiXoTxAF4HQcQ=
|
||||||
|
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1/go.mod h1:FbtygfRFze9usAadmnGJNc8KsP346kEe+y2/oyhGAGc=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/dynamodb v1.38.0 h1:isKhHsjpQR3CypQJ4G1g8QWx7zNpiC/xKw1zjgJYVno=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/dynamodb v1.38.0/go.mod h1:xDvUyIkwBwNtVZJdHEwAuhFly3mezwdEWkbJ5oNYwIw=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/dynamodbstreams v1.24.2 h1:E7Tuo0ipWpBl0f3uThz8cZsuyD5H8jLCnbtbKR4YL2s=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/dynamodbstreams v1.24.2/go.mod h1:txOfweuNPBLhHodsV+C2lvPPRTommVTWbts9SZV6Myc=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/ec2 v1.198.0 h1:ivPJXmGlzAjgy0jLO9naExUWE8IM8lLRcRKLPBEx6Q0=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/ec2 v1.198.0/go.mod h1:00zqVNJFK6UASrTnuvjJHJuaqUdkVz5tW8Ip+VhzuNg=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1 h1:iXtILhvDxB6kPvEXgsDhGaZCSC6LQET5ZHSdJozeI0Y=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1/go.mod h1:9nu0fVANtYiAePIBh2/pFUSwtJ402hLnp854CNoDOeE=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.10.6 h1:nbmKXZzXPJn41CcD4HsHsGWqvKjLKz9kWu6XxvLmf1s=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.10.6/go.mod h1:SJhcisfKfAawsdNQoZMBEjg+vyN2lH6rO6fP+T94z5Y=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.6 h1:50+XsN70RS7dwJ2CkVNXzj7U2L1HKP8nqTd3XWEXBN4=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.6/go.mod h1:WqgLmwY7so32kG01zD8CPTJWVWM+TzJoOVHwTg4aPug=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/lambda v1.69.0 h1:BXt75frE/FYtAmEDBJRBa2HexOw+oAZWZl6QknZEFgg=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/lambda v1.69.0/go.mod h1:guz2K3x4FKSdDaoeB+TPVgJNU9oj2gftbp5cR8ela1A=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/sso v1.24.2 h1:bSYXVyUzoTHoKalBmwaZxs97HU9DWWI3ehHSAMa7xOk=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/sso v1.24.2/go.mod h1:skMqY7JElusiOUjMJMOv1jJsP7YUg7DrhgqZZWuzu1U=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.2 h1:AhmO1fHINP9vFYUE0LHzCWg/LfUWUF+zFPEcY9QXb7o=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.2/go.mod h1:o8aQygT2+MVP0NaV6kbdE1YnnIM8RRVQzoeUH45GOdI=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/sts v1.32.2 h1:CiS7i0+FUe+/YY1GvIBLLrR/XNGZ4CtM1Ll0XavNuVo=
|
||||||
|
github.com/aws/aws-sdk-go-v2/service/sts v1.32.2/go.mod h1:HtaiBI8CjYoNVde8arShXb94UbQQi9L4EMr6D+xGBwo=
|
||||||
|
github.com/aws/smithy-go v1.22.1 h1:/HPHZQ0g7f4eUeK6HKglFz8uwVfZKgoI25rb/J+dnro=
|
||||||
|
github.com/aws/smithy-go v1.22.1/go.mod h1:irrKGvNn1InZwb2d7fkIRNucdfwR8R+Ts3wxYa/cJHg=
|
||||||
|
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||||
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
|
||||||
|
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
|
||||||
|
github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=
|
||||||
|
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
|
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||||
|
github.com/stretchr/testify v1.7.2 h1:4jaiDzPyXQvSd7D0EjG45355tLlV3VOECpq10pLC+8s=
|
||||||
|
github.com/stretchr/testify v1.7.2/go.mod h1:R6va5+xMeoiuVRoj+gSkQ7d3FALtqAAGI1FQKckRals=
|
||||||
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||||
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10=
|
||||||
|
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
@@ -31,7 +31,6 @@ import (
|
|||||||
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
|
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
|
||||||
"github.com/aws/aws-sdk-go-v2/service/ec2"
|
"github.com/aws/aws-sdk-go-v2/service/ec2"
|
||||||
ec2types "github.com/aws/aws-sdk-go-v2/service/ec2/types"
|
ec2types "github.com/aws/aws-sdk-go-v2/service/ec2/types"
|
||||||
awslambda "github.com/aws/aws-sdk-go-v2/service/lambda"
|
|
||||||
"gopkg.in/yaml.v3"
|
"gopkg.in/yaml.v3"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -100,38 +99,36 @@ type spotConfig struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var labelToSpot = map[string]spotConfig{
|
var labelToSpot = map[string]spotConfig{
|
||||||
"go": {InstanceType: "t3.small", MaxPrice: "0.008"},
|
"go": {InstanceType: "t3.small", MaxPrice: "0.02"},
|
||||||
"node": {InstanceType: "t3.small", MaxPrice: "0.008"},
|
"node": {InstanceType: "t3.medium", MaxPrice: "0.04"},
|
||||||
"docker": {InstanceType: "t3.medium", MaxPrice: "0.016"},
|
"docker": {InstanceType: "t3.medium", MaxPrice: "0.04"},
|
||||||
"deploy": {InstanceType: "t3.micro", MaxPrice: "0.004"},
|
"deploy": {InstanceType: "t3.micro", MaxPrice: "0.01"},
|
||||||
"godot": {InstanceType: "t3.medium", MaxPrice: "0.016"},
|
"godot": {InstanceType: "t3.medium", MaxPrice: "0.04"},
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- Config from env ---
|
// --- Config from env ---
|
||||||
|
|
||||||
type cfg struct {
|
type cfg struct {
|
||||||
GiteaURL string
|
GiteaURL string
|
||||||
GiteaToken string
|
GiteaToken string // API token (for fetching workflows, setting commit status)
|
||||||
ExecFnName string
|
RunnerToken string // Runner registration token (for act_runner register)
|
||||||
AMI string // pre-baked AMI with Go, Node, Docker, AWS CLI, act_runner
|
AMI string // pre-baked AMI with Go, Node, Docker, AWS CLI, act_runner
|
||||||
Subnet string
|
Subnet string
|
||||||
SecurityGroup string
|
SecurityGroup string
|
||||||
DDBTable string
|
DDBTable string
|
||||||
InstanceProfile string
|
InstanceProfile string
|
||||||
Region string
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func loadCfg() cfg {
|
func loadCfg() cfg {
|
||||||
return cfg{
|
return cfg{
|
||||||
GiteaURL: os.Getenv("GITEA_URL"),
|
GiteaURL: os.Getenv("GITEA_URL"),
|
||||||
GiteaToken: os.Getenv("GITEA_TOKEN"),
|
GiteaToken: os.Getenv("GITEA_TOKEN"),
|
||||||
ExecFnName: os.Getenv("EXECUTOR_FUNCTION_NAME"),
|
RunnerToken: os.Getenv("RUNNER_TOKEN"),
|
||||||
AMI: os.Getenv("RUNNER_AMI"),
|
AMI: os.Getenv("RUNNER_AMI"),
|
||||||
Subnet: os.Getenv("SUBNET"),
|
Subnet: os.Getenv("SUBNET"),
|
||||||
SecurityGroup: os.Getenv("SECURITY_GROUP"),
|
SecurityGroup: os.Getenv("SECURITY_GROUP"),
|
||||||
DDBTable: os.Getenv("DDB_TABLE"),
|
DDBTable: os.Getenv("DDB_TABLE"),
|
||||||
InstanceProfile: os.Getenv("INSTANCE_PROFILE"), // IAM instance profile ARN
|
InstanceProfile: os.Getenv("INSTANCE_PROFILE"),
|
||||||
Region: os.Getenv("AWS_REGION"),
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -144,31 +141,61 @@ set -euo pipefail
|
|||||||
exec > /var/log/tinqs-ci.log 2>&1
|
exec > /var/log/tinqs-ci.log 2>&1
|
||||||
|
|
||||||
echo "=== Tinqs CI Runner: %s ==="
|
echo "=== Tinqs CI Runner: %s ==="
|
||||||
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
|
|
||||||
REGION=%s
|
|
||||||
|
|
||||||
# Self-termination trap — kill instance on exit (success or failure)
|
# Get instance metadata (IMDSv2)
|
||||||
cleanup() {
|
TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 60")
|
||||||
echo "=== Job done, terminating $INSTANCE_ID ==="
|
INSTANCE_ID=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id)
|
||||||
aws ec2 terminate-instances --instance-ids "$INSTANCE_ID" --region "$REGION" || true
|
REGION=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/region)
|
||||||
}
|
|
||||||
trap cleanup EXIT
|
|
||||||
|
|
||||||
# act_runner is pre-installed in the AMI at /usr/local/bin/act_runner
|
# Self-termination: shutdown triggers terminate (InstanceInitiatedShutdownBehavior=terminate)
|
||||||
cd /tmp
|
# No IAM needed — kernel handles it
|
||||||
|
trap 'echo "=== Job done, shutting down ===" && shutdown -h now' EXIT
|
||||||
|
|
||||||
|
export PATH=$PATH:/usr/local/go/bin:/usr/local/bin
|
||||||
|
export HOME=/root
|
||||||
|
|
||||||
|
# Git auth for private repo clones (checkout action uses git binary which reads this)
|
||||||
|
# Note: act_runner's internal action resolution uses go-git, which does NOT read
|
||||||
|
# gitconfig — so tinqs/ci must stay public for act_runner to clone actions.
|
||||||
|
GITEA_TOKEN="%s"
|
||||||
|
git config --global url."https://token:${GITEA_TOKEN}@tinqs.com/".insteadOf "https://tinqs.com/"
|
||||||
|
|
||||||
|
# Create proper working directory for act_runner
|
||||||
|
mkdir -p /opt/runner && cd /opt/runner
|
||||||
|
|
||||||
# Register as ephemeral runner (picks one job, then exits)
|
# Register as ephemeral runner (picks one job, then exits)
|
||||||
act_runner register --no-interactive \
|
act_runner register --no-interactive \
|
||||||
--instance %s \
|
--instance %s \
|
||||||
--token %s \
|
--token %s \
|
||||||
--name %s \
|
--name %s \
|
||||||
--labels "%s:host"
|
--labels "%s:host" \
|
||||||
|
--ephemeral
|
||||||
|
|
||||||
# Run — blocks until the job completes, then exits (ephemeral mode)
|
# Configure runner
|
||||||
act_runner daemon
|
cat > .runner.yaml << 'RUNCFG'
|
||||||
|
log:
|
||||||
|
level: info
|
||||||
|
runner:
|
||||||
|
capacity: 1
|
||||||
|
timeout: 30m
|
||||||
|
envs:
|
||||||
|
HOME: /root
|
||||||
|
AWS_DEFAULT_REGION: eu-west-1
|
||||||
|
PATH: /usr/local/go/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
|
||||||
|
host:
|
||||||
|
workdir_parent: /opt/runner/work
|
||||||
|
RUNCFG
|
||||||
|
|
||||||
|
mkdir -p /opt/runner/work
|
||||||
|
|
||||||
|
# Run — blocks until the job completes, then exits (ephemeral = one job only)
|
||||||
|
act_runner daemon --config .runner.yaml
|
||||||
|
|
||||||
|
# Safety: if daemon exits without triggering trap (shouldn't happen), force shutdown
|
||||||
|
shutdown -h now
|
||||||
|
|
||||||
echo "=== Runner exited, cleanup will terminate instance ==="
|
echo "=== Runner exited, cleanup will terminate instance ==="
|
||||||
`, runnerName, c.Region, c.GiteaURL, c.GiteaToken, runnerName, label)
|
`, runnerName, c.GiteaToken, c.GiteaURL, c.RunnerToken, runnerName, label)
|
||||||
|
|
||||||
return base64.StdEncoding.EncodeToString([]byte(script))
|
return base64.StdEncoding.EncodeToString([]byte(script))
|
||||||
}
|
}
|
||||||
@@ -236,15 +263,14 @@ func handler(ctx context.Context, request events.APIGatewayProxyRequest) (events
|
|||||||
runID := fmt.Sprintf("%s-%s-%d", push.After[:7], name, time.Now().UnixMilli())
|
runID := fmt.Sprintf("%s-%s-%d", push.After[:7], name, time.Now().UnixMilli())
|
||||||
|
|
||||||
switch label {
|
switch label {
|
||||||
case "deploy":
|
|
||||||
if err := invokeLambdaExec(ctx, c, push, name, content); err != nil {
|
|
||||||
log.Printf("Failed to invoke executor for %s: %v", name, err)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
case "host":
|
case "host":
|
||||||
log.Printf("Skipping %s (runs-on: host — registered runner)", name)
|
log.Printf("Skipping %s (runs-on: host — registered runner)", name)
|
||||||
continue
|
continue
|
||||||
default:
|
default:
|
||||||
|
// All non-host jobs (go/node/docker/deploy/godot) go through Spot.
|
||||||
|
// The old `deploy` → Lambda-executor path was retired when
|
||||||
|
// tinqs-ci-exec was deleted (26 May 2026); deploy now launches
|
||||||
|
// a t3.micro Spot runner labelled `deploy` like any other job.
|
||||||
instanceID, err := startSpotRunner(ctx, c, label, runID)
|
instanceID, err := startSpotRunner(ctx, c, label, runID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Printf("Failed to start spot for %s: %v", name, err)
|
log.Printf("Failed to start spot for %s: %v", name, err)
|
||||||
@@ -276,11 +302,23 @@ func startSpotRunner(ctx context.Context, c cfg, label, runID string) (string, e
|
|||||||
return "", fmt.Errorf("unknown label: %s", label)
|
return "", fmt.Errorf("unknown label: %s", label)
|
||||||
}
|
}
|
||||||
|
|
||||||
runnerName := fmt.Sprintf("spot-%s-%s", label, runID[:12])
|
// runID = "<sha7>-<workflow>-<ms>" already encodes everything unique.
|
||||||
|
// The old runID[:12] truncation collapsed every deploy of a commit to the
|
||||||
|
// same name (e.g. deploy-arikigame + release on one push), confusing task
|
||||||
|
// routing and same-commit reruns. Use the full runID, sanitised for the
|
||||||
|
// runner-name charset (no dots/slashes from workflow filenames).
|
||||||
|
safeID := strings.NewReplacer(".", "-", "/", "-").Replace(runID)
|
||||||
|
runnerName := fmt.Sprintf("spot-%s-%s", label, safeID)
|
||||||
userData := userDataScript(c, label, runnerName)
|
userData := userDataScript(c, label, runnerName)
|
||||||
|
|
||||||
client := ec2.NewFromConfig(awsCfg)
|
client := ec2.NewFromConfig(awsCfg)
|
||||||
|
|
||||||
|
// Docker builds need more disk (monorepo + layers)
|
||||||
|
diskSize := int32(20)
|
||||||
|
if label == "docker" {
|
||||||
|
diskSize = 40
|
||||||
|
}
|
||||||
|
|
||||||
out, err := client.RunInstances(ctx, &ec2.RunInstancesInput{
|
out, err := client.RunInstances(ctx, &ec2.RunInstancesInput{
|
||||||
ImageId: aws.String(c.AMI),
|
ImageId: aws.String(c.AMI),
|
||||||
InstanceType: ec2types.InstanceType(spot.InstanceType),
|
InstanceType: ec2types.InstanceType(spot.InstanceType),
|
||||||
@@ -288,6 +326,16 @@ func startSpotRunner(ctx context.Context, c cfg, label, runID string) (string, e
|
|||||||
MaxCount: aws.Int32(1),
|
MaxCount: aws.Int32(1),
|
||||||
UserData: aws.String(userData),
|
UserData: aws.String(userData),
|
||||||
|
|
||||||
|
// Root volume — bigger for Docker builds
|
||||||
|
BlockDeviceMappings: []ec2types.BlockDeviceMapping{{
|
||||||
|
DeviceName: aws.String("/dev/xvda"),
|
||||||
|
Ebs: &ec2types.EbsBlockDevice{
|
||||||
|
VolumeSize: aws.Int32(diskSize),
|
||||||
|
VolumeType: ec2types.VolumeTypeGp3,
|
||||||
|
DeleteOnTermination: aws.Bool(true),
|
||||||
|
},
|
||||||
|
}},
|
||||||
|
|
||||||
// Spot request
|
// Spot request
|
||||||
InstanceMarketOptions: &ec2types.InstanceMarketOptionsRequest{
|
InstanceMarketOptions: &ec2types.InstanceMarketOptionsRequest{
|
||||||
MarketType: ec2types.MarketTypeSpot,
|
MarketType: ec2types.MarketTypeSpot,
|
||||||
@@ -339,39 +387,6 @@ func startSpotRunner(ctx context.Context, c cfg, label, runID string) (string, e
|
|||||||
return instanceID, nil
|
return instanceID, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- Lambda executor (for deploy-only jobs) ---
|
|
||||||
|
|
||||||
func invokeLambdaExec(ctx context.Context, c cfg, push GiteaPushEvent, workflow, content string) error {
|
|
||||||
if c.ExecFnName == "" {
|
|
||||||
log.Printf("[DRY RUN] Would invoke executor for %s", workflow)
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
awsCfg, err := config.LoadDefaultConfig(ctx)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
payload, _ := json.Marshal(map[string]interface{}{
|
|
||||||
"repo": push.Repository,
|
|
||||||
"ref": push.Ref,
|
|
||||||
"commit_sha": push.After,
|
|
||||||
"pusher": push.Pusher.Login,
|
|
||||||
"workflow_name": workflow,
|
|
||||||
"workflow_yaml": content,
|
|
||||||
"gitea_url": c.GiteaURL,
|
|
||||||
"gitea_token": c.GiteaToken,
|
|
||||||
})
|
|
||||||
|
|
||||||
client := awslambda.NewFromConfig(awsCfg)
|
|
||||||
_, err = client.Invoke(ctx, &awslambda.InvokeInput{
|
|
||||||
FunctionName: aws.String(c.ExecFnName),
|
|
||||||
InvocationType: "Event",
|
|
||||||
Payload: payload,
|
|
||||||
})
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// --- Cancel handler ---
|
// --- Cancel handler ---
|
||||||
|
|
||||||
func handleCancel(ctx context.Context, c cfg, body string) (events.APIGatewayProxyResponse, error) {
|
func handleCancel(ctx context.Context, c cfg, body string) (events.APIGatewayProxyResponse, error) {
|
||||||
|
|||||||
@@ -1,8 +0,0 @@
|
|||||||
module tinqs.com/tinqs/ci/orchestrator/exec
|
|
||||||
|
|
||||||
go 1.23
|
|
||||||
|
|
||||||
require (
|
|
||||||
github.com/aws/aws-lambda-go v1.47.0
|
|
||||||
gopkg.in/yaml.v3 v3.0.1
|
|
||||||
)
|
|
||||||
@@ -1,233 +0,0 @@
|
|||||||
// tinqs/ci orchestrator — executor Lambda
|
|
||||||
//
|
|
||||||
// Handles deploy-only jobs directly in Lambda (no Fargate needed).
|
|
||||||
// Parses workflow YAML, runs shell steps, reports commit status to Gitea.
|
|
||||||
// 15 min timeout, 2 GB memory, 5 GB /tmp.
|
|
||||||
|
|
||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"log"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"os/exec"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/aws/aws-lambda-go/lambda"
|
|
||||||
"gopkg.in/yaml.v3"
|
|
||||||
)
|
|
||||||
|
|
||||||
type ExecPayload struct {
|
|
||||||
Repo GiteaRepo `json:"repo"`
|
|
||||||
Ref string `json:"ref"`
|
|
||||||
CommitSHA string `json:"commit_sha"`
|
|
||||||
Pusher string `json:"pusher"`
|
|
||||||
Workflow string `json:"workflow_name"`
|
|
||||||
WorkflowYAML string `json:"workflow_yaml"`
|
|
||||||
GiteaURL string `json:"gitea_url"`
|
|
||||||
GiteaToken string `json:"gitea_token"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type GiteaRepo struct {
|
|
||||||
FullName string `json:"full_name"`
|
|
||||||
CloneURL string `json:"clone_url"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type Workflow struct {
|
|
||||||
Name string `yaml:"name"`
|
|
||||||
Jobs map[string]WorkflowJob `yaml:"jobs"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type WorkflowJob struct {
|
|
||||||
Steps []WorkflowStep `yaml:"steps"`
|
|
||||||
Env map[string]string `yaml:"env"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type WorkflowStep struct {
|
|
||||||
Name string `yaml:"name"`
|
|
||||||
Run string `yaml:"run"`
|
|
||||||
Uses string `yaml:"uses"`
|
|
||||||
Env map[string]string `yaml:"env"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type StepResult struct {
|
|
||||||
Name string `json:"name"`
|
|
||||||
Status string `json:"status"`
|
|
||||||
Duration string `json:"duration"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type JobResult struct {
|
|
||||||
Workflow string `json:"workflow"`
|
|
||||||
Status string `json:"status"`
|
|
||||||
Steps []StepResult `json:"steps"`
|
|
||||||
Duration string `json:"duration"`
|
|
||||||
}
|
|
||||||
|
|
||||||
func handler(ctx context.Context, payload ExecPayload) (JobResult, error) {
|
|
||||||
sha := payload.CommitSHA
|
|
||||||
if len(sha) > 7 {
|
|
||||||
sha = sha[:7]
|
|
||||||
}
|
|
||||||
log.Printf("Executor: repo=%s workflow=%s sha=%s", payload.Repo.FullName, payload.Workflow, sha)
|
|
||||||
|
|
||||||
start := time.Now()
|
|
||||||
|
|
||||||
var wf Workflow
|
|
||||||
if err := yaml.Unmarshal([]byte(payload.WorkflowYAML), &wf); err != nil {
|
|
||||||
return JobResult{Workflow: payload.Workflow, Status: "failure"}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
setCommitStatus(payload, "pending", "CI running...")
|
|
||||||
|
|
||||||
overallStatus := "success"
|
|
||||||
var steps []StepResult
|
|
||||||
|
|
||||||
for _, job := range wf.Jobs {
|
|
||||||
// Build environment
|
|
||||||
env := os.Environ()
|
|
||||||
env = append(env,
|
|
||||||
"CI=true",
|
|
||||||
fmt.Sprintf("GITHUB_REPOSITORY=%s", payload.Repo.FullName),
|
|
||||||
fmt.Sprintf("GITHUB_REF=%s", payload.Ref),
|
|
||||||
fmt.Sprintf("GITHUB_SHA=%s", payload.CommitSHA),
|
|
||||||
)
|
|
||||||
for k, v := range job.Env {
|
|
||||||
env = append(env, fmt.Sprintf("%s=%s", k, v))
|
|
||||||
}
|
|
||||||
|
|
||||||
workdir := "/tmp/workspace"
|
|
||||||
os.MkdirAll(workdir, 0755)
|
|
||||||
os.RemoveAll(workdir)
|
|
||||||
os.MkdirAll(workdir, 0755)
|
|
||||||
|
|
||||||
for i, step := range job.Steps {
|
|
||||||
stepStart := time.Now()
|
|
||||||
name := step.Name
|
|
||||||
if name == "" {
|
|
||||||
name = fmt.Sprintf("Step %d", i+1)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Handle checkout action
|
|
||||||
if step.Uses != "" && contains(step.Uses, "checkout") {
|
|
||||||
cloneURL := payload.Repo.CloneURL
|
|
||||||
if payload.GiteaToken != "" {
|
|
||||||
cloneURL = fmt.Sprintf("https://token:%s@%s",
|
|
||||||
payload.GiteaToken, cloneURL[8:]) // strip https://
|
|
||||||
}
|
|
||||||
branch := payload.Ref
|
|
||||||
if len(branch) > 11 && branch[:11] == "refs/heads/" {
|
|
||||||
branch = branch[11:]
|
|
||||||
}
|
|
||||||
cmd := exec.CommandContext(ctx, "git", "clone", "--depth=1", "--branch", branch, cloneURL, workdir)
|
|
||||||
if out, err := cmd.CombinedOutput(); err != nil {
|
|
||||||
log.Printf("Checkout failed: %s", string(out))
|
|
||||||
steps = append(steps, StepResult{Name: name, Status: "failure", Duration: since(stepStart)})
|
|
||||||
overallStatus = "failure"
|
|
||||||
break
|
|
||||||
}
|
|
||||||
steps = append(steps, StepResult{Name: name, Status: "success", Duration: since(stepStart)})
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
// Skip unknown actions
|
|
||||||
if step.Uses != "" {
|
|
||||||
steps = append(steps, StepResult{Name: name, Status: "skipped", Duration: since(stepStart)})
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
if step.Run == "" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
// Run shell step
|
|
||||||
stepEnv := make([]string, len(env))
|
|
||||||
copy(stepEnv, env)
|
|
||||||
for k, v := range step.Env {
|
|
||||||
stepEnv = append(stepEnv, fmt.Sprintf("%s=%s", k, v))
|
|
||||||
}
|
|
||||||
|
|
||||||
cmd := exec.CommandContext(ctx, "bash", "-c", step.Run)
|
|
||||||
cmd.Dir = workdir
|
|
||||||
cmd.Env = stepEnv
|
|
||||||
|
|
||||||
var output bytes.Buffer
|
|
||||||
cmd.Stdout = io.MultiWriter(&output, os.Stdout)
|
|
||||||
cmd.Stderr = io.MultiWriter(&output, os.Stderr)
|
|
||||||
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
log.Printf("Step %d FAILED: %v", i+1, err)
|
|
||||||
steps = append(steps, StepResult{Name: name, Status: "failure", Duration: since(stepStart)})
|
|
||||||
overallStatus = "failure"
|
|
||||||
break
|
|
||||||
}
|
|
||||||
|
|
||||||
steps = append(steps, StepResult{Name: name, Status: "success", Duration: since(stepStart)})
|
|
||||||
}
|
|
||||||
|
|
||||||
if overallStatus == "failure" {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
desc := fmt.Sprintf("CI %s in %s", overallStatus, time.Since(start).Round(time.Second))
|
|
||||||
setCommitStatus(payload, overallStatus, desc)
|
|
||||||
|
|
||||||
return JobResult{
|
|
||||||
Workflow: payload.Workflow,
|
|
||||||
Status: overallStatus,
|
|
||||||
Steps: steps,
|
|
||||||
Duration: time.Since(start).Round(time.Second).String(),
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func setCommitStatus(payload ExecPayload, state, description string) {
|
|
||||||
if payload.GiteaURL == "" || payload.GiteaToken == "" {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
url := fmt.Sprintf("%s/api/v1/repos/%s/statuses/%s",
|
|
||||||
payload.GiteaURL, payload.Repo.FullName, payload.CommitSHA)
|
|
||||||
|
|
||||||
body, _ := json.Marshal(map[string]string{
|
|
||||||
"state": state,
|
|
||||||
"description": description,
|
|
||||||
"context": fmt.Sprintf("ci/%s", payload.Workflow),
|
|
||||||
})
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("POST", url, bytes.NewReader(body))
|
|
||||||
req.Header.Set("Authorization", "token "+payload.GiteaToken)
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp, err := (&http.Client{Timeout: 10 * time.Second}).Do(req)
|
|
||||||
if err != nil {
|
|
||||||
log.Printf("Failed to set commit status: %v", err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
resp.Body.Close()
|
|
||||||
}
|
|
||||||
|
|
||||||
func contains(s, sub string) bool {
|
|
||||||
return len(s) >= len(sub) && (s == sub || len(s) > 0 && containsStr(s, sub))
|
|
||||||
}
|
|
||||||
|
|
||||||
func containsStr(s, sub string) bool {
|
|
||||||
for i := 0; i <= len(s)-len(sub); i++ {
|
|
||||||
if s[i:i+len(sub)] == sub {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
func since(t time.Time) string {
|
|
||||||
return time.Since(t).Round(time.Millisecond).String()
|
|
||||||
}
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
lambda.Start(handler)
|
|
||||||
}
|
|
||||||
@@ -1,9 +0,0 @@
|
|||||||
{
|
|
||||||
"body": "{\"ref\":\"refs/heads/main\",\"before\":\"0000000\",\"after\":\"abc1234567890\",\"repository\":{\"id\":1,\"name\":\"studio\",\"full_name\":\"tinqs/studio\",\"clone_url\":\"https://tinqs.com/tinqs/studio.git\",\"html_url\":\"https://tinqs.com/tinqs/studio\"},\"pusher\":{\"login\":\"ozan\",\"email\":\"ozan@tinqs.com\"},\"commits\":[{\"id\":\"abc1234567890\",\"message\":\"test push\",\"added\":[\"cmd/tstudio/main.go\"],\"removed\":[],\"modified\":[]}]}",
|
|
||||||
"headers": {
|
|
||||||
"x-gitea-event": "push",
|
|
||||||
"content-type": "application/json"
|
|
||||||
},
|
|
||||||
"httpMethod": "POST",
|
|
||||||
"path": "/webhook"
|
|
||||||
}
|
|
||||||
@@ -1,8 +1,9 @@
|
|||||||
# tinqs/ci/setup-aws — Tinqs Studio CI
|
# tinqs/ci/setup-aws — Tinqs Studio CI
|
||||||
# Installs AWS CLI and optionally logs into ECR.
|
# Installs AWS CLI and optionally logs into ECR.
|
||||||
# Detects Alpine (musl) vs Debian (glibc) and uses the right install method.
|
# Detects Alpine (apk/pip), Debian (apt-get), and Amazon Linux/RHEL (dnf).
|
||||||
# Credentials come from the runner environment (IAM task role on Fargate, instance
|
# Skips install if AWS CLI is already present (pre-baked AMI).
|
||||||
# profile on EC2/Lightsail) — no explicit key configuration needed.
|
# Credentials come from the runner environment (IAM instance profile on EC2,
|
||||||
|
# task role on Fargate) — no explicit key configuration needed.
|
||||||
# Composite action — runs directly on the host.
|
# Composite action — runs directly on the host.
|
||||||
# Author: Ozan + Claude Code — 2026-05-22
|
# Author: Ozan + Claude Code — 2026-05-22
|
||||||
|
|
||||||
@@ -35,9 +36,8 @@ runs:
|
|||||||
# Alpine — use pip (AWS CLI v2 binary needs glibc)
|
# Alpine — use pip (AWS CLI v2 binary needs glibc)
|
||||||
apk add --no-cache python3 py3-pip
|
apk add --no-cache python3 py3-pip
|
||||||
pip3 install --break-system-packages awscli 2>/dev/null || pip3 install awscli
|
pip3 install --break-system-packages awscli 2>/dev/null || pip3 install awscli
|
||||||
elif command -v apt-get &>/dev/null; then
|
elif command -v dnf &>/dev/null || command -v apt-get &>/dev/null; then
|
||||||
# Debian/Ubuntu — use official installer
|
# Amazon Linux / Debian — official installer (glibc available)
|
||||||
apt-get update && apt-get install -y unzip curl
|
|
||||||
curl -fsSL https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o /tmp/awscli.zip
|
curl -fsSL https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o /tmp/awscli.zip
|
||||||
unzip -q /tmp/awscli.zip -d /tmp/aws-install
|
unzip -q /tmp/awscli.zip -d /tmp/aws-install
|
||||||
/tmp/aws-install/aws/install
|
/tmp/aws-install/aws/install
|
||||||
|
|||||||
+14
-3
@@ -1,7 +1,7 @@
|
|||||||
# tinqs/ci/setup-node — Tinqs Studio CI
|
# tinqs/ci/setup-node — Tinqs Studio CI
|
||||||
# Installs Node.js and optionally pnpm.
|
# Installs Node.js and optionally pnpm.
|
||||||
# Detects Alpine vs Debian and uses the right package manager.
|
# Detects Alpine (apk), Debian (apt-get), and Amazon Linux/RHEL (dnf).
|
||||||
# Skips install if the correct major version is already present (pre-baked runner image).
|
# Skips install if the correct major version is already present (pre-baked AMI).
|
||||||
# Composite action — runs directly on the host.
|
# Composite action — runs directly on the host.
|
||||||
# Author: Ozan + Claude Code — 2026-05-22
|
# Author: Ozan + Claude Code — 2026-05-22
|
||||||
|
|
||||||
@@ -29,17 +29,28 @@ runs:
|
|||||||
if [ "$CURRENT" = "$NODE_VERSION" ]; then
|
if [ "$CURRENT" = "$NODE_VERSION" ]; then
|
||||||
echo "Node $NODE_VERSION already installed"
|
echo "Node $NODE_VERSION already installed"
|
||||||
node --version
|
node --version
|
||||||
if [ "$INSTALL_PNPM" = "true" ] && command -v pnpm &>/dev/null; then
|
if [ "$INSTALL_PNPM" = "true" ]; then
|
||||||
|
if command -v pnpm &>/dev/null; then
|
||||||
|
pnpm --version
|
||||||
|
exit 0
|
||||||
|
else
|
||||||
|
npm install -g pnpm
|
||||||
pnpm --version
|
pnpm --version
|
||||||
exit 0
|
exit 0
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "Installing Node.js $NODE_VERSION..."
|
echo "Installing Node.js $NODE_VERSION..."
|
||||||
if command -v apk &>/dev/null; then
|
if command -v apk &>/dev/null; then
|
||||||
# Alpine
|
# Alpine
|
||||||
apk add --no-cache nodejs npm
|
apk add --no-cache nodejs npm
|
||||||
|
elif command -v dnf &>/dev/null; then
|
||||||
|
# Amazon Linux / RHEL / Fedora
|
||||||
|
curl -fsSL "https://rpm.nodesource.com/setup_${NODE_VERSION}.x" | bash -
|
||||||
|
dnf install -y nodejs
|
||||||
elif command -v apt-get &>/dev/null; then
|
elif command -v apt-get &>/dev/null; then
|
||||||
# Debian/Ubuntu
|
# Debian/Ubuntu
|
||||||
curl -fsSL "https://deb.nodesource.com/setup_${NODE_VERSION}.x" | bash -
|
curl -fsSL "https://deb.nodesource.com/setup_${NODE_VERSION}.x" | bash -
|
||||||
|
|||||||
@@ -0,0 +1,80 @@
|
|||||||
|
# Architecture
|
||||||
|
|
||||||
|
[← Home](README.md) · [DevOps Reference](DevOps-Reference.md) · [Operations](Operations.md) · [Roadmap](Roadmap.md)
|
||||||
|
|
||||||
|
```
|
||||||
|
Push → Gitea webhook → Lambda (tinqs-ci-dispatch) → EC2 Spot → act_runner → job → self-terminate
|
||||||
|
```
|
||||||
|
|
||||||
|
Runners are **ephemeral**: one Spot instance per job, self-terminating on completion. Private-repo clones are authenticated via `git config url.insteadOf` injected in the runner user-data.
|
||||||
|
|
||||||
|
## Key design decisions
|
||||||
|
|
||||||
|
- **Ephemeral Spot instances** (not Fargate, not persistent runners) — cheapest, cleanest, no state to manage.
|
||||||
|
- **`--ephemeral` on `act_runner register`** — the runner exits after one job, triggering `shutdown -h now` → the instance terminates. Without this, runners pile up as zombies (see the 25 May 2026 incident in [Operations](Operations.md)).
|
||||||
|
- **No local action cache** — `act_runner` uses go-git internally, which ignores `~/.gitconfig`. The `url.insteadOf` trick only works for the `git` binary (used by the `checkout` action), so action repos are cloned fresh each run. This is why `tinqs/ci` must stay public.
|
||||||
|
- **`tinqs.com`** — Gitea's `ROOT_URL` is `tinqs.com`. The old `git.tinqs.com` subdomain is retired.
|
||||||
|
|
||||||
|
## Composite actions
|
||||||
|
|
||||||
|
Bash-only composite actions (no Node.js runtime). Resolve via `@v1` (the main branch).
|
||||||
|
|
||||||
|
| Action | What it does |
|
||||||
|
|--------|-------------|
|
||||||
|
| `tinqs/ci/checkout@v1` | Clone a repo from tinqs.com (sparse checkout, depth control, token auth) |
|
||||||
|
| `tinqs/ci/setup-go@v1` | Install Go (skips if pre-baked in the AMI) |
|
||||||
|
| `tinqs/ci/setup-node@v1` | Install Node.js + pnpm (skips if pre-baked) |
|
||||||
|
| `tinqs/ci/setup-aws@v1` | Install AWS CLI + optional ECR login |
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
steps:
|
||||||
|
- uses: tinqs/ci/checkout@v1
|
||||||
|
with:
|
||||||
|
sparse: 'cmd/tstudio'
|
||||||
|
- uses: tinqs/ci/setup-go@v1
|
||||||
|
- uses: tinqs/ci/setup-aws@v1
|
||||||
|
with:
|
||||||
|
ecr-login: 'true'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Dispatcher (Lambda)
|
||||||
|
|
||||||
|
`orchestrator/dispatch/main.go` receives Gitea push webhooks, fetches `.gitea/workflows/*.yml` via the Gitea API, evaluates triggers (branch + path filters), reads each matched workflow's `runs-on` label, and launches a Spot instance with that label. Run state is tracked in DynamoDB.
|
||||||
|
|
||||||
|
Routing by label (`labelToSpot` map in `main.go`):
|
||||||
|
|
||||||
|
| Label | Instance | Use |
|
||||||
|
|-------|----------|-----|
|
||||||
|
| `go` | t3.small | Go builds (tstudio, proxy, docgen) |
|
||||||
|
| `docker` | t3.medium | Docker image builds (platform, bot) |
|
||||||
|
| `deploy` | t3.micro | S3 sync, CloudFront invalidation, SSM template deploy |
|
||||||
|
| `node` | t3.medium | Frontend builds |
|
||||||
|
| `godot` | t3.medium | Game exports (future) |
|
||||||
|
|
||||||
|
`runs-on: host` is skipped by the dispatcher (it's for a standing registered runner, not Spot).
|
||||||
|
|
||||||
|
> **Fixed 2026-06-07:** `deploy`-labelled jobs used to route to a separate executor Lambda (`tinqs-ci-exec`) that was deleted 26 May, so they silently hit a `[DRY RUN] Would invoke executor` no-op and never ran. They now fall through to the normal Spot path like every other label. A second bug — runner names derived from `runID[:12]` collided across same-commit deploys — was also fixed (names now use the full sanitised runID).
|
||||||
|
|
||||||
|
## Runner lifecycle (user-data)
|
||||||
|
|
||||||
|
```
|
||||||
|
boot → git auth config (url.insteadOf with GITEA_TOKEN)
|
||||||
|
→ act_runner register --ephemeral --labels <label>:host
|
||||||
|
→ act_runner daemon (blocks until job completes)
|
||||||
|
→ EXIT trap → shutdown -h now → instance terminates
|
||||||
|
```
|
||||||
|
|
||||||
|
## Runner images
|
||||||
|
|
||||||
|
Dockerfiles in `images/` — lean, purpose-built. Push to ECR with `images/build-all.sh v1`.
|
||||||
|
|
||||||
|
| Image | Contents |
|
||||||
|
|-------|----------|
|
||||||
|
| `base` | Alpine + git + AWS CLI + SSH |
|
||||||
|
| `go` | base + Go |
|
||||||
|
| `node` | base + Node + pnpm |
|
||||||
|
| `docker` | docker:dind + Go + AWS CLI |
|
||||||
|
| `deploy` | base only (lightest) |
|
||||||
|
| `godot` | base + headless Godot |
|
||||||
|
|
||||||
|
> Note: the live Spot runners boot from a **pre-baked AMI** (`RUNNER_AMI`, with Go/Node/Docker/act_runner installed), not these container images. The images exist for purpose-built runner variants; the AMI is the fast path.
|
||||||
@@ -0,0 +1,109 @@
|
|||||||
|
# DevOps Reference
|
||||||
|
|
||||||
|
[← Home](README.md) · [Architecture](Architecture.md) · [Operations](Operations.md) · [Roadmap](Roadmap.md)
|
||||||
|
|
||||||
|
## AWS resources (eu-west-1)
|
||||||
|
|
||||||
|
| Resource | Name/ID | Purpose |
|
||||||
|
|----------|---------|---------|
|
||||||
|
| Lambda | `tinqs-ci-dispatch` | Webhook handler + Spot launcher |
|
||||||
|
| DynamoDB | `tinqs-ci-runs` | Run tracking (repo, run_id, instance_id, status) |
|
||||||
|
| AMI | `tinqs-ci-runner-v2` (ami-00a129385002e4de9) | Pre-baked runner (Go, Node, Docker, act_runner) |
|
||||||
|
| Security Group | sg-030bf74b43d3faac7 | Runner SG (outbound HTTPS) |
|
||||||
|
| Subnet | subnet-04b5aeec9bfc4ec2c | Default VPC subnet |
|
||||||
|
| Instance Profile | `tinqs-ci-runner` → role `tinqs-git-task` | Runner IAM role (S3, ECR, SSM) |
|
||||||
|
| CloudWatch | /aws/lambda/tinqs-ci-dispatch | Dispatcher logs |
|
||||||
|
| API Gateway | `q4ohxovfr8…/webhook` | Receives the per-repo Gitea push webhook |
|
||||||
|
|
||||||
|
### Platform host (NOT CI — context)
|
||||||
|
|
||||||
|
| Resource | Name/ID | Purpose |
|
||||||
|
|----------|---------|---------|
|
||||||
|
| EC2 | `tinqs-prod-gitea` (i-0d085288f467083e0, t3.medium) | Runs tinqs.com as a single `docker` Gitea container |
|
||||||
|
| ALB | `tinqs-git` | Fronts the platform |
|
||||||
|
| ECR | `tinqs-git:latest` | Platform image (built by `build.yml` → CodeBuild) |
|
||||||
|
| RDS | `tinqs-prod` (PostgreSQL) | Platform DB |
|
||||||
|
|
||||||
|
The platform mounts host `/data`; `GITEA_CUSTOM=/data/gitea`, so **custom templates live at `/data/gitea/templates/`**. Template-only changes deploy here via SSM — see [Operations](Operations.md).
|
||||||
|
|
||||||
|
### Retired resources
|
||||||
|
|
||||||
|
| Resource | When / why |
|
||||||
|
|----------|------------|
|
||||||
|
| ECS Cluster `tinqs-git` | Deleted **2026-06-05** — platform moved to the `tinqs-prod-gitea` EC2 box |
|
||||||
|
| EFS `tinqs-git-repos` | Retired in the 2026-06-05 EC2 migration (repos now on instance `/data`) |
|
||||||
|
| Lambda `tinqs-ci-exec` | Deleted **26 May 2026** — never ran a build; deploy jobs go through Spot now |
|
||||||
|
| CloudWatch `/aws/lambda/tinqs-ci-exec`, `/ecs/tinqs-runner` | Log groups for the above / the Fargate era |
|
||||||
|
| Fargate runner service | Scaled to 0 then removed |
|
||||||
|
|
||||||
|
## Webhook flow
|
||||||
|
|
||||||
|
```
|
||||||
|
Gitea (tinqs.com)
|
||||||
|
└─ per-repo webhook on push
|
||||||
|
└─ POST https://<api-gw>/webhook
|
||||||
|
└─ Lambda tinqs-ci-dispatch
|
||||||
|
├─ Fetch .gitea/workflows/*.yml via Gitea API
|
||||||
|
├─ Evaluate triggers (branch + path filters)
|
||||||
|
├─ For each matched workflow:
|
||||||
|
│ ├─ Read runs-on label
|
||||||
|
│ └─ RunInstances (Spot, ephemeral) [host → skipped]
|
||||||
|
└─ Track in DynamoDB
|
||||||
|
```
|
||||||
|
|
||||||
|
## Spot instance lifecycle
|
||||||
|
|
||||||
|
```
|
||||||
|
1. Lambda calls RunInstances (Spot, InstanceInitiatedShutdownBehavior=terminate)
|
||||||
|
2. User-data runs:
|
||||||
|
a. Configure git auth (url.insteadOf with GITEA_TOKEN)
|
||||||
|
b. act_runner register --ephemeral --labels <label>:host
|
||||||
|
c. act_runner daemon (blocks until job completes)
|
||||||
|
d. EXIT trap fires → shutdown -h now → instance terminates
|
||||||
|
3. DynamoDB record: running → completed (or timeout after 30 min cleanup)
|
||||||
|
```
|
||||||
|
|
||||||
|
Offline runners listed in Gitea admin → Actions → Runners are **normal** — they're spent ephemeral registrations, not a fault.
|
||||||
|
|
||||||
|
## Cleanup cron
|
||||||
|
|
||||||
|
The dispatcher Lambda also handles cleanup when invoked with an empty body or `{"action":"cleanup"}`. Triggered by EventBridge every 5 minutes.
|
||||||
|
|
||||||
|
- Scans DynamoDB for runs older than 30 min with `status=running`
|
||||||
|
- Terminates matching EC2 instances
|
||||||
|
- Sweeps for orphan instances (tagged `tinqs-ci`, running > 30 min)
|
||||||
|
|
||||||
|
## Lambda env vars
|
||||||
|
|
||||||
|
Configured in the AWS console, not in code:
|
||||||
|
|
||||||
|
| Var | Purpose |
|
||||||
|
|-----|---------|
|
||||||
|
| `GITEA_URL` | `https://tinqs.com` |
|
||||||
|
| `GITEA_TOKEN` | API token — fetches workflows AND provides runner git auth |
|
||||||
|
| `RUNNER_TOKEN` | act_runner registration token (Gitea admin → Runners) |
|
||||||
|
| `RUNNER_AMI` | Pre-baked AMI ID |
|
||||||
|
| `SUBNET` | VPC subnet for Spot instances |
|
||||||
|
| `SECURITY_GROUP` | SG allowing outbound HTTPS |
|
||||||
|
| `DDB_TABLE` | DynamoDB run-tracking table (`tinqs-ci-runs`) |
|
||||||
|
| `INSTANCE_PROFILE` | IAM instance profile for runners |
|
||||||
|
|
||||||
|
## Runner IAM role (`tinqs-git-task`)
|
||||||
|
|
||||||
|
Inline policies of note:
|
||||||
|
|
||||||
|
- `tinqs-ci-s3` — R/W on `tinqs-cli-releases`, `arikigame-com-website`, `docs.tinqs.com` *(corrected 2026-06-07: was the non-existent `arikigame.com`, which broke the arikigame deploy)*
|
||||||
|
- `tinqs-git-s3` — R/W on `tinqs-git-lfs`, `tinqs-git-preview`
|
||||||
|
- `tinqs-ci-deploy` — ECR push, CloudFront `CreateInvalidation`, (legacy ECS update)
|
||||||
|
- `tinqs-ci-ssm-deploy` — `ec2:DescribeInstances` + `ssm:SendCommand` **scoped to the `tinqs-prod-gitea` instance** (added 2026-06-07 for template deploys)
|
||||||
|
- `ssm-exec` — Session Manager channels · `ec2-self-terminate` — terminate own `tinqs-ci`-tagged instance
|
||||||
|
|
||||||
|
## Cost
|
||||||
|
|
||||||
|
| Component | Estimated monthly cost |
|
||||||
|
|-----------|----------------------|
|
||||||
|
| Spot instances (t3.small, ~10 min/build, ~5 builds/day) | ~$1–2 |
|
||||||
|
| Lambda (< 1000 invocations/month) | ~$0 (free tier) |
|
||||||
|
| DynamoDB (< 1 GB, low RCU/WCU) | ~$0 (free tier) |
|
||||||
|
| CloudWatch logs | ~$0.50 |
|
||||||
|
| **Total CI** | **~$2–3/month** |
|
||||||
@@ -0,0 +1,105 @@
|
|||||||
|
# Operations
|
||||||
|
|
||||||
|
[← Home](README.md) · [Architecture](Architecture.md) · [DevOps Reference](DevOps-Reference.md) · [Roadmap](Roadmap.md)
|
||||||
|
|
||||||
|
## Deploy the dispatcher
|
||||||
|
|
||||||
|
The dispatcher Lambda can't CI itself — deploy manually:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd orchestrator/dispatch
|
||||||
|
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o bootstrap -ldflags "-s -w" .
|
||||||
|
# Windows: powershell -Command "Compress-Archive -Path bootstrap -DestinationPath function.zip -Force"
|
||||||
|
# Mac/Linux: zip -j function.zip bootstrap
|
||||||
|
aws lambda update-function-code --region eu-west-1 \
|
||||||
|
--function-name tinqs-ci-dispatch --zip-file fileb://function.zip
|
||||||
|
# Verify: push a change to cmd/tstudio/ in tinqs/studio and watch the pipeline
|
||||||
|
```
|
||||||
|
|
||||||
|
## Deploy templates to prod (no rebuild)
|
||||||
|
|
||||||
|
Template-only changes don't need a platform rebuild. `tinqs/studio/.gitea/workflows/deploy-templates.yml` (label `deploy`) tars `templates/` → `s3://tinqs-git-lfs/custom-templates.tar.gz`, then over **SSM** tells `tinqs-prod-gitea` to pull + extract into `/data/gitea/templates/` and `docker restart gitea`.
|
||||||
|
|
||||||
|
> Repointed to SSM/EC2 on **2026-06-07**. It previously ran `aws ecs update-service --cluster tinqs-git`, which failed with `ClusterNotFoundException` after the cluster was deleted on 06-05 — that's why the repo Wiki tab and theme CSS never went live. The runner role gained a scoped `ssm:SendCommand` (prod-gitea only).
|
||||||
|
|
||||||
|
Manual one-off (admin creds):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
tar czf /tmp/custom-templates.tar.gz -C templates .
|
||||||
|
aws s3 cp /tmp/custom-templates.tar.gz s3://tinqs-git-lfs/custom-templates.tar.gz --region eu-west-1
|
||||||
|
IID=$(aws ec2 describe-instances --region eu-west-1 \
|
||||||
|
--filters "Name=tag:Name,Values=tinqs-prod-gitea" "Name=instance-state-name,Values=running" \
|
||||||
|
--query "Reservations[0].Instances[0].InstanceId" --output text)
|
||||||
|
aws ssm send-command --region eu-west-1 --instance-ids "$IID" \
|
||||||
|
--document-name AWS-RunShellScript \
|
||||||
|
--parameters 'commands=["aws s3 cp s3://tinqs-git-lfs/custom-templates.tar.gz /tmp/ct.tar.gz --region eu-west-1","tar xzf /tmp/ct.tar.gz -C /data/gitea/templates","docker restart gitea"]'
|
||||||
|
```
|
||||||
|
|
||||||
|
> Note: a template change does **not** bump the platform version string in the footer (that tracks the Go binary build). Unchanged footer ≠ failed deploy.
|
||||||
|
|
||||||
|
## Rotate `GITEA_TOKEN`
|
||||||
|
|
||||||
|
1. Generate a new token in Gitea: Settings → Applications → Generate Token
|
||||||
|
2. `aws lambda update-function-configuration --function-name tinqs-ci-dispatch --environment ...`
|
||||||
|
3. Old token is burned into running instances — they die within 30 min
|
||||||
|
|
||||||
|
## Rotate `RUNNER_TOKEN`
|
||||||
|
|
||||||
|
1. Gitea admin → Actions → Runners → Create new registration token
|
||||||
|
2. Update the Lambda env var
|
||||||
|
3. Running instances keep their existing registration until they die
|
||||||
|
|
||||||
|
## Build a new AMI
|
||||||
|
|
||||||
|
```bash
|
||||||
|
aws ec2 run-instances --image-id ami-00a129385002e4de9 \
|
||||||
|
--instance-type t3.small --key-name <your-key> --region eu-west-1 \
|
||||||
|
--query 'Instances[0].InstanceId'
|
||||||
|
# SSH in, update tools (Go, Node, Docker, act_runner), then:
|
||||||
|
aws ec2 create-image --instance-id <id> --name tinqs-ci-runner-v3
|
||||||
|
aws lambda update-function-configuration --function-name tinqs-ci-dispatch \
|
||||||
|
--environment "Variables={...,RUNNER_AMI=ami-NEW,...}"
|
||||||
|
aws ec2 terminate-instances --instance-id <id>
|
||||||
|
```
|
||||||
|
|
||||||
|
## Add CI to a new repo
|
||||||
|
|
||||||
|
1. Create `.gitea/workflows/<name>.yml` in the repo
|
||||||
|
2. Add a per-repo webhook in Gitea: Settings → Webhooks → Add Webhook
|
||||||
|
- URL: the dispatcher API Gateway URL · Events: Push · Content type: `application/json`
|
||||||
|
3. Push a change matching the workflow trigger
|
||||||
|
|
||||||
|
## Monitoring
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Zombie check (should be 0 except during active builds)
|
||||||
|
aws ec2 describe-instances --region eu-west-1 \
|
||||||
|
--filters "Name=tag:tinqs-ci,Values=true" "Name=instance-state-name,Values=running" \
|
||||||
|
--query 'Reservations[].Instances[].InstanceId'
|
||||||
|
|
||||||
|
# Dispatcher logs (MSYS_NO_PATHCONV=1 on Windows/Git Bash; or use PowerShell)
|
||||||
|
MSYS_NO_PATHCONV=1 aws logs tail '/aws/lambda/tinqs-ci-dispatch' --region eu-west-1
|
||||||
|
|
||||||
|
# Build/job logs
|
||||||
|
curl -s "https://tinqs.com/api/v1/repos/tinqs/studio/actions/jobs/<JOB_ID>/logs" \
|
||||||
|
-H "Authorization: token <gitea-token>"
|
||||||
|
|
||||||
|
# Stale DynamoDB runs
|
||||||
|
aws dynamodb scan --region eu-west-1 --table-name tinqs-ci-runs \
|
||||||
|
--filter-expression "#s = :r" \
|
||||||
|
--expression-attribute-names '{"#s":"status"}' \
|
||||||
|
--expression-attribute-values '{":r":{"S":"running"}}' --query Count
|
||||||
|
```
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
**New composite action:** create `<name>/action.yml` (`using: composite`, `shell: bash`), keep it bash-only, add a `<name>/README.md`, list it in [Architecture](Architecture.md), push to main (resolves via `@v1`).
|
||||||
|
|
||||||
|
**Modify the dispatcher:** edit `orchestrator/dispatch/main.go`, `go build .` to catch errors, deploy manually (above), verify with a push to `tinqs/studio`.
|
||||||
|
|
||||||
|
**New runner label:** add to `labelToSpot` in `main.go`, create `images/<label>/Dockerfile` if needed, build/push (`cd images && ./build-all.sh v1`), deploy the Lambda, add `runs-on: <label>` to the consuming workflow.
|
||||||
|
|
||||||
|
## Incidents
|
||||||
|
|
||||||
|
- **25 May 2026** — 18 zombie runners DDoS-ing Gitea. Root cause: no `--ephemeral` on registration + no git auth after repos went private. Fix: `--ephemeral` + `url.insteadOf` git auth in user-data.
|
||||||
|
- **07 Jun 2026** — all `runs-on: deploy` jobs silently dry-running (dead `tinqs-ci-exec` route) + arikigame IAM bucket mismatch + template deploy pointing at the deleted ECS cluster. All fixed; see [Architecture](Architecture.md) and the template-deploy note above.
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
# tinqs/ci — CI Toolchain
|
||||||
|
|
||||||
|
> **📖 This is the team wiki.** Standard: the in-repo **`wiki/`** folder is the home for team/architecture docs in every repo (distinct from `.agents/` = agent operating context, and `docs/` = public product docs at tinqs.com/docs). Plain markdown, rendered by Gitea — no separate wiki repo, no build. Cross-link with `[Title](Page-Name.md)`.
|
||||||
|
|
||||||
|
**The CI system for Tinqs Studio: composite Gitea Actions + a Lambda dispatcher that launches ephemeral EC2 Spot runners, one per job.** Status baked in — ✅ live · 🔨 built · 📋 planned. Last updated 2026-06-07.
|
||||||
|
|
||||||
|
> ⚠️ **This repo must stay public.** `act_runner` (go-git) clones action repos without auth; every other tinqs repo is private. If `tinqs/ci` goes private, every workflow that does `uses: tinqs/ci/...` breaks.
|
||||||
|
|
||||||
|
```
|
||||||
|
Push → Gitea webhook → Lambda (tinqs-ci-dispatch) → EC2 Spot → act_runner → job → self-terminate
|
||||||
|
```
|
||||||
|
|
||||||
|
## Pages
|
||||||
|
|
||||||
|
- [Architecture](Architecture.md) — design decisions, the dispatcher, runner labels & images, runner lifecycle
|
||||||
|
- [DevOps Reference](DevOps-Reference.md) — AWS resources, webhook flow, Spot lifecycle, cleanup cron, cost, Lambda env vars
|
||||||
|
- [Operations](Operations.md) — deploy the dispatcher, rotate tokens, build an AMI, add CI to a repo, monitoring, incidents
|
||||||
|
- [Roadmap](Roadmap.md) — what's done, what's next
|
||||||
|
|
||||||
|
## Key facts
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| **Runners** | Ephemeral EC2 Spot, one per job, self-terminate (`--ephemeral` + `shutdown -h now`) |
|
||||||
|
| **Dispatcher** | `tinqs-ci-dispatch` Lambda (`orchestrator/dispatch/main.go`), Go, `provided.al2023` |
|
||||||
|
| **Routing** | Workflow `runs-on` label → Spot instance type (see [Architecture](Architecture.md)) |
|
||||||
|
| **Auth** | `GITEA_TOKEN` injected into runner user-data via `git config url.insteadOf` |
|
||||||
|
| **Region** | eu-west-1 |
|
||||||
|
| **Cost** | ~$2–3/month |
|
||||||
|
|
||||||
|
> **2026-06-05 — platform moved off ECS.** tinqs.com now runs as a single `docker` container on the standalone EC2 box **`tinqs-prod-gitea`** (behind ALB `tinqs-git`, image from ECR `tinqs-git:latest`, state on RDS `tinqs-prod` + local `/data`). The old ECS cluster `tinqs-git` and EFS `tinqs-git-repos` were retired. Any workflow that still referenced ECS (e.g. template deploy) was repointed at the EC2 host via SSM — see [Operations](Operations.md).
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# Roadmap
|
||||||
|
|
||||||
|
[← Home](README.md) · [Architecture](Architecture.md) · [DevOps Reference](DevOps-Reference.md) · [Operations](Operations.md)
|
||||||
|
|
||||||
|
## Done
|
||||||
|
|
||||||
|
- [x] Composite actions: `checkout`, `setup-go`, `setup-node`, `setup-aws`
|
||||||
|
- [x] Lambda dispatcher with Spot instance routing by `runs-on` label
|
||||||
|
- [x] Ephemeral runners (one job, self-terminate)
|
||||||
|
- [x] Git auth for private repos (`url.insteadOf`)
|
||||||
|
- [x] DynamoDB run tracking + cleanup cron
|
||||||
|
- [x] Runner image Dockerfiles: base, go, node, docker, deploy, godot
|
||||||
|
- [x] Zombie runner incident resolved (25 May 2026)
|
||||||
|
- [x] `deploy`-label jobs routed through Spot (was dead-Lambda dry-run) (07 Jun 2026)
|
||||||
|
- [x] Unique Spot runner names per dispatch (07 Jun 2026)
|
||||||
|
- [x] Template deploy repointed off deleted ECS → EC2 via SSM (07 Jun 2026)
|
||||||
|
|
||||||
|
## Next
|
||||||
|
|
||||||
|
| Priority | Task | Impact |
|
||||||
|
|----------|------|--------|
|
||||||
|
| P1 | Pre-warm Go module + build cache in the AMI | −30s build time |
|
||||||
|
| P1 | Automate AMI build (Packer or script) | Repeatable, no manual SSH |
|
||||||
|
| P2 | Internal DNS for git clones | Faster than public HTTPS |
|
||||||
|
| P2 | CloudWatch agent on the runner AMI | Persistent logs after instance death |
|
||||||
|
| P3 | `tinqs/ci/deploy-s3` action | S3 sync + CloudFront invalidation wrapper |
|
||||||
|
| P3 | `tinqs/ci/deploy-ssm` action | Reusable SSM-to-prod deploy (generalise the template-deploy step) |
|
||||||
|
| P3 | `tinqs/ci/notify` action | Post build status to GChat |
|
||||||
|
|
||||||
|
## Watch / cleanup
|
||||||
|
|
||||||
|
- **Repo size** — `tinqs/studio` now commits the arikigame site assets (~75 MB) as regular files because the CI `checkout` does no `git lfs pull`. If this grows, add `git lfs pull` to the checkout action, then LFS-track `web/arikigame/public/img/**`.
|
||||||
|
- **DEVOPS doc drift** — keep this wiki current when AWS topology changes (the ECS→EC2 move went unnoticed in docs for two days and broke deploys).
|
||||||
Reference in New Issue
Block a user