Compare commits
25 Commits
v0.259.1
...
nektos/mas
| Author | SHA1 | Date | |
|---|---|---|---|
|
e3da912e2b | ||
|
|
361b7e9f1a | ||
|
|
f825e42ce2 | ||
|
|
d9a19c8b02 | ||
|
|
3949d74af5 | ||
|
|
b9382a2c4e | ||
|
|
f56dd65ff6 | ||
|
|
069720abff | ||
|
|
8c83d57212 | ||
|
|
119ceb81d9 | ||
|
|
352ad41ad2 | ||
|
|
75e4ad93f4 | ||
|
|
934b13a7a1 | ||
|
|
d3c8664d3d | ||
|
|
c79f59f802 | ||
|
|
68597d68f6 | ||
|
|
e1cd7c915f | ||
|
|
ac6a465e27 | ||
|
|
9e6ce121bc | ||
|
|
5601fb0e13 | ||
|
|
852959e1e1 | ||
|
|
27eb79bb5a | ||
|
|
8277584f00 | ||
|
|
f2e65e1d40 | ||
|
|
6e80373eb6 |
@@ -26,7 +26,7 @@
|
||||
|
||||
## Images based on [`actions/virtual-environments`][gh/actions/virtual-environments]
|
||||
|
||||
**Note: `nektos/act-environments-ubuntu` have been last updated in February, 2020. It's recommended to update the image manually after `docker pull` if you decide to to use it.**
|
||||
**Note: `nektos/act-environments-ubuntu` have been last updated in February, 2020. It's recommended to update the image manually after `docker pull` if you decide to use it.**
|
||||
|
||||
| Image | Size | GitHub Repository |
|
||||
| --------------------------------------------------------------------------------- | -------------------------------------------------------------------------- | ------------------------------------------------------- |
|
||||
|
||||
454
README.md
454
README.md
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Overview [](https://github.com/nektos/act/actions) [](https://gitter.im/nektos/act?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [](https://goreportcard.com/report/github.com/nektos/act) [](https://github.com/jonico/awesome-runners)
|
||||
|
||||
@@ -15,462 +15,12 @@ When you run `act` it reads in your GitHub Actions from `.github/workflows/` and
|
||||
|
||||
Let's see it in action with a [sample repo](https://github.com/cplee/github-actions-demo)!
|
||||
|
||||
|
||||
|
||||
|
||||
# Act User Guide
|
||||
|
||||
Please look at the [act user guide](https://nektosact.com) for more documentation.
|
||||
|
||||
# Installation
|
||||
|
||||
## Necessary prerequisites for running `act`
|
||||
|
||||
`act` depends on `docker` to run workflows.
|
||||
|
||||
If you are using macOS, please be sure to follow the steps outlined in [Docker Docs for how to install Docker Desktop for Mac](https://docs.docker.com/docker-for-mac/install/).
|
||||
|
||||
If you are using Windows, please follow steps for [installing Docker Desktop on Windows](https://docs.docker.com/docker-for-windows/install/).
|
||||
|
||||
If you are using Linux, you will need to [install Docker Engine](https://docs.docker.com/engine/install/).
|
||||
|
||||
`act` is currently not supported with `podman` or other container backends (it might work, but it's not guaranteed). Please see [#303](https://github.com/nektos/act/issues/303) for updates.
|
||||
|
||||
## Installation through package managers
|
||||
|
||||
### [Homebrew](https://brew.sh/) (Linux/macOS)
|
||||
|
||||
[](https://github.com/Homebrew/homebrew-core/blob/master/Formula/act.rb)
|
||||
|
||||
```shell
|
||||
brew install act
|
||||
```
|
||||
|
||||
or if you want to install version based on latest commit, you can run below (it requires compiler to be installed but Homebrew will suggest you how to install it, if you don't have it):
|
||||
|
||||
```shell
|
||||
brew install act --HEAD
|
||||
```
|
||||
|
||||
### [MacPorts](https://www.macports.org) (macOS)
|
||||
|
||||
[](https://repology.org/project/act-run-github-actions/versions)
|
||||
|
||||
```shell
|
||||
sudo port install act
|
||||
```
|
||||
|
||||
### [Chocolatey](https://chocolatey.org/) (Windows)
|
||||
|
||||
[](https://community.chocolatey.org/packages/act-cli)
|
||||
|
||||
```shell
|
||||
choco install act-cli
|
||||
```
|
||||
|
||||
### [Scoop](https://scoop.sh/) (Windows)
|
||||
|
||||
[](https://github.com/ScoopInstaller/Main/blob/master/bucket/act.json)
|
||||
|
||||
```shell
|
||||
scoop install act
|
||||
```
|
||||
|
||||
### [Winget](https://learn.microsoft.com/en-us/windows/package-manager/) (Windows)
|
||||
|
||||
[](https://repology.org/project/act-run-github-actions/versions)
|
||||
|
||||
```shell
|
||||
winget install nektos.act
|
||||
```
|
||||
|
||||
### [AUR](https://aur.archlinux.org/packages/act/) (Linux)
|
||||
|
||||
[](https://aur.archlinux.org/packages/act/)
|
||||
|
||||
```shell
|
||||
yay -Syu act
|
||||
```
|
||||
|
||||
### [COPR](https://copr.fedorainfracloud.org/coprs/rubemlrm/act-cli/) (Linux)
|
||||
|
||||
```shell
|
||||
dnf copr enable rubemlrm/act-cli
|
||||
dnf install act-cli
|
||||
```
|
||||
|
||||
### [Nix](https://nixos.org) (Linux/macOS)
|
||||
|
||||
[Nix recipe](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/tools/misc/act/default.nix)
|
||||
|
||||
Global install:
|
||||
|
||||
```sh
|
||||
nix-env -iA nixpkgs.act
|
||||
```
|
||||
|
||||
or through `nix-shell`:
|
||||
|
||||
```sh
|
||||
nix-shell -p act
|
||||
```
|
||||
|
||||
Using the latest [Nix command](https://nixos.wiki/wiki/Nix_command), you can run directly :
|
||||
|
||||
```sh
|
||||
nix run nixpkgs#act
|
||||
```
|
||||
|
||||
## Installation as GitHub CLI extension
|
||||
|
||||
Act can be installed as a [GitHub CLI](https://cli.github.com/) extension:
|
||||
|
||||
```sh
|
||||
gh extension install https://github.com/nektos/gh-act
|
||||
```
|
||||
|
||||
## Other install options
|
||||
|
||||
### Bash script
|
||||
|
||||
Run this command in your terminal:
|
||||
|
||||
```shell
|
||||
curl -s https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash
|
||||
```
|
||||
|
||||
### Manual download
|
||||
|
||||
Download the [latest release](https://github.com/nektos/act/releases/latest) and add the path to your binary into your PATH.
|
||||
|
||||
# Example commands
|
||||
|
||||
```sh
|
||||
# Command structure:
|
||||
act [<event>] [options]
|
||||
If no event name passed, will default to "on: push"
|
||||
If actions handles only one event it will be used as default instead of "on: push"
|
||||
|
||||
# List all actions for all events:
|
||||
act -l
|
||||
|
||||
# List the actions for a specific event:
|
||||
act workflow_dispatch -l
|
||||
|
||||
# List the actions for a specific job:
|
||||
act -j test -l
|
||||
|
||||
# Run the default (`push`) event:
|
||||
act
|
||||
|
||||
# Run a specific event:
|
||||
act pull_request
|
||||
|
||||
# Run a specific job:
|
||||
act -j test
|
||||
|
||||
# Collect artifacts to the /tmp/artifacts folder:
|
||||
act --artifact-server-path /tmp/artifacts
|
||||
|
||||
# Run a job in a specific workflow (useful if you have duplicate job names)
|
||||
act -j lint -W .github/workflows/checks.yml
|
||||
|
||||
# Run in dry-run mode:
|
||||
act -n
|
||||
|
||||
# Enable verbose-logging (can be used with any of the above commands)
|
||||
act -v
|
||||
```
|
||||
|
||||
## First `act` run
|
||||
|
||||
When running `act` for the first time, it will ask you to choose image to be used as default.
|
||||
It will save that information to `~/.actrc`, please refer to [Configuration](#configuration) for more information about `.actrc` and to [Runners](#runners) for information about used/available Docker images.
|
||||
|
||||
## `GITHUB_TOKEN`
|
||||
|
||||
GitHub [automatically provides](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#about-the-github_token-secret) a `GITHUB_TOKEN` secret when running workflows inside GitHub.
|
||||
|
||||
If your workflow depends on this token, you need to create a [personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) and pass it to `act` as a secret:
|
||||
|
||||
```bash
|
||||
act -s GITHUB_TOKEN=[insert token or leave blank and omit equals for secure input]
|
||||
```
|
||||
|
||||
If [GitHub CLI](https://cli.github.com/) is installed, the [`gh auth token`](https://cli.github.com/manual/gh_auth_token) command can be used to automatically pass the token to act
|
||||
|
||||
```bash
|
||||
act -s GITHUB_TOKEN="$(gh auth token)"
|
||||
```
|
||||
|
||||
**WARNING**: `GITHUB_TOKEN` will be logged in shell history if not inserted through secure input or (depending on your shell config) the command is prefixed with a whitespace.
|
||||
|
||||
# Known Issues
|
||||
|
||||
## Services
|
||||
|
||||
Services are not currently supported but are being worked on. See: [#173](https://github.com/nektos/act/issues/173)
|
||||
|
||||
## `MODULE_NOT_FOUND`
|
||||
|
||||
A `MODULE_NOT_FOUND` during `docker cp` command [#228](https://github.com/nektos/act/issues/228) can happen if you are relying on local changes that have not been pushed. This can get triggered if the action is using a path, like:
|
||||
|
||||
```yaml
|
||||
- name: test action locally
|
||||
uses: ./
|
||||
```
|
||||
|
||||
In this case, you _must_ use `actions/checkout@v2` with a path that _has the same name as your repository_. If your repository is called _my-action_, then your checkout step would look like:
|
||||
|
||||
```yaml
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v2
|
||||
with:
|
||||
path: "my-action"
|
||||
```
|
||||
|
||||
If the `path:` value doesn't match the name of the repository, a `MODULE_NOT_FOUND` will be thrown.
|
||||
|
||||
## `docker context` support
|
||||
|
||||
The current `docker context` isn't respected ([#583](https://github.com/nektos/act/issues/583)).
|
||||
|
||||
You can work around this by setting `DOCKER_HOST` before running `act`, with e.g:
|
||||
|
||||
```bash
|
||||
export DOCKER_HOST=$(docker context inspect --format '{{.Endpoints.docker.Host}}')
|
||||
```
|
||||
|
||||
# Runners
|
||||
|
||||
GitHub Actions offers managed [virtual environments](https://help.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners) for running workflows. In order for `act` to run your workflows locally, it must run a container for the runner defined in your workflow file. Here are the images that `act` uses for each runner type and size:
|
||||
|
||||
| GitHub Runner | Micro Docker Image | Medium Docker Image | Large Docker Image |
|
||||
| --------------- | -------------------------------- | ------------------------------------------------- | -------------------------------------------------- |
|
||||
| `ubuntu-latest` | [`node:16-buster-slim`][micro] | [`catthehacker/ubuntu:act-latest`][docker_images] | [`catthehacker/ubuntu:full-latest`][docker_images] |
|
||||
| `ubuntu-22.04` | [`node:16-bullseye-slim`][micro] | [`catthehacker/ubuntu:act-22.04`][docker_images] | `unavailable` |
|
||||
| `ubuntu-20.04` | [`node:16-buster-slim`][micro] | [`catthehacker/ubuntu:act-20.04`][docker_images] | [`catthehacker/ubuntu:full-20.04`][docker_images] |
|
||||
| `ubuntu-18.04` | [`node:16-buster-slim`][micro] | [`catthehacker/ubuntu:act-18.04`][docker_images] | [`catthehacker/ubuntu:full-18.04`][docker_images] |
|
||||
|
||||
[micro]: https://hub.docker.com/_/buildpack-deps
|
||||
[docker_images]: https://github.com/catthehacker/docker_images
|
||||
|
||||
Windows and macOS based platforms are currently **unsupported and won't work** (see issue [#97](https://github.com/nektos/act/issues/97))
|
||||
|
||||
## Please see [IMAGES.md](./IMAGES.md) for more information about the Docker images that can be used with `act`
|
||||
|
||||
## Default runners are intentionally incomplete
|
||||
|
||||
These default images do **not** contain **all** the tools that GitHub Actions offers by default in their runners.
|
||||
Many things can work improperly or not at all while running those image.
|
||||
Additionally, some software might still not work even if installed properly, since GitHub Actions are running in fully virtualized machines while `act` is using Docker containers (e.g. Docker does not support running `systemd`).
|
||||
In case of any problems [please create issue](https://github.com/nektos/act/issues/new/choose) in respective repository (issues with `act` in this repository, issues with `nektos/act-environments-ubuntu:18.04` in [`nektos/act-environments`](https://github.com/nektos/act-environments) and issues with any image from user `catthehacker` in [`catthehacker/docker_images`](https://github.com/catthehacker/docker_images))
|
||||
|
||||
## Alternative runner images
|
||||
|
||||
If you need an environment that works just like the corresponding GitHub runner then consider using an image provided by [nektos/act-environments](https://github.com/nektos/act-environments):
|
||||
|
||||
- [`nektos/act-environments-ubuntu:18.04`](https://hub.docker.com/r/nektos/act-environments-ubuntu/tags) - built from the Packer file GitHub uses in [actions/virtual-environments](https://github.com/actions/runner).
|
||||
|
||||
:warning: :elephant: `*** WARNING - this image is >18GB 😱***`
|
||||
|
||||
- [`catthehacker/ubuntu:full-*`](https://github.com/catthehacker/docker_images/pkgs/container/ubuntu) - built from Packer template provided by GitHub, see [catthehacker/virtual-environments-fork](https://github.com/catthehacker/virtual-environments-fork) or [catthehacker/docker_images](https://github.com/catthehacker/docker_images) for more information
|
||||
|
||||
## Using local runner images
|
||||
|
||||
The `--pull` flag is set to true by default due to a breaking on older default docker images. This would pull the docker image everytime act is executed.
|
||||
|
||||
Set `--pull` to false if a local docker image is needed
|
||||
```sh
|
||||
act --pull=false
|
||||
```
|
||||
|
||||
## Use an alternative runner image
|
||||
|
||||
To use a different image for the runner, use the `-P` option.
|
||||
|
||||
```sh
|
||||
act -P <platform>=<docker-image>
|
||||
```
|
||||
|
||||
If your workflow uses `ubuntu-18.04`, consider below line as an example for changing Docker image used to run that workflow:
|
||||
|
||||
```sh
|
||||
act -P ubuntu-18.04=nektos/act-environments-ubuntu:18.04
|
||||
```
|
||||
|
||||
If you use multiple platforms in your workflow, you have to specify them to change which image is used.
|
||||
For example, if your workflow uses `ubuntu-18.04`, `ubuntu-16.04` and `ubuntu-latest`, specify all platforms like below
|
||||
|
||||
```sh
|
||||
act -P ubuntu-18.04=nektos/act-environments-ubuntu:18.04 -P ubuntu-latest=ubuntu:latest -P ubuntu-16.04=node:16-buster-slim
|
||||
```
|
||||
|
||||
# Secrets
|
||||
|
||||
To run `act` with secrets, you can enter them interactively, supply them as environment variables or load them from a file. The following options are available for providing secrets:
|
||||
|
||||
- `act -s MY_SECRET=somevalue` - use `somevalue` as the value for `MY_SECRET`.
|
||||
- `act -s MY_SECRET` - check for an environment variable named `MY_SECRET` and use it if it exists. If the environment variable is not defined, prompt the user for a value.
|
||||
- `act --secret-file my.secrets` - load secrets values from `my.secrets` file.
|
||||
- secrets file format is the same as `.env` format
|
||||
|
||||
# Vars
|
||||
|
||||
To run `act` with repository variables that are acessible inside the workflow via ${{ vars.VARIABLE }}, you can enter them interactively or load them from a file. The following options are available for providing github repository variables:
|
||||
|
||||
- `act --var VARIABLE=somevalue` - use `somevalue` as the value for `VARIABLE`.
|
||||
- `act --var-file my.variables` - load variables values from `my.variables` file.
|
||||
- variables file format is the same as `.env` format
|
||||
|
||||
# Configuration
|
||||
|
||||
You can provide default configuration flags to `act` by either creating a `./.actrc` or a `~/.actrc` file. Any flags in the files will be applied before any flags provided directly on the command line. For example, a file like below will always use the `nektos/act-environments-ubuntu:18.04` image for the `ubuntu-latest` runner:
|
||||
|
||||
```sh
|
||||
# sample .actrc file
|
||||
-P ubuntu-latest=nektos/act-environments-ubuntu:18.04
|
||||
```
|
||||
|
||||
Additionally, act supports loading environment variables from an `.env` file. The default is to look in the working directory for the file but can be overridden by:
|
||||
|
||||
```sh
|
||||
act --env-file my.env
|
||||
```
|
||||
|
||||
`.env`:
|
||||
|
||||
```env
|
||||
MY_ENV_VAR=MY_ENV_VAR_VALUE
|
||||
MY_2ND_ENV_VAR="my 2nd env var value"
|
||||
```
|
||||
|
||||
# Skipping jobs
|
||||
|
||||
You cannot use the `env` context in job level if conditions, but you can add a custom event property to the `github` context. You can use this method also on step level if conditions.
|
||||
|
||||
```yml
|
||||
on: push
|
||||
jobs:
|
||||
deploy:
|
||||
if: ${{ !github.event.act }} # skip during local actions testing
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- run: exit 0
|
||||
```
|
||||
|
||||
And use this `event.json` file with act otherwise the Job will run:
|
||||
|
||||
```json
|
||||
{
|
||||
"act": true
|
||||
}
|
||||
```
|
||||
|
||||
Run act like
|
||||
|
||||
```sh
|
||||
act -e event.json
|
||||
```
|
||||
|
||||
_Hint: you can add / append `-e event.json` as a line into `./.actrc`_
|
||||
|
||||
# Skipping steps
|
||||
|
||||
Act adds a special environment variable `ACT` that can be used to skip a step that you
|
||||
don't want to run locally. E.g. a step that posts a Slack message or bumps a version number.
|
||||
**You cannot use this method in job level if conditions, see [Skipping jobs](#skipping-jobs)**
|
||||
|
||||
```yml
|
||||
- name: Some step
|
||||
if: ${{ !env.ACT }}
|
||||
run: |
|
||||
...
|
||||
```
|
||||
|
||||
# Events
|
||||
|
||||
Every [GitHub event](https://developer.github.com/v3/activity/events/types) is accompanied by a payload. You can provide these events in JSON format with the `--eventpath` to simulate specific GitHub events kicking off an action. For example:
|
||||
|
||||
```json
|
||||
{
|
||||
"pull_request": {
|
||||
"head": {
|
||||
"ref": "sample-head-ref"
|
||||
},
|
||||
"base": {
|
||||
"ref": "sample-base-ref"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```sh
|
||||
act pull_request -e pull-request.json
|
||||
```
|
||||
|
||||
Act will properly provide `github.head_ref` and `github.base_ref` to the action as expected.
|
||||
|
||||
# Pass Inputs to Manually Triggered Workflows
|
||||
|
||||
Example workflow file
|
||||
|
||||
```yaml
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
NAME:
|
||||
description: "A random input name for the workflow"
|
||||
type: string
|
||||
SOME_VALUE:
|
||||
description: "Some other input to pass"
|
||||
type: string
|
||||
|
||||
jobs:
|
||||
test:
|
||||
name: Test
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Test with inputs
|
||||
run: |
|
||||
echo "Hello ${{ github.event.inputs.NAME }} and ${{ github.event.inputs.SOME_VALUE }}!"
|
||||
```
|
||||
|
||||
## via input or input-file flag
|
||||
|
||||
- `act --input NAME=somevalue` - use `somevalue` as the value for `NAME` input.
|
||||
- `act --input-file my.input` - load input values from `my.input` file.
|
||||
- input file format is the same as `.env` format
|
||||
|
||||
## via JSON
|
||||
|
||||
Example JSON payload file conveniently named `payload.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"inputs": {
|
||||
"NAME": "Manual Workflow",
|
||||
"SOME_VALUE": "ABC"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Command for triggering the workflow
|
||||
|
||||
```sh
|
||||
act workflow_dispatch -e payload.json
|
||||
```
|
||||
|
||||
# GitHub Enterprise
|
||||
|
||||
Act supports using and authenticating against private GitHub Enterprise servers.
|
||||
To use your custom GHE server, set the CLI flag `--github-instance` to your hostname (e.g. `github.company.com`).
|
||||
|
||||
Please note that if your GHE server requires authentication, we will use the secret provided via `GITHUB_TOKEN`.
|
||||
|
||||
Please also see the [official documentation for GitHub actions on GHE](https://docs.github.com/en/enterprise-server@3.0/admin/github-actions/about-using-actions-in-your-enterprise) for more information on how to use actions.
|
||||
|
||||
# Support
|
||||
|
||||
Need help? Ask on [Gitter](https://gitter.im/nektos/act)!
|
||||
|
||||
@@ -59,6 +59,7 @@ type Input struct {
|
||||
logPrefixJobID bool
|
||||
networkName string
|
||||
useNewActionCache bool
|
||||
localRepository []string
|
||||
}
|
||||
|
||||
func (i *Input) resolve(path string) string {
|
||||
|
||||
98
cmd/root.go
98
cmd/root.go
@@ -32,7 +32,7 @@ import (
|
||||
// Execute is the entry point to running the CLI
|
||||
func Execute(ctx context.Context, version string) {
|
||||
input := new(Input)
|
||||
var rootCmd = &cobra.Command{
|
||||
rootCmd := &cobra.Command{
|
||||
Use: "act [event name to run] [flags]\n\nIf no event name passed, will default to \"on: push\"\nIf actions handles only one event it will be used as default instead of \"on: push\"",
|
||||
Short: "Run GitHub actions locally by specifying the event name (e.g. `push`) or an action name directly.",
|
||||
Args: cobra.MaximumNArgs(1),
|
||||
@@ -100,6 +100,7 @@ func Execute(ctx context.Context, version string) {
|
||||
rootCmd.PersistentFlags().BoolVarP(&input.actionOfflineMode, "action-offline-mode", "", false, "If action contents exists, it will not be fetch and pull again. If turn on this,will turn off force pull")
|
||||
rootCmd.PersistentFlags().StringVarP(&input.networkName, "network", "", "host", "Sets a docker network name. Defaults to host.")
|
||||
rootCmd.PersistentFlags().BoolVarP(&input.useNewActionCache, "use-new-action-cache", "", false, "Enable using the new Action Cache for storing Actions locally")
|
||||
rootCmd.PersistentFlags().StringArrayVarP(&input.localRepository, "local-repository", "", []string{}, "Replaces the specified repository and ref with a local folder (e.g. https://github.com/test/test@v0=/home/act/test or test/test@v0=/home/act/test, the latter matches any hosts or protocols)")
|
||||
rootCmd.SetArgs(args())
|
||||
|
||||
if err := rootCmd.Execute(); err != nil {
|
||||
@@ -125,34 +126,6 @@ func configLocations() []string {
|
||||
return []string{specPath, homePath, invocationPath}
|
||||
}
|
||||
|
||||
var commonSocketPaths = []string{
|
||||
"/var/run/docker.sock",
|
||||
"/run/podman/podman.sock",
|
||||
"$HOME/.colima/docker.sock",
|
||||
"$XDG_RUNTIME_DIR/docker.sock",
|
||||
"$XDG_RUNTIME_DIR/podman/podman.sock",
|
||||
`\\.\pipe\docker_engine`,
|
||||
"$HOME/.docker/run/docker.sock",
|
||||
}
|
||||
|
||||
// returns socket path or false if not found any
|
||||
func socketLocation() (string, bool) {
|
||||
if dockerHost, exists := os.LookupEnv("DOCKER_HOST"); exists {
|
||||
return dockerHost, true
|
||||
}
|
||||
|
||||
for _, p := range commonSocketPaths {
|
||||
if _, err := os.Lstat(os.ExpandEnv(p)); err == nil {
|
||||
if strings.HasPrefix(p, `\\.\`) {
|
||||
return "npipe://" + filepath.ToSlash(os.ExpandEnv(p)), true
|
||||
}
|
||||
return "unix://" + filepath.ToSlash(os.ExpandEnv(p)), true
|
||||
}
|
||||
}
|
||||
|
||||
return "", false
|
||||
}
|
||||
|
||||
func args() []string {
|
||||
actrc := configLocations()
|
||||
|
||||
@@ -185,7 +158,7 @@ func bugReport(ctx context.Context, version string) error {
|
||||
|
||||
report += sprintf("Docker host:", dockerHost)
|
||||
report += fmt.Sprintln("Sockets found:")
|
||||
for _, p := range commonSocketPaths {
|
||||
for _, p := range container.CommonSocketLocations {
|
||||
if _, err := os.Lstat(os.ExpandEnv(p)); err != nil {
|
||||
continue
|
||||
} else if _, err := os.Stat(os.ExpandEnv(p)); err != nil {
|
||||
@@ -356,18 +329,6 @@ func parseMatrix(matrix []string) map[string]map[string]bool {
|
||||
return matrixes
|
||||
}
|
||||
|
||||
func isDockerHostURI(daemonPath string) bool {
|
||||
if protoIndex := strings.Index(daemonPath, "://"); protoIndex != -1 {
|
||||
scheme := daemonPath[:protoIndex]
|
||||
if strings.IndexFunc(scheme, func(r rune) bool {
|
||||
return (r < 'a' || r > 'z') && (r < 'A' || r > 'Z')
|
||||
}) == -1 {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
//nolint:gocyclo
|
||||
func newRunCommand(ctx context.Context, input *Input) func(*cobra.Command, []string) error {
|
||||
return func(cmd *cobra.Command, args []string) error {
|
||||
@@ -378,27 +339,12 @@ func newRunCommand(ctx context.Context, input *Input) func(*cobra.Command, []str
|
||||
if ok, _ := cmd.Flags().GetBool("bug-report"); ok {
|
||||
return bugReport(ctx, cmd.Version)
|
||||
}
|
||||
|
||||
// Prefer DOCKER_HOST, don't override it
|
||||
socketPath, hasDockerHost := os.LookupEnv("DOCKER_HOST")
|
||||
if !hasDockerHost {
|
||||
// a - in containerDaemonSocket means don't mount, preserve this value
|
||||
// otherwise if input.containerDaemonSocket is a filepath don't use it as socketPath
|
||||
skipMount := input.containerDaemonSocket == "-" || !isDockerHostURI(input.containerDaemonSocket)
|
||||
if input.containerDaemonSocket != "" && !skipMount {
|
||||
socketPath = input.containerDaemonSocket
|
||||
} else {
|
||||
socket, found := socketLocation()
|
||||
if !found {
|
||||
log.Errorln("daemon Docker Engine socket not found and containerDaemonSocket option was not set")
|
||||
} else {
|
||||
socketPath = socket
|
||||
}
|
||||
if !skipMount {
|
||||
input.containerDaemonSocket = socketPath
|
||||
}
|
||||
}
|
||||
os.Setenv("DOCKER_HOST", socketPath)
|
||||
if ret, err := container.GetSocketAndHost(input.containerDaemonSocket); err != nil {
|
||||
log.Warnf("Couldn't get a valid docker connection: %+v", err)
|
||||
} else {
|
||||
os.Setenv("DOCKER_HOST", ret.Host)
|
||||
input.containerDaemonSocket = ret.Socket
|
||||
log.Infof("Using docker host '%s', and daemon socket '%s'", ret.Host, ret.Socket)
|
||||
}
|
||||
|
||||
if runtime.GOOS == "darwin" && runtime.GOARCH == "arm64" && input.containerArchitecture == "" {
|
||||
@@ -616,9 +562,29 @@ func newRunCommand(ctx context.Context, input *Input) func(*cobra.Command, []str
|
||||
Matrix: matrixes,
|
||||
ContainerNetworkMode: docker_container.NetworkMode(input.networkName),
|
||||
}
|
||||
if input.useNewActionCache {
|
||||
config.ActionCache = &runner.GoGitActionCache{
|
||||
Path: config.ActionCacheDir,
|
||||
if input.useNewActionCache || len(input.localRepository) > 0 {
|
||||
if input.actionOfflineMode {
|
||||
config.ActionCache = &runner.GoGitActionCacheOfflineMode{
|
||||
Parent: runner.GoGitActionCache{
|
||||
Path: config.ActionCacheDir,
|
||||
},
|
||||
}
|
||||
} else {
|
||||
config.ActionCache = &runner.GoGitActionCache{
|
||||
Path: config.ActionCacheDir,
|
||||
}
|
||||
}
|
||||
if len(input.localRepository) > 0 {
|
||||
localRepositories := map[string]string{}
|
||||
for _, l := range input.localRepository {
|
||||
k, v, _ := strings.Cut(l, "=")
|
||||
localRepositories[k] = v
|
||||
}
|
||||
config.ActionCache = &runner.LocalRepositoryCache{
|
||||
Parent: config.ActionCache,
|
||||
LocalRepositories: localRepositories,
|
||||
CacheDirCache: map[string]string{},
|
||||
}
|
||||
}
|
||||
}
|
||||
r, err := runner.New(config)
|
||||
|
||||
34
go.mod
34
go.mod
@@ -10,10 +10,10 @@ require (
|
||||
github.com/creack/pty v1.1.21
|
||||
github.com/docker/cli v24.0.7+incompatible
|
||||
github.com/docker/distribution v2.8.3+incompatible
|
||||
github.com/docker/docker v24.0.7+incompatible // 24.0 branch
|
||||
github.com/docker/docker v24.0.9+incompatible // 24.0 branch
|
||||
github.com/docker/go-connections v0.4.0
|
||||
github.com/go-git/go-billy/v5 v5.5.0
|
||||
github.com/go-git/go-git/v5 v5.11.0
|
||||
github.com/go-git/go-git/v5 v5.12.0
|
||||
github.com/imdario/mergo v0.3.16
|
||||
github.com/joho/godotenv v1.5.1
|
||||
github.com/julienschmidt/httprouter v1.3.0
|
||||
@@ -21,18 +21,18 @@ require (
|
||||
github.com/mattn/go-isatty v0.0.20
|
||||
github.com/moby/buildkit v0.12.5
|
||||
github.com/moby/patternmatcher v0.6.0
|
||||
github.com/opencontainers/image-spec v1.1.0-rc3
|
||||
github.com/opencontainers/image-spec v1.1.0
|
||||
github.com/opencontainers/selinux v1.11.0
|
||||
github.com/pkg/errors v0.9.1
|
||||
github.com/rhysd/actionlint v1.6.26
|
||||
github.com/rhysd/actionlint v1.6.27
|
||||
github.com/sabhiram/go-gitignore v0.0.0-20210923224102-525f6e181f06
|
||||
github.com/sirupsen/logrus v1.9.3
|
||||
github.com/spf13/cobra v1.8.0
|
||||
github.com/spf13/pflag v1.0.5
|
||||
github.com/stretchr/testify v1.8.4
|
||||
github.com/stretchr/testify v1.9.0
|
||||
github.com/timshannon/bolthold v0.0.0-20210913165410-232392fc8a6a
|
||||
go.etcd.io/bbolt v1.3.8
|
||||
golang.org/x/term v0.16.0
|
||||
go.etcd.io/bbolt v1.3.9
|
||||
golang.org/x/term v0.18.0
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
gotest.tools/v3 v3.5.1
|
||||
)
|
||||
@@ -40,7 +40,7 @@ require (
|
||||
require (
|
||||
dario.cat/mergo v1.0.0 // indirect
|
||||
github.com/Microsoft/go-winio v0.6.1 // indirect
|
||||
github.com/ProtonMail/go-crypto v0.0.0-20230828082145-3c4c8a2d2371 // indirect
|
||||
github.com/ProtonMail/go-crypto v1.0.0 // indirect
|
||||
github.com/cloudflare/circl v1.3.7 // indirect
|
||||
github.com/containerd/containerd v1.7.2 // indirect
|
||||
github.com/cyphar/filepath-securejoin v0.2.4 // indirect
|
||||
@@ -49,7 +49,7 @@ require (
|
||||
github.com/docker/docker-credential-helpers v0.7.0 // indirect
|
||||
github.com/docker/go-units v0.5.0 // indirect
|
||||
github.com/emirpasic/gods v1.18.1 // indirect
|
||||
github.com/fatih/color v1.15.0 // indirect
|
||||
github.com/fatih/color v1.16.0 // indirect
|
||||
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
|
||||
github.com/gogo/protobuf v1.3.2 // indirect
|
||||
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
|
||||
@@ -69,20 +69,20 @@ require (
|
||||
github.com/opencontainers/runc v1.1.12 // indirect
|
||||
github.com/pjbgf/sha1cd v0.3.0 // indirect
|
||||
github.com/pmezard/go-difflib v1.0.0 // indirect
|
||||
github.com/rivo/uniseg v0.4.4 // indirect
|
||||
github.com/rivo/uniseg v0.4.7 // indirect
|
||||
github.com/robfig/cron/v3 v3.0.1 // indirect
|
||||
github.com/sergi/go-diff v1.2.0 // indirect
|
||||
github.com/skeema/knownhosts v1.2.1 // indirect
|
||||
github.com/stretchr/objx v0.5.0 // indirect
|
||||
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
|
||||
github.com/skeema/knownhosts v1.2.2 // indirect
|
||||
github.com/stretchr/objx v0.5.2 // indirect
|
||||
github.com/xanzy/ssh-agent v0.3.3 // indirect
|
||||
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
|
||||
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
|
||||
github.com/xeipuuv/gojsonschema v1.2.0 // indirect
|
||||
golang.org/x/crypto v0.17.0 // indirect
|
||||
golang.org/x/crypto v0.21.0 // indirect
|
||||
golang.org/x/mod v0.12.0 // indirect
|
||||
golang.org/x/net v0.19.0 // indirect
|
||||
golang.org/x/sync v0.3.0 // indirect
|
||||
golang.org/x/sys v0.16.0 // indirect
|
||||
golang.org/x/net v0.22.0 // indirect
|
||||
golang.org/x/sync v0.6.0 // indirect
|
||||
golang.org/x/sys v0.18.0 // indirect
|
||||
golang.org/x/text v0.14.0 // indirect
|
||||
golang.org/x/tools v0.13.0 // indirect
|
||||
gopkg.in/warnings.v0 v0.1.2 // indirect
|
||||
|
||||
74
go.sum
74
go.sum
@@ -13,8 +13,8 @@ github.com/Microsoft/go-winio v0.6.1/go.mod h1:LRdKpFKfdobln8UmuiYcKPot9D2v6svN5
|
||||
github.com/Microsoft/hcsshim v0.10.0-rc.8 h1:YSZVvlIIDD1UxQpJp0h+dnpLUw+TrY0cx8obKsp3bek=
|
||||
github.com/Netflix/go-expect v0.0.0-20220104043353-73e0943537d2 h1:+vx7roKuyA63nhn5WAunQHLTznkw5W8b1Xc0dNjp83s=
|
||||
github.com/Netflix/go-expect v0.0.0-20220104043353-73e0943537d2/go.mod h1:HBCaDeC1lPdgDeDbhX8XFpy1jqjK0IBG8W5K+xYqA0w=
|
||||
github.com/ProtonMail/go-crypto v0.0.0-20230828082145-3c4c8a2d2371 h1:kkhsdkhsCvIsutKu5zLMgWtgh9YxGCNAw8Ad8hjwfYg=
|
||||
github.com/ProtonMail/go-crypto v0.0.0-20230828082145-3c4c8a2d2371/go.mod h1:EjAoLdwvbIOoOQr3ihjnSoLZRtE8azugULFRteWMNc0=
|
||||
github.com/ProtonMail/go-crypto v1.0.0 h1:LRuvITjQWX+WIfr930YHG2HNfjR1uOfyf5vE0kC2U78=
|
||||
github.com/ProtonMail/go-crypto v1.0.0/go.mod h1:EjAoLdwvbIOoOQr3ihjnSoLZRtE8azugULFRteWMNc0=
|
||||
github.com/adrg/xdg v0.4.0 h1:RzRqFcjH4nE5C6oTAxhBtoE2IRyjBSa62SCbyPidvls=
|
||||
github.com/adrg/xdg v0.4.0/go.mod h1:N6ag73EX4wyxeaoeHctc1mas01KZgsj5tYiAIwqJE/E=
|
||||
github.com/andreaskoch/go-fswatch v1.0.0 h1:la8nP/HiaFCxP2IM6NZNUCoxgLWuyNFgH0RligBbnJU=
|
||||
@@ -42,8 +42,8 @@ github.com/docker/cli v24.0.7+incompatible h1:wa/nIwYFW7BVTGa7SWPVyyXU9lgORqUb1x
|
||||
github.com/docker/cli v24.0.7+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
|
||||
github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk=
|
||||
github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
|
||||
github.com/docker/docker v24.0.7+incompatible h1:Wo6l37AuwP3JaMnZa226lzVXGA3F9Ig1seQen0cKYlM=
|
||||
github.com/docker/docker v24.0.7+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
|
||||
github.com/docker/docker v24.0.9+incompatible h1:HPGzNmwfLZWdxHqK9/II92pyi1EpYKsAqcl4G0Of9v0=
|
||||
github.com/docker/docker v24.0.9+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
|
||||
github.com/docker/docker-credential-helpers v0.7.0 h1:xtCHsjxogADNZcdv1pKUHXryefjlVRqWqIhk/uXJp0A=
|
||||
github.com/docker/docker-credential-helpers v0.7.0/go.mod h1:rETQfLdHNT3foU5kuNkFR1R1V12OJRRO5lzt2D1b5X0=
|
||||
github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
|
||||
@@ -53,16 +53,16 @@ github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDD
|
||||
github.com/elazarl/goproxy v0.0.0-20230808193330-2592e75ae04a h1:mATvB/9r/3gvcejNsXKSkQ6lcIaNec2nyfOdlTBR2lU=
|
||||
github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=
|
||||
github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ=
|
||||
github.com/fatih/color v1.15.0 h1:kOqh6YHBtK8aywxGerMG2Eq3H6Qgoqeo13Bk2Mv/nBs=
|
||||
github.com/fatih/color v1.15.0/go.mod h1:0h5ZqXfHYED7Bhv2ZJamyIOUej9KtShiJESRwBDUSsw=
|
||||
github.com/gliderlabs/ssh v0.3.5 h1:OcaySEmAQJgyYcArR+gGGTHCyE7nvhEMTlYY+Dp8CpY=
|
||||
github.com/fatih/color v1.16.0 h1:zmkK9Ngbjj+K0yRhTVONQh1p/HknKYSlNT+vZCzyokM=
|
||||
github.com/fatih/color v1.16.0/go.mod h1:fL2Sau1YI5c0pdGEVCbKQbLXB6edEj1ZgiY4NijnWvE=
|
||||
github.com/gliderlabs/ssh v0.3.7 h1:iV3Bqi942d9huXnzEF2Mt+CY9gLu8DNM4Obd+8bODRE=
|
||||
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI=
|
||||
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic=
|
||||
github.com/go-git/go-billy/v5 v5.5.0 h1:yEY4yhzCDuMGSv83oGxiBotRzhwhNr8VZyphhiu+mTU=
|
||||
github.com/go-git/go-billy/v5 v5.5.0/go.mod h1:hmexnoNsr2SJU1Ju67OaNz5ASJY3+sHgFRpCtpDCKow=
|
||||
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399 h1:eMje31YglSBqCdIqdhKBW8lokaMrL3uTkpGYlE2OOT4=
|
||||
github.com/go-git/go-git/v5 v5.11.0 h1:XIZc1p+8YzypNr34itUfSvYJcv+eYdTnTvOZ2vD3cA4=
|
||||
github.com/go-git/go-git/v5 v5.11.0/go.mod h1:6GFcX2P3NM7FPBfpePbpLd21XxsgdAt+lKqXmCUiUCY=
|
||||
github.com/go-git/go-git/v5 v5.12.0 h1:7Md+ndsjrzZxbddRDZjF14qK+NN56sy6wkqaVrjZtys=
|
||||
github.com/go-git/go-git/v5 v5.12.0/go.mod h1:FTM9VKtnI2m65hNI/TenDDDnUf2Q9FHnXYjuz9i5OEY=
|
||||
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
|
||||
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
|
||||
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
|
||||
@@ -124,8 +124,8 @@ github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
|
||||
github.com/onsi/gomega v1.27.10 h1:naR28SdDFlqrG6kScpT8VWpu1xWY5nJRCF3XaYyBjhI=
|
||||
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
|
||||
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
|
||||
github.com/opencontainers/image-spec v1.1.0-rc3 h1:fzg1mXZFj8YdPeNkRXMg+zb88BFV0Ys52cJydRwBkb8=
|
||||
github.com/opencontainers/image-spec v1.1.0-rc3/go.mod h1:X4pATf0uXsnn3g5aiGIsVnJBR4mxhKzfwmvK/B2NTm8=
|
||||
github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
|
||||
github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM=
|
||||
github.com/opencontainers/runc v1.1.12 h1:BOIssBaW1La0/qbNZHXOOa71dZfZEQOzW7dqQf3phss=
|
||||
github.com/opencontainers/runc v1.1.12/go.mod h1:S+lQwSfncpBha7XTy/5lBwWgm5+y5Ma/O44Ekby9FK8=
|
||||
github.com/opencontainers/selinux v1.11.0 h1:+5Zbo97w3Lbmb3PeqQtpmTkMwsW5nRI3YaLpt7tQ7oU=
|
||||
@@ -137,25 +137,25 @@ github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
|
||||
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/rhysd/actionlint v1.6.26 h1:zi7jPZf3Ks14gCXYAAL47uBziyFlX7+Xwilqhexct9g=
|
||||
github.com/rhysd/actionlint v1.6.26/go.mod h1:TIj1DlCgtYLOv5CH9wCK+WJTOr1qAdnFzkGi0IgSCO4=
|
||||
github.com/rhysd/actionlint v1.6.27 h1:xxwe8YmveBcC8lydW6GoHMGmB6H/MTqUU60F2p10wjw=
|
||||
github.com/rhysd/actionlint v1.6.27/go.mod h1:m2nFUjAnOrxCMXuOMz9evYBRCLUsMnKY2IJl/N5umbk=
|
||||
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
|
||||
github.com/rivo/uniseg v0.4.4 h1:8TfxU8dW6PdqD27gjM8MVNuicgxIjxpm4K7x4jp8sis=
|
||||
github.com/rivo/uniseg v0.4.4/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
|
||||
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
|
||||
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
|
||||
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
|
||||
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
|
||||
github.com/rogpeppe/go-internal v1.11.0 h1:cWPaGQEPrBb5/AsnsZesgZZ9yb1OQ+GOISoDNXVBh4M=
|
||||
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
|
||||
github.com/sabhiram/go-gitignore v0.0.0-20210923224102-525f6e181f06 h1:OkMGxebDjyw0ULyrTYWeN0UNCCkmCWfjPnIA2W6oviI=
|
||||
github.com/sabhiram/go-gitignore v0.0.0-20210923224102-525f6e181f06/go.mod h1:+ePHsJ1keEjQtpvf9HHw0f4ZeJ0TLRsxhunSI2hYJSs=
|
||||
github.com/sergi/go-diff v1.2.0 h1:XU+rvMAioB0UC3q1MFrIQy4Vo5/4VsRDQQXHsEya6xQ=
|
||||
github.com/sergi/go-diff v1.2.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
|
||||
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 h1:n661drycOFuPLCN3Uc8sB6B/s6Z4t2xvBgU1htSHuq8=
|
||||
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
|
||||
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
|
||||
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
|
||||
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
|
||||
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
|
||||
github.com/skeema/knownhosts v1.2.1 h1:SHWdIUa82uGZz+F+47k8SY4QhhI291cXCpopT1lK2AQ=
|
||||
github.com/skeema/knownhosts v1.2.1/go.mod h1:xYbVRSPxqBZFrdmDyMmsOs+uX1UZC3nTN3ThzgDxUwo=
|
||||
github.com/skeema/knownhosts v1.2.2 h1:Iug2P4fLmDw9f41PB6thxUkNUkJzB5i+1/exaj40L3A=
|
||||
github.com/skeema/knownhosts v1.2.2/go.mod h1:xYbVRSPxqBZFrdmDyMmsOs+uX1UZC3nTN3ThzgDxUwo=
|
||||
github.com/spf13/cobra v1.8.0 h1:7aJaZx1B85qltLMc546zn58BxxfZdR/W22ej9CFoEf0=
|
||||
github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho=
|
||||
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
|
||||
@@ -163,18 +163,15 @@ github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
|
||||
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||
github.com/stretchr/objx v0.5.0 h1:1zr/of2m5FGMsad5YfcqgdqdWrIhu+EBEJRhR1U7z/c=
|
||||
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
||||
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
|
||||
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
|
||||
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
|
||||
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
||||
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
|
||||
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
||||
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
|
||||
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||
github.com/timshannon/bolthold v0.0.0-20210913165410-232392fc8a6a h1:oIi7H/bwFUYKYhzKbHc+3MvHRWqhQwXVB4LweLMiVy0=
|
||||
github.com/timshannon/bolthold v0.0.0-20210913165410-232392fc8a6a/go.mod h1:iSvujNDmpZ6eQX+bg/0X3lF7LEmZ8N77g2a/J/+Zt2U=
|
||||
github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
|
||||
@@ -190,8 +187,8 @@ github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9de
|
||||
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
|
||||
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
|
||||
go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4=
|
||||
go.etcd.io/bbolt v1.3.8 h1:xs88BrvEv273UsB79e0hcVrlUWmS0a8upikMFhSyAtA=
|
||||
go.etcd.io/bbolt v1.3.8/go.mod h1:N9Mkw9X8x5fupy0IKsmuqVtoGDyxsaDlbk4Rd05IAQw=
|
||||
go.etcd.io/bbolt v1.3.9 h1:8x7aARPEXiXbHmtUwAIv7eV2fQFHrLLavdiJ3uzJXoI=
|
||||
go.etcd.io/bbolt v1.3.9/go.mod h1:zaO32+Ti0PK1ivdPtgMESzuzL2VPoIG1PCQNvOdo/dE=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
|
||||
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
|
||||
@@ -199,8 +196,8 @@ golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5y
|
||||
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
|
||||
golang.org/x/crypto v0.3.1-0.20221117191849-2c476679df9a/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
|
||||
golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU=
|
||||
golang.org/x/crypto v0.17.0 h1:r8bRNjWL3GshPW3gkd+RpvzWrZAwPS49OmTGZ/uhM4k=
|
||||
golang.org/x/crypto v0.17.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
|
||||
golang.org/x/crypto v0.21.0 h1:X31++rzVUdKhX5sWmSOFZxx8UW/ldWx55cbf08iNAMA=
|
||||
golang.org/x/crypto v0.21.0/go.mod h1:0BP7YvVV9gBbVKyeTG0Gyn+gZm94bibOW5BjDEYAOMs=
|
||||
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
|
||||
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
|
||||
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
|
||||
@@ -218,15 +215,15 @@ golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug
|
||||
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
|
||||
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
|
||||
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
|
||||
golang.org/x/net v0.19.0 h1:zTwKpTd2XuCqf8huc7Fo2iSy+4RHPd10s4KzeTnVr1c=
|
||||
golang.org/x/net v0.19.0/go.mod h1:CfAk/cbD4CthTvqiEl8NpboMuiuOYsAr/7NOjZJtv1U=
|
||||
golang.org/x/net v0.22.0 h1:9sGLhx7iRIHEiX0oAJ3MRZMUCElJgy7Br1nO+AMN3Tc=
|
||||
golang.org/x/net v0.22.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
|
||||
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.3.0 h1:ftCYgMx6zT/asHUrPw8BLLscYtGznsLAnjq5RH9P66E=
|
||||
golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
|
||||
golang.org/x/sync v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
|
||||
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
|
||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
@@ -248,15 +245,15 @@ golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.16.0 h1:xWw16ngr6ZMtmxDyKyIgsE93KNKz5HKmMa3b8ALHidU=
|
||||
golang.org/x/sys v0.16.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
|
||||
golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
|
||||
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
||||
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
|
||||
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
|
||||
golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
|
||||
golang.org/x/term v0.16.0 h1:m+B6fahuftsE9qjo0VWp2FW0mB3MTJvR0BaMQrq0pmE=
|
||||
golang.org/x/term v0.16.0/go.mod h1:yn7UURbUtPyrVJPGPq404EukNFxcm/foM+bV/bfcDsY=
|
||||
golang.org/x/term v0.18.0 h1:FcHjZXDMxI8mM3nwhX9HlKop4C0YQvCVCdwYl2wOtE8=
|
||||
golang.org/x/term v0.18.0/go.mod h1:ILwASektA3OnRv7amZ1xhE/KTR+u50pbXfZ03+6Nx58=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
@@ -286,7 +283,6 @@ gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntN
|
||||
gopkg.in/warnings.v0 v0.1.2 h1:wFXVbFY8DY5/xOe1ECiWdKCzZlxgshcYVNkBHstARME=
|
||||
gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=
|
||||
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
|
||||
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
|
||||
@@ -35,7 +35,7 @@ type Handler struct {
|
||||
server *http.Server
|
||||
logger logrus.FieldLogger
|
||||
|
||||
gcing int32 // TODO: use atomic.Bool when we can use Go 1.19
|
||||
gcing atomic.Bool
|
||||
gcAt time.Time
|
||||
|
||||
outboundIP string
|
||||
@@ -170,7 +170,7 @@ func (h *Handler) find(w http.ResponseWriter, r *http.Request, _ httprouter.Para
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
cache, err := h.findCache(db, keys, version)
|
||||
cache, err := findCache(db, keys, version)
|
||||
if err != nil {
|
||||
h.responseJSON(w, r, 500, err)
|
||||
return
|
||||
@@ -206,32 +206,17 @@ func (h *Handler) reserve(w http.ResponseWriter, r *http.Request, _ httprouter.P
|
||||
api.Key = strings.ToLower(api.Key)
|
||||
|
||||
cache := api.ToCache()
|
||||
cache.FillKeyVersionHash()
|
||||
db, err := h.openDB()
|
||||
if err != nil {
|
||||
h.responseJSON(w, r, 500, err)
|
||||
return
|
||||
}
|
||||
defer db.Close()
|
||||
if err := db.FindOne(cache, bolthold.Where("KeyVersionHash").Eq(cache.KeyVersionHash)); err != nil {
|
||||
if !errors.Is(err, bolthold.ErrNotFound) {
|
||||
h.responseJSON(w, r, 500, err)
|
||||
return
|
||||
}
|
||||
} else {
|
||||
h.responseJSON(w, r, 400, fmt.Errorf("already exist"))
|
||||
return
|
||||
}
|
||||
|
||||
now := time.Now().Unix()
|
||||
cache.CreatedAt = now
|
||||
cache.UsedAt = now
|
||||
if err := db.Insert(bolthold.NextSequence(), cache); err != nil {
|
||||
h.responseJSON(w, r, 500, err)
|
||||
return
|
||||
}
|
||||
// write back id to db
|
||||
if err := db.Update(cache.ID, cache); err != nil {
|
||||
if err := insertCache(db, cache); err != nil {
|
||||
h.responseJSON(w, r, 500, err)
|
||||
return
|
||||
}
|
||||
@@ -364,56 +349,51 @@ func (h *Handler) middleware(handler httprouter.Handle) httprouter.Handle {
|
||||
}
|
||||
|
||||
// if not found, return (nil, nil) instead of an error.
|
||||
func (h *Handler) findCache(db *bolthold.Store, keys []string, version string) (*Cache, error) {
|
||||
if len(keys) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
key := keys[0] // the first key is for exact match.
|
||||
|
||||
cache := &Cache{
|
||||
Key: key,
|
||||
Version: version,
|
||||
}
|
||||
cache.FillKeyVersionHash()
|
||||
|
||||
if err := db.FindOne(cache, bolthold.Where("KeyVersionHash").Eq(cache.KeyVersionHash)); err != nil {
|
||||
if !errors.Is(err, bolthold.ErrNotFound) {
|
||||
return nil, err
|
||||
func findCache(db *bolthold.Store, keys []string, version string) (*Cache, error) {
|
||||
cache := &Cache{}
|
||||
for _, prefix := range keys {
|
||||
// if a key in the list matches exactly, don't return partial matches
|
||||
if err := db.FindOne(cache,
|
||||
bolthold.Where("Key").Eq(prefix).
|
||||
And("Version").Eq(version).
|
||||
And("Complete").Eq(true).
|
||||
SortBy("CreatedAt").Reverse()); err == nil || !errors.Is(err, bolthold.ErrNotFound) {
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("find cache: %w", err)
|
||||
}
|
||||
return cache, nil
|
||||
}
|
||||
} else if cache.Complete {
|
||||
return cache, nil
|
||||
}
|
||||
stop := fmt.Errorf("stop")
|
||||
|
||||
for _, prefix := range keys[1:] {
|
||||
found := false
|
||||
prefixPattern := fmt.Sprintf("^%s", regexp.QuoteMeta(prefix))
|
||||
re, err := regexp.Compile(prefixPattern)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if err := db.ForEach(bolthold.Where("Key").RegExp(re).And("Version").Eq(version).SortBy("CreatedAt").Reverse(), func(v *Cache) error {
|
||||
if !strings.HasPrefix(v.Key, prefix) {
|
||||
return stop
|
||||
}
|
||||
if v.Complete {
|
||||
cache = v
|
||||
found = true
|
||||
return stop
|
||||
}
|
||||
return nil
|
||||
}); err != nil {
|
||||
if !errors.Is(err, stop) {
|
||||
return nil, err
|
||||
if err := db.FindOne(cache,
|
||||
bolthold.Where("Key").RegExp(re).
|
||||
And("Version").Eq(version).
|
||||
And("Complete").Eq(true).
|
||||
SortBy("CreatedAt").Reverse()); err != nil {
|
||||
if errors.Is(err, bolthold.ErrNotFound) {
|
||||
continue
|
||||
}
|
||||
return nil, fmt.Errorf("find cache: %w", err)
|
||||
}
|
||||
if found {
|
||||
return cache, nil
|
||||
}
|
||||
return cache, nil
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func insertCache(db *bolthold.Store, cache *Cache) error {
|
||||
if err := db.Insert(bolthold.NextSequence(), cache); err != nil {
|
||||
return fmt.Errorf("insert cache: %w", err)
|
||||
}
|
||||
// write back id to db
|
||||
if err := db.Update(cache.ID, cache); err != nil {
|
||||
return fmt.Errorf("write back id to db: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (h *Handler) useCache(id int64) {
|
||||
db, err := h.openDB()
|
||||
if err != nil {
|
||||
@@ -428,14 +408,21 @@ func (h *Handler) useCache(id int64) {
|
||||
_ = db.Update(cache.ID, cache)
|
||||
}
|
||||
|
||||
const (
|
||||
keepUsed = 30 * 24 * time.Hour
|
||||
keepUnused = 7 * 24 * time.Hour
|
||||
keepTemp = 5 * time.Minute
|
||||
keepOld = 5 * time.Minute
|
||||
)
|
||||
|
||||
func (h *Handler) gcCache() {
|
||||
if atomic.LoadInt32(&h.gcing) != 0 {
|
||||
if h.gcing.Load() {
|
||||
return
|
||||
}
|
||||
if !atomic.CompareAndSwapInt32(&h.gcing, 0, 1) {
|
||||
if !h.gcing.CompareAndSwap(false, true) {
|
||||
return
|
||||
}
|
||||
defer atomic.StoreInt32(&h.gcing, 0)
|
||||
defer h.gcing.Store(false)
|
||||
|
||||
if time.Since(h.gcAt) < time.Hour {
|
||||
h.logger.Debugf("skip gc: %v", h.gcAt.String())
|
||||
@@ -444,37 +431,18 @@ func (h *Handler) gcCache() {
|
||||
h.gcAt = time.Now()
|
||||
h.logger.Debugf("gc: %v", h.gcAt.String())
|
||||
|
||||
const (
|
||||
keepUsed = 30 * 24 * time.Hour
|
||||
keepUnused = 7 * 24 * time.Hour
|
||||
keepTemp = 5 * time.Minute
|
||||
)
|
||||
|
||||
db, err := h.openDB()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
// Remove the caches which are not completed for a while, they are most likely to be broken.
|
||||
var caches []*Cache
|
||||
if err := db.Find(&caches, bolthold.Where("UsedAt").Lt(time.Now().Add(-keepTemp).Unix())); err != nil {
|
||||
h.logger.Warnf("find caches: %v", err)
|
||||
} else {
|
||||
for _, cache := range caches {
|
||||
if cache.Complete {
|
||||
continue
|
||||
}
|
||||
h.storage.Remove(cache.ID)
|
||||
if err := db.Delete(cache.ID, cache); err != nil {
|
||||
h.logger.Warnf("delete cache: %v", err)
|
||||
continue
|
||||
}
|
||||
h.logger.Infof("deleted cache: %+v", cache)
|
||||
}
|
||||
}
|
||||
|
||||
caches = caches[:0]
|
||||
if err := db.Find(&caches, bolthold.Where("UsedAt").Lt(time.Now().Add(-keepUnused).Unix())); err != nil {
|
||||
if err := db.Find(&caches, bolthold.
|
||||
Where("UsedAt").Lt(time.Now().Add(-keepTemp).Unix()).
|
||||
And("Complete").Eq(false),
|
||||
); err != nil {
|
||||
h.logger.Warnf("find caches: %v", err)
|
||||
} else {
|
||||
for _, cache := range caches {
|
||||
@@ -487,8 +455,11 @@ func (h *Handler) gcCache() {
|
||||
}
|
||||
}
|
||||
|
||||
// Remove the old caches which have not been used recently.
|
||||
caches = caches[:0]
|
||||
if err := db.Find(&caches, bolthold.Where("CreatedAt").Lt(time.Now().Add(-keepUsed).Unix())); err != nil {
|
||||
if err := db.Find(&caches, bolthold.
|
||||
Where("UsedAt").Lt(time.Now().Add(-keepUnused).Unix()),
|
||||
); err != nil {
|
||||
h.logger.Warnf("find caches: %v", err)
|
||||
} else {
|
||||
for _, cache := range caches {
|
||||
@@ -500,6 +471,55 @@ func (h *Handler) gcCache() {
|
||||
h.logger.Infof("deleted cache: %+v", cache)
|
||||
}
|
||||
}
|
||||
|
||||
// Remove the old caches which are too old.
|
||||
caches = caches[:0]
|
||||
if err := db.Find(&caches, bolthold.
|
||||
Where("CreatedAt").Lt(time.Now().Add(-keepUsed).Unix()),
|
||||
); err != nil {
|
||||
h.logger.Warnf("find caches: %v", err)
|
||||
} else {
|
||||
for _, cache := range caches {
|
||||
h.storage.Remove(cache.ID)
|
||||
if err := db.Delete(cache.ID, cache); err != nil {
|
||||
h.logger.Warnf("delete cache: %v", err)
|
||||
continue
|
||||
}
|
||||
h.logger.Infof("deleted cache: %+v", cache)
|
||||
}
|
||||
}
|
||||
|
||||
// Remove the old caches with the same key and version, keep the latest one.
|
||||
// Also keep the olds which have been used recently for a while in case of the cache is still in use.
|
||||
if results, err := db.FindAggregate(
|
||||
&Cache{},
|
||||
bolthold.Where("Complete").Eq(true),
|
||||
"Key", "Version",
|
||||
); err != nil {
|
||||
h.logger.Warnf("find aggregate caches: %v", err)
|
||||
} else {
|
||||
for _, result := range results {
|
||||
if result.Count() <= 1 {
|
||||
continue
|
||||
}
|
||||
result.Sort("CreatedAt")
|
||||
caches = caches[:0]
|
||||
result.Reduction(&caches)
|
||||
for _, cache := range caches[:len(caches)-1] {
|
||||
if time.Since(time.Unix(cache.UsedAt, 0)) < keepOld {
|
||||
// Keep it since it has been used recently, even if it's old.
|
||||
// Or it could break downloading in process.
|
||||
continue
|
||||
}
|
||||
h.storage.Remove(cache.ID)
|
||||
if err := db.Delete(cache.ID, cache); err != nil {
|
||||
h.logger.Warnf("delete cache: %v", err)
|
||||
continue
|
||||
}
|
||||
h.logger.Infof("deleted cache: %+v", cache)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (h *Handler) responseJSON(w http.ResponseWriter, r *http.Request, code int, v ...any) {
|
||||
|
||||
@@ -10,9 +10,11 @@ import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"github.com/timshannon/bolthold"
|
||||
"go.etcd.io/bbolt"
|
||||
)
|
||||
|
||||
@@ -78,6 +80,9 @@ func TestHandler(t *testing.T) {
|
||||
t.Run("duplicate reserve", func(t *testing.T) {
|
||||
key := strings.ToLower(t.Name())
|
||||
version := "c19da02a2bd7e77277f1ac29ab45c09b7d46a4ee758284e26bb3045ad11d9d20"
|
||||
var first, second struct {
|
||||
CacheID uint64 `json:"cacheId"`
|
||||
}
|
||||
{
|
||||
body, err := json.Marshal(&Request{
|
||||
Key: key,
|
||||
@@ -89,10 +94,8 @@ func TestHandler(t *testing.T) {
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, 200, resp.StatusCode)
|
||||
|
||||
got := struct {
|
||||
CacheID uint64 `json:"cacheId"`
|
||||
}{}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&got))
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&first))
|
||||
assert.NotZero(t, first.CacheID)
|
||||
}
|
||||
{
|
||||
body, err := json.Marshal(&Request{
|
||||
@@ -103,8 +106,13 @@ func TestHandler(t *testing.T) {
|
||||
require.NoError(t, err)
|
||||
resp, err := http.Post(fmt.Sprintf("%s/caches", base), "application/json", bytes.NewReader(body))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, 400, resp.StatusCode)
|
||||
assert.Equal(t, 200, resp.StatusCode)
|
||||
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&second))
|
||||
assert.NotZero(t, second.CacheID)
|
||||
}
|
||||
|
||||
assert.NotEqual(t, first.CacheID, second.CacheID)
|
||||
})
|
||||
|
||||
t.Run("upload with bad id", func(t *testing.T) {
|
||||
@@ -341,9 +349,9 @@ func TestHandler(t *testing.T) {
|
||||
version := "c19da02a2bd7e77277f1ac29ab45c09b7d46a4ee758284e26bb3045ad11d9d20"
|
||||
key := strings.ToLower(t.Name())
|
||||
keys := [3]string{
|
||||
key + "_a",
|
||||
key + "_a_b",
|
||||
key + "_a_b_c",
|
||||
key + "_a_b",
|
||||
key + "_a",
|
||||
}
|
||||
contents := [3][]byte{
|
||||
make([]byte, 100),
|
||||
@@ -354,6 +362,7 @@ func TestHandler(t *testing.T) {
|
||||
_, err := rand.Read(contents[i])
|
||||
require.NoError(t, err)
|
||||
uploadCacheNormally(t, base, keys[i], version, contents[i])
|
||||
time.Sleep(time.Second) // ensure CreatedAt of caches are different
|
||||
}
|
||||
|
||||
reqKeys := strings.Join([]string{
|
||||
@@ -361,29 +370,33 @@ func TestHandler(t *testing.T) {
|
||||
key + "_a_b",
|
||||
key + "_a",
|
||||
}, ",")
|
||||
var archiveLocation string
|
||||
{
|
||||
resp, err := http.Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, reqKeys, version))
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 200, resp.StatusCode)
|
||||
got := struct {
|
||||
Result string `json:"result"`
|
||||
ArchiveLocation string `json:"archiveLocation"`
|
||||
CacheKey string `json:"cacheKey"`
|
||||
}{}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&got))
|
||||
assert.Equal(t, "hit", got.Result)
|
||||
assert.Equal(t, keys[1], got.CacheKey)
|
||||
archiveLocation = got.ArchiveLocation
|
||||
}
|
||||
{
|
||||
resp, err := http.Get(archiveLocation) //nolint:gosec
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 200, resp.StatusCode)
|
||||
got, err := io.ReadAll(resp.Body)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, contents[1], got)
|
||||
}
|
||||
|
||||
resp, err := http.Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, reqKeys, version))
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 200, resp.StatusCode)
|
||||
|
||||
/*
|
||||
Expect `key_a_b` because:
|
||||
- `key_a_b_x" doesn't match any caches.
|
||||
- `key_a_b" matches `key_a_b` and `key_a_b_c`, but `key_a_b` is newer.
|
||||
*/
|
||||
except := 1
|
||||
|
||||
got := struct {
|
||||
Result string `json:"result"`
|
||||
ArchiveLocation string `json:"archiveLocation"`
|
||||
CacheKey string `json:"cacheKey"`
|
||||
}{}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&got))
|
||||
assert.Equal(t, "hit", got.Result)
|
||||
assert.Equal(t, keys[except], got.CacheKey)
|
||||
|
||||
contentResp, err := http.Get(got.ArchiveLocation)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 200, contentResp.StatusCode)
|
||||
content, err := io.ReadAll(contentResp.Body)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, contents[except], content)
|
||||
})
|
||||
|
||||
t.Run("case insensitive", func(t *testing.T) {
|
||||
@@ -409,6 +422,110 @@ func TestHandler(t *testing.T) {
|
||||
assert.Equal(t, key+"_abc", got.CacheKey)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("exact keys are preferred (key 0)", func(t *testing.T) {
|
||||
version := "c19da02a2bd7e77277f1ac29ab45c09b7d46a4ee758284e26bb3045ad11d9d20"
|
||||
key := strings.ToLower(t.Name())
|
||||
keys := [3]string{
|
||||
key + "_a",
|
||||
key + "_a_b_c",
|
||||
key + "_a_b",
|
||||
}
|
||||
contents := [3][]byte{
|
||||
make([]byte, 100),
|
||||
make([]byte, 200),
|
||||
make([]byte, 300),
|
||||
}
|
||||
for i := range contents {
|
||||
_, err := rand.Read(contents[i])
|
||||
require.NoError(t, err)
|
||||
uploadCacheNormally(t, base, keys[i], version, contents[i])
|
||||
time.Sleep(time.Second) // ensure CreatedAt of caches are different
|
||||
}
|
||||
|
||||
reqKeys := strings.Join([]string{
|
||||
key + "_a",
|
||||
key + "_a_b",
|
||||
}, ",")
|
||||
|
||||
resp, err := http.Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, reqKeys, version))
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 200, resp.StatusCode)
|
||||
|
||||
/*
|
||||
Expect `key_a` because:
|
||||
- `key_a` matches `key_a`, `key_a_b` and `key_a_b_c`, but `key_a` is an exact match.
|
||||
- `key_a_b` matches `key_a_b` and `key_a_b_c`, but previous key had a match
|
||||
*/
|
||||
expect := 0
|
||||
|
||||
got := struct {
|
||||
ArchiveLocation string `json:"archiveLocation"`
|
||||
CacheKey string `json:"cacheKey"`
|
||||
}{}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&got))
|
||||
assert.Equal(t, keys[expect], got.CacheKey)
|
||||
|
||||
contentResp, err := http.Get(got.ArchiveLocation)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 200, contentResp.StatusCode)
|
||||
content, err := io.ReadAll(contentResp.Body)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, contents[expect], content)
|
||||
})
|
||||
|
||||
t.Run("exact keys are preferred (key 1)", func(t *testing.T) {
|
||||
version := "c19da02a2bd7e77277f1ac29ab45c09b7d46a4ee758284e26bb3045ad11d9d20"
|
||||
key := strings.ToLower(t.Name())
|
||||
keys := [3]string{
|
||||
key + "_a",
|
||||
key + "_a_b_c",
|
||||
key + "_a_b",
|
||||
}
|
||||
contents := [3][]byte{
|
||||
make([]byte, 100),
|
||||
make([]byte, 200),
|
||||
make([]byte, 300),
|
||||
}
|
||||
for i := range contents {
|
||||
_, err := rand.Read(contents[i])
|
||||
require.NoError(t, err)
|
||||
uploadCacheNormally(t, base, keys[i], version, contents[i])
|
||||
time.Sleep(time.Second) // ensure CreatedAt of caches are different
|
||||
}
|
||||
|
||||
reqKeys := strings.Join([]string{
|
||||
"------------------------------------------------------",
|
||||
key + "_a",
|
||||
key + "_a_b",
|
||||
}, ",")
|
||||
|
||||
resp, err := http.Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, reqKeys, version))
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 200, resp.StatusCode)
|
||||
|
||||
/*
|
||||
Expect `key_a` because:
|
||||
- `------------------------------------------------------` doesn't match any caches.
|
||||
- `key_a` matches `key_a`, `key_a_b` and `key_a_b_c`, but `key_a` is an exact match.
|
||||
- `key_a_b` matches `key_a_b` and `key_a_b_c`, but previous key had a match
|
||||
*/
|
||||
expect := 0
|
||||
|
||||
got := struct {
|
||||
ArchiveLocation string `json:"archiveLocation"`
|
||||
CacheKey string `json:"cacheKey"`
|
||||
}{}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&got))
|
||||
assert.Equal(t, keys[expect], got.CacheKey)
|
||||
|
||||
contentResp, err := http.Get(got.ArchiveLocation)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 200, contentResp.StatusCode)
|
||||
content, err := io.ReadAll(contentResp.Body)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, contents[expect], content)
|
||||
})
|
||||
}
|
||||
|
||||
func uploadCacheNormally(t *testing.T, base, key, version string, content []byte) {
|
||||
@@ -469,3 +586,112 @@ func uploadCacheNormally(t *testing.T, base, key, version string, content []byte
|
||||
assert.Equal(t, content, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandler_gcCache(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "artifactcache")
|
||||
handler, err := StartHandler(dir, "", 0, nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
defer func() {
|
||||
require.NoError(t, handler.Close())
|
||||
}()
|
||||
|
||||
now := time.Now()
|
||||
|
||||
cases := []struct {
|
||||
Cache *Cache
|
||||
Kept bool
|
||||
}{
|
||||
{
|
||||
// should be kept, since it's used recently and not too old.
|
||||
Cache: &Cache{
|
||||
Key: "test_key_1",
|
||||
Version: "test_version",
|
||||
Complete: true,
|
||||
UsedAt: now.Unix(),
|
||||
CreatedAt: now.Add(-time.Hour).Unix(),
|
||||
},
|
||||
Kept: true,
|
||||
},
|
||||
{
|
||||
// should be removed, since it's not complete and not used for a while.
|
||||
Cache: &Cache{
|
||||
Key: "test_key_2",
|
||||
Version: "test_version",
|
||||
Complete: false,
|
||||
UsedAt: now.Add(-(keepTemp + time.Second)).Unix(),
|
||||
CreatedAt: now.Add(-(keepTemp + time.Hour)).Unix(),
|
||||
},
|
||||
Kept: false,
|
||||
},
|
||||
{
|
||||
// should be removed, since it's not used for a while.
|
||||
Cache: &Cache{
|
||||
Key: "test_key_3",
|
||||
Version: "test_version",
|
||||
Complete: true,
|
||||
UsedAt: now.Add(-(keepUnused + time.Second)).Unix(),
|
||||
CreatedAt: now.Add(-(keepUnused + time.Hour)).Unix(),
|
||||
},
|
||||
Kept: false,
|
||||
},
|
||||
{
|
||||
// should be removed, since it's used but too old.
|
||||
Cache: &Cache{
|
||||
Key: "test_key_3",
|
||||
Version: "test_version",
|
||||
Complete: true,
|
||||
UsedAt: now.Unix(),
|
||||
CreatedAt: now.Add(-(keepUsed + time.Second)).Unix(),
|
||||
},
|
||||
Kept: false,
|
||||
},
|
||||
{
|
||||
// should be kept, since it has a newer edition but be used recently.
|
||||
Cache: &Cache{
|
||||
Key: "test_key_1",
|
||||
Version: "test_version",
|
||||
Complete: true,
|
||||
UsedAt: now.Add(-(keepOld - time.Minute)).Unix(),
|
||||
CreatedAt: now.Add(-(time.Hour + time.Second)).Unix(),
|
||||
},
|
||||
Kept: true,
|
||||
},
|
||||
{
|
||||
// should be removed, since it has a newer edition and not be used recently.
|
||||
Cache: &Cache{
|
||||
Key: "test_key_1",
|
||||
Version: "test_version",
|
||||
Complete: true,
|
||||
UsedAt: now.Add(-(keepOld + time.Second)).Unix(),
|
||||
CreatedAt: now.Add(-(time.Hour + time.Second)).Unix(),
|
||||
},
|
||||
Kept: false,
|
||||
},
|
||||
}
|
||||
|
||||
db, err := handler.openDB()
|
||||
require.NoError(t, err)
|
||||
for _, c := range cases {
|
||||
require.NoError(t, insertCache(db, c.Cache))
|
||||
}
|
||||
require.NoError(t, db.Close())
|
||||
|
||||
handler.gcAt = time.Time{} // ensure gcCache will not skip
|
||||
handler.gcCache()
|
||||
|
||||
db, err = handler.openDB()
|
||||
require.NoError(t, err)
|
||||
for i, v := range cases {
|
||||
t.Run(fmt.Sprintf("%d_%s", i, v.Cache.Key), func(t *testing.T) {
|
||||
cache := &Cache{}
|
||||
err = db.Get(v.Cache.ID, cache)
|
||||
if v.Kept {
|
||||
assert.NoError(t, err)
|
||||
} else {
|
||||
assert.ErrorIs(t, err, bolthold.ErrNotFound)
|
||||
}
|
||||
})
|
||||
}
|
||||
require.NoError(t, db.Close())
|
||||
}
|
||||
|
||||
@@ -1,10 +1,5 @@
|
||||
package artifactcache
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
type Request struct {
|
||||
Key string `json:"key" `
|
||||
Version string `json:"version"`
|
||||
@@ -29,16 +24,11 @@ func (c *Request) ToCache() *Cache {
|
||||
}
|
||||
|
||||
type Cache struct {
|
||||
ID uint64 `json:"id" boltholdKey:"ID"`
|
||||
Key string `json:"key" boltholdIndex:"Key"`
|
||||
Version string `json:"version" boltholdIndex:"Version"`
|
||||
KeyVersionHash string `json:"keyVersionHash" boltholdUnique:"KeyVersionHash"`
|
||||
Size int64 `json:"cacheSize"`
|
||||
Complete bool `json:"complete"`
|
||||
UsedAt int64 `json:"usedAt" boltholdIndex:"UsedAt"`
|
||||
CreatedAt int64 `json:"createdAt" boltholdIndex:"CreatedAt"`
|
||||
}
|
||||
|
||||
func (c *Cache) FillKeyVersionHash() {
|
||||
c.KeyVersionHash = fmt.Sprintf("%x", sha256.Sum256([]byte(fmt.Sprintf("%s:%s", c.Key, c.Version))))
|
||||
ID uint64 `json:"id" boltholdKey:"ID"`
|
||||
Key string `json:"key" boltholdIndex:"Key"`
|
||||
Version string `json:"version" boltholdIndex:"Version"`
|
||||
Size int64 `json:"cacheSize"`
|
||||
Complete bool `json:"complete" boltholdIndex:"Complete"`
|
||||
UsedAt int64 `json:"usedAt" boltholdIndex:"UsedAt"`
|
||||
CreatedAt int64 `json:"createdAt" boltholdIndex:"CreatedAt"`
|
||||
}
|
||||
|
||||
@@ -97,7 +97,7 @@ func NewParallelExecutor(parallel int, executors ...Executor) Executor {
|
||||
errs := make(chan error, len(executors))
|
||||
|
||||
if 1 > parallel {
|
||||
log.Infof("Parallel tasks (%d) below minimum, setting to 1", parallel)
|
||||
log.Debugf("Parallel tasks (%d) below minimum, setting to 1", parallel)
|
||||
parallel = 1
|
||||
}
|
||||
|
||||
|
||||
@@ -679,7 +679,7 @@ func (cr *containerReference) CopyTarStream(ctx context.Context, destPath string
|
||||
tw := tar.NewWriter(buf)
|
||||
_ = tw.WriteHeader(&tar.Header{
|
||||
Name: destPath,
|
||||
Mode: 777,
|
||||
Mode: 0o777,
|
||||
Typeflag: tar.TypeDir,
|
||||
})
|
||||
tw.Close()
|
||||
|
||||
@@ -2,7 +2,9 @@ package container
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"strings"
|
||||
@@ -75,6 +77,11 @@ func (m *mockDockerClient) ContainerExecInspect(ctx context.Context, execID stri
|
||||
return args.Get(0).(types.ContainerExecInspect), args.Error(1)
|
||||
}
|
||||
|
||||
func (m *mockDockerClient) CopyToContainer(ctx context.Context, id string, path string, content io.Reader, options types.CopyToContainerOptions) error {
|
||||
args := m.Called(ctx, id, path, content, options)
|
||||
return args.Error(0)
|
||||
}
|
||||
|
||||
type endlessReader struct {
|
||||
io.Reader
|
||||
}
|
||||
@@ -165,5 +172,77 @@ func TestDockerExecFailure(t *testing.T) {
|
||||
client.AssertExpectations(t)
|
||||
}
|
||||
|
||||
func TestDockerCopyTarStream(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
|
||||
conn := &mockConn{}
|
||||
|
||||
client := &mockDockerClient{}
|
||||
client.On("CopyToContainer", ctx, "123", "/", mock.Anything, mock.AnythingOfType("types.CopyToContainerOptions")).Return(nil)
|
||||
client.On("CopyToContainer", ctx, "123", "/var/run/act", mock.Anything, mock.AnythingOfType("types.CopyToContainerOptions")).Return(nil)
|
||||
cr := &containerReference{
|
||||
id: "123",
|
||||
cli: client,
|
||||
input: &NewContainerInput{
|
||||
Image: "image",
|
||||
},
|
||||
}
|
||||
|
||||
_ = cr.CopyTarStream(ctx, "/var/run/act", &bytes.Buffer{})
|
||||
|
||||
conn.AssertExpectations(t)
|
||||
client.AssertExpectations(t)
|
||||
}
|
||||
|
||||
func TestDockerCopyTarStreamErrorInCopyFiles(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
|
||||
conn := &mockConn{}
|
||||
|
||||
merr := fmt.Errorf("Failure")
|
||||
|
||||
client := &mockDockerClient{}
|
||||
client.On("CopyToContainer", ctx, "123", "/", mock.Anything, mock.AnythingOfType("types.CopyToContainerOptions")).Return(merr)
|
||||
client.On("CopyToContainer", ctx, "123", "/", mock.Anything, mock.AnythingOfType("types.CopyToContainerOptions")).Return(merr)
|
||||
cr := &containerReference{
|
||||
id: "123",
|
||||
cli: client,
|
||||
input: &NewContainerInput{
|
||||
Image: "image",
|
||||
},
|
||||
}
|
||||
|
||||
err := cr.CopyTarStream(ctx, "/var/run/act", &bytes.Buffer{})
|
||||
assert.ErrorIs(t, err, merr)
|
||||
|
||||
conn.AssertExpectations(t)
|
||||
client.AssertExpectations(t)
|
||||
}
|
||||
|
||||
func TestDockerCopyTarStreamErrorInMkdir(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
|
||||
conn := &mockConn{}
|
||||
|
||||
merr := fmt.Errorf("Failure")
|
||||
|
||||
client := &mockDockerClient{}
|
||||
client.On("CopyToContainer", ctx, "123", "/", mock.Anything, mock.AnythingOfType("types.CopyToContainerOptions")).Return(nil)
|
||||
client.On("CopyToContainer", ctx, "123", "/var/run/act", mock.Anything, mock.AnythingOfType("types.CopyToContainerOptions")).Return(merr)
|
||||
cr := &containerReference{
|
||||
id: "123",
|
||||
cli: client,
|
||||
input: &NewContainerInput{
|
||||
Image: "image",
|
||||
},
|
||||
}
|
||||
|
||||
err := cr.CopyTarStream(ctx, "/var/run/act", &bytes.Buffer{})
|
||||
assert.ErrorIs(t, err, merr)
|
||||
|
||||
conn.AssertExpectations(t)
|
||||
client.AssertExpectations(t)
|
||||
}
|
||||
|
||||
// Type assert containerReference implements ExecutionsEnvironment
|
||||
var _ ExecutionsEnvironment = &containerReference{}
|
||||
|
||||
134
pkg/container/docker_socket.go
Normal file
134
pkg/container/docker_socket.go
Normal file
@@ -0,0 +1,134 @@
|
||||
package container
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
var CommonSocketLocations = []string{
|
||||
"/var/run/docker.sock",
|
||||
"/run/podman/podman.sock",
|
||||
"$HOME/.colima/docker.sock",
|
||||
"$XDG_RUNTIME_DIR/docker.sock",
|
||||
"$XDG_RUNTIME_DIR/podman/podman.sock",
|
||||
`\\.\pipe\docker_engine`,
|
||||
"$HOME/.docker/run/docker.sock",
|
||||
}
|
||||
|
||||
// returns socket URI or false if not found any
|
||||
func socketLocation() (string, bool) {
|
||||
if dockerHost, exists := os.LookupEnv("DOCKER_HOST"); exists {
|
||||
return dockerHost, true
|
||||
}
|
||||
|
||||
for _, p := range CommonSocketLocations {
|
||||
if _, err := os.Lstat(os.ExpandEnv(p)); err == nil {
|
||||
if strings.HasPrefix(p, `\\.\`) {
|
||||
return "npipe://" + filepath.ToSlash(os.ExpandEnv(p)), true
|
||||
}
|
||||
return "unix://" + filepath.ToSlash(os.ExpandEnv(p)), true
|
||||
}
|
||||
}
|
||||
|
||||
return "", false
|
||||
}
|
||||
|
||||
// This function, `isDockerHostURI`, takes a string argument `daemonPath`. It checks if the
|
||||
// `daemonPath` is a valid Docker host URI. It does this by checking if the scheme of the URI (the
|
||||
// part before "://") contains only alphabetic characters. If it does, the function returns true,
|
||||
// indicating that the `daemonPath` is a Docker host URI. If it doesn't, or if the "://" delimiter
|
||||
// is not found in the `daemonPath`, the function returns false.
|
||||
func isDockerHostURI(daemonPath string) bool {
|
||||
if protoIndex := strings.Index(daemonPath, "://"); protoIndex != -1 {
|
||||
scheme := daemonPath[:protoIndex]
|
||||
if strings.IndexFunc(scheme, func(r rune) bool {
|
||||
return (r < 'a' || r > 'z') && (r < 'A' || r > 'Z')
|
||||
}) == -1 {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
type SocketAndHost struct {
|
||||
Socket string
|
||||
Host string
|
||||
}
|
||||
|
||||
func GetSocketAndHost(containerSocket string) (SocketAndHost, error) {
|
||||
log.Debugf("Handling container host and socket")
|
||||
|
||||
// Prefer DOCKER_HOST, don't override it
|
||||
dockerHost, hasDockerHost := socketLocation()
|
||||
socketHost := SocketAndHost{Socket: containerSocket, Host: dockerHost}
|
||||
|
||||
// ** socketHost.Socket cases **
|
||||
// Case 1: User does _not_ want to mount a daemon socket (passes a dash)
|
||||
// Case 2: User passes a filepath to the socket; is that even valid?
|
||||
// Case 3: User passes a valid socket; do nothing
|
||||
// Case 4: User omitted the flag; set a sane default
|
||||
|
||||
// ** DOCKER_HOST cases **
|
||||
// Case A: DOCKER_HOST is set; use it, i.e. do nothing
|
||||
// Case B: DOCKER_HOST is empty; use sane defaults
|
||||
|
||||
// Set host for sanity's sake, when the socket isn't useful
|
||||
if !hasDockerHost && (socketHost.Socket == "-" || !isDockerHostURI(socketHost.Socket) || socketHost.Socket == "") {
|
||||
// Cases: 1B, 2B, 4B
|
||||
socket, found := socketLocation()
|
||||
socketHost.Host = socket
|
||||
hasDockerHost = found
|
||||
}
|
||||
|
||||
// A - (dash) in socketHost.Socket means don't mount, preserve this value
|
||||
// otherwise if socketHost.Socket is a filepath don't use it as socket
|
||||
// Exit early if we're in an invalid state (e.g. when no DOCKER_HOST and user supplied "-", a dash or omitted)
|
||||
if !hasDockerHost && socketHost.Socket != "" && !isDockerHostURI(socketHost.Socket) {
|
||||
// Cases: 1B, 2B
|
||||
// Should we early-exit here, since there is no host nor socket to talk to?
|
||||
return SocketAndHost{}, fmt.Errorf("DOCKER_HOST was not set, couldn't be found in the usual locations, and the container daemon socket ('%s') is invalid", socketHost.Socket)
|
||||
}
|
||||
|
||||
// Default to DOCKER_HOST if set
|
||||
if socketHost.Socket == "" && hasDockerHost {
|
||||
// Cases: 4A
|
||||
log.Debugf("Defaulting container socket to DOCKER_HOST")
|
||||
socketHost.Socket = socketHost.Host
|
||||
}
|
||||
// Set sane default socket location if user omitted it
|
||||
if socketHost.Socket == "" {
|
||||
// Cases: 4B
|
||||
socket, _ := socketLocation()
|
||||
// socket is empty if it isn't found, so assignment here is at worst a no-op
|
||||
log.Debugf("Defaulting container socket to default '%s'", socket)
|
||||
socketHost.Socket = socket
|
||||
}
|
||||
|
||||
// Exit if both the DOCKER_HOST and socket are fulfilled
|
||||
if hasDockerHost {
|
||||
// Cases: 1A, 2A, 3A, 4A
|
||||
if !isDockerHostURI(socketHost.Socket) {
|
||||
// Cases: 1A, 2A
|
||||
log.Debugf("DOCKER_HOST is set, but socket is invalid '%s'", socketHost.Socket)
|
||||
}
|
||||
return socketHost, nil
|
||||
}
|
||||
|
||||
// Set a sane DOCKER_HOST default if we can
|
||||
if isDockerHostURI(socketHost.Socket) {
|
||||
// Cases: 3B
|
||||
log.Debugf("Setting DOCKER_HOST to container socket '%s'", socketHost.Socket)
|
||||
socketHost.Host = socketHost.Socket
|
||||
// Both DOCKER_HOST and container socket are valid; short-circuit exit
|
||||
return socketHost, nil
|
||||
}
|
||||
|
||||
// Here there is no DOCKER_HOST _and_ the supplied container socket is not a valid URI (either invalid or a file path)
|
||||
// Cases: 2B <- but is already handled at the top
|
||||
// I.e. this path should never be taken
|
||||
return SocketAndHost{}, fmt.Errorf("no DOCKER_HOST and an invalid container socket '%s'", socketHost.Socket)
|
||||
}
|
||||
150
pkg/container/docker_socket_test.go
Normal file
150
pkg/container/docker_socket_test.go
Normal file
@@ -0,0 +1,150 @@
|
||||
package container
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
log "github.com/sirupsen/logrus"
|
||||
assert "github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func init() {
|
||||
log.SetLevel(log.DebugLevel)
|
||||
}
|
||||
|
||||
var originalCommonSocketLocations = CommonSocketLocations
|
||||
|
||||
func TestGetSocketAndHostWithSocket(t *testing.T) {
|
||||
// Arrange
|
||||
CommonSocketLocations = originalCommonSocketLocations
|
||||
dockerHost := "unix:///my/docker/host.sock"
|
||||
socketURI := "/path/to/my.socket"
|
||||
os.Setenv("DOCKER_HOST", dockerHost)
|
||||
|
||||
// Act
|
||||
ret, err := GetSocketAndHost(socketURI)
|
||||
|
||||
// Assert
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, SocketAndHost{socketURI, dockerHost}, ret)
|
||||
}
|
||||
|
||||
func TestGetSocketAndHostNoSocket(t *testing.T) {
|
||||
// Arrange
|
||||
dockerHost := "unix:///my/docker/host.sock"
|
||||
os.Setenv("DOCKER_HOST", dockerHost)
|
||||
|
||||
// Act
|
||||
ret, err := GetSocketAndHost("")
|
||||
|
||||
// Assert
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, SocketAndHost{dockerHost, dockerHost}, ret)
|
||||
}
|
||||
|
||||
func TestGetSocketAndHostOnlySocket(t *testing.T) {
|
||||
// Arrange
|
||||
socketURI := "/path/to/my.socket"
|
||||
os.Unsetenv("DOCKER_HOST")
|
||||
CommonSocketLocations = originalCommonSocketLocations
|
||||
defaultSocket, defaultSocketFound := socketLocation()
|
||||
|
||||
// Act
|
||||
ret, err := GetSocketAndHost(socketURI)
|
||||
|
||||
// Assert
|
||||
assert.NoError(t, err, "Expected no error from GetSocketAndHost")
|
||||
assert.Equal(t, true, defaultSocketFound, "Expected to find default socket")
|
||||
assert.Equal(t, socketURI, ret.Socket, "Expected socket to match common location")
|
||||
assert.Equal(t, defaultSocket, ret.Host, "Expected ret.Host to match default socket location")
|
||||
}
|
||||
|
||||
func TestGetSocketAndHostDontMount(t *testing.T) {
|
||||
// Arrange
|
||||
CommonSocketLocations = originalCommonSocketLocations
|
||||
dockerHost := "unix:///my/docker/host.sock"
|
||||
os.Setenv("DOCKER_HOST", dockerHost)
|
||||
|
||||
// Act
|
||||
ret, err := GetSocketAndHost("-")
|
||||
|
||||
// Assert
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, SocketAndHost{"-", dockerHost}, ret)
|
||||
}
|
||||
|
||||
func TestGetSocketAndHostNoHostNoSocket(t *testing.T) {
|
||||
// Arrange
|
||||
CommonSocketLocations = originalCommonSocketLocations
|
||||
os.Unsetenv("DOCKER_HOST")
|
||||
defaultSocket, found := socketLocation()
|
||||
|
||||
// Act
|
||||
ret, err := GetSocketAndHost("")
|
||||
|
||||
// Assert
|
||||
assert.Equal(t, true, found, "Expected a default socket to be found")
|
||||
assert.Nil(t, err, "Expected no error from GetSocketAndHost")
|
||||
assert.Equal(t, SocketAndHost{defaultSocket, defaultSocket}, ret, "Expected to match default socket location")
|
||||
}
|
||||
|
||||
// Catch
|
||||
// > Your code breaks setting DOCKER_HOST if shouldMount is false.
|
||||
// > This happens if neither DOCKER_HOST nor --container-daemon-socket has a value, but socketLocation() returns a URI
|
||||
func TestGetSocketAndHostNoHostNoSocketDefaultLocation(t *testing.T) {
|
||||
// Arrange
|
||||
mySocketFile, tmpErr := os.CreateTemp("", "act-*.sock")
|
||||
mySocket := mySocketFile.Name()
|
||||
unixSocket := "unix://" + mySocket
|
||||
defer os.RemoveAll(mySocket)
|
||||
assert.NoError(t, tmpErr)
|
||||
os.Unsetenv("DOCKER_HOST")
|
||||
|
||||
CommonSocketLocations = []string{mySocket}
|
||||
defaultSocket, found := socketLocation()
|
||||
|
||||
// Act
|
||||
ret, err := GetSocketAndHost("")
|
||||
|
||||
// Assert
|
||||
assert.Equal(t, unixSocket, defaultSocket, "Expected default socket to match common socket location")
|
||||
assert.Equal(t, true, found, "Expected default socket to be found")
|
||||
assert.Nil(t, err, "Expected no error from GetSocketAndHost")
|
||||
assert.Equal(t, SocketAndHost{unixSocket, unixSocket}, ret, "Expected to match default socket location")
|
||||
}
|
||||
|
||||
func TestGetSocketAndHostNoHostInvalidSocket(t *testing.T) {
|
||||
// Arrange
|
||||
os.Unsetenv("DOCKER_HOST")
|
||||
mySocket := "/my/socket/path.sock"
|
||||
CommonSocketLocations = []string{"/unusual", "/socket", "/location"}
|
||||
defaultSocket, found := socketLocation()
|
||||
|
||||
// Act
|
||||
ret, err := GetSocketAndHost(mySocket)
|
||||
|
||||
// Assert
|
||||
assert.Equal(t, false, found, "Expected no default socket to be found")
|
||||
assert.Equal(t, "", defaultSocket, "Expected no default socket to be found")
|
||||
assert.Equal(t, SocketAndHost{}, ret, "Expected to match default socket location")
|
||||
assert.Error(t, err, "Expected an error in invalid state")
|
||||
}
|
||||
|
||||
func TestGetSocketAndHostOnlySocketValidButUnusualLocation(t *testing.T) {
|
||||
// Arrange
|
||||
socketURI := "unix:///path/to/my.socket"
|
||||
CommonSocketLocations = []string{"/unusual", "/location"}
|
||||
os.Unsetenv("DOCKER_HOST")
|
||||
defaultSocket, found := socketLocation()
|
||||
|
||||
// Act
|
||||
ret, err := GetSocketAndHost(socketURI)
|
||||
|
||||
// Assert
|
||||
// Default socket locations
|
||||
assert.Equal(t, "", defaultSocket, "Expect default socket location to be empty")
|
||||
assert.Equal(t, false, found, "Expected no default socket to be found")
|
||||
// Sane default
|
||||
assert.Nil(t, err, "Expect no error from GetSocketAndHost")
|
||||
assert.Equal(t, socketURI, ret.Host, "Expect host to default to unusual socket")
|
||||
}
|
||||
@@ -343,7 +343,7 @@ func environment(yml yaml.Node) map[string]string {
|
||||
return env
|
||||
}
|
||||
|
||||
// Environments returns string-based key=value map for a job
|
||||
// Environment returns string-based key=value map for a job
|
||||
func (j *Job) Environment() map[string]string {
|
||||
return environment(j.Env)
|
||||
}
|
||||
@@ -578,7 +578,7 @@ func (s *Step) String() string {
|
||||
return s.ID
|
||||
}
|
||||
|
||||
// Environments returns string-based key=value map for a step
|
||||
// Environment returns string-based key=value map for a step
|
||||
func (s *Step) Environment() map[string]string {
|
||||
return environment(s.Env)
|
||||
}
|
||||
|
||||
@@ -6,7 +6,6 @@ import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
"path"
|
||||
@@ -43,17 +42,7 @@ func (c GoGitActionCache) Fetch(ctx context.Context, cacheDir, url, ref, token s
|
||||
return "", err
|
||||
}
|
||||
branchName := hex.EncodeToString(tmpBranch)
|
||||
var refSpec config.RefSpec
|
||||
spec := config.RefSpec(ref + ":" + branchName)
|
||||
tagOrSha := false
|
||||
if spec.IsExactSHA1() {
|
||||
refSpec = spec
|
||||
} else if strings.HasPrefix(ref, "refs/") {
|
||||
refSpec = config.RefSpec(ref + ":refs/heads/" + branchName)
|
||||
} else {
|
||||
tagOrSha = true
|
||||
refSpec = config.RefSpec("refs/*/" + ref + ":refs/heads/*/" + branchName)
|
||||
}
|
||||
|
||||
var auth transport.AuthMethod
|
||||
if token != "" {
|
||||
auth = &http.BasicAuth{
|
||||
@@ -71,35 +60,17 @@ func (c GoGitActionCache) Fetch(ctx context.Context, cacheDir, url, ref, token s
|
||||
return "", err
|
||||
}
|
||||
defer func() {
|
||||
if refs, err := gogitrepo.References(); err == nil {
|
||||
_ = refs.ForEach(func(r *plumbing.Reference) error {
|
||||
if strings.Contains(r.Name().String(), branchName) {
|
||||
return gogitrepo.DeleteBranch(r.Name().String())
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
_ = gogitrepo.DeleteBranch(branchName)
|
||||
}()
|
||||
if err := remote.FetchContext(ctx, &git.FetchOptions{
|
||||
RefSpecs: []config.RefSpec{
|
||||
refSpec,
|
||||
config.RefSpec(ref + ":" + branchName),
|
||||
},
|
||||
Auth: auth,
|
||||
Force: true,
|
||||
}); err != nil {
|
||||
if tagOrSha && errors.Is(err, git.NoErrAlreadyUpToDate) {
|
||||
return "", fmt.Errorf("couldn't find remote ref \"%s\"", ref)
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
if tagOrSha {
|
||||
for _, prefix := range []string{"refs/heads/tags/", "refs/heads/heads/"} {
|
||||
hash, err := gogitrepo.ResolveRevision(plumbing.Revision(prefix + branchName))
|
||||
if err == nil {
|
||||
return hash.String(), nil
|
||||
}
|
||||
}
|
||||
}
|
||||
hash, err := gogitrepo.ResolveRevision(plumbing.Revision(branchName))
|
||||
if err != nil {
|
||||
return "", err
|
||||
|
||||
41
pkg/runner/action_cache_offline_mode.go
Normal file
41
pkg/runner/action_cache_offline_mode.go
Normal file
@@ -0,0 +1,41 @@
|
||||
package runner
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"path"
|
||||
|
||||
git "github.com/go-git/go-git/v5"
|
||||
"github.com/go-git/go-git/v5/plumbing"
|
||||
)
|
||||
|
||||
type GoGitActionCacheOfflineMode struct {
|
||||
Parent GoGitActionCache
|
||||
}
|
||||
|
||||
func (c GoGitActionCacheOfflineMode) Fetch(ctx context.Context, cacheDir, url, ref, token string) (string, error) {
|
||||
sha, fetchErr := c.Parent.Fetch(ctx, cacheDir, url, ref, token)
|
||||
gitPath := path.Join(c.Parent.Path, safeFilename(cacheDir)+".git")
|
||||
gogitrepo, err := git.PlainOpen(gitPath)
|
||||
if err != nil {
|
||||
return "", fetchErr
|
||||
}
|
||||
refName := plumbing.ReferenceName("refs/action-cache-offline/" + ref)
|
||||
r, err := gogitrepo.Reference(refName, true)
|
||||
if fetchErr == nil {
|
||||
if err != nil || sha != r.Hash().String() {
|
||||
if err == nil {
|
||||
refName = r.Name()
|
||||
}
|
||||
ref := plumbing.NewHashReference(refName, plumbing.NewHash(sha))
|
||||
_ = gogitrepo.Storer.SetReference(ref)
|
||||
}
|
||||
} else if err == nil {
|
||||
return r.Hash().String(), nil
|
||||
}
|
||||
return sha, fetchErr
|
||||
}
|
||||
|
||||
func (c GoGitActionCacheOfflineMode) GetTarArchive(ctx context.Context, cacheDir, sha, includePrefix string) (io.ReadCloser, error) {
|
||||
return c.Parent.GetTarArchive(ctx, cacheDir, sha, includePrefix)
|
||||
}
|
||||
@@ -18,20 +18,60 @@ func TestActionCache(t *testing.T) {
|
||||
Path: os.TempDir(),
|
||||
}
|
||||
ctx := context.Background()
|
||||
sha, err := cache.Fetch(ctx, "christopherhx/script", "https://github.com/christopherhx/script", "main", "")
|
||||
a.NoError(err)
|
||||
a.NotEmpty(sha)
|
||||
atar, err := cache.GetTarArchive(ctx, "christopherhx/script", sha, "node_modules")
|
||||
a.NoError(err)
|
||||
a.NotEmpty(atar)
|
||||
mytar := tar.NewReader(atar)
|
||||
th, err := mytar.Next()
|
||||
a.NoError(err)
|
||||
a.NotEqual(0, th.Size)
|
||||
buf := &bytes.Buffer{}
|
||||
// G110: Potential DoS vulnerability via decompression bomb (gosec)
|
||||
_, err = io.Copy(buf, mytar)
|
||||
a.NoError(err)
|
||||
str := buf.String()
|
||||
a.NotEmpty(str)
|
||||
cacheDir := "nektos/act-test-actions"
|
||||
repo := "https://github.com/nektos/act-test-actions"
|
||||
refs := []struct {
|
||||
Name string
|
||||
CacheDir string
|
||||
Repo string
|
||||
Ref string
|
||||
}{
|
||||
{
|
||||
Name: "Fetch Branch Name",
|
||||
CacheDir: cacheDir,
|
||||
Repo: repo,
|
||||
Ref: "main",
|
||||
},
|
||||
{
|
||||
Name: "Fetch Branch Name Absolutely",
|
||||
CacheDir: cacheDir,
|
||||
Repo: repo,
|
||||
Ref: "refs/heads/main",
|
||||
},
|
||||
{
|
||||
Name: "Fetch HEAD",
|
||||
CacheDir: cacheDir,
|
||||
Repo: repo,
|
||||
Ref: "HEAD",
|
||||
},
|
||||
{
|
||||
Name: "Fetch Sha",
|
||||
CacheDir: cacheDir,
|
||||
Repo: repo,
|
||||
Ref: "de984ca37e4df4cb9fd9256435a3b82c4a2662b1",
|
||||
},
|
||||
}
|
||||
for _, c := range refs {
|
||||
t.Run(c.Name, func(t *testing.T) {
|
||||
sha, err := cache.Fetch(ctx, c.CacheDir, c.Repo, c.Ref, "")
|
||||
if !a.NoError(err) || !a.NotEmpty(sha) {
|
||||
return
|
||||
}
|
||||
atar, err := cache.GetTarArchive(ctx, c.CacheDir, sha, "js")
|
||||
if !a.NoError(err) || !a.NotEmpty(atar) {
|
||||
return
|
||||
}
|
||||
mytar := tar.NewReader(atar)
|
||||
th, err := mytar.Next()
|
||||
if !a.NoError(err) || !a.NotEqual(0, th.Size) {
|
||||
return
|
||||
}
|
||||
buf := &bytes.Buffer{}
|
||||
// G110: Potential DoS vulnerability via decompression bomb (gosec)
|
||||
_, err = io.Copy(buf, mytar)
|
||||
a.NoError(err)
|
||||
str := buf.String()
|
||||
a.NotEmpty(str)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -27,7 +27,7 @@ func evaluateCompositeInputAndEnv(ctx context.Context, parent *RunContext, step
|
||||
envKey := regexp.MustCompile("[^A-Z0-9-]").ReplaceAllString(strings.ToUpper(inputID), "_")
|
||||
envKey = fmt.Sprintf("INPUT_%s", strings.ToUpper(envKey))
|
||||
|
||||
// lookup if key is defined in the step but the the already
|
||||
// lookup if key is defined in the step but the already
|
||||
// evaluated value from the environment
|
||||
_, defined := step.getStepModel().With[inputID]
|
||||
if value, ok := stepEnv[envKey]; defined && ok {
|
||||
|
||||
@@ -106,7 +106,7 @@ func (rc *RunContext) NewExpressionEvaluatorWithEnv(ctx context.Context, env map
|
||||
//go:embed hashfiles/index.js
|
||||
var hashfiles string
|
||||
|
||||
// NewExpressionEvaluator creates a new evaluator
|
||||
// NewStepExpressionEvaluator creates a new evaluator
|
||||
func (rc *RunContext) NewStepExpressionEvaluator(ctx context.Context, step step) ExpressionEvaluator {
|
||||
// todo: cleanup EvaluationEnvironment creation
|
||||
job := rc.Run.Job()
|
||||
|
||||
91
pkg/runner/local_repository_cache.go
Normal file
91
pkg/runner/local_repository_cache.go
Normal file
@@ -0,0 +1,91 @@
|
||||
package runner
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
goURL "net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/nektos/act/pkg/filecollector"
|
||||
)
|
||||
|
||||
type LocalRepositoryCache struct {
|
||||
Parent ActionCache
|
||||
LocalRepositories map[string]string
|
||||
CacheDirCache map[string]string
|
||||
}
|
||||
|
||||
func (l *LocalRepositoryCache) Fetch(ctx context.Context, cacheDir, url, ref, token string) (string, error) {
|
||||
if dest, ok := l.LocalRepositories[fmt.Sprintf("%s@%s", url, ref)]; ok {
|
||||
l.CacheDirCache[fmt.Sprintf("%s@%s", cacheDir, ref)] = dest
|
||||
return ref, nil
|
||||
}
|
||||
if purl, err := goURL.Parse(url); err == nil {
|
||||
if dest, ok := l.LocalRepositories[fmt.Sprintf("%s@%s", strings.TrimPrefix(purl.Path, "/"), ref)]; ok {
|
||||
l.CacheDirCache[fmt.Sprintf("%s@%s", cacheDir, ref)] = dest
|
||||
return ref, nil
|
||||
}
|
||||
}
|
||||
return l.Parent.Fetch(ctx, cacheDir, url, ref, token)
|
||||
}
|
||||
|
||||
func (l *LocalRepositoryCache) GetTarArchive(ctx context.Context, cacheDir, sha, includePrefix string) (io.ReadCloser, error) {
|
||||
// sha is mapped to ref in fetch if there is a local override
|
||||
if dest, ok := l.CacheDirCache[fmt.Sprintf("%s@%s", cacheDir, sha)]; ok {
|
||||
srcPath := filepath.Join(dest, includePrefix)
|
||||
buf := &bytes.Buffer{}
|
||||
tw := tar.NewWriter(buf)
|
||||
defer tw.Close()
|
||||
srcPath = filepath.Clean(srcPath)
|
||||
fi, err := os.Lstat(srcPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tc := &filecollector.TarCollector{
|
||||
TarWriter: tw,
|
||||
}
|
||||
if fi.IsDir() {
|
||||
srcPrefix := srcPath
|
||||
if !strings.HasSuffix(srcPrefix, string(filepath.Separator)) {
|
||||
srcPrefix += string(filepath.Separator)
|
||||
}
|
||||
fc := &filecollector.FileCollector{
|
||||
Fs: &filecollector.DefaultFs{},
|
||||
SrcPath: srcPath,
|
||||
SrcPrefix: srcPrefix,
|
||||
Handler: tc,
|
||||
}
|
||||
err = filepath.Walk(srcPath, fc.CollectFiles(ctx, []string{}))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else {
|
||||
var f io.ReadCloser
|
||||
var linkname string
|
||||
if fi.Mode()&fs.ModeSymlink != 0 {
|
||||
linkname, err = os.Readlink(srcPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else {
|
||||
f, err = os.Open(srcPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
}
|
||||
err := tc.WriteFile(fi.Name(), fi, linkname, f)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return io.NopCloser(buf), nil
|
||||
}
|
||||
return l.Parent.GetTarArchive(ctx, cacheDir, sha, includePrefix)
|
||||
}
|
||||
@@ -52,7 +52,7 @@ func Masks(ctx context.Context) *[]string {
|
||||
return &[]string{}
|
||||
}
|
||||
|
||||
// WithLogger adds a value to the context for the logger
|
||||
// WithMasks adds a value to the context for the logger
|
||||
func WithMasks(ctx context.Context, masks *[]string) context.Context {
|
||||
return context.WithValue(ctx, masksContextKeyVal, masks)
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
@@ -14,6 +15,7 @@ import (
|
||||
"github.com/joho/godotenv"
|
||||
log "github.com/sirupsen/logrus"
|
||||
assert "github.com/stretchr/testify/assert"
|
||||
"gopkg.in/yaml.v3"
|
||||
|
||||
"github.com/nektos/act/pkg/common"
|
||||
"github.com/nektos/act/pkg/model"
|
||||
@@ -187,6 +189,7 @@ func (j *TestJobFileInfo) runTest(ctx context.Context, t *testing.T, cfg *Config
|
||||
GitHubInstance: "github.com",
|
||||
ContainerArchitecture: cfg.ContainerArchitecture,
|
||||
Matrix: cfg.Matrix,
|
||||
ActionCache: cfg.ActionCache,
|
||||
}
|
||||
|
||||
runner, err := New(runnerConfig)
|
||||
@@ -209,6 +212,10 @@ func (j *TestJobFileInfo) runTest(ctx context.Context, t *testing.T, cfg *Config
|
||||
fmt.Println("::endgroup::")
|
||||
}
|
||||
|
||||
type TestConfig struct {
|
||||
LocalRepositories map[string]string `yaml:"local-repositories"`
|
||||
}
|
||||
|
||||
func TestRunEvent(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("skipping integration test")
|
||||
@@ -307,6 +314,9 @@ func TestRunEvent(t *testing.T) {
|
||||
{workdir, "services", "push", "", platforms, secrets},
|
||||
{workdir, "services-host-network", "push", "", platforms, secrets},
|
||||
{workdir, "services-with-container", "push", "", platforms, secrets},
|
||||
|
||||
// local remote action overrides
|
||||
{workdir, "local-remote-action-overrides", "push", "", platforms, secrets},
|
||||
}
|
||||
|
||||
for _, table := range tables {
|
||||
@@ -320,6 +330,22 @@ func TestRunEvent(t *testing.T) {
|
||||
config.EventPath = eventFile
|
||||
}
|
||||
|
||||
testConfigFile := filepath.Join(workdir, table.workflowPath, "config.yml")
|
||||
if file, err := os.ReadFile(testConfigFile); err == nil {
|
||||
testConfig := &TestConfig{}
|
||||
if yaml.Unmarshal(file, testConfig) == nil {
|
||||
if testConfig.LocalRepositories != nil {
|
||||
config.ActionCache = &LocalRepositoryCache{
|
||||
Parent: GoGitActionCache{
|
||||
path.Clean(path.Join(workdir, "cache")),
|
||||
},
|
||||
LocalRepositories: testConfig.LocalRepositories,
|
||||
CacheDirCache: map[string]string{},
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
table.runTest(ctx, t, config)
|
||||
})
|
||||
}
|
||||
|
||||
3
pkg/runner/testdata/local-remote-action-overrides/config.yml
vendored
Normal file
3
pkg/runner/testdata/local-remote-action-overrides/config.yml
vendored
Normal file
@@ -0,0 +1,3 @@
|
||||
local-repositories:
|
||||
https://github.com/nektos/test-override@a: testdata/actions/node20
|
||||
nektos/test-override@b: testdata/actions/node16
|
||||
9
pkg/runner/testdata/local-remote-action-overrides/push.yml
vendored
Normal file
9
pkg/runner/testdata/local-remote-action-overrides/push.yml
vendored
Normal file
@@ -0,0 +1,9 @@
|
||||
name: basic
|
||||
on: push
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: nektos/test-override@a
|
||||
- uses: nektos/test-override@b
|
||||
4
pkg/runner/testdata/networking/push.yml
vendored
4
pkg/runner/testdata/networking/push.yml
vendored
@@ -7,8 +7,8 @@ jobs:
|
||||
- name: Install tools
|
||||
run: |
|
||||
apt update
|
||||
apt install -y bind9-host
|
||||
apt install -y iputils-ping
|
||||
- name: Run hostname test
|
||||
run: |
|
||||
hostname -f
|
||||
host $(hostname -f)
|
||||
ping -c 4 $(hostname -f)
|
||||
|
||||
Reference in New Issue
Block a user