Support go plugins for forges and agent backends (#2751)

As of #2520 

Support to load new forges and agent backends at runtime using go's
plugin system. (https://pkg.go.dev/plugin)

I also added a simple example addon (a new forge which just prints log
statements), it should be removed later of course, but you can see an
example.

---------

Co-authored-by: Michalis Zampetakis <mzampetakis@gmail.com>
Co-authored-by: Anbraten <anton@ju60.de>

docs/docs/30-administration/10-server-config.md

@@ -521,6 +521,12 @@ Supported variables:
 - `owner`: the repo's owner
 - `repo`: the repo's name
 
+### WOODPECKER_ADDONS
+
+> Default: empty
+
+List of addon files. See [addons](./75-addons/00-overview.md).
+
 ---
 
 ### `WOODPECKER_LIMIT_MEM_SWAP`

docs/docs/30-administration/15-agent-config.md

@@ -178,6 +178,12 @@ Configures if the gRPC server certificate should be verified, only valid when `W
 
 Configures the backend engine to run pipelines on. Possible values are `auto-detect`, `docker`, `local` or `kubernetes`.
 
+### WOODPECKER_ADDONS
+
+> Default: empty
+
+List of addon files. See [addons](./75-addons/00-overview.md).
+
 ### `WOODPECKER_BACKEND_DOCKER_*`
 
 See [Docker backend configuration](./22-backends/10-docker.md#configuration)

docs/docs/30-administration/75-addons/00-overview.md

@@ -0,0 +1,48 @@
+# Addons
+
+:::warning
+Addons are still experimental. Their implementation can change and break at any time.
+:::
+
+:::danger
+You need to trust the author of the addons you use. Depending on their type, addons can access forge authentication codes, your secrets or other sensitive information.
+:::
+
+To adapt Woodpecker to your needs beyond the [configuration](../10-server-config.md), Woodpecker has its own **addon** system, built ontop of [Go's internal plugin system](https://go.dev/pkg/plugin).
+
+Addons can be used for:
+
+- Forges
+- Agent backends
+
+## Restrictions
+
+Addons are restricted by how Go plugins work. This includes the following restrictions:
+
+- only supported on Linux, FreeBSD and macOS
+- addons must have been built for the correct Woodpecker version. If an addon is not provided specifically for this version, you likely won't be able to use it.
+
+## Usage
+
+To use an addon, download the addon version built for your Woodpecker version. Then, you can add the following to your configuration:
+
+```diff
+# docker-compose.yml
+version: '3'
+
+services:
+  woodpecker-server:
+    [...]
+    environment:
++     - WOODPECKER_ADDONS=/path/to/your/addon/file.so
+```
+
+In case you run Woodpecker as container, you probably want to mount the addon binaries to `/opt/addons/`.
+
+You can list multiple addons, Woodpecker will automatically determine their type. If you specify multiple addons with the same type, only the first one will be used.
+
+Using an addon always overwrites Woodpecker's internal setup. This means, that a forge addon will be used if specified, no matter what's configured for the forges natively supported by Woodpecker.
+
+### Bug reports
+
+If you experience bugs, please check which component has the issue. If it's the addon, **do not raise an issue in the main repository**, but rather use the separate addon repositories. To check which component is responsible for the bug, look at the logs. Logs from addons are marked with a special field `addon` containing their addon file name.

docs/docs/30-administration/75-addons/20-creating-addons.md

@@ -0,0 +1,88 @@
+# Creating addons
+
+Addons are written in Go.
+
+## Writing your code
+
+An addon consists of two variables/functions in Go.
+
+1. The `Type` variable. Specifies the type of the addon and must be directly accessed from `shared/addons/types/types.go`.
+2. The `Addon` function which is the main point of your addon.
+   This function takes two arguments:
+
+   1. The zerolog logger you should use to log errors, warnings etc.
+   2. A slice of strings with the environment variables used as configuration.
+
+   It returns two values:
+
+   1. The actual addon. For type reference see [table below](#return-types).
+   2. An error. If this error is not `nil`, Woodpecker exits.
+
+Directly import Woodpecker's Go package (`go.woodpecker-ci.org/woodpecker/woodpecker/v2`) and use the interfaces and types defined there.
+
+### Return types
+
+| Addon type | Return type                                                                      |
+| ---------- | -------------------------------------------------------------------------------- |
+| `Forge`    | `"go.woodpecker-ci.org/woodpecker/woodpecker/v2/server/forge".Forge`             |
+| `Backend`  | `"go.woodpecker-ci.org/woodpecker/woodpecker/v2/pipeline/backend/types".Backend` |
+
+## Compiling
+
+After you write your addon code, compile your addon:
+
+```sh
+go build -buildmode plugin
+```
+
+The output file is your addon which is now ready to be used.
+
+## Restrictions
+
+Addons must directly depend on Woodpecker's core (`go.woodpecker-ci.org/woodpecker/woodpecker/v2`).
+The addon must have been built with **excatly the same code** as the Woodpecker instance you'd like to use it on. This means: If you build your addon with a specific commit from Woodpecker `next`, you can likely only use it with the Woodpecker version compiled from this commit.
+Also, if you change something inside of Woodpecker without committing, it might fail because you need to recompile your addon with this code first.
+
+In addition to this, addons are only supported on Linux, FreeBSD and macOS.
+
+:::info
+It is recommended to at least support the latest released version of Woodpecker.
+:::
+
+### Compile for different versions
+
+As long as there are no changes to Woodpecker's interfaces or they are backwards-compatible, you can easily compile the addon for multiple version by changing the version of `go.woodpecker-ci.org/woodpecker/woodpecker/v2` using `go get` before compiling.
+
+## Logging
+
+The entrypoint receives a `zerolog.Logger` as input. **Do not use any other logging solution.** This logger follows the configuration of the Woodpecker instance and adds a special field `addon` to the log entries which allows users to find out which component is writing the log messages.
+
+## Example structure
+
+```go
+package main
+
+import (
+  "context"
+  "net/http"
+
+  "github.com/rs/zerolog"
+  "go.woodpecker-ci.org/woodpecker/woodpecker/v2/server/forge"
+  forge_types "go.woodpecker-ci.org/woodpecker/woodpecker/v2/server/forge/types"
+  "go.woodpecker-ci.org/woodpecker/woodpecker/v2/server/model"
+  addon_types "go.woodpecker-ci.org/woodpecker/woodpecker/v2/shared/addon/types"
+)
+
+var Type = addon_types.TypeForge
+
+func Addon(logger zerolog.Logger, env []string) (forge.Forge, error) {
+  logger.Info().Msg("hello world from addon")
+  return &config{l: logger}, nil
+}
+
+type config struct {
+  l zerolog.Logger
+}
+
+// ... in this case, `config` must implement `forge.Forge`. You must directly use Woodpecker's packages - see imports above.
+```

docs/docs/30-administration/75-addons/_category_.yml

@@ -0,0 +1,6 @@
+label: 'Addons'
+collapsible: true
+collapsed: true
+link:
+  type: 'doc'
+  id: 'overview'