From f3bd63fb21d16a585d0db762cde3ba95eaf82c01 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 13:50:47 +0300 Subject: [PATCH 01/34] tweaking... Signed-off-by: Alex A. Naanou --- config.global.example | 3 --- host/make.sh | 4 +++- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/config.global.example b/config.global.example index 8ad34f1..52f7783 100644 --- a/config.global.example +++ b/config.global.example @@ -23,8 +23,6 @@ # Usually this is the default bridge created in Proxmox, so there is no # need to touch this. BOOTSTRAP_BRIDGE=0 -# XXX -#BOOTSTRAP_PORT=none # CT interface bridge configuration. @@ -43,7 +41,6 @@ BOOTSTRAP_BRIDGE=0 # ADMIN_BRIDGE=3 # LAN_BRIDGE=10 # -# XXX revise numbering... ADMIN_BRIDGE=_admin WAN_BRIDGE=_wan LAN_BRIDGE=_lan diff --git a/host/make.sh b/host/make.sh index e16c240..8c459eb 100755 --- a/host/make.sh +++ b/host/make.sh @@ -38,7 +38,9 @@ SOFTWARE=( INTERFACES=/etc/network/interfaces -BRIDGES_TPL=bridges.tpl +BOOTSTRAP_PORT=${BOOTSTRAP_PORT:-none} + +BRIDGES_TPL=${BRIDGES_TPL:-bridges.tpl} # XXX #readVars From 2691fb7faab9f70c3cef3fc60e9e0bcb6724f016 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 14:15:05 +0300 Subject: [PATCH 02/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 38 +++++++++++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 0b127e2..bbc174e 100644 --- a/README.md +++ b/README.md @@ -77,12 +77,34 @@ Goals: +---------------------------------------------------------------+ ``` -XXX +The system defines two networks: +- LAN + Hosts all the service CT's (`*.srv`) +- ADMIN + Used for administration (`*.adm`) +The ADMIN network is connected to the admin port. -### Services +Both networks are provided DNS and DHCP services by the `ns` CT. -XXX +Services on both networks are connected to the outside world (WAN) via +a NAT router implemented by the `gate` CT (`iptables`). + +The `gate` CT also implements a reverse proxy (`traefik`), routing requests +from the WAN ($WAN_IP) to appropriate service CT's on the LAN. + +Services expose their administration interfaces only on the ADMIN network +when possible. + +The host Proxmox (`pve.adm`) is only accessible through the ADMIN network. + +The `gate` and `ns` CT's are only accessible for administration from the +host (i.e. via `lxc-attach ..`). + +Three ways of access to the ADMIN network are provided: +- `ssh` service (CT) via the `gate` reverse proxy +- `wireguard` vpn via `gate` reverse proxy +- `ssh` service (CT) via the direct `$WAN_SSH_IP` (fail-safe) @@ -92,11 +114,13 @@ XXX Install Proxmox and connect it to your device/network. + +#### Notes + This setup will use three IP addresses: -1. IP address used for setup only, this is the static (usually) IP - initially assigned to Proxmox on install and it will not be used after - setup is done, -2. WAN IP adress to be used for the main set of applications, this is +1. The static (usually) IP initially assigned to Proxmox on install. This + will not be used after setup is done, +2. WAN IP address to be used for the main set of applications, this is the address that all the requests will be routed from to various services internally, 3. Fail-safe ssh IP address, this is the connection used for recovery From 26259ab3865da34b06871c2f986211a1c0d4a92a Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 14:18:57 +0300 Subject: [PATCH 03/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index bbc174e..835c681 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,8 @@ Ansible version will be implemented next as a direct comparison._ ## Architecture -Goals: +### Goals + - Separate concerns Preferably one service/role per CT - Keep things as light as possible From 8c24030f6d78e80665b55df3402c9f5ba4c6afef Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 14:22:16 +0300 Subject: [PATCH 04/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 835c681..5c5bd98 100644 --- a/README.md +++ b/README.md @@ -91,8 +91,9 @@ Both networks are provided DNS and DHCP services by the `ns` CT. Services on both networks are connected to the outside world (WAN) via a NAT router implemented by the `gate` CT (`iptables`). -The `gate` CT also implements a reverse proxy (`traefik`), routing requests -from the WAN ($WAN_IP) to appropriate service CT's on the LAN. +The `gate` CT also implements a reverse proxy ([`traefik`](https://traefik.io/traefik/)), +routing requests from the WAN (`$WAN_IP`) to appropriate service CT's on +the LAN. Services expose their administration interfaces only on the ADMIN network when possible. From fc7d23b8603014db4d15660cd999211dd65b49c0 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 14:23:50 +0300 Subject: [PATCH 05/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5c5bd98..1ed2267 100644 --- a/README.md +++ b/README.md @@ -105,7 +105,7 @@ host (i.e. via `lxc-attach ..`). Three ways of access to the ADMIN network are provided: - `ssh` service (CT) via the `gate` reverse proxy -- `wireguard` vpn via `gate` reverse proxy +- `wireguard` VPN (CT) via `gate` reverse proxy - `ssh` service (CT) via the direct `$WAN_SSH_IP` (fail-safe) From ee2f88bf0b3a7cf32a2ce8ddad9c2e02c4ae8d95 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 15:58:03 +0300 Subject: [PATCH 06/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 44 +++++++++++++++ config.global.example | 128 +++++++++++++++++++++++------------------- gate-traefik/make.sh | 4 +- ssh/make.sh | 8 +-- 4 files changed, 121 insertions(+), 63 deletions(-) diff --git a/README.md b/README.md index 1ed2267..2a833e3 100644 --- a/README.md +++ b/README.md @@ -174,10 +174,52 @@ This will connecting it to a configured network can lead to unexpected behavior.* + #### Accessing the host +The simplest way is to connect to `wireguard` VPN and open http://pve.adm:8006 +in a browser. + +The second approach is to `ssh` to either: + +```shell +ssh -p 23 @ +``` + +or: +```shell +ssh @ +``` + +The later will also work if the `gate` CT is down or not accessible. + + +And from the `ssh` CT: +```shell +ssh root@pve +``` + +_WARNING: NEVER store any ssh keys on the `ssh` CT, use `ssh-agent` instead!_ + + + +#### Configuration + XXX +The following CT's interfaces can not be configured in the Proxmox UI: +- `gate` +- `ns` +- `nextcloud` +- `wireguard` + +This is done mostly to keep Proxmox from touching the `hostname $(hostname)` +directive (used by the DNS server to assigned predefined IP's) and in +the case of `gate` and `wireguard` to keep it from touching the additional +bridges or interfaces defined. +(XXX this restriction may be lifted in the future) + + #### Setup additional services @@ -212,6 +254,8 @@ XXX traefik rules + + ### Manual setup diff --git a/config.global.example b/config.global.example index 52f7783..fac9161 100644 --- a/config.global.example +++ b/config.global.example @@ -1,4 +1,4 @@ -#------------------------------------------------------------------------ +#---------------------------------------------------------------------- # # Global config file # @@ -16,7 +16,68 @@ # It is not recomended to set passwords here or in other config files. # # -#------------------------------------------------------------------------ +#---------------------------------------------------------------------- +# These options need to be revised or changed... +# (remove "DFL_" prefix to disable promting) +# + +# Domain and email configuration +# +DFL_DOMAIN=example.com +DFL_EMAIL=user@example.com + + +# Network configuration... +# +# NOTE: it is simpler to statically assign these than to configure DHCP +# plus port forewarding to the dynamically assigned IP. +# NOTE: if installing on a different network than the target, these can +# be changed for target deployment in: +# - gate CT's /etc/network/interfaces (NOT in the Proxmox UI) +# - ssh CT's network configuration (Proxmox UI) +DFL_WAN_IP=192.168.1.101/24 +DFL_WAN_GATE=192.168.1.252 + +# IP used for fail-safe conection to the ADMIN network +DFL_WAN_SSH_IP=192.168.1.102/24 + + +# Web app/service domain configuration +# +# Here two optional variables are provided per service: +# - _DOMAIN=... +# Overrides the $DOMAIN option above for +# - _SUBDOMAIN=... +# Sets the subdomain of $DOMAIN (or $_DOMAIN) for + +# Nextcloud +#NEXTCLOUD_DOMAIN= +NEXTCLOUD_SUBDOMAIN=nc. + +# Gitea +#GITEA_DOMAIN= +#GITEA_SUBDOMAIN=git. + + +# Extra options passed to each CT when created. +# +# This can be used for passing in ssh keys, etc... +# +# see: +# man pct +# +# Example: +# DFL_PCT_EXTRA="--ssh-public-keys /path/to/autohrized_keys" +# +DFL_PCT_EXTRA=SKIP + + + +#---------------------------------------------------------------------- +# +# Options afetr this point are sane defaults and in the general case +# can be left as-is. +# # Bootsrap configuration... # @@ -37,70 +98,21 @@ BOOTSTRAP_BRIDGE=0 # bridges with numbers greater than X (10 in the example below) # # Example: -# WAN_BRIDGE=0 -# ADMIN_BRIDGE=3 -# LAN_BRIDGE=10 +# ADMIN_BRIDGE=_admin +# WAN_BRIDGE=_wan +# LAN_BRIDGE=_lan # ADMIN_BRIDGE=_admin WAN_BRIDGE=_wan LAN_BRIDGE=_lan -# NOTE: it is simpler to statically assign these than to configure dhcp -# plus port forewarding to the dynamically assigned IP. -DFL_WAN_IP=192.168.1.101/24 -DFL_WAN_GATE=192.168.1.252 - -DFL_WAN_SSH_IP=192.168.1.102/24 - - -# Domain and email configuration -# -DOMAIN=example.com -EMAIL=user@example.com - - -# Web app/service domain configuration -# -# Here two optional variables are provided per service: -# - _DOMAIN=... -# Overrides the $DOMAIN option above for -# - _SUBDOMAIN=... -# Sets the subdomain of $DOMAIN (or $_DOMAIN) for -# - -# Nextcloud -#NEXTCLOUD_DOMAIN= -NEXTCLOUD_SUBDOMAIN=nc. - -# Gitea -#GITEA_DOMAIN= -#GITEA_SUBDOMAIN=git. - - -# Extra options passed to each CT created. -# -# This can be used for passing in ssh keys, etc... -# -# see: -# man pct -# -# Example: -# DFL_PCT_EXTRA="--ssh-public-keys /path/to/autohrized_keys" -# -DFL_PCT_EXTRA=SKIP - - - -#------------------------------------------------------------------------ -# -# Options afetr this point are sane defaults and in the general case -# can be left as-is. -# +# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - # host HOST_ADMIN_IP=10.0.0.254/24 + # Nameserver NS_HOSTNAME=ns NS_ID=100 @@ -110,6 +122,7 @@ NS_LAN_IP=10.1.1.1/24 RESERVE_NS_ID=101 TEMPLATE_NS_ID=200 + # Gateway / Reverse proxy GATE_HOSTNAME=gate GATE_ID=110 @@ -120,4 +133,5 @@ RESERVE_GATE_ID=111 TEMPLATE_GATE_ID=210 -#------------------------------------------------------------------------ + +#---------------------------------------------------------------------- diff --git a/gate-traefik/make.sh b/gate-traefik/make.sh index 8ff7a92..3050c1f 100755 --- a/gate-traefik/make.sh +++ b/gate-traefik/make.sh @@ -25,8 +25,8 @@ RAM=128 SWAP=$RAM DRIVE=0.5 -DFL_WAN_IP=${DFL_WAN_IP} -DFL_WAN_GATE=${DFL_WAN_GATE} +#DFL_WAN_IP=${DFL_WAN_IP} +#DFL_WAN_GATE=${DFL_WAN_GATE} # XXX revise... DFL_ADMIN_IP=${GATE_ADMIN_IP:=${DFL_ADMIN_IP:=10.0.0.2/24}} diff --git a/ssh/make.sh b/ssh/make.sh index c9d9a5e..247ae5c 100755 --- a/ssh/make.sh +++ b/ssh/make.sh @@ -24,10 +24,10 @@ readConfig DFL_ID=${DFL_ID:=120} DFL_CTHOSTNAME=${DFL_CTHOSTNAME:=ssh} -DFL_CORES=${DFL_CORES:=1} -DFL_RAM=${DFL_RAM:=1024} -DFL_SWAP=${DFL_SWAP:=${DFL_RAM}} -DFL_DRIVE=${DFL_DRIVE:=16} +DFL_CORES=${DFL_SSH_CORES:=1} +DFL_RAM=${DFL_SSH_RAM:=512} +DFL_SWAP=${DFL_SSH_SWAP:=${RAM}} +DFL_DRIVE=${DFL_SSH_DRIVE:=16} WAN_IP=SKIP WAN_GATE=SKIP From 78b2932f661ae0e41a4fdf03a8e523bb6b1cdc80 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 16:02:55 +0300 Subject: [PATCH 07/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 2a833e3..48f17ac 100644 --- a/README.md +++ b/README.md @@ -116,6 +116,9 @@ Three ways of access to the ADMIN network are provided: Install Proxmox and connect it to your device/network. +Proxmox will need to have access to the internet to download assets and +updates. + #### Notes @@ -168,10 +171,12 @@ This will - setup firewall rules. Note that the firewall will not be enabled, this should be done manually after rule review. - -*Note that the ADMIN port is configured for direct connections only (DHCP), -connecting it to a configured network can lead to unexpected behavior.* +After this stage disconnect from the ADMIN port and connect the WAN port +to the network. + +_Note that the ADMIN port is configured for direct connections only (DHCP), +connecting it to a configured network can lead to unexpected behavior._ From 12b1029989cf05e40046b6f163d9b4bf6ce931a5 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 16:15:22 +0300 Subject: [PATCH 08/34] docs... Signed-off-by: Alex A. Naanou --- .github/workflows/toc.yml | 8 ++++++++ README.md | 20 +++++++++++--------- 2 files changed, 19 insertions(+), 9 deletions(-) create mode 100644 .github/workflows/toc.yml diff --git a/.github/workflows/toc.yml b/.github/workflows/toc.yml new file mode 100644 index 0000000..8481c7e --- /dev/null +++ b/.github/workflows/toc.yml @@ -0,0 +1,8 @@ +on: push +name: TOC Generator +jobs: + generateTOC: + name: TOC Generator + runs-on: ubuntu-latest + steps: + - uses: technote-space/toc-generator@v4 diff --git a/README.md b/README.md index 48f17ac..0579292 100644 --- a/README.md +++ b/README.md @@ -3,18 +3,20 @@ A set of scripts for automating setup and tasks in proxmox. ## TODO -- revise defaults +- CT updates +- backup/restore +- mail +- which is better? + - Makefile (a-la ./wireguard/templates/root/Makefile) + - shell (a-la ./shadow/templates/root/update-shadowsocks.sh) - separate templates/assets into distribution and user directories ...this is needed to allow the user to change the configs without the fear of them being overwritten by git (similar to how config is handlerd) -- might be a good idea to export a specific ct script that can be used - for updates for that ct -- which is better? - - Makefile (a-la wireguard) - - shell (a-la shadow) -- ct updates -- backup/restore -- mail + + + + + ## Motivation From f4c54d2cba98bca2402ca6720dab8c0a662b827d Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 16:23:43 +0300 Subject: [PATCH 09/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index 0579292..6565b69 100644 --- a/README.md +++ b/README.md @@ -174,8 +174,6 @@ This will Note that the firewall will not be enabled, this should be done manually after rule review. -After this stage disconnect from the ADMIN port and connect the WAN port -to the network. _Note that the ADMIN port is configured for direct connections only (DHCP), connecting it to a configured network can lead to unexpected behavior._ From bf30e7714a55b45f058db93a6a5b6025f95f2763 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 16:30:35 +0300 Subject: [PATCH 10/34] disabled toc generator... Signed-off-by: Alex A. Naanou --- .github/workflows/toc.yml | 8 -------- README.md | 3 ++- 2 files changed, 2 insertions(+), 9 deletions(-) delete mode 100644 .github/workflows/toc.yml diff --git a/.github/workflows/toc.yml b/.github/workflows/toc.yml deleted file mode 100644 index 8481c7e..0000000 --- a/.github/workflows/toc.yml +++ /dev/null @@ -1,8 +0,0 @@ -on: push -name: TOC Generator -jobs: - generateTOC: - name: TOC Generator - runs-on: ubuntu-latest - steps: - - uses: technote-space/toc-generator@v4 diff --git a/README.md b/README.md index 6565b69..779d497 100644 --- a/README.md +++ b/README.md @@ -183,7 +183,8 @@ connecting it to a configured network can lead to unexpected behavior._ #### Accessing the host The simplest way is to connect to `wireguard` VPN and open http://pve.adm:8006 -in a browser. +in a browser (a profile was created during the setup process and stored +in the `/root/clients/` directory on the `wireguard` CT). The second approach is to `ssh` to either: From fb0005e9d1e7d2469450f923dd2bffd8751a8463 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 16:34:30 +0300 Subject: [PATCH 11/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/README.md b/README.md index 779d497..35b1000 100644 --- a/README.md +++ b/README.md @@ -137,6 +137,13 @@ This setup will use three IP addresses: ### Semi-automated setup +Open a terminal on the host (`ssh` or via the UI). + +Optionally, set a desired default editor (defaults to `nano`) via: +```shell +export EDITOR=.. +``` + Download the [`bootstrap.sh`](./scripts/bootstrap.sh) script and execute it: ```shell curl 'https://raw.githubusercontent.com/flynx/proxmox-utils/refs/heads/master/scripts/bootstrap.sh' | sudo bash From 752675a5cae86e698985357ab07e115706fbc874 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 16:37:19 +0300 Subject: [PATCH 12/34] docs and tweaks... Signed-off-by: Alex A. Naanou --- Makefile | 2 +- README.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index 6ad588c..33dbd0e 100644 --- a/Makefile +++ b/Makefile @@ -10,7 +10,7 @@ # #---------------------------------------------------------------------- -EDITOR ?= vim +EDITOR ?= nano # CTs... diff --git a/README.md b/README.md index 35b1000..cd501d9 100644 --- a/README.md +++ b/README.md @@ -139,9 +139,9 @@ This setup will use three IP addresses: Open a terminal on the host (`ssh` or via the UI). -Optionally, set a desired default editor (defaults to `nano`) via: +Optionally, set a desired default editor via: ```shell -export EDITOR=.. +export EDITOR=nano ``` Download the [`bootstrap.sh`](./scripts/bootstrap.sh) script and execute it: From a844845c4f31b91ab5869a5f4425c89be83acb17 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 17:43:01 +0300 Subject: [PATCH 13/34] bugfix... Signed-off-by: Alex A. Naanou --- ssh/make.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ssh/make.sh b/ssh/make.sh index 247ae5c..e9d4a0d 100755 --- a/ssh/make.sh +++ b/ssh/make.sh @@ -26,7 +26,7 @@ DFL_CTHOSTNAME=${DFL_CTHOSTNAME:=ssh} DFL_CORES=${DFL_SSH_CORES:=1} DFL_RAM=${DFL_SSH_RAM:=512} -DFL_SWAP=${DFL_SSH_SWAP:=${RAM}} +DFL_SWAP=${DFL_SSH_SWAP:=${DFL_RAM}} DFL_DRIVE=${DFL_SSH_DRIVE:=16} WAN_IP=SKIP From 8e8c89f50a1b13dbd660c44a7e9886f9617210ea Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 18:11:02 +0300 Subject: [PATCH 14/34] make finalize not uses nohup... Signed-off-by: Alex A. Naanou --- Makefile | 17 +++++++++++++++-- README.md | 2 ++ 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 33dbd0e..0ad7ab6 100644 --- a/Makefile +++ b/Makefile @@ -93,11 +93,24 @@ bootstrap-clean: host-bootstrap-clean # Finalize: reconect admin port/bridge correctly... -.PHONY: finalize -finalize: bootstrap-clean gate-bootstrap-clean +.PHONY: _finalize +_finalize: bootstrap-clean gate-bootstrap-clean make host-bootstrap-clean +.PHONY: finalize +finalize: + ### + ### This will break the connection to the server, to continue: + ### - Detach the ADMIN port, + ### - Connect the WAN port, + ### - $ ssh @ + ### from there: + ### $ ssh root@pve + ### + nohup make _finalize & tail -f nohup.out + + #---------------------------------------------------------------------- # Shorthands... diff --git a/README.md b/README.md index cd501d9..ae42438 100644 --- a/README.md +++ b/README.md @@ -173,6 +173,8 @@ To finalize the setup run: make finalize ``` +Note that this will break the ssh connection. + This will - detach the host from any external ports and make it accessible only from the internal network. From c8c09ce8d5d80778bf0bad2f0b8b98fcbdd8708c Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 18:44:03 +0300 Subject: [PATCH 15/34] switched to tmux instad of nohup... Signed-off-by: Alex A. Naanou --- Makefile | 4 +++- README.md | 7 ++++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 0ad7ab6..99f0921 100644 --- a/Makefile +++ b/Makefile @@ -107,8 +107,10 @@ finalize: ### - $ ssh @ ### from there: ### $ ssh root@pve + ### then: + ### # tmux a ### - nohup make _finalize & tail -f nohup.out + tmux 'make _finalize' diff --git a/README.md b/README.md index ae42438..70949e4 100644 --- a/README.md +++ b/README.md @@ -173,7 +173,12 @@ To finalize the setup run: make finalize ``` -Note that this will break the ssh connection. +This will break the ssh connection, reconnect via the WAN port +and reattach to the `tmux` session via: + +```shell +tmux a +``` This will - detach the host from any external ports and make it accessible only From aeb2bbde824f4bfcad4d741c4d581f6b265516c6 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 18:46:30 +0300 Subject: [PATCH 16/34] ... Signed-off-by: Alex A. Naanou --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 99f0921..96ac4a3 100644 --- a/Makefile +++ b/Makefile @@ -110,7 +110,7 @@ finalize: ### then: ### # tmux a ### - tmux 'make _finalize' + tmux new-session 'make _finalize' From b7f56623d9996324ee1fa555a0b621f3ac49b646 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 18:58:41 +0300 Subject: [PATCH 17/34] ... Signed-off-by: Alex A. Naanou --- Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/Makefile b/Makefile index 96ac4a3..c56df20 100644 --- a/Makefile +++ b/Makefile @@ -109,6 +109,7 @@ finalize: ### $ ssh root@pve ### then: ### # tmux a + ### - When done this will reboot the system. ### tmux new-session 'make _finalize' From 66b2b7ea7f15e4559e8b0daa3a80cc0e9ed6cda6 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 19:15:53 +0300 Subject: [PATCH 18/34] ... Signed-off-by: Alex A. Naanou --- Makefile | 33 +++++++++++++++++---------------- host/make.sh | 28 +++++++++++++++++++--------- 2 files changed, 36 insertions(+), 25 deletions(-) diff --git a/Makefile b/Makefile index c56df20..8c8e313 100644 --- a/Makefile +++ b/Makefile @@ -93,25 +93,26 @@ bootstrap-clean: host-bootstrap-clean # Finalize: reconect admin port/bridge correctly... -.PHONY: _finalize -_finalize: bootstrap-clean gate-bootstrap-clean +.PHONY: finalize +finalize: gate-bootstrap-clean + make host-bootstrap-clean make host-bootstrap-clean -.PHONY: finalize -finalize: - ### - ### This will break the connection to the server, to continue: - ### - Detach the ADMIN port, - ### - Connect the WAN port, - ### - $ ssh @ - ### from there: - ### $ ssh root@pve - ### then: - ### # tmux a - ### - When done this will reboot the system. - ### - tmux new-session 'make _finalize' +#.PHONY: finalize +#finalize: +# ### +# ### This will break the connection to the server, to continue: +# ### - Detach the ADMIN port, +# ### - Connect the WAN port, +# ### - $ ssh @ +# ### from there: +# ### $ ssh root@pve +# ### then: +# ### # tmux a +# ### - When done this will reboot the system. +# ### +# tmux new-session 'make _finalize' diff --git a/host/make.sh b/host/make.sh index 8c459eb..673a5a4 100755 --- a/host/make.sh +++ b/host/make.sh @@ -52,6 +52,18 @@ BRIDGES_TPL=${BRIDGES_TPL:-bridges.tpl} if ! [ -z $BOOTSTRAP_CLEAN ] ; then @ cp "$INTERFACES"{,.bak} + __finalize(){ + if reviewApplyChanges "$INTERFACES" ; then + # XXX this must be done in nohup to avoid breaking on connection lost... + if ! @ ifreload -a ; then + # reset settings back if ifreload fails... + @ cp "$INTERFACES"{.bak,} + @ ifreload -a + fi + fi + unset -f __finalize + } + # stage 1: bootstrap -> clean if [ -e "$INTERFACES".clean ] ; then @ mv "$INTERFACES"{.clean,.new} @@ -62,6 +74,8 @@ if ! [ -z $BOOTSTRAP_CLEAN ] ; then DFL_DNS=1 DFL_FIREWALL=SKIP + __finalize + # stage 2: clean -> final elif [ -e "$INTERFACES".final ] ; then @ mv "$INTERFACES"{.final,.new} @@ -79,15 +93,6 @@ if ! [ -z $BOOTSTRAP_CLEAN ] ; then exit fi - if reviewApplyChanges "$INTERFACES" ; then - # XXX this must be done in nohup to avoid breaking on connection lost... - if ! @ ifreload -a ; then - # reset settings back if ifreload fails... - @ cp "$INTERFACES"{.bak,} - @ ifreload -a - fi - fi - # Bootstrap... elif ! [ -z $BOOTSTRAP ] ; then DFL_BOOTSTRAP_PORT=${DFL_BOOTSTRAP_PORT:-none} @@ -257,6 +262,11 @@ showNotes echo "# Done." +if [[ $( type -t __finalize ) == function ]] ; then + __finalize +fi + + if ! [ -z $REBOOT ] ; then echo "# Rebooting..." @ reboot From 39b42883fcd64050bd034d95fbb9b1bf16efec4d Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 21:13:44 +0300 Subject: [PATCH 19/34] cleanup... Signed-off-by: Alex A. Naanou --- README.md | 19 ++++++++----------- host/make.sh | 16 ++++++++++++---- 2 files changed, 20 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 70949e4..6ba9ff3 100644 --- a/README.md +++ b/README.md @@ -173,20 +173,17 @@ To finalize the setup run: make finalize ``` -This will break the ssh connection, reconnect via the WAN port -and reattach to the `tmux` session via: - -```shell -tmux a -``` - This will -- detach the host from any external ports and make it accessible only - from the internal network. - See: [Architecture](#architecture) and [Bootstrapping](#bootstrapping) -- setup firewall rules. +- Setup firewall rules. Note that the firewall will not be enabled, this should be done manually after rule review. +- Detach the host from any external ports and make it accessible only + from the internal network. + See: [Architecture](#architecture) and [Bootstrapping](#bootstrapping) + +This will break the ssh connection when done, reconnect via the WAN port +to continue (see: [Accessing the host](#accessing-the-host)), or connect +directly to the ADMIN port (DHCP) and ssh into `$HOST_ADMIN_IP` (default: 10.0.0.254). _Note that the ADMIN port is configured for direct connections only (DHCP), diff --git a/host/make.sh b/host/make.sh index 673a5a4..69634ac 100755 --- a/host/make.sh +++ b/host/make.sh @@ -49,6 +49,7 @@ BRIDGES_TPL=${BRIDGES_TPL:-bridges.tpl} # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - # Bootstrap... +# cleanup... if ! [ -z $BOOTSTRAP_CLEAN ] ; then @ cp "$INTERFACES"{,.bak} @@ -61,6 +62,7 @@ if ! [ -z $BOOTSTRAP_CLEAN ] ; then @ ifreload -a fi fi + # clear self to avoid a second deffered execution... unset -f __finalize } @@ -74,6 +76,7 @@ if ! [ -z $BOOTSTRAP_CLEAN ] ; then DFL_DNS=1 DFL_FIREWALL=SKIP + # NOTE: in general this is non-destructive and can be done inline. __finalize # stage 2: clean -> final @@ -86,6 +89,8 @@ if ! [ -z $BOOTSTRAP_CLEAN ] ; then DFL_DNS=SKIP DFL_FIREWALL=1 + # NOTE: __finalize is deferred to just before reboot... + REBOOT=1 # done @@ -112,20 +117,20 @@ fi #---------------------------------------------------------------------- -# System... +# system... if xreadYes "# Update system?" UPDATE ; then @ apt update @ apt upgrade fi -# Tools... +# tools... if xreadYes "# Install additional apps?" APPS ; then @ apt install ${SOFTWARE[@]} fi -# Bridges... +# bridges... if xreadYes "# Create bridges?" BRIDGES ; then xread "WAN port: " WAN_PORT xread "ADMIN port: " ADMIN_PORT @@ -262,11 +267,14 @@ showNotes echo "# Done." -if [[ $( type -t __finalize ) == function ]] ; then +# finalize... +if [[ $( type -t __finalize ) == "function" ]] ; then + echo "# Finalizing ${INTERFACES}..." __finalize fi +# reboot... if ! [ -z $REBOOT ] ; then echo "# Rebooting..." @ reboot From f3d3b3fe323681603aa531281a3fd07459c71a79 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Mon, 21 Oct 2024 21:52:00 +0300 Subject: [PATCH 20/34] now revewApplyChanges accepts a default option... Signed-off-by: Alex A. Naanou --- .pct-helpers | 60 ++++++++++++++++++++++++++++++++++++++++++++++------ Makefile | 18 ++-------------- host/make.sh | 10 ++++----- 3 files changed, 61 insertions(+), 27 deletions(-) diff --git a/.pct-helpers b/.pct-helpers index 108aa1a..4c79715 100644 --- a/.pct-helpers +++ b/.pct-helpers @@ -302,33 +302,81 @@ xreadpass(){ } +# Like cat but a prettier... +# +# listFile PATH +# +listFile(){ + if [ -e "$1" ] ; then + echo "--- $1 ---" + cat "$1" + echo '---' + else + echo "$FUNCNAME: $1: No such file or directory." + return 1 + fi +} + + # Review changes in PATH.new, then edit/apply changes to PATH # -# reviewApplyChanges PATH +# reviewApplyChanges PATH [apply|edit|skip] +# # # NOTE: if changes are not applied this will return non-zero making this # usable in conditionals... reviewApplyChanges(){ local file=$1 + if ! [ -e "$file".new ] ; then + echo "$FUNCNAME: $1: No such file or directory." + return 1 + fi + + # default option... + local dfl= + local a=a + local e=e + local s=s + case "${2,,}" in + a|apply) + a=A + dfl=a + ;; + e|edit) + e=E + dfl=e + ;; + s|skip) + s=S + dfl=s + ;; + esac + echo "# Review updated: ${file}.new:" - @ cat ${file}.new - echo '---' + listFile ${file}.new local res while true ; do - read -ep "# [a]pply, [e]dit, [s]kip? " res + read -ep "# [$a]pply, [$e]dit, [$s]kip? " res + if [ -z $res ] ; then + if [ -z $dfl ] ; then + continue + fi + res=$dfl + fi case "${res,,}" in a|apply) break ;; e|edit) ${EDITOR} "${file}.new" + listFile ${file}.new ;; s|skip) - echo "# file saved as: ${file}.new" + echo "# Changes kept as: ${file}.new" return 1 ;; *) - echo "ERROR: unknown command: \"$res\"" >&2 + echo "ERROR: Unknown command: \"$res\"" >&2 continue ;; esac diff --git a/Makefile b/Makefile index 8c8e313..e14e455 100644 --- a/Makefile +++ b/Makefile @@ -95,26 +95,12 @@ bootstrap-clean: host-bootstrap-clean # Finalize: reconect admin port/bridge correctly... .PHONY: finalize finalize: gate-bootstrap-clean + # cleanup: stage 1... make host-bootstrap-clean + # cleanup: stage 2... make host-bootstrap-clean -#.PHONY: finalize -#finalize: -# ### -# ### This will break the connection to the server, to continue: -# ### - Detach the ADMIN port, -# ### - Connect the WAN port, -# ### - $ ssh @ -# ### from there: -# ### $ ssh root@pve -# ### then: -# ### # tmux a -# ### - When done this will reboot the system. -# ### -# tmux new-session 'make _finalize' - - #---------------------------------------------------------------------- # Shorthands... diff --git a/host/make.sh b/host/make.sh index 69634ac..fb77960 100755 --- a/host/make.sh +++ b/host/make.sh @@ -54,7 +54,7 @@ if ! [ -z $BOOTSTRAP_CLEAN ] ; then @ cp "$INTERFACES"{,.bak} __finalize(){ - if reviewApplyChanges "$INTERFACES" ; then + if reviewApplyChanges "$INTERFACES" apply ; then # XXX this must be done in nohup to avoid breaking on connection lost... if ! @ ifreload -a ; then # reset settings back if ifreload fails... @@ -213,7 +213,7 @@ if xreadYes "# Create bridges?" BRIDGES ; then fi # interfaces - if reviewApplyChanges "$INTERFACES" ; then + if reviewApplyChanges "$INTERFACES" apply ; then # XXX this must be done in nohup to avoid breaking on connection lost... if ! @ ifreload -a ; then # reset settings back if ifreload fails... @@ -231,7 +231,7 @@ if xreadYes "# Update /etc/hosts?" HOSTS ; then @ sed -i \ -e 's/^[^#].* \(pve.local.*\)$/'${HOST_ADMIN_IP/\/*}' \1/' \ /etc/hosts.new - reviewApplyChanges /etc/hosts + reviewApplyChanges /etc/hosts apply fi @@ -250,7 +250,7 @@ if xreadYes "# Update DNS?" DNS ; then build file=/etc/resolv.conf @ cp "staging/${file}" "${file}".new - reviewApplyChanges "${file}" + reviewApplyChanges "${file}" apply fi @@ -259,7 +259,7 @@ if xreadYes "# Update firewall rules?" FIREWALL ; then build file=/etc/pve/firewall/cluster.fw @ cp "staging/${file}" "${file}".new - reviewApplyChanges "${file}" + reviewApplyChanges "${file}" apply fi From c3ccb5eb0c6eb07e505ef87086ab6cb46c807767 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 02:22:25 +0300 Subject: [PATCH 21/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 6ba9ff3..c772105 100644 --- a/README.md +++ b/README.md @@ -27,6 +27,8 @@ functionality in Ansible. _NOTE: for a fair assessment of viability of further development an Ansible version will be implemented next as a direct comparison._ +Fun. + ## Architecture From ed943f754b3ce3cde41b65ad78427fdbb87c6707 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 02:55:50 +0300 Subject: [PATCH 22/34] minor cleanup to ns... Signed-off-by: Alex A. Naanou --- README.md | 42 +++++++++++++++++------------------ ns/templates/etc/dnsmasq.conf | 9 ++++++-- 2 files changed, 27 insertions(+), 24 deletions(-) diff --git a/README.md b/README.md index c772105..f7d5fea 100644 --- a/README.md +++ b/README.md @@ -34,17 +34,17 @@ Fun. ### Goals -- Separate concerns +- _Separate concerns_ Preferably one service/role per CT -- Keep things as light as possible +- _Keep things as light as possible_ This for the most part rules out Docker as a nested virtualization - layer under Proxmox while preferring light distributions like Alpine + layer under Proxmox, and preferring light distributions like Alpine Linux -- Pragmatic simplicity +- _Pragmatic simplicity_ This goal yields some compromises to previous goals, for example [TKL]() is used as a base for [Nextcloud]() effectively simplifying the setup and administration of all the related components at the cost of a - heavier CT transparently integrating multiple related services + heavier CT, transparently integrating multiple related services ### Network @@ -108,9 +108,9 @@ The `gate` and `ns` CT's are only accessible for administration from the host (i.e. via `lxc-attach ..`). Three ways of access to the ADMIN network are provided: -- `ssh` service (CT) via the `gate` reverse proxy -- `wireguard` VPN (CT) via `gate` reverse proxy -- `ssh` service (CT) via the direct `$WAN_SSH_IP` (fail-safe) +- `wireguard` VPN (CT) via `gate` reverse proxy, +- `ssh` service (CT) via the `gate` reverse proxy, +- `ssh` service (CT) via the direct `$WAN_SSH_IP` (fail-safe). @@ -131,7 +131,7 @@ This setup will use three IP addresses: will not be used after setup is done, 2. WAN IP address to be used for the main set of applications, this is the address that all the requests will be routed from to various - services internally, + services on the LAN network, 3. Fail-safe ssh IP address, this is the connection used for recovery in case the internal routing fails. @@ -139,9 +139,9 @@ This setup will use three IP addresses: ### Semi-automated setup -Open a terminal on the host (`ssh` or via the UI). +Open a terminal on the host, either `ssh` (recommended) or via the UI. -Optionally, set a desired default editor via: +Optionally, set a desired default editor (default: `nano`) via: ```shell export EDITOR=nano ``` @@ -151,22 +151,20 @@ Download the [`bootstrap.sh`](./scripts/bootstrap.sh) script and execute it: curl 'https://raw.githubusercontent.com/flynx/proxmox-utils/refs/heads/master/scripts/bootstrap.sh' | sudo bash ``` +_It is recommended to review the script/code before starting._ + This will: -- Install basic dependencies -- Clone this repo -- Run `make bootstrap` on the repo +- Install basic dependencies, +- Clone this repo, +- Run `make bootstrap` on the repo. -After the basic setup is done connect the device to the network via the -selcted WAN port and **disconnect** the ADMIN port. - -The WAN interface exposes two IPs: +At this point WAN interface exposes two IPs: - Main server (config: `$DFL_WAN_IP` / `$WAN_IP`) - ssh:23 - wireguard:51820 - Fail-safe ssh (config: `$DFL_WAN_SSH_IP` / `$WAN_SSH_IP`) - ssh:22 - The Proxmox administrative interface is available behind the Wireguard proxy or on the ADMIN port, both on https://10.0.0.254:8006. @@ -187,9 +185,9 @@ This will break the ssh connection when done, reconnect via the WAN port to continue (see: [Accessing the host](#accessing-the-host)), or connect directly to the ADMIN port (DHCP) and ssh into `$HOST_ADMIN_IP` (default: 10.0.0.254). - -_Note that the ADMIN port is configured for direct connections only (DHCP), -connecting it to a configured network can lead to unexpected behavior._ +_Note that the ADMIN port is configured for direct connections only, +connecting it to a configured network can lead to unexpected behavior -- +DHCP races, IP clashes... etc._ diff --git a/ns/templates/etc/dnsmasq.conf b/ns/templates/etc/dnsmasq.conf index b8df181..9afbc86 100644 --- a/ns/templates/etc/dnsmasq.conf +++ b/ns/templates/etc/dnsmasq.conf @@ -43,12 +43,16 @@ dhcp-range=interface:admin,10.0.0.20,10.0.0.200,12h dhcp-range=interface:lan,10.1.1.20,10.1.1.200,12h # ns -address=/${CTHOSTNAME}/${ADMIN_IPn} address=/${CTHOSTNAME}/${LAN_IPn} +address=/${CTHOSTNAME}.srv/${LAN_IPn} +#address=/${CTHOSTNAME}/${ADMIN_IPn} +address=/${CTHOSTNAME}.adm/${ADMIN_IPn} # gate -address=/${GATE_HOSTNAME}/${GATE_ADMIN_IPn} address=/${GATE_HOSTNAME}/${LAN_GATE} +address=/${GATE_HOSTNAME}.srv/${LAN_GATE} +#address=/${GATE_HOSTNAME}/${GATE_ADMIN_IPn} +address=/${GATE_HOSTNAME}.adm/${GATE_ADMIN_IPn} dhcp-option=admin,option:router,${GATE_ADMIN_IPn} dhcp-option=lan,option:router,${LAN_GATE} dhcp-host=admin,gate,${GATE_ADMIN_IPn},infinite @@ -56,6 +60,7 @@ dhcp-host=lan,gate,${LAN_GATE},infinite # pve address=/pve/10.0.0.254 +address=/pve.adm/10.0.0.254 From bbfa30fd5246777b097db384b397dddc1b43f1ab Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 03:03:15 +0300 Subject: [PATCH 23/34] commented out docs that need rewriting/refactoring... Signed-off-by: Alex A. Naanou --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f7d5fea..179c4d9 100644 --- a/README.md +++ b/README.md @@ -272,7 +272,7 @@ XXX traefik rules - + ## Extending From fc2bf66aa94b08924791e8ca67efcc3ea56ae072 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 03:08:22 +0300 Subject: [PATCH 24/34] refactoring... Signed-off-by: Alex A. Naanou --- README.md | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 179c4d9..169b4de 100644 --- a/README.md +++ b/README.md @@ -114,7 +114,7 @@ Three ways of access to the ADMIN network are provided: -## Setup +## Getting started ### Prerequisites @@ -137,7 +137,7 @@ This setup will use three IP addresses: -### Semi-automated setup +### Setup Open a terminal on the host, either `ssh` (recommended) or via the UI. @@ -238,7 +238,7 @@ bridges or interfaces defined. -#### Setup additional services +## Services XXX @@ -251,21 +251,35 @@ make dev ``` -Or individually: -```shell -make nextcloud -``` +### Syncthing ```shell make syncthing ``` +XXX + + +### Nextcloud + +```shell +make nextcloud +``` + +XXX + + +### Gitea + ```shell make gitea ``` +XXX -#### Setup and configure custom services + + +### Custom services XXX traefik rules From c97810c1bf1d39698ab34859d9550acc21defce2 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 03:11:26 +0300 Subject: [PATCH 25/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 169b4de..67edaa4 100644 --- a/README.md +++ b/README.md @@ -425,11 +425,14 @@ XXX firewall ### Directory structure ``` -/ +proxmox-utils/ +- / | +- templates/ +| | +- ... | +- assets/ +| | +- ... | +- staging/ +| | +- ... | +- make.sh | +- config | +- config.last-run From 890e4c2dd27c66af25945283a534580fdf275bb3 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 03:22:51 +0300 Subject: [PATCH 26/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 67edaa4..2c8a812 100644 --- a/README.md +++ b/README.md @@ -4,8 +4,16 @@ A set of scripts for automating setup and tasks in proxmox. ## TODO - CT updates + Right now the simplest way to update the infrastructure CT's if the + sources changed is to simply rebuild them -- add rebuild command. + - backup + - build (new reserve) + - destroy + - clone + - cleanup - backup/restore -- mail +- config manager -- save/use/.. +- mail server - which is better? - Makefile (a-la ./wireguard/templates/root/Makefile) - shell (a-la ./shadow/templates/root/update-shadowsocks.sh) From 1f9b24147c7848b0ef9507980467a861965078e2 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 03:23:43 +0300 Subject: [PATCH 27/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2c8a812..8757bda 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ A set of scripts for automating setup and tasks in proxmox. ## TODO -- CT updates +- CT updates / upgrades Right now the simplest way to update the infrastructure CT's if the sources changed is to simply rebuild them -- add rebuild command. - backup From 28334831a6805f39f66d76d237293ae877ed78e3 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 03:36:04 +0300 Subject: [PATCH 28/34] ... Signed-off-by: Alex A. Naanou --- Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/Makefile b/Makefile index e14e455..2d2bdb5 100644 --- a/Makefile +++ b/Makefile @@ -62,6 +62,7 @@ FORCE: %: config %/make.sh FORCE $*/make.sh + @echo %.config: %/config.example From e404ea7d77b0a875487e5c08959d225b1af52cc9 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 03:39:05 +0300 Subject: [PATCH 29/34] ... Signed-off-by: Alex A. Naanou --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 8757bda..bffb28b 100644 --- a/README.md +++ b/README.md @@ -44,7 +44,7 @@ Fun. - _Separate concerns_ Preferably one service/role per CT -- _Keep things as light as possible_ +- _Keep things as light as possible_ This for the most part rules out Docker as a nested virtualization layer under Proxmox, and preferring light distributions like Alpine Linux From 456078c1f5f71fac3cb3a485f44cd39c8f494e9c Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 13:37:06 +0300 Subject: [PATCH 30/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index bffb28b..05f5926 100644 --- a/README.md +++ b/README.md @@ -100,7 +100,7 @@ The ADMIN network is connected to the admin port. Both networks are provided DNS and DHCP services by the `ns` CT. -Services on both networks are connected to the outside world (WAN) via +Services on either network are connected to the outside world (WAN) via a NAT router implemented by the `gate` CT (`iptables`). The `gate` CT also implements a reverse proxy ([`traefik`](https://traefik.io/traefik/)), @@ -116,7 +116,7 @@ The `gate` and `ns` CT's are only accessible for administration from the host (i.e. via `lxc-attach ..`). Three ways of access to the ADMIN network are provided: -- `wireguard` VPN (CT) via `gate` reverse proxy, +- [`wireguard`](https://www.wireguard.com/) VPN (CT) via `gate` reverse proxy, - `ssh` service (CT) via the `gate` reverse proxy, - `ssh` service (CT) via the direct `$WAN_SSH_IP` (fail-safe). @@ -164,7 +164,9 @@ _It is recommended to review the script/code before starting._ This will: - Install basic dependencies, - Clone this repo, -- Run `make bootstrap` on the repo. +- Run `make bootstrap` on the repo: + - bootstrap configure the network (2 out of 3 stages) + - build and infrastructure start CT's (`gate`, `ns`, `ssh`, and `wireguard`) At this point WAN interface exposes two IPs: - Main server (config: `$DFL_WAN_IP` / `$WAN_IP`) @@ -173,8 +175,11 @@ At this point WAN interface exposes two IPs: - Fail-safe ssh (config: `$DFL_WAN_SSH_IP` / `$WAN_SSH_IP`) - ssh:22 -The Proxmox administrative interface is available behind the Wireguard -proxy or on the ADMIN port, both on https://10.0.0.254:8006. +The Proxmox administrative interface is available behind the +[Wireguard](https://www.wireguard.com/) proxy or on the ADMIN port, both +on https://10.0.0.254:8006. + +Additional administrative tasks can be performed now if needed. To finalize the setup run: ```shell @@ -248,16 +253,25 @@ bridges or interfaces defined. ## Services -XXX - +Install all user services: ```shell make all ``` +Includes: +- [`syncthing`](#syncthing) +- [`nextcloud`](#nextcloud) + + +Install development services: ```shell make dev ``` +Includes: +- [`gitea`](#gitea) + + ### Syncthing @@ -267,6 +281,8 @@ make syncthing XXX +For more info see: https://syncthing.net/ + ### Nextcloud @@ -276,6 +292,8 @@ make nextcloud XXX +For more info see: https://nextcloud.com/ + ### Gitea @@ -286,6 +304,8 @@ make gitea XXX +For more info see: https://about.gitea.com/ + ### Custom services From 7ae851da8401cf19d27e3ce442ff33e87aad83c2 Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 15:12:00 +0300 Subject: [PATCH 31/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 05f5926..da89999 100644 --- a/README.md +++ b/README.md @@ -49,8 +49,9 @@ Fun. layer under Proxmox, and preferring light distributions like Alpine Linux - _Pragmatic simplicity_ - This goal yields some compromises to previous goals, for example [TKL]() - is used as a base for [Nextcloud]() effectively simplifying the setup + This goal yields some compromises to previous goals, for example + [TKL](https://www.turnkeylinux.org/) is used as a base for + [Nextcloud](https://nextcloud.com/) effectively simplifying the setup and administration of all the related components at the cost of a heavier CT, transparently integrating multiple related services @@ -279,7 +280,12 @@ Includes: make syncthing ``` -XXX +Syncthing administration interface is accessible via https://syncthing.adm/ +on the ADMIN network, it is recommended to set an admin password on +the web interface as soon as possible. + +No additional routing or network configuration is required, Syncthing is +smart enough to handle its own connections itself. For more info see: https://syncthing.net/ @@ -290,9 +296,16 @@ For more info see: https://syncthing.net/ make nextcloud ``` -XXX +Nextcloud will get mapped to subdomain `$NEXTCLOUD_SUBDOMAIN` of +`$NEXTCLOUD_DOMAIN` (defaulting to `$DOMAIN`, if not defined). -For more info see: https://nextcloud.com/ +For basic configuration see: ./config.global and ./config.global.example + +For deeper management use the [TKL](https://www.turnkeylinux.org/) consoles +(via https://nextcloud.srv, on the LAN network) and `ssh`, for more details +see: https://www.turnkeylinux.org/nextcloud + +For more info on Nextcloud see: https://nextcloud.com/ ### Gitea @@ -301,10 +314,12 @@ For more info see: https://nextcloud.com/ make gitea ``` -XXX +Gitea is mapped to the subdomain `$GITEA_SUBDOMAIN` of `$GITEA_DOMAIN` +or `$DOMAIN` if the former is not defined. +For basic configuration see: ./config.global and ./config.global.example -For more info see: https://about.gitea.com/ +For more info see: https://gitea.com/ ### Custom services From c6ad52f759a6f24f994e504a114cfcbbccbf4d6e Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 15:13:35 +0300 Subject: [PATCH 32/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index da89999..912f1fc 100644 --- a/README.md +++ b/README.md @@ -317,7 +317,7 @@ make gitea Gitea is mapped to the subdomain `$GITEA_SUBDOMAIN` of `$GITEA_DOMAIN` or `$DOMAIN` if the former is not defined. -For basic configuration see: ./config.global and ./config.global.example +For basic configuration see: [./config.global] and [./config.global.example] For more info see: https://gitea.com/ From 57450cb86694bac7e7c1c397d5ea29584da84afe Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 15:15:13 +0300 Subject: [PATCH 33/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 912f1fc..6f19d2e 100644 --- a/README.md +++ b/README.md @@ -317,7 +317,8 @@ make gitea Gitea is mapped to the subdomain `$GITEA_SUBDOMAIN` of `$GITEA_DOMAIN` or `$DOMAIN` if the former is not defined. -For basic configuration see: [./config.global] and [./config.global.example] +For basic configuration edit the generated: [config.global](./config.global) +and for defaults: [config.global.example](./config.global.example). For more info see: https://gitea.com/ From 8c0b785f43560474ac9081083b279681f10365ee Mon Sep 17 00:00:00 2001 From: "Alex A. Naanou" Date: Tue, 22 Oct 2024 15:17:19 +0300 Subject: [PATCH 34/34] docs... Signed-off-by: Alex A. Naanou --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 6f19d2e..ed12063 100644 --- a/README.md +++ b/README.md @@ -299,7 +299,8 @@ make nextcloud Nextcloud will get mapped to subdomain `$NEXTCLOUD_SUBDOMAIN` of `$NEXTCLOUD_DOMAIN` (defaulting to `$DOMAIN`, if not defined). -For basic configuration see: ./config.global and ./config.global.example +For basic configuration edit the generated: [config.global](./config.global) +and for defaults: [config.global.example](./config.global.example). For deeper management use the [TKL](https://www.turnkeylinux.org/) consoles (via https://nextcloud.srv, on the LAN network) and `ssh`, for more details