HAProxy's reloading speed is very fast, and I don't feel that there is a restart. It is really convenient for all credentials to be handled by HAProxy.

Host environment requirements, please confirm that you have the following two commands
(How to install Ubuntu? apt-get install -y haproxy cerbot, I guess, if not, please Google)

 root@ptest:/usr/local/scripts# which certbot
/usr/bin/certbot
root@ptest:/usr/local/scripts# which haproxy
 /usr/sbin/haproxy

This article introduces how I use bash to generate the whole process of credentials.

Step 1: Create a bash for generating certificates, I named it gen_cert.sh

First of all, I wrote a bash called gen_cert.sh for generating certificates. My habit is to put it under /usr/local/scripts, and only root can access it.

 #!/bin/bash
/usr/bin/certbot\
--email unknown@ccc.tc\
--no-eff-email \
--webroot -w /var/www/html \
-d ccc.tc \
-d www.ccc.tc\
in the middle...skip a bunch
-d chat.ccc.tc \
-d speed.ccc.tc \
-d chat.ccc.tc \
-d video.ccc.tc \
 cert only

Here --webroot -w is the root directory of the verification file, which means that when the command is executed, the directory below will be placed in the verification file
/var/www/html/.well-known/acme-challenge
The other -d parameter is the domain name of the certificate you want to apply for

Before starting, let me explain a little bit, the DNS Record of my domain name is set to *.ccc.tc,
In other words, if all subdomains do not specify A Record, the IP will point to the IP location of 220.134.172.98.

OK, I'll try it out for everyone

 root@ptest:/usr/local/scripts# host -ta eewoieoiewo.ccc.tc
 eewoieoiewo.ccc.tc has address 220.134.172.98

因為這樣，大家或許猜到了，當我要申請域名時，只要在gen_cert上，補上-d <我要的域名即可>，再執行./gen_cert.sh即可。 Because of this, you may have guessed that when I want to apply for a domain name, I just need to add -d on the gen_cert, and then execute ./gen_cert.sh.
Example: -d dtest.ccc.tc \

 /usr/bin/certbot \
--email unknown@ccc.tc\
--no-eff-email \
--webroot -w /var/www/html \
-d ccc.tc \
-d www.ccc.tc\
in the middle...skip a bunch
-d chat.ccc.tc\
-d speed.ccc.tc\
-d chat.ccc.tc\
-d video.ccc.tc \
-d dtest.ccc.tc \
 cert only

Above, the bash description of this production certificate is completed.
That is to say, when Let's encrpyt needs to be verified, no matter what the subdomain name is, as long as the following path is /.well-known/acme-challenge, it will always lead to the local HTTP Server I set.

Step 2: We need to install a local HTTP Server
Here my example uses apache2, and the root directory of the website is /var/www/html, which is the location specified by bash above (--webroot -w /var/www/html).
You can use any Web Server or Docker you can install as long as you like, but one thing to pay attention to is not to open port 80, because this is reserved for HAProxy.

 root@ptest:/etc/apache2/sites-enabled# systemctl status apache2
● apache2.service - The Apache HTTP Server
Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2022-11-26 21:47:31 CST; 2 weeks 0 days ago
Docs: https://httpd.apache.org/docs/2.4/
Main PID: 701 (apache2)
Tasks: 55 (limit: 4915)
CGroup: /system.slice/apache2.service
├─ 701 /usr/sbin/apache2 -k start
├─19344 /usr/sbin/apache2 -k start
 └─19345 /usr/sbin/apache2 -k start

In my environment, apache2 was lucky to start the connection port default is port 8080 instead of port 80.

 root@ptest:/etc/apache2/sites-enabled# head -n1 000-default.conf

root@ptest:/etc/apache2/sites-enabled# netstat -ntlp|grep :8080
tcp6 0 0 :::8080 :::* LISTEN 701/apache2
 root@ptest:/etc/apache2/sites-enabled#

Step 3: Set HAProxy access rules
The HAProyx configuration file should be in /etc/haproxy/haproxy.cfg
Please use your favorite editor to open and edit. Before you start, let's take a look at the following two important settings.

 acl acme path_beg -i /.well-known/acme-challenge
 redirect scheme https code 301 if !{ ssl_fc } !acme

Article 1: Set the starting path as /.well-known/acme-challenge to use the acme access rule. Article 2: If it does not meet the acme rules, it will automatically jump to HTTPS, that is,
If the URL is long like http://abc.ccc.tc/.well-known/acme-challenge, it will not be redirected.
In order to make everyone feel more comfortable, the domain names below are just typed up by me. Basically, they all open the same file. You can try it out.
Even if you change the name of the subdomain name casually, you can see the Hello World text.

http://iewoieoieo.ccc.tc/.well-known/acme-challenge/test.txt

http://lksjlkjkalfjlsjlj.ccc.tc/.well-known/acme-challenge/test.txt

Now, let's take a look at the content of /etc/haproxy/haproxy.cfg, which is probably like the following, I only list the key points,
Have you noticed? Use_backend local if acme is set at the bottom of the content.
What he means is very intuitive. No matter what the domain name is, as long as the path is /.well-known/acme-challenge and meets the access rules of acme, the local backend service (use_backend local) will be used.
The name "local" is a name I made at random, you can choose the name you want.

 frontend proxy
mode http
bind :443 ssl crt /etc/ssl/ccc.tc/ccc.tc.pem alpn h2,http/1.1 crt /etc/ssl/ccc.tc/demo.ccc.tc.pem alpn h2,http/1.1 crt / etc/ssl/ccc.tc/e-course.app.pem alpn h2,http/1.1
bind *:80
acl acme path_beg -i /.well-known/acme-challenge
redirect scheme https code 301 if !{ ssl_fc } !acme

#acme-challenge You should have many backends like me, and the lcoal server of acme should be placed on the top
use_backend local if acme
use_backend material_server if material_url
use_backend wss_server if wss_url
use_backend webrtc_server if webrtc_url
 use_backend linebot_server if linebot_url

In use_backend local if acme, it points to the host set in the "second step" above

 backend local
mode http
balance round robin
 server ptest 127.0.0.1:8080

My own habit is to create an alias verification file to check whether the settings of haproyx are wrong

 alias hat='haproxy -f /etc/haproxy/haproxy.cfg -c'

like this

 root@ptest:~# hat
 Configuration file is valid

If there is no problem, basically, I want to generate certificates and sub-domain names, just adjust gen_cert.sh to generate them, super convenient.

To confirm whether your settings are valid, you can touch a file under the verified directory (/var/www/html/.well-known/acme-challenge),
Then open it from the host side through the URL to see if it works.

Because I have already generated a lot, the example below will ask me whether to expand the certificate. You can also see that there are a bunch of subdomains in the screen below.
So much so that I have forgotten what it is for 😆, I know how easy it is for me to use this environment to generate credentials.

 root@ptest:/usr/local/scripts# ./gen_cert.sh
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Attempting to parse the version 1.22.0 renewal configuration file found at /etc/letsencrypt/renewal/ccc.tc.conf with version 0.31.0 of Certbot. This might not work.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
You have an existing certificate that contains a portion of the domains you
requested (ref: /etc/letsencrypt/renewal/ccc.tc-0003.conf)

It contains these names: ccc.tc, 627csizs.ccc.tc, admin.ccc.tc, ai.ccc.tc,
auth2.ccc.tc, bbb.ccc.tc, cbk.ccc.tc, ccc-chat.ccc.tc, chat.ccc.tc,
chatroom.ccc.tc, cool.ccc.tc, d2.ccc.tc, ddm.ccc.tc, demo.ccc.tc, drive.ccc.tc,
dtest.ccc.tc, fb.ccc.tc, git.ccc.tc, gw.ccc.tc, imac.ccc.tc, ip.ccc.tc,
laravel.ccc.tc, line.ccc.tc, linebot.ccc.tc, linux.ccc.tc, mail.ccc.tc,
material.ccc.tc, nas.ccc.tc, notes.ccc.tc, pdf.ccc.tc, photo.ccc.tc,
phpenv.ccc.tc, pi4.ccc.tc, push.ccc.tc, reg.ccc.tc, sagent.ccc.tc,
sldrive.ccc.tc, speed.ccc.tc, station.ccc.tc, test.ccc.tc, tn.ccc.tc,
tools.ccc.tc, vbk.ccc.tc, video.ccc.tc, vm.ccc.tc, webrtc.ccc.tc, wss.ccc.tc,
www.ccc.tc, yty.ccc.tc

You requested these names for the new certificate: ccc.tc, 627csizs.ccc.tc,
admin.ccc.tc, chat.ccc.tc, ddm.ccc.tc, drive.ccc.tc, fb.ccc.tc, imac.ccc.tc,
ip.ccc.tc, laravel.ccc.tc, line.ccc.tc, linux.ccc.tc, nas.ccc.tc, notes.ccc.tc,
station.ccc.tc, pdf.ccc.tc, reg.ccc.tc, sagent.ccc.tc, www.ccc.tc, vm.ccc.tc,
cool.ccc.tc, photo.ccc.tc, bbb.ccc.tc, yty.ccc.tc, cbk.ccc.tc, auth2.ccc.tc,
webrtc.ccc.tc, vbk.ccc.tc, d2.ccc.tc, git.ccc.tc, push.ccc.tc, video.ccc.tc,
dtest.ccc.tc, mail.ccc.tc, ccc-chat.ccc.tc, tools.ccc.tc, sldrive.ccc.tc,
tn.ccc.tc, pi4.ccc.tc, gw.ccc.tc, wss.ccc.tc, demo.ccc.tc, test.ccc.tc,
ai.ccc.tc, speed.ccc.tc, linebot.ccc.tc, chatroom.ccc.tc, material.ccc.tc

Do you want to expand and replace this existing certificate with the new
certificate?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(E)xpand/(C)ancel:

I press the uppercase E to expand the certificate.

Step 4: Credential update renew.sh or credential reload reload.sh
Since the certificate is controlled by HAProxy, when a new certificate is generated, we need to reorganize the content of the new certificate to HAProyx, and reload the haproxy service.
You can create two files, one is used for renewing certificates, and the other does not have a renew command. In this way, the execution of reload.sh is faster when generating new certificates, and there is no need to check whether the certificate needs to be updated.

 #!/bin/bash
#Update Credentials
#/usr/bin/docker run --rm \
# -v /etc/letsencrypt:/etc/letsencrypt \
# -v /var/www/html:/html \
# certbot/certbot:arm32v6-latest \
# --webroot -w /html/ \
#renew

/usr/bin/certbot\
--webroot -w /var/www/html/ \
renew

#Reorganization
FOLDER=ccc.tc
cat /etc/letsencrypt/live/${FOLDER}/fullchain.pem \
/etc/letsencrypt/live/${FOLDER}/privkey.pem > \
/etc/ssl/ccc.tc/${FOLDER}.pem

# restart HAProxykk
 systemctl reload haproxy

Step 5: If it is confirmed that there is no problem with manual execution of renew.sh, put it into the schedule and run automatic update every day

 0 0 * * * /usr/local/scripts/renew.sh

Summary
Assuming your settings are correct and you understand my instructions in this article, all backend hosts passing through this HAProxy will be encrypted by HAProyx,
All back-end hosts do not need to set credentials at all, and the credentials of all back-end hosts will be automatically updated.

The following is a schematic description

 use_backend local if acme

use_backend material_server if material_url
use_backend wss_server if wss_url
use_backend webrtc_server if webrtc_url
use_backend linebot_server if linebot_url

backend ai_server
mode http
balance round robin
fullconn 10000
cookie SITEID insert indirect nocache
server tplink 192.168.99.181:8000

backend gw_router
mode http
balance round robin
fullconn 10000
cookie SITEID insert indirect nocache
server tplink 192.168.99.254:80

backend nas
mode http
balance round robin
fullconn 10000
cookie SITEID insert indirect nocache
server nas 192.168.99.130:5000 check cookie synology