higress使用了解

了解下 http-echo、httpbin 镜像使用

未按文档实际搭建，但大致是这样

官方文档：https://higress.io/zh-cn/docs/overview/what-is-higress

了解：利用sealos快速安装kubernetes集群：https://note.youdao.com/s/M2z4OzsL

# 使用Helm 安装Higress
[root@vm ~]# helm repo add higress.io https://higress.io/helm-charts
[root@vm ~]# helm install higress -n higress-system higress.io/higress --create-namespace --render-subchart-notes --set higress-console.domain=console.higress.io

# 登录密码查看
[root@vm ~]# export ADMIN_USERNAME=$(kubectl get secret --namespace higress-system higress-console -o jsonpath="{.data.adminUsername}" | base64 -d)
[root@vm ~]# export ADMIN_PASSWORD=$(kubectl get secret --namespace higress-system higress-console -o jsonpath="{.data.adminPassword}" | base64 -d)
[root@vm ~]# echo -e "Username: ${ADMIN_USERNAME}\nPassword: ${ADMIN_PASSWORD}"

1、测试

#创建一个pod
kind: Pod
apiVersion: v1
metadata:
  name: foo-app
  labels:
    app: foo
spec:
  containers:
  - name: foo-app
    image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/http-echo:0.2.4-alpine
    args:
    - "-text=foo"    # 访问返回 foo
---
kind: Service
apiVersion: v1
metadata:
  name: foo-service
spec:
  selector:
    app: foo
  ports:
  # Default port used by the image
  - port: 5678

# 使用 Ingress CRD 进行路由配置，编写foo-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: foo
spec:
  ingressClassName: higress # 使用higress 只需要在ingress里添加ingressClassName
  rules:
  - host: foo.bar.com
    http:
      paths:
      - pathType: Prefix
        path: "/foo"
        backend:
          service:
            name: foo-service
            port:
              number: 5678 #

# 集群内访问  
curl http://10.96.1.19/foo -H 'host: foo.bar.com'    # foo

2、Higress流量治理实战

在Higress上可以使用Ingress并借助Annotation实现高阶流量治理

2.1 灰度发布

Higress提供复杂的路由处理能力，支持基于Header、Cookie以及权重的灰度发布功能。灰度发布功能可以通过设置注解来实现，为了启用灰度发布功能，
需要设置注解：higress.io/canary: "true"。通过不同注解可以实现不同的灰度发布功能。
说明：当多种方式同时配置时，灰度方式选择优先级为：基于Header > 基于Cookie > 基于权重（从高到低）。

部署两个版本的服务

1）在集群中部署第一个版本的 Deployment，本文以 nginx-v1 为例。YAML 示例如下：

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-v1
spec:
  type: ClusterIP
  ports:
  - port: 80
    protocol: TCP
    name: http
  selector:
    app: nginx
    version: v1
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v1
  template:
    metadata:
      labels:
        app: nginx
        version: v1
    spec:
      containers:
      - name: nginx
        image: "openresty/openresty:centos"
        ports:
        - name: http
          protocol: TCP
          containerPort: 80
        volumeMounts:
        - mountPath: /usr/local/openresty/nginx/conf/nginx.conf
          name: config
          subPath: nginx.conf
      volumes:
      - name: config
        configMap:
          name: nginx-v1

---
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: nginx
    version: v1
  name: nginx-v1
data:
  nginx.conf: |-
    worker_processes  1;

    events {
        accept_mutex on;
        multi_accept on;
        use epoll;
        worker_connections  1024;
    }

    http {
        ignore_invalid_headers off;
        server {
            listen 80;
            location / {
                access_by_lua '
                    local header_str = ngx.say("nginx-v1")
                ';
            }
            location /hello {
                access_by_lua '
                    local header_str = ngx.say("hello nginx-v1")
                ';
            }
        }
    }

2）再部署第二个版本的 Deployment，本文以 nginx-v2 为例。YAML 示例如下：

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-v2
spec:
  type: ClusterIP
  ports:
  - port: 80
    protocol: TCP
    name: http
  selector:
    app: nginx
    version: v2
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v2
  template:
    metadata:
      labels:
        app: nginx
        version: v2
    spec:
      containers:
      - name: nginx
        image: "openresty/openresty:centos"
        ports:
        - name: http
          protocol: TCP
          containerPort: 80
        volumeMounts:
        - mountPath: /usr/local/openresty/nginx/conf/nginx.conf
          name: config
          subPath: nginx.conf
      volumes:
      - name: config
        configMap:
          name: nginx-v2
---
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: nginx
    version: v2
  name: nginx-v2
data:
  nginx.conf: |-
    worker_processes  1;

    events {
        accept_mutex on;
        multi_accept on;
        use epoll;
        worker_connections  1024;
    }

    http {
        ignore_invalid_headers off;
        server {
            listen 80;
            location / {
                access_by_lua '
                    local header_str = ngx.say("nginx-v2")
                ';
            }
            location /hello {
                access_by_lua '
                    local header_str = ngx.say("hello nginx-v2")
                ';
            }
        }
    }

基于Header灰度发布

• 只配置higress.io/canary-by-header：基于Request Header的名称进行流量切分。当请求包含该Header并且值为always时，请求流量会被分配到灰度服务入口；其他情况时，请求流量不会分配到灰度服务。
• 同时配置higress.io/canary-by-header和higress.io/canary-by-header-value：基于Request Header的名称和值进行流量切分。当请求中的header的名称和header的值与该配置匹配时，请求流量会被分配到灰度服务；其他情况时，请求流量不会分配到灰度服务。

1、请求Header为higress: always时将访问灰度服务nginx-v2；其他情况将访问正式服务nginx-v1，配置如下：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    higress.io/canary: "true"
    higress.io/canary-by-header: "higress"
  name: higress-demo-canary
spec:
  ingressClassName: higress
  rules:
    - http:
        paths:
          - backend:
              service:
                name: nginx-v2
                port: 
                  number: 80
            path: /hello
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: higress-demo
spec:
  ingressClassName: higress
  rules:
    - http:
        paths:
          - backend:
              service:
                name: nginx-v1
                port: 
                  number: 80
            path: /hello
            pathType: Exact  

curl  -H "higress: always" http://10.96.1.19/hello   # hello nginx-v2

2、请求Header为higress: v2和higress: always时将访问灰度服务nginx-v2；其他情况将访问正式服务nginx-v1，配置如下：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    higress.io/canary: "true"
    higress.io/canary-by-header: "higress"
    higress.io/canary-by-header-value: "v2"
  name: higress-demo-canary
spec:
  ingressClassName: higress
  rules:
    - http:
        paths:
          - backend:
              service:
                name: nginx-v2
                port: 
                  number: 80
            path: /hello
            pathType: Exact

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: higress-demo
spec:
  ingressClassName: higress
  rules:
    - http:
        paths:
          - backend:
              service:
                name: nginx-v1
                port: 
                  number: 80
            path: /hello
            pathType: Exact

curl  -H "higress: v2" http://10.96.1.19/hello  # hello nginx-v2
curl  -H "higress: always" http://10.96.1.19/hello  # hello nginx-v2
curl  -H "higress: v1" http://10.96.1.19/hello  # hello nginx-v1
curl  -H "higress: always1" http://10.96.1.19/hello  # hello nginx-v1

基于Cookie灰度发布

• higress.io/canary-by-cookie：基于Cookie的Key进行流量切分。当请求的Cookie中含有该Key且其值为always时，请求流量将被分配到灰度服务；其他情况时，请求流量将不会分配到灰度服务。
  说明：基于Cookie的灰度发布不支持自定义设置Key对应的值，只能是always。
  请求的Cookie为demo=always时将访问灰度服务nginx-v2；其他情况将访问正式服务nginx-v1。配置如下：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    higress.io/canary: "true"
    higress.io/canary-by-cookie: "demo"
  name: higress-demo-canary
spec:
  ingressClassName: higress
  rules:
    - http:
        paths:
          - backend:
              service:
                name: nginx-v2
                port: 
                  number: 80
            path: /hello
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: higress-demo
spec:
  ingressClassName: higress
  rules:
    - http:
        paths:
          - backend:
              service:
                name: nginx-v1
                port: 
                  number: 80
            path: /hello
            pathType: Exact

curl  --cookie "demo=always" http://10.96.1.19/hello

基于权重灰度发布

• higress.io/canary-weight：设置请求到指定服务的百分比（值为0~100的整数）
• higress.io/canary-weight-totatl：设置权重总和，默认为100
  配置灰度服务nginx-v2的权重为30%，配置正式服务nginx-v1的权重为70%。

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    higress.io/canary: "true"
    higress.io/canary-weight: "30"
  name: higress-demo-canary
spec:
  ingressClassName: higress
  rules:
    - http:
        paths:
          - backend:
              service:
                name: nginx-v2
                port: 
                  number: 80
            path: /hello
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: higress-demo
spec:
  ingressClassName: higress
  rules:
    - http:
        paths:
          - backend:
              service:
                name: nginx-v1
                port: 
                  number: 80
            path: /hello
            pathType: Exact

2.2 跨域测试

测试页面 保存为html文件，浏览器打开即可

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
    <script src="http://apps.bdimg.com/libs/jquery/1.9.1/jquery.min.js"></script>
</head>
<body>
  <h3 id="demo"></h3>
</body>
<script>
    $.get('http://192.168.65.130:30332/hello',function(data){
        console.log(data)
        $('#demo').html(data);
    });
</script>
</html>

跨域资源共享CORS（Cross-Origin Resource Sharing）是指允许Web应用服务器进行跨域访问控制，从而实现跨域数据安全传输。

• higress.io/enable-cors：“true” or “false”。开启或关闭跨域。
• higress.io/cors-allow-origin：允许的第三方站点，支持泛域名，逗号分隔；支持通配符。
• higress.io/cors-allow-methods：允许的请求方法，如GET、POST，逗号分隔；支持通配符*。默认值为GET, PUT, POST, DELETE, PATCH, OPTIONS。
• higress.io/cors-allow-headers：允许的请求头部，逗号分隔；支持通配符*。默认值为DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization。
• higress.io/cors-expose-headers：允许的响应头部，逗号分隔。
• higress.io/cors-allow-credentials：“true” or “false”。是否允许携带凭证信息。默认允许。
• higress.io/cors-max-age：预检结果的最大缓存时间，单位为秒；默认值为1728000。

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    higress.io/enable-cors: "true"
    higress.io/cors-allow-origin: "*"
    higress.io/cors-allow-methods: "GET,POST"
    
    higress.io/cors-allow-credentials: "false"
  name: higress-demo
spec:
  ingressClassName: higress
  rules:
    - http:
        paths:
          - backend:
              service:
                name: nginx-v1
                port: 
                  number: 80
            path: /hello
            pathType: Exact

部署之后测试，可以正常访问

2.3 Rewrite重写

在请求转发给目标后端服务之前，重写可以修改原始请求的路径（Path）和主机域（Host)。

• higress.io/rewrite-target：重写Path。
• higress.io/upstream-vhost：重写Host。
  准备测试服务httpbin
  httpbin.org 一个简单的 HTTP 请求和响应服务，用 Python + Flask 编写。 #调试 各种响应结果
  官网地址：https://httpbin.org/

使用helm安装httpbin
helm repo add rgnu https://gitlab.com/mulesoft-int/helm-repository/-/raw/master/
helm install my-httpbin rgnu/httpbin --version 1.0.0
cur 10.96.1.15/get?name=fox  #访问httpbin

Rewrite重写Path

1. 将请求example.com/test在转发至后端服务之前，重写为example.com/get

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    higress.io/rewrite-target: "/get"
  name: higress-demo
spec:
  ingressClassName: higress
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: my-httpbin
                port: 
                  number: 80
            path: /test
            pathType: Exact

curl 10.96.1.19/test?name=fox -H "host: example.com"   # 响应 http://example.com/get

2.将请求example.com/v1/get在转发至后端服务之前，去掉Path前缀/v1

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    higress.io/rewrite-target: "/$2"
  name: higress-demo
spec:
  ingressClassName: higress
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: my-httpbin
                port: 
                  number: 80
            path: /v1(/|$)(.*) # $1表示路径中正则表达式匹配的第一个()的内容，$2为第二个，以此类推。
            pathType: ImplementationSpecific

curl 10.96.1.19/v1/get?name=fox -H "host: example.com"  #响应 http://example.com/get

2.4 Header控制

通过Header控制，您可以在转发请求到后端服务之前对请求Header进行增删改，在收到响应转发给客户端时对响应Header进行增删改。

请求Header控制

• higress.io/request-header-control-add：请求在转发给后端服务时，添加指定Header。若该Header存在，则其值拼接在原有值后面。语法如下：
• 单个Header：Key Value
• 多个Header：使用yaml特殊符号 |，每对Key Value单独处于一行
• higress.io/request-header-control-update：请求在转发给后端服务时，修改指定Header。若该header存在，则其值覆盖原有值。语法如下：
• 单个Header：Key Value
• 多个Header：使用yaml特殊符号 |，每对Key Value单独处于一行
• higress.io/request-header-control-remove：请求在转发给后端服务时，删除指定Header。语法如下：
• 单个Header：Key
• 多个Header：逗号分隔

1.对于请求example.com/headers添加两个Header，foo: bar和test: true。

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    higress.io/request-header-control-add: |
      foo: bar
      test: true
  name: demo
spec:
  ingressClassName: higress
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: my-httpbin
                port: 
                  number: 80
            path: /headers
            pathType: Exact

curl 10.96.1.19/headers -H "host: example.com"   #  添加了请求头 foo test

2. Header控制可以结合灰度发布，对灰度流量进行染色。请求Header为higress：v2时将访问灰度服务nginx-v2，并添加Header，stage: gray；其他情况将访问正式服务nignx-v1，并添加Header，stage: prod。配置如下：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    higress.io/canary: "true"
    higress.io/canary-by-header: "higress"
    higress.io/canary-by-header-value: "v2"
    higress.io/request-header-control-add: "stage gray"
  name: higress-demo-canary
spec:
  ingressClassName: higress
  rules:
    - http:
        paths:
          - backend:
              service:
                name: my-httpbin-canary
                port: 
                  number: 80
            path: /headers
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    higress.io/request-header-control-add: "stage prod"
  name: higress-demo
spec:
  ingressClassName: higress
  rules:
    - http:
        paths:
          - backend:
              service:
                name: my-httpbin
                port: 
                  number: 80
            path: /headers
            pathType: Exact

helm install my-httpbin-canary rgnu/httpbin --version 1.0.0
部署之后执行以下命令进行测试
curl 10.96.1.19/headers -H "higress: v2"    # 请求头带 stage:gray

响应Header控制

• higress.io/response-header-control-add：请求在收到后端服务响应之后并且转发响应给客户端之前，添加指定Header。若该Header存在，则其值拼接在原有值后面。语法如下：
• 单个Header：Key Value
• 多个Header：使用yaml特殊符号 |，每对Key Value单独处于一行
• higress.io/response-header-control-update：请求在收到后端服务响应之后并且转发响应给客户端之前，修改指定Header。若该header存在，则其值覆盖原有值。语法如下：
• 单个Header：Key Value
• 多个Header：使用yaml特殊符号 |，每对Key Value单独处于一行
• higress.io/response-header-control-remove：请求在收到后端服务响应之后并且转发响应给客户端之前，删除指定Header。语法如下：
• 单个Header：Key
• 多个Header：逗号分隔
  对于请求example.com/headers的响应删除Header：req-cost-time。

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    higress.io/response-header-control-remove: "req-cost-time"
  name: higress-demo
spec:
  ingressClassName: higress
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: my-httpbin
                port: 
                  number: 80
            path: /headers
            pathType: Exact

正常的响应结果
# -I 打印响应头
curl 10.96.1.19/headers -H "host: example.com" -I

Higress插件实战

https://higress.io/zh-cn/docs/plugins/intro

https://higress.io/zh-cn/docs/plugins/intro
3.1 通过 Higress 控制台进行配置
Higress 控制台提供了 3 个入口进行插件配置:
1.全局配置：插件市场->选择插件进行配置
2.域名级配置：域名管理->选择域名->点击策略->选择插件进行配置
3.路由级配置: 路由配置->选择路由->点击策略->选择插件进行配置
这三个配置的生效优先级是: 路由级 > 域名级 > 全局
3.2 通过 Higress WasmPlugin CRD 进行配置
Higress WasmPlugin CRD 在 Istio WasmPlugin CRD 的基础上进行了扩展，新增以下配置字段：

全局插件，全局限流  路由--策略配置限流
wasm插件使用

kubectl get wasmPlugin -n higress-system
kubectl describe wasmPlugin key-rate-limit-1.0.0 -n higress-system

参考：https://note.youdao.com/s/cAE1VMEN