Management Service MQTT Messaging
Send kubectl
This section describes the MQTT messages and topics the Management Service handles to perform the "send-kubectl" action. When using AnythingDB in Studio all of this is taken care of within the product.
Sending a Message to the Management Service
The MQTT topic the Management Service subscribes to, for running kubectl commands
is:
spaces/<space>/categories/cluster/things/<thing-id>/actions/send-kubectlThe message format for publishing is as follows:
{
"send-kubectl": {
"input": {
"correlationId": "{create-a-unique-id-for-each-message}",
"command": "kubectl",
"arguments": [
"{each}",
"{new}",
"{argument}",
"{needs}",
"{its}",
"{own}",
"{array}",
"{entry}"
]
},
"status": "pending",
"timeRequested": "<datetime>",
"href": "/spaces/<space>/categories/cluster/things/<thing-id>/actions/send-kubectl/<action-id>"
}
}
For example:
{
"send-kubectl": {
"input": {
"correlationId": "666",
"command": "kubectl",
"arguments": [
"get",
"nodes",
"-o",
"wide"
]
},
"status": "pending",
"timeRequested": "2020-05-26 15:37:46+0000",
"href": "/spaces/my-space/categories/cluster/things/my-thing-id/actions/send-kubectl/1234"
}
}
It is important that each argument is put in a new array entry. If there is a space in the complete command then that means a new argument.
Status Update After Receiving Action Request
MQTT topic the message is sent
to:
spaces/<space>/categories/cluster/things/<thing-id>/actionsMessage:
{
"send-kubectl": {
"href": "/spaces/<space>/categories/cluster/things/<thing-id>/actions/send-kubectl/<action-id>",
"status": "received"
}
}
Event Publish After Running Action
MQTT topic the message is sent
to:
spaces/<space>/categories/cluster/things/<thing-id>/eventsMessage:
{
"kubectl-logs": {
"data": {
"correlationId": "{the-unique-id-from-the-request}",
"statusCode": "{an-http-status-code}",
"statusMessage": "{optional descriptive message}",
"response": "{the output in plain text format}"
}
}
}
Example:
{
"kubectl-logs": {
"data": {
"correlationId": "666",
"statusCode": 200,
"statusMessage": "Command executed successfully",
"response": "NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME\nmartin-ag150 Ready master 50d v1.18.2+k3s1 10.10.10.100 \u003cnone\u003e Ubuntu 20.04 LTS 5.4.0-40-generic containerd://1.3.3-k3s2\n"
}
}
}You need to use the correlationId to match the response to the
request.
Status Update After Completing Action
MQTT topic the message is sent
to:
spaces/<space>/categories/cluster/things/<thing-id>/actionsMessage:
{
"send-kubectl": {
"href": "/spaces/<space>/categories/cluster/things/<thing-id>/actions/send-kubectl/<action-id>",
"status": "completed"
}
}Send kubernetes API
This section describes the MQTT messages and topics the Management Service handles to
perform the send-kubernetes-api action. When using AnythingDB in Studio all
of this is taken care of within the product.
Sending a Message to the Management Service
The MQTT topic the Management Service subscribes to, for running kubernetes API
requests
is:
spaces/<space>/categories/cluster/things/<thing-id>/actions/send-kubernetes-apiThe message format for publishing is as follows:
{
"send-kubernetes-api": {
"input": {
"correlationId": "{create-a-unique-id-for-each-message}",
"requestMethod": "{GET|POST|PUT|PATCH|DELETE}",
"href": "{see API docs}",
"requestBody": "{body if required}"
},
"status": "pending",
"timeRequested": "<datetime>",
"href": "/spaces/<space>/categories/cluster/things/<thing-id>/actions/send-kubernetes-api/<action-id>"
}
}The requestBody is a string. For a JSON requestBody
this means it has to be stringified first. This results in "double
JSON-encoding".
For example:
{
"send-kubernetes-api":{
"input":{
"correlationId": "666",
"requestMethod": "GET",
"href": "/api/v1/nodes",
"requestBody": ""
},
"status": "pending",
"timeRequested": "2020-05-26 15:37:46+0000",
"href": "/spaces/may-space/categories/cluster/things/my-thing-id/actions/send-kubernetes-api/1234"
}
}
Status Update After Receiving Action Request
MQTT topic the message is sent
to:
spaces/<space>/categories/cluster/things/<thing-id>/actionsMessage:
{
"send-kubernetes-api": {
"href": "/spaces/<space>/categories/cluster/things/<thing-id>/actions/send-kubernetes-api/<action-id>",
"status": "received"
}
}Event Publish After Running Action
MQTT topic the message is sent
to:
spaces/<space>/categories/cluster/things/<thing-id>/eventsMessage:
{
"send-kubernetes-api": {
"data": {
"correlationId": "{the-unique-id-from-the-request}",
"statusCode": "{an-http-status-code}",
"statusMessage": "{optional descriptive message}",
"response": "{the output of the API request}"
}
}
}Example:
{
"send-kubernetes-api": {
"data": {
"correlationId": "666",
"statusCode": 200,
"statusMessage": "Command executed successfully",
"response": ""
}
}
}You need to use the correlationId to match the response to the
request.
Status Update After Completing Action
MQTT topic the message is sent
to:
spaces/<space>/categories/cluster/things/<thing-id>/actionsMessage:
{
"send-kubernetes-api": {
"href": "/spaces/<space>/categories/cluster/things/<thing-id>/actions/send-kubernetes-api/<action-id>",
"status": "completed"
}
}
Metrics
This section describes the MQTT messages and topics the Management Service handles to perform the "run-stats" and "stop-stats" actions. When using AnythingDB in Studio all of this is taken care of within the product.
Metrics / Statistics Data
The Management Service publishes metrics for the nodes and deployments of the cluster at a regular interval (default: 30 seconds). This data is obtained from the Kubernetes Metrics Server which is installed by default in a K3s installation.
Start the Metrics / Update the Interval
The metrics publishing can be started, or the interval can be updated, by sending a
message to the following
topic:
status/<space>/categories/cluster/things/<thing-id>/actions/run-statsThe message format for publishing is as follows:
{
"run-stats": {
"input": {
"interval": {in-seconds}
},
"status": "pending",
"timeRequested": "<datetime>",
"href": "<space>/categories/cluster/things/<thing-id>/actions/run-stats/<action-id>"
}
}For example:
{
"run-stats": {
"input": {
"interval": 15
},
"status": "pending",
"timeRequested": "2020-06-18 10:32:46+0000",
"href": "martin/things/ag150/actions/run-stats/1234"
}
}Stop the Metrics
The metrics publishing can be stopped by sending a message to the following
topic:
status/<space>/categories/cluster/things/<thing-id>/actions/stop-statsThe message format for publishing is as follows:
{
"stop-stats": {
"status": "pending",
"timeRequested": "<datetime>",
"href": "<space>/categories/cluster/things/<thing-id>/actions/stop-stats/<action-id>"
}
}For example:
{
"stop-stats": {
"status": "pending",
"timeRequested": "2020-06-18 10:36:46+0000",
"href": "martin/things/ag150/actions/stop-stats/1234"
}
}Example of Node-stats
{
"nodes-stats": {
"data": [
{
"capacities": {
"cpu": 4000000000,
"memory": 1937969152,
"pods": 110,
"storage": 31084183552
},
"conditions": [
{
"lastHeartbeatTime": 1607879049,
"lastTransitionTime": 1607879049,
"message": "Flannel is running on this node",
"reason": "FlannelIsUp",
"status": "False",
"type": "NetworkUnavailable"
},
{
"lastHeartbeatTime": 1608055458,
"lastTransitionTime": 1607592799,
"message": "kubelet has sufficient memory available",
"reason": "KubeletHasSufficientMemory",
"status": "False",
"type": "MemoryPressure"
},
{
"lastHeartbeatTime": 1608055458,
"lastTransitionTime": 1607592799,
"message": "kubelet has no disk pressure",
"reason": "KubeletHasNoDiskPressure",
"status": "False",
"type": "DiskPressure"
},
{
"lastHeartbeatTime": 1608055458,
"lastTransitionTime": 1607592799,
"message": "kubelet has sufficient PID available",
"reason": "KubeletHasSufficientPID",
"status": "False",
"type": "PIDPressure"
},
{
"lastHeartbeatTime": 1608055458,
"lastTransitionTime": 1607879059,
"message": "kubelet is posting ready status. AppArmor enabled",
"reason": "KubeletReady",
"status": "True",
"type": "Ready"
}
],
"name": "ubuntu",
"stats": {
"cpu": 867753808,
"cpuPercentage": 21.693845200000002,
"memory": 1239392256,
"memoryPercentage": 63.95314676298831,
"timestamp": 1608143932,
"window": 30
}
}
]
}
}
Example of Deployments-stats
{
"deployments-stats": {
"data": {
"nodes": [
{
"deployments": [
{
"name": "ase-core-message",
"pods": [
{
"containers": [
{
"name": "ase-core-message",
"stats": {
"cpu": 105314661,
"cpuPercentage": 2.6328665250000003,
"memory": 140861440,
"memoryPercentage": 7.268507852905184,
"timestamp": 1608143929,
"window": 30
}
}
],
"name": "ase-core-message-866947bfb6-59g4f"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1605289453,
"lastUpdateTime": 1607448834,
"message": "ReplicaSet \"ase-core-message-866947bfb6\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1607879075,
"lastUpdateTime": 1607879075,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 4,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-core-management",
"pods": [
{
"containers": [
{
"name": "ase-core-management",
"stats": {
"cpu": 804966,
"cpuPercentage": 0.02012415,
"memory": 15163392,
"memoryPercentage": 0.7824372221999125,
"timestamp": 1608143927,
"window": 30
}
}
],
"name": "ase-core-management-764f8d8b66-pv2gf"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1605795610,
"lastUpdateTime": 1607678997,
"message": "ReplicaSet \"ase-core-management-764f8d8b66\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1607879075,
"lastUpdateTime": 1607879075,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 23,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-core-cache",
"pods": [
{
"containers": [
{
"name": "ase-core-cache",
"stats": {
"cpu": 3803112,
"cpuPercentage": 0.0950778,
"memory": 2957312,
"memoryPercentage": 0.15259850740905911,
"timestamp": 1608143924,
"window": 30
}
}
],
"name": "ase-core-cache-676b54585d-jrncd"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1605289503,
"lastUpdateTime": 1607452737,
"message": "ReplicaSet \"ase-core-cache-676b54585d\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1607879075,
"lastUpdateTime": 1607879075,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 2,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-virtual-meter",
"pods": [
{
"containers": [
{
"name": "ase-virtual-meter",
"stats": {
"cpu": 1428236,
"cpuPercentage": 0.0357059,
"memory": 7585792,
"memoryPercentage": 0.391429966373376,
"timestamp": 1608143930,
"window": 30
}
}
],
"name": "ase-virtual-meter-655b56d958-vw7cn"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1607879074,
"lastUpdateTime": 1607879074,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": 1606748304,
"lastUpdateTime": 1607972823,
"message": "ReplicaSet \"ase-virtual-meter-655b56d958\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
}
],
"observedGeneration": 9,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-core-cloud",
"pods": [
{
"containers": [
{
"name": "ase-core-cloud",
"stats": {
"cpu": 861411,
"cpuPercentage": 0.021535275,
"memory": 6254592,
"memoryPercentage": 0.3227395025119574,
"timestamp": 1608143924,
"window": 30
}
}
],
"name": "ase-core-cloud-57486fd5f5-5drcv"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1607695991,
"lastUpdateTime": 1607696007,
"message": "ReplicaSet \"ase-core-cloud-57486fd5f5\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1607879068,
"lastUpdateTime": 1607879068,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 1,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-core-health",
"pods": [
{
"containers": [
{
"name": "ase-core-health",
"stats": {
"cpu": 2052389,
"cpuPercentage": 0.051309725,
"memory": 7409664,
"memoryPercentage": 0.38234168961632675,
"timestamp": 1608143930,
"window": 30
}
}
],
"name": "ase-core-health-7c4f9d966d-vw9mt"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1605289626,
"lastUpdateTime": 1607696276,
"message": "ReplicaSet \"ase-core-health-7c4f9d966d\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1607879074,
"lastUpdateTime": 1607879074,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 5,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
}
],
"name": "ubuntu"
}
]
}
}
}