Crmsh usage in high availability cluster corosync + pacemaker (2)

Time:2020-10-28

In the last blog, we talked about the installation of crmsh and the usage of related commands for configuring a resource to the corosync + pacemaker high availability cluster. Please refer to the https://www.cnblogs.com/qiuhom-1874/p/13592484.html Today, we will continue to talk about the usage of other common commands of crmsh;

Common commands of node

Show: view the definition information of the specified node. If no node is specified, it means to view the definition information of all nodes in the cluster;

Standby: set the specified node as standby state; if it is not specified by default, it means that the current crmsh node is set to standby state; standby state means that the node is in standby state and does not run any resources;

Command syntax format

Tip: lifetime specifies the life cycle of standby state. Reboot means to set the corresponding node to standby state until it is restarted. It means that as long as the corresponding node is restarted, the standby state will be converted to online state; and “forever” means that the specified node will always be in standby state as long as it is not manually set to online state, even if it is not set to online state manually Restart the corresponding node; if lifetime is not specified by default, it means that the specified node will be set to standby state permanently;

Example: set node01 to standby state

Tip: under normal circumstances, as long as the specified node is set to standby state, all resources running on it will be migrated to other nodes. If all nodes in the cluster are in standby mode, the corresponding resources will not be able to run. Therefore, we can see that the web service hosted on is in the stopped state;

Ready: set the specified node to non maintenance mode, that is, to set the specified node to be ready to go online;

Tip: if the specified node is set to ready state, it will add an attribute of maintenance = off in the node definition information by default, which means that the node is in non maintenance state. However, if the corresponding node is in standby state, even if we set the specified host as ready state, it will not go online. We can understand that the priority of standby state is higher than that of ready state Thinking is that as long as the corresponding node is in standby state, it does not work when the node is set to ready;

Online: set the specified node to be online;

Tip: after setting the specified online state, the corresponding resources hosted on the cluster will run automatically;

Attribute: set the specified definition attribute of the specified node;

Tip: you can see that after changing the standby attribute of node02 to off, the corresponding node is online; the corresponding resource is also migrated from node01 to node02; normally, the resource will not migrate by default without setting any resource’s preference to node. The reason for migration is that when we create a group, it generates a resource’s inclination to node in the configuration information; As shown below

Tip: the configuration in the red box indicates that the group resource of WebService has unlimited tendency to node02, which means that as long as node02 is online, the group resource of WebService will run to node02;

Server: view the host name list of the current cluster node;

Common commands of resource

Ban: remove the specified resource from the specified node, similar to manually moving the specified resource out of the specified node;

Command syntax format

Tips: RSC is the name of the specified resource; lifetime specifies the life cycle of its ban operation, which is similar to the lifecycle of standby command; force means mandatory operation;

Example: moving webip out of node01

Tip: we can see that after we move webip from node01, it will create a rule in the configuration file for the resource’s preference to the node specified by our ban. This means that as long as other nodes can run, the corresponding resource will never run on node01. Because webip, webserver and webstore belong to webservice, webip will not run on node01 Therefore, the resources in the corresponding group will also migrate with the migration of webip;

Cleanup: clear the status information of the specified node;

Example: clear the error information in the process of resource migration;

Tip: by default, the specified resource is not in the group. By default, it will only clear the status information related to the specified resource. From the above screenshot, we can see that the specified resource is in the group, so it will clear the status information of all resources in the group;

Clear: clear the resource relationship;

Tip: you can see that the location constraint relationship of the specified resource has been cleared. In fact, when we manually migrate resources, it will create location constraint configuration for us in the configuration information by default. Clear is equivalent to clearing the constraint configuration generated by migration;

Constraints: displays the constraints that affect the specified resource;

Locate: displays the location where the resource is running;

Migrate / move: manually migrate the specified resource to the specified node;

Command syntax format

Example: migrating webip to node02;

Tip: you can see that the corresponding webip resource has not been migrated. In fact, webip is in the same group as webserver and webstore, and webserver has negative infinity for node02, so WebService cannot be migrated to node02. Therefore, webip in the group is constrained by the group, and it will not be signed;

Tip: in fact, when you execute the migrate / move operation, it will generate a location constraint configuration in the configuration file; and it will configure that the propensity of the migrated node is positive infinity; if the resource is in the same group, the propensity rule of the resource in the group to the same node is: negative infinity is greater than positive infinity, and positive infinity is greater than a specific score; calculating the propensity of a group to nodes, the It is the sum of the propensities of the resources in the group to the nodes; therefore, the reason why the migration is not successful is that the web server’s propensity to node02 is negative infinity;

Delete the web server’s preference for node02 and see if webip will migrate to node02?

Tip: we can see that webip moved from node01 to node02 after deleting the web server’s preference for node02, and the corresponding group also moved to node02;

Stop: stop the specified resource;

Status: view the status information of the specified resource

Start: start the specified resource;

Restart: restart the specified resource;

Configure related instructions

Delete: delete the specified resource definition;

Tip: after the group resource is deleted, the corresponding resource will be redistributed to each node. If there is a location constraint, it will run according to the location constraint. If there is no location constraint, it will be distributed to each node of the cluster in a load balancing manner, as shown below

Location: sets the preference of the specified resource to the node;

Command syntax help

Usage:

location   [] {|}

rsc :: //
        | { resource_sets }
        | 

attributes :: role= | resource-discovery=always|never|exclusive

node_pref :: : 

rules ::
  rule [id_spec] [$role=] : 
  [rule [id_spec] [$role=] :  ...]

id_spec :: $id= | $id-ref=
score ::  |  | [-]inf
expression ::  [  ...]
bool_op :: or | and
simple_exp ::  [type:] 
          |  
          | date 
type :: string | version | number
binary_op :: lt | gt | lte | gte | eq | ne
unary_op :: defined | not_defined

date_expr :: lt 
         | gt 
         | in start= end=
         | in start= 
         | spec 
duration|date_spec ::
         hours=
         | monthdays=
         | weekdays=
         | yearsdays=
         | months=
         | weeks=
         | years=
         | weekyears=
         | moon=

Examples:

location conn_1 internal_www 100: node1

location conn_1 internal_www \
  rule 50: #uname eq node1 \
  rule pingd: defined pingd

location conn_2 dummy_float \
  rule -inf: not_defined pingd or pingd number:lte 0

# never probe for rsc1 on node1
location no-probe rsc1 resource-discovery=never -inf: node1

Example: set the propensity of webstore to node01 as 100 points;

Example: set the propensity of webserver to node01 as positive infinity;

Tip: in addition to defining the propensity of a resource to a node, it also depends on the stickiness to the node; as shown below

Conclusion: if the cluster has two nodes, node01 and node02, webip has a tendency of 100 for node01 and 200 for webserver;

Verification: set node01 to standby state to see if the corresponding webip and webserver will migrate to node02?

Tip: webip and webserver are running on node02 normally;

If node01 goes online again, webip and webserver will migrate back to node01?

Tip: by default, they are moved back to node01;

Set the stickiness of the default resource to the node to be 500, and then set node01 to standby mode again, and then go online to see if the corresponding resources will migrate back from node02 to node01?

crm(live)configure# property default-resource-stickiness=
default-resource-stickiness (integer, [(null)]): 
    Deprecated (use resource-stickiness in rsc_defaults instead)

crm(live)configure# property default-resource-stickiness=500
 crm(live)configure# verify
ERROR: Warnings found during check: config may not be valid
crm(live)configure# commit
ERROR: Warnings found during check: config may not be valid
Do you still want to commit (y/n)? y
crm(live)configure# cd
crm(live)# status
Stack: corosync
Current DC: node01.test.org (version 1.1.21-4.el7-f14e36fd43) - partition with quorum
Last updated: Thu Sep  3 22:56:52 2020
Last change: Thu Sep  3 22:56:39 2020 by root via cibadmin on node01.test.org

2 nodes configured
3 resources configured

Online: [ node01.test.org node02.test.org ]

Full list of resources:

 webstore       (ocf::heartbeat:Filesystem):    Started node02.test.org
 webserver      (systemd:httpd):        Started node01.test.org
 webip  (ocf::heartbeat:IPaddr):        Started node01.test.org

crm(live)# node standby node01.test.org
crm(live)# status
Stack: corosync
Current DC: node01.test.org (version 1.1.21-4.el7-f14e36fd43) - partition with quorum
Last updated: Thu Sep  3 22:57:16 2020
Last change: Thu Sep  3 22:57:10 2020 by root via crm_attribute on node01.test.org

2 nodes configured
3 resources configured

Node node01.test.org: standby
Online: [ node02.test.org ]

Full list of resources:

 webstore       (ocf::heartbeat:Filesystem):    Started node02.test.org
 webserver      (systemd:httpd):        Started node02.test.org
 webip  (ocf::heartbeat:IPaddr):        Started node02.test.org

crm(live)# node online node01.test.org
crm(live)# status
Stack: corosync
Current DC: node01.test.org (version 1.1.21-4.el7-f14e36fd43) - partition with quorum
Last updated: Thu Sep  3 22:57:34 2020
Last change: Thu Sep  3 22:57:26 2020 by root via crm_attribute on node01.test.org

2 nodes configured
3 resources configured

Online: [ node01.test.org node02.test.org ]

Full list of resources:

 webstore       (ocf::heartbeat:Filesystem):    Started node02.test.org
 webserver      (systemd:httpd):        Started node02.test.org
 webip  (ocf::heartbeat:IPaddr):        Started node02.test.org

crm(live)# 

Tip: when setting the default resource stickiness, an error will be prompted when checking. The error means that if there is a warning message, the configuration may not take effect. In fact, we can directly submit without caring about it. After modifying the default stickiness of resources to nodes, node01 is set to standby again, and the resources are moved to node02. Node01 is launched again, but the resources are not migrated back to node E01; this is the reason why the default stickiness of resources to nodes that we set just now. Only when the propensity of resources to node01 differs from node02 by more than 500, will resources move back from node02 to node01 after node01 returns to normal;

Colonization: define the tendency of resources and resources together;

Command syntax format help

Usage:

colocation  : [:] [:]
  [node-attribute=]

colocation  : 
  [node-attribute=]

resource_sets ::  [ ...]

resource_set :: ["("|"["] [:] [[:] ...] \
                []  [")"|"]"]

attributes :: [require-all=(true|false)] [sequential=(true|false)]


Example:

colocation never_put_apache_with_dummy -inf: apache dummy
colocation c1 inf: A ( B C )

Example: the tendency to define webip, webserver and webstore together is positive infinity

It is suggested that the definition of the tendency of resources and resources together as positive infinity means that these resources must be together and cannot be separated; if it is negative infinity, it means the opposite;

Verification: check the cluster status information to see whether webip, webserver and webstore are running on the same node?

Verification: manually migrate webip to node01 to see if webserver and webstore will follow node01?

Tip: you can see that webip is migrated manually, and webserver and webstore will also migrate along with it, which is a bit similar to the group relationship;

Order: defines the resource order constraint;

Command syntax help

Usage:

order  [{kind|}:] first then [symmetrical=]

order  [{kind|}:] resource_sets [symmetrical=]

kind :: Mandatory | Optional | Serialize

first :: [:]

then ::  [:]

resource_sets :: resource_set [resource_set ...]

resource_set :: ["["|"("] [:] [[:] ...] \
                [attributes] ["]"|")"]

attributes :: [require-all=(true|false)] [sequential=(true|false)]


Example:

order o-1 Mandatory: apache:start ip_1
order o-2 Serialize: A ( B C )
order o-3 inf: [ A B ] C
order o-4 first-resource then-resource

Tips: symmetric indicates whether to start the symmetric stop sequence, and the default is true; the so-called symmetric stop sequence is based on the start sequence defined by us, which starts first and then stops, and then starts first; mandatory means mandatory constraint; optional indicates optional; serialize indicates serialization, sequential start and sequential stop; mandatory is commonly used to constrain the start sequence; mandatory is used to constrain the start sequence;

Example: forcing webstore to start before webserver and webserver to start before webip

Tip: the above configuration indicates that webstore starts first, followed by webserver, and finally webip; the stop sequence is in the reverse order, first stop webip, then webserver, and finally webstore; if you do not want to define a symmetric stop sequence, you can set symmetric to false;

Verification: set node02 to standby state and check the cluster status to see if the start and stop modes of the three resources are in the order defined by us?

Tip: you can see that after node02 is set to standby state, all resources are moved to node01; webip is the first to stop, followed by webserver, and finally webstore; when starting, start webstore first, then webserver, and finally webip; the starting sequence is the same as that defined by us;