专栏名称: 企业存储技术
企业存储、服务器、SSD、灾备等领域技术分享，交流 | @唐僧_huangliang （新浪微博 ）
目录
相关文章推荐
51好读  ›  专栏  ›  企业存储技术

Multipath切换分布式存储iSCSI主路径

企业存储技术  · 公众号  ·  · 2020-01-04 14:10

正文

请到「今天看啥」查看全文



本文转载自一位专家朋友Bean Li的博客，原文链接见文末。

前言

本文记录multipath相关指令, Multipath的path path_grouping_policy – Paths are grouped into path groups. The policy determines how path groups are formed. There are five different policies.

  • failover: One path per priority group

  • multibus: All paths in one priority group. This is the default.

  • group_by_serial: One priority group per storage controller (serial number)

  • group_by_prio: One priority group per priority value

  • group_by_node_name: One priority group per target node name

有时候使用failover，需要切换主路径，本文记录切换主路径的方法

查看当前配置 

root@host243:~# multipath -ll
2c2c5d201aba719ee dm-0 Bigtera ,VirtualStor_Scal
size=5.0T features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 49:0:0:0 sdg 8:96 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 48:0:0:0 sdf 8:80 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 46:0:0:0 sdd 8:48 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 47:0:0:0 sde 8:64 active ready running

灰色部分按住可左右拖动，以下同

切换主路径

从上面的输出可以看出，sdd/sde/sdf/sdg 都是同一个设备的4条不同的路径。如果设备是分布式存储提供的iSCSI设备，每条路径可能对应一个Gateway。有时候可能需要切换主路径，比如某个存储节点要关机维护，这种情况，要怎么做呢？

首先执行如下语句，进入交互页面： 

multipathd -k

通过如下指令可以实时查看某个multipath的多个group信息： 

multipathd> show topology
2c2c5d201aba719ee dm-0 Bigtera ,VirtualStor_Scal
size=5.0T features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio




    
=1 status=active
| `- 49:0:0:0 sdg 8:96 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 48:0:0:0 sdf 8:80 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 46:0:0:0 sdd 8:48 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 47:0:0:0 sde 8:64 active ready running
multipathd>

对于我这个环境比较简单，如果存在多个，可以通过如下指令指定： 

multipathd> show map 2c2c5d201aba719ee topology
2c2c5d201aba719ee dm-0 Bigtera ,VirtualStor_Scal
size=5.0T features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 49:0:0:0 sdg 8:96 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 48:0:0:0 sdf 8:80 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 46:0:0:0 sdd 8:48 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 47:0:0:0 sde 8:64 active ready running
multipathd>

从上面的输出可以看出，我们设备下有多条路径，第一条sdg 为group 1，第二个sdf为group 2， 如果想切换主路径到sdd，那么需要执行：

执行完毕后，可以执行quit 退出multipathd的交互，然后执行multipath -ll来确认主路径是否发生了变化。

有些时候，没有I/O流量的情况下，multipath -ll的输出并不会及时发生变化，不过没关系，引入一些IO，就可以看到active 路径发生了变化。 

oot@host243:~# multipath -ll
2c2c5d201aba719ee dm-0 Bigtera ,VirtualStor_Scal
size=5.0T features='0' hwhandler=




    
'0' wp=rw
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 49:0:0:0 sdg 8:96 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 48:0:0:0 sdf 8:80 active ready running
|-+- policy='round-robin 0' prio=1 status=active
| `- 46:0:0:0 sdd 8:48 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 47:0:0:0 sde 8:64 active ready running

验证

对于分布式存储的iSCSI，每一条路径对应某一个存储Gateway，如何判断sdX和存储Gateway的映射关系呢？ 

root@host243:~# iscsiadm  -m session -P 3
iSCSI Transport Class version 2.0-870
version 2.0-873
Target: iqn.2020-01.lenovo:test
	Current Portal: 10.16.172.105:3260,1
	Persistent Portal: 10.16.172.105:3260,1
		**********
		Interface:
		**********
		Iface Name: default
		Iface Transport: tcp
		Iface Initiatorname: iqn.1993-08.org.debian:01:f077d70d498
		Iface IPaddress: 10.16.172.243
		Iface HWaddress: 
		Iface Netdev: 
		SID: 16
		iSCSI Connection State: LOGGED IN
		iSCSI Session State: LOGGED_IN
		Internal iscsid Session State: NO CHANGE
		*********
		Timeouts:
		*********
		Recovery Timeout: 120
		Target Reset Timeout: 30
		LUN Reset Timeout: 30
		Abort Timeout: 15
		*****
		CHAP:
		*****
		username: 
		password: ********
		username_in: 
		password_in: ********
		************************
		Negotiated iSCSI params:
		************************
		HeaderDigest: None
		DataDigest: None
		MaxRecvDataSegmentLength: 262144
		MaxXmitDataSegmentLength: 1048576
		FirstBurstLength: 262144
		MaxBurstLength: 1048576
		ImmediateData: Yes
		InitialR2T: No
		MaxOutstandingR2T: 1
		************************
		Attached SCSI devices:
		************************
		Host Number: 46	State: running
		scsi46 Channel 00 Id 0 Lun: 0
			Attached scsi disk sdd		State: running
	Current Portal: 10.16.172.106:3260,1
	Persistent Portal: 10.16.172.106:3260,1
		**********
		Interface:
		**********
		Iface Name: default
		Iface Transport: tcp
		Iface Initiatorname: iqn.1993-08.org.debian:01:f077d70d498
		Iface IPaddress: 10.16.172.243
		Iface HWaddress: 
		Iface Netdev: 
		SID: 17
		iSCSI Connection State: LOGGED IN
		iSCSI Session State: LOGGED_IN
		Internal iscsid Session State: NO CHANGE
		*********
		Timeouts:
		*********
		Recovery Timeout: 120
		Target Reset Timeout: 30
		LUN Reset Timeout: 30
		Abort Timeout: 15
		*****
		CHAP:
		*****
		username: 
		password: ********
		username_in: 
		password_in: ********
		************************
		Negotiated iSCSI params:
		************************
		HeaderDigest: None
		DataDigest: None
		MaxRecvDataSegmentLength: 262144
		MaxXmitDataSegmentLength: 1048576
		FirstBurstLength: 262144
		MaxBurstLength: 1048576
		ImmediateData: Yes
		InitialR2T: No
		MaxOutstandingR2T: 1
		************************
		Attached SCSI devices:
		************************
		Host Number: 47	State: running
		scsi47 Channel 00 Id 0 Lun: 0
			Attached scsi disk sde		State: running
	Current Portal: 10.16.172.107:3260,1
	Persistent Portal: 10.16.172.107:3260,1
		**********
		Interface:
		**********
		Iface Name: default
		Iface Transport: tcp
		Iface Initiatorname: iqn.1993-08.org.debian:01:f077d70d498
		Iface IPaddress: 10.16.172.243
		Iface HWaddress: 
		Iface Netdev: 
		SID: 18
		iSCSI Connection State: LOGGED IN
		iSCSI Session State: LOGGED_IN
		Internal iscsid Session State: NO CHANGE
		*********
		Timeouts:
		*********
		Recovery Timeout: 120
		Target Reset Timeout: 30
		LUN Reset Timeout: 30
		Abort Timeout: 15
		*****
		CHAP:
		*****
		username: 
		password: ********
		username_in: 
		password_in: ********
		************************
		Negotiated iSCSI params:
		************************
		HeaderDigest: None
		DataDigest: None
		MaxRecvDataSegmentLength: 262144
		MaxXmitDataSegmentLength: 1048576
		FirstBurstLength: 262144
		MaxBurstLength: 1048576
		ImmediateData: Yes
		InitialR2T: No
		MaxOutstandingR2T: 1
		************************
		Attached SCSI devices:
		************************
		Host Number: 48	State: running
		scsi48 Channel 00 Id 0 Lun: 0
			Attached scsi disk sdf		State: running
	Current Portal: 10.16.172.108:3260,1
	Persistent Portal: 10.16.172.108:3260,1
		**********
		Interface:
		**********
		Iface Name: default
		Iface Transport: tcp
		Iface Initiatorname: iqn.1993-08.org.debian:01:f077d70d498
		Iface IPaddress: 10.16.172.243
		Iface HWaddress: 
		Iface Netdev: 
		SID: 19
		iSCSI Connection State: LOGGED IN
		iSCSI Session State: LOGGED_IN
		Internal iscsid Session State: NO CHANGE
		*********
		Timeouts:
		*********
		Recovery Timeout: 120
		Target Reset Timeout: 30
		LUN Reset Timeout: 30
		Abort Timeout: 15
		*****
		CHAP:
		*****
		username: 
		password: ********
		username_in: 
		password_in: ********
		************************
		Negotiated iSCSI params:
		************************
		HeaderDigest: None
		DataDigest: None
		MaxRecvDataSegmentLength: 262144
		MaxXmitDataSegmentLength: 1048576
		FirstBurstLength: 262144
		MaxBurstLength: 1048576
		ImmediateData: Yes
		InitialR2T: No
		MaxOutstandingR2T: 1
		************************
		Attached SCSI devices:
		************************
		Host Number: 49	State: running
		scsi49 Channel 00 Id 0 Lun: 0
			Attached scsi disk sdg		State: running
root@host243:~#

通过上面的输出，可以看到 /dev/sdd对应的是10.16.172.105:3260 ，因此找到了对应关系。

修改完毕了主路径，如何从网路上判定已经发生了切换呢？ 

iftop -i bond0

通过iftop，查看网络流向，确定是否流向了正确的Gateway。


点开后双击放大显示


原文链接：https://bean-li.github.io/multipath-change-active/



注 ：本文只代表作者个人观点，与任何组织机构无关，如有错误和不足之处欢迎在留言中批评指正。 进一步交流 技术 ， 可以 加我的 QQ/ 微信： 490834312 。如果您想在这个公众号上分享自己的技术干货，也欢迎联系我：）


感谢您的阅读和支持！《企业存储技术》微信公众号： HL_Storage

长按二维码可直接识别关注


历史文章汇总 ： http://chuansong.me/account/huangliang_storage







请到「今天看啥」查看全文


推荐文章
萌宠萌  ·  当流浪猫看着主人怀了幸福的喵喵，那表情，眼睛都湿了！
8 年前
半导体行业观察  ·  上海浦东IC产业发展概况，占中国集成电路产值17%
8 年前
良仓  ·  美国可伸缩时尚环保水杯，可大可小随心变
8 年前
哎咆科技  ·  姑娘你冷么？我给你暖暖
8 年前
新财富  ·  华为工资单出炉：人均收入60万 万人年薪超百万 但45岁退休！你羡慕么？
8 年前
Sov5搜索   ·   小百科   ·   今天看啥   ·   移动版
51好读 - 好文章就要读起来!