正文文本
Logical System Concepts - Partitions
Data distribution only changes when the system changes
Equilibrium is kept when new hardware is added
Equilibrium is kept when old hardware is removed
Equilibrium is kept after a hardware failure
Module 2
Module 3
Module 1
[ hardware upgrade ]
Logical System Concepts – Partitions (Cont.)
Data distribution only changes when the system changes
Equilibrium is kept when new hardware is added
Equilibrium is kept when old hardware is removed
Equilibrium is kept after a hardware failure
Module 2
Module 3
Module 4
[ hardware failure ]
The fact that distribution is full and automatic makes sure all spindles join the effort of data re-distribution after configuration change.
Tremendous performance gains are seen in recovery/optimization times thanks to this fact.
Module 1
Logical System Concepts – Slices
XIV data distribution architecture uses Slices for partition copies
Slices are spread across all disk drives in the system
Each slice has two copies: Primary copy and a Secondary copy
There are 16,384 slices (times 2 copies is a total of 32,768 slices)
Each disk holds approx 182 slices [(16384 x 2)/180]
A Primary and it’s Secondary Slice will never reside on the same module
Module 2
Logical System Concepts – Slices (Cont.)
Module 1
Module 2
When creating a Volume it must span across all drives in the system
The minimum size of a volume is 17GB
1MB Partition = 2^20 (1,048,576) x 16384 = ~17GB
Each Partition is numbered with a Logical Partition number for it’s specific Volume
9
19
23
14
26
11
21
6
24
3
1
4
17
16
18
8
13
20
0
27
12
15
2
7
5
10
25
22
When creating another volume which is bigger than the minimum 17GB, the system will allocate several 17GB chunks for it
In this example it is a 51GB Volume built from 3 x 17GB chunks
Logical Partition numbers are also assigned here
Numbering is always in modulus to 16384
9
19
23
14
26
11
21
6
24
3
1
4
17
16
18
8
13
20
0
27
12
15
2
7
5
10
25
22
9
19
23
14
26
11
21
6
24
3
1
4
17
16
18
8
13
20
0
27
12
15
2
7
5
10
25
22
9
19
23
14
26
11
21
6
24
3
1
4
17
16
18
8
13
20
0
27
12
15
2
7
5
10
25
22
+16384
+32768
+0
Logical System Concepts – Logical Volumes
Every logical volume is comprised of 1MB pieces of data
Interface modules will not hold partition copies from other interface modules
This is called “FC Proof”
Implemented because Interface modules are more prune to problems than Data modules
The physical capacity associated with a logical volume is always a multiple of 17GB
The maximum number of volumes that can be concurrently defined on the system is 4,605
The same address space for both volumes and snapshots is 16,377
Logical volumes are administratively managed within the context of Storage Pools
Data Distribution
All data in an XIV system is broken into 1MB partitions
Each partition belongs to a bucket of partitions, called a slice
There are 16384 slices
Partition X of any volume belongs to slice (X % 16384)
Each slice appears on two disks, that reside on different modules
One is designated as the primary copy
Second is designated as the secondary copy
The Slice Table holds these two disk ID’s for each one of the slices
Read Operations
Host sends a read request to one of the i_nodes
i_node consults with the Slice Table, determines primary node ID and disk #
A read request will always be directed to the primary copy of the data
i_node forwards request to relevant module’s cache node
Cache node consults with its local Partition Table, determines physical location on disk
Cache node reads the data from the memory cache, if there, or from the disk
Cache node sends data to i_node, which gives it to the host
Read Operations (Cont.)
Host sends a read request to one of the i_nodes
READ
i_node consults with slice table to determine primary node ID and disk #
The request is then forwarded to the correct module’s Cache node
Cache node consults with its local PT to determine physical location on disk
Partition’s data is read from either cache or disk
Cache node sends the data back to the i_node, who then sends it back to the host
XIV Slice Table
………
found Primary Slice on Disk 7, Node 12
READ
XIV Cache Node
………
found the Physical locationof the Logical partition
READ
Write Operations
Host sends a write request to one of the i_nodes
i_node consults with the Slice Table, determines primary node ID and disk #
i_node forwards request to relevant module’s primary cache node
Primary cache node consults with the Slice Table, forwards request to the secondary cache node
Both cache nodes consult with their local Partition Table to determine physical location on disks
Both cache nodes save the written buffer in their memory cache
Secondary cache node send an ack to the primary cache, which then acks the i_node, which then acks the host
Write Operations (Cont.)
Host sends a write request to one of the i_nodes
WRITE
i_node consults with the Slice Table to determines Primary Node ID and Disk #
XIV Slice Table
………
found Primary Slice on Disk 3, Node 14
i_node forwards request to relevant module’s primary cache node
WRITE
Primary cache node consults with the Slice Table, and forwards the request to the secondary cache node
WRITE
XIV Slice Table
………
found Secondary Slice on Disk 11, Node 7
Both cache nodes consult with their local Partition Table to determine physical location on disks
Both XIV Cache Nodes
………
found the Physical locationof the Logical partition
Both cache nodes save the written buffer in their memory cache
WRITE
WRITE
Secondary cache node send an ack to the primary cache, which then acks the i_node, which then acks the host
ACK
ACK
The Slice Table
Steady State
(Full Redundancy)
Example
Slice 5
Primary copy on N8D11Secondary copy on N2D1
Slice 5Secondary
Slice 5Primary
Disk Failure
(for Secondary Slice copy)
Example
Slice 5
Disk 1 on Module 2 failed
Slice 5Secondary
System creates a new Goal Distribution
Missing slices are distributed across all disks
Example System Chooses N14D7 for Slice 5
System enters a Rebuild state
All partitions are copied from the Primary copy to the secondary copy until it’s back to Steady State
To enable continued normal operation of the XIV system, the Secondary Slice copy is immediately transformed to become the Primary Slice copy
The Slice Table (Cont.)
Disk Failure
(for Primary Slice copy)
Example
Slice 5
Primary copy on N8D11Secondary copy on N2D1
Slice 5Secondary
Slice 5Primary
Example
Slice 5
Disk 11 on Module 8 failed
Slice 5Secondary
System creates a new Goal Distribution
Missing slices are distributed across all disks
Example System Chooses N6D11 for Slice 5
System enters a Rebuild state
All partitions are copied from the Primary copy to the secondary copy until is back to Steady State
Slice 5Primary
The Slice Table (Cont.)
Once a phase out has been initiated, all it’s slices are immediately set as Secondary copies, while setting their counterpart as Primary Copies
Disk Phase Out
System creates a new Goal Distribution
Marked Slices are distributed across all disks
Example System Chooses N11D2 for Slice 5
System enters a Redist state
All partitions are copied from the Primary copy to the Second secondary copy until all data is copied
Slice 5Secondary
Slice 5Primary
Slice 5Secondary
Slice 5Primary
Slice 5SecondSecondary
Once all slices from phased out disk have the second Secondary copy populated, second Secondary copies are marked as Secondary and disk is marked failed
Example for Slice 5
Disk 11 of Module 8 is being phased out of the system
Slice 5Secondary
The Slice Table (Cont.)
Disk Phase In
N2D1
Slice 5Secondary
Slice 5Primary
Example for Slice 5
Disk 12 of Module 15 is being phased into the system
When phasing in a new disk, a new Goal Distribution is created
The system redistribute all the slices across all disks, including the newly added disk
N15D12
Slice 5SecondSecondary
The Redist algorithm decides which slices will be transferred to the newly added disk
ExampleSlice 5 Secondary copy is selected to be transferred
ExampleAll slices on the new disk are initially marked as second Secondary copies
Data Redist operation is initiated from all slices
Once all the data is fully transferred, the system will mark slices as Primary or Secondary copies
ExampleSlice 5 onN15D12 is marked as Secondary
Slice 5Secondary
Free space left on each disk will be logically allocated to each one of the remaining slices
ImportantSystem capacity changes accordingly
