Overview
Responsibility
Chaper 11
Sending and receiving Address Frames
Sending and receiving the Identification Sequence
Management of Primitives
Chaper 12
Clock Compensation
CRC Generation and Checking
Scrambling
?
Connection Management
Generation of Idle Cycles
Byte Ordering
Introduction
Hard reset
SSP port
Cause a Transport Reset event notification to the Application Layer
Result in a SCSI device reset
STP port
equal to a power on hardware reset (a Phy reset)
Address Frames
OPEN Address Frame
Initiator Connection Tag
Facilitate quickly looking up the context when the reply returns from the target at some later time
Target are not required to check for consistency on this
Destination SAS Address
Identify the intended recipient of this connection request
Expander
Port selection
Check the other arguments to verify that the matching port will actually support the request
Recipient
Verify that the address matches its own SAS address
Source SAS Address
Only AWT and source address are compared to see which will win access to that link
Pathway Blocked Count
The number of times this request has been forced to retry
It received an OPEN_REJECT (PATHWAY BLOCKED) response
Ensure that a request which has previously been retried will have a higher priority next time
Arbitration Wait Time (AWT)
microseconds
milliseconds
Used to assign arbitration priority
Zero or at most 8000h
unfair arbitration
Most significant consideration in determining the priority in the expander
IDENTIFY Address Frame
The information that a SAS Phy passes to its neighbor over the link during a link reset (after Phy reset)
The two devices on either end of a link exchange IDENTIFY frames with each other
SATA devices implement a Phy reset
But no identification sequence
Primitives
General
dwords
four bytes
four characters
Special meaning to the system
The first character
K28.5
SAS
K28.3
SATA
K28.6
A bridge to a SATA device
Receive an error in the flow of traffic to the SATA target
Endian Notation
SAS
Big-Endian
The highest-order byte first
SATA
Little-Endian
The lowest-order byte first
Not a factor for primitives
Error Portection
Not include CRC protection
Chose combination of characters that would maximize the Hamming distance between any two valid primitives
Usage
Indicate address frames for connection and identification
Start and end frames
Ensure proper clock compensation
Perform flow control
Contexts
Non Protocol Specific
outside of a connection
AIP
Used by expanders
Report that the arbitration process for a connection request is under way
BREAK
Terminate a connection without the normal close handshake process
CLOSE
Close out a connection
Free the resources for otheer connections
OPEN_ACCEPT
Sent by the recipient of a connection request
OPEN_REJECT
Sent by the recipient of a connection request
SOAF, EOAF
within a connection
ALIGN
Clock skew management
inside
outside
NOTIFY
BROADCAST
Sent out on all expander ports
Except the one experiencing the change to tell initiator in the system that they need to take steps to discover the change
ERROR
Send by Expander
When forwarding dwords that have been detected on the incoming 'Phy as invalid dwords
HARD_RESET
Force a device-level reset
Within SSP and SMP Connections
Manage aspects of the Link Layer protocol
SOP and EOF
DONE
The connection is being prepared for closing
ACK, NAK
INdicate status of SSP frame reception
RRDY
Indicate readiness to receive a frame
Used in SSP
CREDIT_BLOCKED
No more RRDYs to be sent in this connection
Single Primitives
Only one instance of the primitive to be sent
Only one needs to be detected to achieve a valid result
Used exclusively within SSP and SMP are this type
Repeated Primitives
Repeated several times by the sender
Detected several times in a row by the receiver to be valid
STP connections when responding to power management requests
Continued Primitives
Only used in SATA connections
Allow primitives to repeat indefinitely
Triple Primitives
During normal operation, the same meaning, encoded differently
Sent three times by the transmitter
Seen three times in a row at the receiver
ALIGN and NOTIFY can be mixed into the sequence with no loss of continuity
The four versions of CLOSE
Redundatnt Primitives
Sent 6 times in a row
At least 3 fo those must be seen consecutively by the receiver
The eight versions of BROADCAST
HARD_RESET
ALIGN can be mixed into the sequence with no loss of continuity
State Machine - Link Layer Identification and Reset
General
Control the generation and receipt of the identification sequence
TRansmitter and Receiver
SL_IR Transmitter: Told to send
Identify frame
Hard Reset
Idle dwords
SL_IR Receiver: Report receipt of Identify or Reset sequences
SOAF
data dwords
EOAF
ERROR
Invalid dword
Hard Reset
SL_IR_TIR
Transmit IDENTIFY or HARD_RESET
After the Phy Layer enables the Link Layer, Send:
One IDENTIFY address frame
Hard Reset
SL_IR_RIF
Receive IDENTIFY address Frame
Verify the receipt of a valid IDENTIFY address frame
Start SL_IR Receiver confirmation from the Pjhy Layer has been received
The dword synchroniztion has been achieved
SOAF received confirmation has been received
An address frame has arrived
SL_IR_IRC
Identification and Hard Reset Control
Serial Support
Clock Compensation
General
Embed the clock into the data at the transmitter and recover it at the receiver
The transmitter and receiver clocks are essentially asynchronous to each other
Solution: Elastic Buffer
Elastic Buffer
Crossing Clock Domains
Frequnecy Difference Can Change
ALIGN Injection/Remove is done in Phy Layer of Transmitter
Transmitter's clock is faster
Overflow
Remove ALIGN
Receiver's clock is faster
Underflow
Inject more ALIGNs into the buffer
Metastability Risk
Synchronizer
No method that can completely eliminate the possibility of metastability
Adding more input register
Exist in the Elastic Buffer
Around 4 entries deep
Elastic Buffer Design
Always keep the buffer half full using the incoming data stream
Justification for an Elastic Buffer
Sufficient for ALIGN injection only?
No! Elastic needed to handle both overflow and underflow
CRC Generation and Checking
General
Error checking scheme
Hash function
Create a 32-bit result which is appended at the end of the frame
The primitives are not considered part of the payload and are therefore not involved in CRC calculations
Incoming Frames Verification
Duplicate the process that the transmitter used and compare wtih the received CRC
Process the entiring frame including the received CRC to confirm whether it is the expected constant value
Robust Error Dectection
Discard the entire frame if a CRC error is detected
SSP
NAK (CRC ERROR)
SMP
BREAK
Error Coverage
Based on the number of dwords that are transmitted in a given frame
Scrambling
Reduce the EMI
All data dwords for both SAS and SATA are scrambled
Primitives are not scrambled
multi-version Primitives
ALIGN
ALIGN(0)
ALIGN(1)
ALIGN(2)
ALIGN(3)
Reset to its initial value every time an SOF or SOAF primitive appears
Connection Management
Introduction
Opening the Connection
Rate Matching
Reduced performance on the links that were capable of faster operation
Allow mixing older and newer parts in the same system
State Machines
Introduction
Begin when they receive an Enable Disable SAS Link (Enable) message from the SL_IR
SL Transmitter and Receiver
SL Transmitter
Receive instructions about what needs to be transmitted
No request are pending: Send Idle dwords
Multiple requests are queue up (↓)
BREAK
CLOSE
OPEN_ACCEPT or REJECT
SOAF, or data dword, or EOAF
Idle dword
SL Receiver
Take in the incoming dword stream
Pass messages about the status of what has been received to SL state machines
SL_RA State Machine
Receive open Address frame
Only one state
Function
Detect incoming OPEN address frames
Evaluate them for correctness
Receive from SL Receiver
SOAF Received
Data Dword Received
EOAF Received
Check correctness
8 data dwords received between SOAF and EOAF
Destination address match?
CRC
If correct
Send an OPEN Address Frame Received to SL_CC
SL_CC State Machine
Connection Control
The highest level control in the Link Layer
Arbitration
Arbitration Fairness
Arguments used for arbitration
Arbitration Wait Time(AWT)
Source SAS Address
Connection Rate
Retry Priority Status
Partial Pathway Timeout Status
Pathway Blocked Count
Arbitration
Competition for Narrow Port (↓)
AWT
PBC
Source SAS address
Connection rate
Partial Pathway Counter
If this timer expired, it will increase its own PBC to have a higher priority for the next time
Backoff-Reverse Path
The incoming request has a source address matching the destination address of the outgoing request
OPEN after receiving AIP(NORMAL)
its outgoing request has lost arbitration
Crossing Requests
Backoff-Retry
Deadlock and Livelock
Deadlock
Several devices attempt to access a set of pathways at the same time
The requests end up blocking each other without any of them succeeding
Necessary conditions
Mutual Exclusion
a resource that cannot be used by more than one process at a time
Topic
Prevent: Require shareing of resources
Not a possibility in SAS
Hold and Wait
processes already holding resources may request new resources
Prevent: Require that requesters hold and release all their resources at the same time
Not work in SAS
No Preemption
No resource can be forcibly removed from a process holding it, resources can be released only by the explicit action of the process
Prevent: Let an initiator override a connection or connection request from another initiator
One initiator could cause another to starve for bus access
Difficult to take over resources in other expander
Circular Wait
two or more processes form a circular chain where each process waits for a resource that the next process in the chain holds
Prevent: Require imposing some form of global ordering to ensure that all requesters request resources in order
Recovery
Detect situations that might indicate that a deadlock has occurred and recover from it
Track how long a partially-completed connection waits
Measured by the Partial Pathway Timeout timer
Livelock
Variation of deadlock
All arrive at the same time
Back up by the same amount at the same time and then try again
Protocol Differences
Flow Control
SSP and STP
SSP supports full-duplex traffic
STP only half-duplex
SSP flow control rules ordinarily require that no frames arrive before the receiver explicitly indicates readiness to receive them
Exception Case
the First Burst mechanism
Normal Case
One RRDY → One frame up to 1KB of data
RRDY are primitives
No flow control permission is needed to send primitives
Frame Acknowledgement (ACN/NAK Handshake)
Interlocked Frames
All frames except data frames are interlocked
Another frame can't be sent until receipt of the previous frame has been acknowledge with ACK or NAK
Non-interlocked Frames
Data frames
Several non-interlocked data frames may be sent consecutively without waiting on an ACK/NAK response
Count how many frames have been sent and how may ACK/NAK have been returned
Balance Required
Balance Not Required
Use the same logical unit
The same tag value as the previous Tx Frame message
Only one reason for NAK
CRC ERROR
SSP Connection
State Machines
4 for frame transmission
5 for frame reception
Transmit State MAchines
TIM
Transmit Interlocked Frame Monitor
Count the number of frames sent and the number of ACK/NAKs received
Send a Tx Balance Status message to the SSP_TF2:Tx_Wait state
Automatically set equal to each other whenever receive
Request Close
Request Break
SSP Enable Disable (Enable)
TF
Transmit Frame control
TCM
Transmit frame Credit Monitor
Track the number of frame credits available to make sure that there is credit before a frame is transmitted
Add one credit for every RRDY received
Subtract one for every Tx Credit Used message it receives
D
Done control
Make sure a DONE has been both sent and received before disabling the SSP state machines
Receive State Machines
RF
Receive Frame control
Check incoming frames to verify that they are legal and should be accepted
RCM
Receive frame Credit Monitor
Track the number of RRDYs sent versus the number of frames received
TC
Transmit Credit control
Oversee the requests to transmit an RRDY or CREDIT_BLOCKED
TAN
Transmit ACK/NAK control
Control the sending of ACKs or NAKs in response to received frames
The sending of ACKs and NAKs is reported to SSP_RIM
RIM
Receive Interlocked frame Monitor
Track the ACK.NAK balance at the receiver
Report that status with an Rx Balance Status message
SMP Connections
SMP_IP
Initiator Port
SMP_TP
Target Port