A Taste of OFI

A Taste of OFI

A Taste of OFI Sean Hefty #OFADevWorkshop Taste of OFI Elegant and intellectually satisfying with subtle undertones? Selected analysis of the Open ...

1MB Sizes 0 Downloads 12 Views

Recommend Documents

A Taste of Shakespeare
This curriculum unit is designed to introduce students to William Shakespeare, his works,. Elizabethan England, and to t

A Taste of Yass Valley
perfect getaway! Please note that this is just a small selection of many amazing experiences on offer in Yass. Valley. P

5 ofi free $10 ofi $1 ofi $5 ofi
With this coupon. Valid at San Dimas & Chino locations. Excludes purchases of Gift Cards. One coupon per person, per vis

OFI
RESIDUAL CURRENT CIRCUIT BREAKERS OFI. Residual current circuit breakers 4-pole, type AC. ▫ They react to sine-wave re

It's a matter of taste - Tetley Tea
drink in Britain is actually a tea blend, containing a variety of teas? Here at Tetley, each popular ... takes place –

A Taste of HTTP Botnets - Team Cymru
Botnets come in many flavors. As one might expect, these flavors all taste different. A lot of Internet users have had t

A Taste of Tupperware - Bev's Country Cottage
Tupperware Corporation, a $1.1 billion multinational company, is one of the world's leading ... It's just a taste of wha

A natomy of Taste - Columbia Medicine Magazine
Columbia neuroscience, a field that has become increasingly interdisciplinary. The P&S faculty who are members of the Mo

the taste of a travel - Turismo Catania
called minnuzzi 'i Sant'Aita (*1) - both ricotta based, are famous as well, together ..... fishers reminding us of Verga

A Taste of the Coming Year - Squarespace
Oct 7, 2016 - lunch at The Union League Club (65 West Jackson Blvd.) 2. Richard Beddome. President. First Friday. Club o

A Taste of OFI Sean Hefty #OFADevWorkshop

Taste of OFI Elegant and intellectually satisfying with subtle undertones?

Selected analysis of the Open Fabrics Interfaces architecture and implementation

March 15 – 18, 2015

#OFADevWorkshop

2

Development Rough conceptual model

Quarterly release cycle

~200 requirements MPI, PGAS, SHMEM, DBMS, sockets, …

Requirement analysis Input from wide variety of devices

Deployment

Iterative design and implementation Collective feedback from OFIWG

March 15 – 18, 2015

#OFADevWorkshop

3

Application Requirements Give us a high-level interface! Give us a low-level interface!

And this was just the MPI developers! Try talking to the government! March 15 – 18, 2015

#OFADevWorkshop

4

Design

Implementation Agnostic

EASY

• Enable simple, basic usage • Move functionality under OFI

GURU

• Advanced application constructs • Expose abstract HW capabilities Range of usage models

March 15 – 18, 2015

#OFADevWorkshop

5

Architecture MPI

SHMEM

PGAS

...

OFI Enabled Applications

Control Services

Communication Completion Data Transfer Services Services Services

Discovery

Connection Management

fi_info

Address Vectors

March 15 – 18, 2015

Event Queues

Message Queues

RMA

Counters

Tag Matching

Atomics

#OFADevWorkshop

Triggered Operations

Open Fabrics Interfaces (OFI)

6

EASY

Fabric Information

Endpoint Types • MSG – Reliable connected

• DGRAM – Datagram

• RDM – Reliable datagram messages – Reliable unconnected

Capabilities • Message queue – FIFO

• RMA • Tagged messages – Sends match with specific receive buffers

• Atomics Select desired endpoint type and capabilities

March 15 – 18, 2015

#OFADevWorkshop

7

EASY

App 1 RDM Message Queue

Fabric Information App n

App 2 RDM Message Queue

...

RDM Message Queue

OFI Enabled Applications

RDM Message Queue Common Implementation DGRAM Message Queue March 15 – 18, 2015

#OFADevWorkshop

8

GURU

Fabric Information

• Capabilities – Application desired features and permissions – Primary capabilities • Must be requested by application

– Secondary capabilities • May be requested by application • May be offered by provider

• Attributes – Defines the limits and behavior of selected interfaces – Negotiated

• Mode – Provider request on application March 15 – 18, 2015

#OFADevWorkshop

9

Threading Options

Is synchronization needed?

EASY

Fully thread safe

GURU

Identify resource usage constraints needed for lockless access Example: thread 1: {endpoint 1, CQ 1} thread 2: {endpoint 2, CQs 2-3}

March 15 – 18, 2015

#OFADevWorkshop

10

Progress

EASY

GURU

What thread context does a request use to complete?

Automatic – submit and forget

Manual – application thread used to complete request Timeouts and retries, ack processing, atomic operations, data placement, etc.

March 15 – 18, 2015

#OFADevWorkshop

11

Resource Management EASY

GURU

Flow control and queuing

Enabled – prevent overrunning local and remote queues, including completion queues Disabled – application responsible for preventing overruns

App: behavior that can be relied on. Provider: requirements and potential optimizations not restrictions. March 15 – 18, 2015

#OFADevWorkshop

12

Ordering

Sequence of request and completion processing

EASY

Strict – requests are processed and completed in order

GURU

Relaxed – enable out of order processing Dynamic routing, optimized completion processing, parallel data transfers, optimized retry algorithms, etc.

March 15 – 18, 2015

#OFADevWorkshop

13

Mode Bits

Provider hints on how it should best be used

EASY

None! – provider does all the work!

GURU

Application support may improve performance – Local MR – must register buffers for local operations (send/receives) – Context – app provides ‘scratch space’ for provider to track request (full or partial onload models) – Buffer prefix – app provides space for network headers (usnic, IB GRH)

March 15 – 18, 2015

#OFADevWorkshop

14

Architecture MPI

SHMEM

PGAS

...

OFI Enabled Applications

Control Services

Communication Completion Data Transfer Services Services Services

Discovery

Connection Management

fi_info

Address Vectors

March 15 – 18, 2015

Event Queues

Message Queues

RMA

Counters

Tag Matching

Atomics

#OFADevWorkshop

Triggered Operations

Open Fabrics Interfaces (OFI)

15

Endpoints

Addressable communication portal

EASY

Conceptually similar to a socket or QP

Endpoint

transmit

receive

completions

Sequence of request and completion processing March 15 – 18, 2015

#OFADevWorkshop

16

Shared Tx/Rx Contexts Enable resource manager to direct use of HW resources

GURU

Endpoint

Number of endpoints greater than available resources March 15 – 18, 2015

Endpoint

transmit

Endpoint

receive

#OFADevWorkshop

Endpoint

Map to command queues or HW limits (caching)

17

Scalable Endpoints GURU

transmit

transmit

Endpoint

transmit

- Multi-threading - Ordering - Progress - Completions March 15 – 18, 2015

Multiple Tx/Rx contexts per endpoint

transmit

receive

receive

receive

receive

Incoming requests may be able to target a specific receive context #OFADevWorkshop

18

OFI Is a Full Meal Deal Hello, my name is: OFI

EASY

• Select a main dish and a side

GURU

• OFI tells you today’s specials • You select the ingredients and tell OFI how to assemble them

flair March 15 – 18, 2015

We treat you right! #OFADevWorkshop

19

OFI 1.0 • Framework – libfabric – Interfaces & data structures definitions – ‘Spec’ = man pages

• Functional implementation – Quickly enable hardware and fabrics • Portions layer over vendor interfaces

– Allow for application development – Amount and quality of support is provider specific

Important to distinguish between architecture and direction versus current implementations March 15 – 18, 2015

#OFADevWorkshop

20

OFI 1.0 Providers • Sockets

• PSM

– Implement all interfaces and functionality – App. development & debug

• Verbs

• USNIC

– Targets any verbs HW • Not optimized for a specific device

– Only common verbs functionality supported

March 15 – 18, 2015

– Targets non-verbs HW – Expands capabilities beyond lower software driver – Targets non-verbs HW – Cisco will address Input from verbs derivative and non-verbs providers also fed into OFI design

#OFADevWorkshop

21

OFI 1.x • Address other requirements – Multicast – Virtualization – Features cut from 1.0 release

• Expand and optimize providers – Native providers – Additional hardware

March 15 – 18, 2015

#OFADevWorkshop

22

Thank You

#OFADevWorkshop