Designing for Fast Networks

• Computer Architecture

• Avoiding Data Transfers

• Avoiding Data Manipulation (ILP)

• API design

• Resource Reservation

• RSVP

Network-Computer Interface

Network is only part of the picture:

• data is produced by a process

• encoded

• given to protocol stack – checksum/CRC

– sent to network device

• received by protocol stack – stored in memory

– checksum/CRC

• copied to user buffer

• decoded

• consumed by a process

High-Throughput Architecture

CPU 95MB/s 114MB/s 560MB/s 2000MB/s  mem I/O L2 L1

• OC-12 (622Mb/s) = 77MB/s

• If data must cross memory bus twice, cannot keep up with OC-12

• Could do lots of manipulation of data if guaranteed to stay in L2 cache

• peak I/O bandwidth (GNTE) may not be same as actual I/O bandwidth

Avoiding Data Transfer

• Direct-Memory Access (DMA)

• Programmed I/O

• Use application buffers for processing • Avoid crossing domains

• Merge data manipulation loops

Integrated Layer Processing (ILP)

Fusion of loops from different layers

for (i = 0; i < data_size * 4; i++)
  checksum += short[i];
...
for (i = 0; i < data_size; i++)
  llong[i] = decrypt(llong [i]);

becomes

for (i = 0; i < data_size; i++) {
  value = loadllong();
  checksum += valuep[0] + valuep[1]
            + valuep[2] + valuep[3];
  dest[i] = decrypt(value); }

User Buffering

• Must be able to make decision about where to put the data based on first few bytes, e.g.

– ATM VPI/VCI

– IP source, dest, type, TCP/UDP ports, TCP sequence

• API must permit common network operations (e.g. discard after checksum)

• User process must set aside enough buffer for kernel

API Considerations

• Cross-Domain data transfers page-aligned • Security concerns of partial page re-mapping • Abstract data type must allow efficient

operations

• High performance available only to optimized applications

• Standard is very desirable

Real-Time Applications

• Audio Telephony

• Video Telephony

• Conferencing

• Maximum delay more important than correctness

• Playback: Delay variation within bounds • New service model:

– Real-Time

– Bounded-Delay

– “Traditional”, Elastic

Strict Real-Time

• Dropping data as bad as being late ⇒ Reliability less important than timeliness

• If cannot drop: guaranteed service class

• If can drop: predictive service class – Analogous to lossy compression

– Must be able to deal with lost data

• Use reservation, priority within routers

• Can often guarantee Constant Bit Rate (CBR)

• Need guarantees about total delay, data loss • May be able to adapt to available bandwidth

(e.g. by lowering quality)

Bounded-Delay

• Example: Video/Audio playback

• Receiver can buffer to mask delay variations • Buffer size corresponds to maximum delay

variation

• Can sometimes guarantee CBR, other times not (e.g. MPEG)

• Can usually adapt to different maximum delay variations

• Might be able to retransmit within maximum delay

• May be able to adapt to available bandwidth (e.g. by lowering quality)

Traditional

• Available Bit Rate (ABR)

• Whole range of delay requirements:

– Telnet

– NFS

– WWW

– FTP

– e-mail

– usenet

• Data corruption (loss) not acceptable

• Adapts to available bandwidth: congestion control

RSVP

• Work in progress (IETF)

• Specifying the application’s needs: flowspec • Network decision: admission control

• Communication: signaling

Flowspec

• Analogous to ATM QoS

• Rspec: requested service (e.g. how much delay)

• Tspec: traffic characteristics.

Leaky bucket:

– rate r

– maximum buffer b

Admission Control

• must make yes/no decision

• need suitable algorithm for guaranteed service • could use statistics for predictive service

• after flow is admitted, might be policed to insure it obeys its Tspec

RSVP Signaling

• connectionless

• soft state in the routers

• reservation is terminated after 60 seconds if not renewed

• reservation increases can fail gracefully

• receiver-controlled signaling

• supports multicast

• sender sends PATH with Tspec every 30 seconds

• receiver(s) reverse the path and send RESV for Tspec

Summary

• Latency becomes relatively large, network is unresponsive

• Network throughput becomes comparable to memory bus throughput

• Avoid memory-to-memory data transfers • As throughput and user base grow, new

applications appear

• New service models

• Reservations and guarantees