Implementation Aspects of Communication Protocols in Active Network Nodes for the

Implementation Aspects of Communication
Protocols in Active Network Nodes for the
Dynamic Installation of Performance Enhancing
Protocols1
Michael Eyrich, Morten Schläger, Prof. Dr. Adam Wolisz
Telecommunication Networks Group, Technische Universität Berlin
[email protected]
Feb. 13 – Feb. 15, 2002
1
Work in progress of the FlexiNet Project;
Acknowledgments: Bundesministerium für Bildung und Forschung (BMB+F)
Outline
•
•
•
•
•
Dagstuhl Seminar 2002
Why Active Networks?
ReSoA Architecture
Implementation Aspects
Measurement Evaluation
Outlook
1
What’s it all about?
Dagstuhl Seminar 2002
Performance Enhancing Proxies are an appealing solution for:
• Coupling of heterogenous networks (e.g. wireless Internet access).
• Many different (usefull) proxies developed ⇒ not yet deployed?
Active Networks as a deployment booster . . .
• Scanning of network traffic (network attacks, e.g., DoS)
• Flow conversion (Transcoding)
• Proxies
– Can it be integrated in an AN?
– Which problems are raised by the integration?
– How does it perform?
A case study within FlexiNet at TU Berlin
• Integration of ReSoA into AN environment
2
Our Performance Enhancing Proxy
Dagstuhl Seminar 2002
ReSoA: REmote SOcket Architecture
• Transport Protocol Proxy
– Improved performance of wireless Internet access
– Export of socket interface
• Flow decoupling:
– LHP between ReSoA server and wireless station (end-system)
– TCP between correspondent host and ReSoA server
• Last Hop Protocol (LHP)
– Reliable service in case of TCP, flow dependent in case of UDP
– Technologie dependent
– Exchangeable
∗ can be provided by manufacturer
3
ReSoA– Model
wireless system
Dagstuhl Seminar 2002
access point = active node
correspondent node
Execution Environment
Export Socket Server
application
application
user
interface
I2
Local Socket Module
LHP
WLAN
I3
LHP
WLAN
user
interface
I1
socket interface
TCP
IP
LAN
TCP
IP
LAN
Internet
4
Necessities for proxy downloads
Dagstuhl Seminar 2002
Execution Level
• Kernel vs. User space
Interfaces to the system
• Packet multiplexing and demultiplexing
• Access to service of static protocol stack (e.g. transport layer)
• Interface to control behavior of static protocol stack
(e.g. delay ACKS)
• System services (e.g., timer, process interaction)
5
Necessities for proxy downloads (cont.)
Dagstuhl Seminar 2002
Signalling environment
• Where to find a proxy
• How to communicate version, architecture, etc.
Security and Fairness
• Is it safe to execute the code I got?
• Enforcement of beeing cooperative (memory, processing time, . . . )
6
Execution Environments
Feature
Signalling
Security/Fairness
Modularized
Module instances
De-/Multiplexing
Transport layer data interface
Transport layer control interface
Network layer interface
MAC layer interface
Dagstuhl Seminar 2002
X
X
Crossbow
Cobra
–/??
–
required
for PEP
(X)
(X)
X
–
X
(socket)
–
X
X
X
X
X
(socket)
–
X
X
X
–
–
X
(X)
X
X
Linux Kernel
AMnet
–
–
X
–
X
(socket)
–
X
X
→ Specialized EE only for management (signalling, security, fairness)
→ Additional services needed not provided by EEs.
7
Implementation Design
Dagstuhl Seminar 2002
Kernel modules
• ReSoA server and client
• Last Hop Protocol (LHP) environment
– Registration of and wrapper to LHP
– Protocol and address independent interface to LHP
• Last Hop Protocols
– currently IPLHP, i.e., based on IP
– LLP: MAC Link Layer Protocol extension
Interfaces
• Transport layer access: sockets (I1)
• Internal Interface: ReSoA– LHP (I2)
• De-/Multiplexing: New protocol familiy for LHP (I3)
8
Interface to local TCP
Dagstuhl Seminar 2002
What does it mean to take Sockets?
• Socket requires copying of data (protection of kernel space)
– no way to inject skb’s into TCP stack
• Socket requires process context (I1)
– Queueing of packets necessary (Socket callback lock)
– Explicit tagging of process to get runable
– Explicit scheduling of process
(non-preemptive scheduling for threads)
→ Delay therefore not predictive (load dependent)
• Socket structure depends on configured protocols!
(Bug or Feature?)
• Socket allows for waiting on events (reception of data, . . . )
9
In-kernel socket Interface
Dagstuhl Seminar 2002
Access to functions
• Standard socket creation (sock create())
• Direct function calls (not via sys socketcall)
Access to data
• sock sendmsg requires an iovec
• Explicit switch to kernel data segment required
Example
mm_segment_t oldfs;
struct msghdr msg;
...
oldfs = get_fs(); set_fs(KERNEL_DS);
err = sock_sendmsg(sock, &msg, len);
set_fs(oldfs);
10
Measurement Environment
Dagstuhl Seminar 2002
What is to be measured?
•
•
•
•
Data transfer from FH to WH via AN
I1: Time to read data from TCP (sock recvmsg)
I2: Scheduling delay
Socket Module
I3: Total time within ReSoA
(sock sendmsg)
scheduling
Interface I1
TCP
ioctl.
Interface I2
LHP
Snuffle
kernel level IP socket
•
•
•
•
•
Kernel level TCP socket
Interface I3
a protocol state monitoring extension
IP
IP
In-kernel tracepoints
In-kernel buffer to keep data
Java application for data collection (read at regular intervals)
How to measure time between tracepoints? → skb address
11
General throughput
Dagstuhl Seminar 2002
7.0
6.5
6.0
5.0
5.5
Throughput [Mbit/s]
7.5
8.0
Comparision TCP vs. ReSoA
0
50
100
150
200
250
Index
Comparision
• Implementation of ReSoA (red) behaves well with regard to
throughput of TCP (black)
12
Socket receive (I1)
Dagstuhl Seminar 2002
1500
0
500
Frequency
2500
Histogramm
0
20
40
60
80
100
time [usec]
Time to copy from socket to skb
• Median 5 µsec for average size of 1448 kbyte
13
Scheduling delays (I2)
Dagstuhl Seminar 2002
80 100
60
40
0
20
delay [usec]
Athlon/1500
0
500
1000
1500
2000
Index
Scheduling delay (µsec) per packet
• Median: 55 µsec (red line)
• saw-tooth shape caused by increasing load
14
Stay-time within ReSoA framework
15
10
5
0
total time [sec]
20
Dagstuhl Seminar 2002
0
1000
2000
3000
4000
5000
6000
Index
Per Packet stay-time within ReSoA
• Links of different quality (100 MBit/s vs. 10 MBit/s “wireless”)
• backpressure too slowly propagated to TCP (full socket receive
buffer)?
• We need an interface to control the sending speed of TCP ACKs
• Work in progress!
15
Conclusion
Dagstuhl Seminar 2002
We have . . .
• a dynamic loadable implementation of ReSoA
• an internal interface for the exchange of a LHP (address and
protocol independent)
• a measurement setup including in-kernel tracepoints
We conclude . . .
•
•
•
•
•
•
Integration is possible
Throughput behaves well
System does not scale because of copying and scheduling
Neither environment provides TL interface with copy-avoidance
Kernel patches required to modify TCP’s behavior (netfilter?)
All environments provide network layer access
16
Outlook
Dagstuhl Seminar 2002
Further work
•
•
•
•
•
LLP integration
Socket extensions for reduction of copy operations
Signalling integration
Parameters for signalling (e.g. Kernel version)
Name space collisions between modules
Fairness?
• Avoid starving of other
processes
• Cleanup memory
• Never sleep in interupt context
Modifications to protocol state
• filter rules for delaying ACK
17