What I’ll be talking about... • • • • • • •
Project JXTA Overview Steve Krasinsky
Introduction Some context (a brief look at other P2P architectures) JXTA PeerGroups JXTA Peers JXTA Pipes Resource discovery Security
What is JXTA? What is P2P? “In general P2P describes an environment where computers (hosts) connect to each other in a distributed environment that does not use a centralized control point to route or connect data traffic.” - Bill Yeager Project JXTA Chief Technology Officer
“… an open-source project that defines a set of protocols for ad hoc, pervasive, peer-to-peer computing” from “Project JXTA Virtual Network”
• • • • •
from “Secure Peer-to-Peer Networking: The JXTA Example” Yeager/Williams http://csdl.computer.org/comp/mags/it/2002/02/f2053abs.htm
http://www.jxta.org/docs/JXTAprotocols.pdf
announced by SUN in April 2001 first to include standard protocols and multi-language implementations current version: 2.2.1 over 2 million downloads over 100 active projects
Some context:
What are the goals of Project JXTA? Interoperability. Interconnected peers must easily locate and communicate with each other; participate in community-based activities; and offer services to each other seamlessly across different P2P systems and communities.
Brief look at some common unstructured P2P file sharing applications
Platform independence. P2P must be independent of programming languages, such as C or Java; system platforms, such as the Microsoft Windows and Unix operating systems; and networking platforms, such as TCP/IP (transmission control protocol/Internet protocol) or Bluetooth.
•
Napster
•
Gnutella
Ubiquity. P2P has its broadest reach when it works with any device having a digital heartbeat, including sensors, consumer electronics, PDAs (personal digital assistants), appliances, network routers, desktop computers, data center servers, and storage systems
•
Kazaa
•
Bit Torrent
Performance? from “Secure Peer-to-Peer Networking: The JXTA Example” Yeager/Williams http://csdl.computer.org/comp/mags/it/2002/02/f2053abs.htm
Napster requester
Gnuntella provider
provider requestor
Kazaa
Bit Torrent
provider
provider provider
provider
provider
requestor
requestor/ provider torrent provider
*Around 50% current internet traffic!!!
Some more context: Structured Distributed Hash Table P2P approaches • • • • • •
Pastry Chord CAN Tapestry Kademlia many more ...
Common characteristics of DHT-based approaches • A hash function is used to assign a unique identifier to a resource on the network. The range of the hash function is distributed equally among nodes in the network. Nodes can cache copies of the resource or a pointer to the resource. • Structured network topology - each network node keeps a list of pointers to a set of neighbor nodes • Flat hierarchy - each node on the network is assumes the same responsibilities
DHT-based approaches (cont’d) Goals minimize hop count needed to satisfy the query locality • finding the closest copy of the resource • a topology that exhibits good locality will ensure that neighbor nodes are close to each other Challenges What happens when nodes join and leave? Resource knowledge must be redistributed throughout the network. We’ll revisit Pastry later and compare it to JXTA’s default mechanism for resource discovery...
JXTA Protocols Core Protocols (required for communication) • Peer Endpoint Protocol (PEP) - route discovery • Peer Resolver Protocol (PRP) - query peers Standard Services Protocols (optional but common) • Peer Discovery Protocol (PDP) • Rendezvous Protocol (RVP) • Peer Information Protocol (PIP) • Pipe Binding Protocol (PDP) •
All protocols are provided as services in JXTA
Back to JXTA! The Stack P2P Applications: IM, File Sharing, VOP2P etc... Services: Pipe, Rendezvous Peers, Peer Groups, Pipes, Endpoints, Advertisements Protocols
adapted from “Self-Constructing P2P Networks on the Project JXTA Platform”Bill Yeager http://netseminar.stanford.edu/sessions/yeager-small.pdf
Advertisements All entities in JXTA (services, peers, peergroups, pipes) are represented as advertisements. Advertisements are XML documents 6 Basic Types: • Peer advertisement • Peergroup advertisement • Pipe advertisement • Service advertisement • Content advertisement • Endpoint advertisement
JXTA Virtual Network
Peer Groups Peers in JXTA self-organize into peer groups - peers which have a common set of interests, and have agreed upon a common set of policies (membership, content, exchange etc.). Motivation: • To create secure domains • To create a scoping environment (subdivide the network) namespaces for all IDs for Peers, Services, and Content are unique • To create a monitoring environment (traffic etc.) Some rules: • Any peer can create, join and leave peer groups • A peer can be a member of more than one peer group • All searches are applied to the peer’s current peer group
Interoperability, Platform Independence, Ubiquity
Peer Groups (cont’d) NetPeerGroup
PeerGroup A
PeerGroup B
PeerGroup C • default peer group joined when you boot jxta • can discover and communicate with other peers in that peer group
NetPeerGroup
Types of Peers
Peers Any device on the JXTA virtual network Peers have: • PeerName • Peer ID a unique URN - virtual address • J2SE Reference implementation uses 128 bit URNs containing UUIDs which are generated using java.security.SecureRandom • ex. urn:jxta:idform-1234567890
Edge Peer • Minimal Edge Peer does not cache advertisements or route messages PDAs, cell phones • Edge Peer caches advertisements, replies to discovery requests does not forward discovery requests Super Peers • Rendezvous Super Peer caches advertisements and forwards discovery requests • Relay Super Peer maintains routing information about peers that cannot directly connect to other peers
Pipes
Pipes and endpoints
JXTA peers use communicate via pipes abstraction allows location-independent communication peers can use pipes to communicate with other peers even if they reside behind a firewall or NAT or even physically different networks (TCP to token ring) HTTP
Pipes are not bound to a physical location (IP address) so a peer can carry its pipe with itself even if it’s physical network location changes
•
Pipes have endpoints
•
Endpoints are the bottom most element in network transport abstraction defined by JXTA
•
Endpoints are bound to physical network addresses
•
A pipe can have more than one endpoint enabling communication over more than one transport
•
More than one pipe can terminate at one endpoint so messages are queued on the destination peer by pipeID containted in each message
HTTP
JXTA Pipe (direc t TCP Peer C
JXTA Pipe (2 diff ’t tran sports )
conne ction)
Peer D TCP
Peer B (relay)
Peer A
firewall
•
Peer E
firewall
Peer F
Interoperability... Adapted from “JXTA In A Nutshell” Oaks, Traversat & Gong O’Reilly 2002
Pipes (cont’d)
Pipes (cont’d)
Peer communication example 1. Peer A creates an input pipe using a pipe advertisement for a Weather Service. Peer A associates the input pipe with an endpoint. Starts to listen to the endpoint.
Peer A
What protocols/services are being used here? 2. Peer B wants to use the Weather Service, so it sends a Pipe Binding Query over the JXTA network using the Discovery Service.
JXTA Network
3. Peer A receives the Pipe Binding Query and sends Peer B a reply.
Peer B
4. Peer B receives the Pipe Binding Answer which contains Peer A’s Peer Advertisement which lists one or more available endpoints for Peer A.
Pipes (cont’d) • Pipes are unidirectional and asynchronous by default so for 2 way communication, you need to create an input pipe and an output pipe • Pipe Communication Mode Point-to-Point Propigate pipe - overcasting, can use IP multicast in the local subnet *note JXTA J2SE reference implementation has a JXTA Socket (bi-directional pipe) and a multicast socket
1. Peer A creates an input pipe using a pipe advertisement for a Weather Service. Peer A associates the input pipe with an endpoint. Starts to listen to the endpoint for messages containing the PipeID.
Peer A
3. Peer A receives the Pipe Binding Query and sends Peer B a reply.
Peer A uses the Pipe Service to create the input pipe.
2. Peer B wants to use the Weather Service, so it sends a Pipe Binding Query over the JXTA network using the Discovery Service.
JXTA Network Peer A uses the Pipe Service (PBP) to reply to Peer B.
Peer B
4. Peer B receives the Pipe Binding Answer which contains Peer A’s Peer Advertisement which lists one or more available endpoints for Peer A.
Resource Discovery How does discovery work? What is going on in the
JXTA Network
Peer B uses the Discovery service(PDP) find the pipe adv.
?
Peer A uses the Pipe Service (PBP)to bind Pipe to endpoint on Peer A.
Super Peer Network
SuperPeers and SRDI (Shared Resource Distributed Index) • J2SE reference implementation resource discovery framework • Special peers called Rendezvous Peers have extra responsibilities for resource discovery • SuperPeers (Rendezvous Peers) are points of centralization in the network and they maintain an index of advertisement indices (pointers to edge peers that cache corresponding advertisements).
from “Project JXTA 2.0 Super-Peer Virtual Network” Traversat, Arora, Abdelaziz, Duigou, Haywood, Hugly, Pouyoul, Yeager www.jxta.org/project/www/docs/JXTA2.0protocols1.pdf
RPV
Edge peers and Rendezvous peers
Rendezvous Peer View (RPV) - an ordered list of other known Rendezvous in the peer group by PeerID
•
The RPV is maintained by a loosely coupled algorithm: • Periodically select random list of known rendezvous • Send a random list of known rendezvous • Ping neighbors and purge unresponsive peers from RPV
• Edge peers only maintain 1 rendezvous connection for each peer group that they belong to • Edge peers either know about a rendezvous from a previous connection, use a seed rendezvous, or can use IP Multicast to discover a rendezvous peer. • Edge peers can use a seeding rendezvous to find out about other rendezvous.
Loosely-Consistent DHT Rendezvous Walker Algorithm used by rendezvous peers to find search for advertisement indices 1. Peer P1 uses SRDI to index an advertisement on its rendezvous R2. 2. R2 uses a DHT function to map the index to a rendezvous in its local RPV.
Loosely-Consistent DHT Rendezvous Walker 1. P2 sends a query to R3 2. R3 hashes the query and forwards to R5 3. R5 sends the query to P1 4. P1 sends a response to P2
3. The index hashes to R5 4. R2 pushes the index to R5 and its neighbors R4 and R6 from “Project JXTA 2.0 Super-Peer Virtual Network”
from “Project JXTA 2.0 Super-Peer Virtual Network”
Traversat, Arora, Abdelaziz, Duigou, Haywood, Hugly, Pouyoul, Yeager
Traversat, Arora, Abdelaziz, Duigou, Haywood, Hugly, Pouyoul, Yeager
www.jxta.org/project/www/docs/JXTA2.0protocols1.pdf
www.jxta.org/project/www/docs/JXTA2.0protocols1.pdf
Loosely-Consistent DHT Rendezvous Walker
Loosely-Consistent DHT Rendezvous Walker
1. R5 leaves the network
Massive changes example
2. R6 becomes R5 in R3’s RPV
1. P2 sends query to R3
3. P2 sends query to its rendezvous R3
2. R3 hashes query and forwards to R5
4. R3 forwards to new R5
3. R5 does not have the index to the advertisement
5. Replicated advertisement index is found. 6. P1 responds to P2
4. Search is propigated in up and down directions from R5 until hop count is reached
from “Project JXTA 2.0 Super-Peer Virtual Network”
from “Project JXTA 2.0 Super-Peer Virtual Network”
Traversat, Arora, Abdelaziz, Duigou, Haywood, Hugly, Pouyoul, Yeager
Traversat, Arora, Abdelaziz, Duigou, Haywood, Hugly, Pouyoul, Yeager
www.jxta.org/project/www/docs/JXTA2.0protocols1.pdf
www.jxta.org/project/www/docs/JXTA2.0protocols1.pdf
Pastry revisited Each Pastry node has a routing table with
Pastry revisited What is the join/leave cost for a JXTA node? • 0 messages
[log2b N] rows and 2b - 1 entries
What about stale advertisement indices in JXTA?
As config parameter b ! size of the routing table goes up and the number of hops req’d decreases.
• Every JXTA advertisement has an expiration date
Expected number of hops to satisfy a query is [log2b N] What is the join/leave cost?
Ubiquity - doesn’t put any demands on peers for routing (except super peers) hierarchy makes this possible Can use Drawbacks - does not exhibit good locality, routing is not deterministic
"(log2b N) messages from “Pastry: Scalable, decentralized object location and routing for large-scale peer-to-peer systems” Rowstron and Druschel
Pure DHT can be used in place of SRDI - Pastry, CAN etc...
http://research.microsoft.com/~antr/PAST/pastry.pdf
JXTA Security
Goals: • Privacy • Authentication • Integrity • Non-repudiation
JXTA Security JXTA uses TLS • Chosen for interoperability • Symmetric cryptography: 3DES, RC4 • Secure hash functions: MD5, SHA • Asymmetric Public Key cryptography: RSA • Certificates X509 version 3 • Digital Signatures: DSS Java 1.3 TLS implementation • Claymore Systems pureTLS • Bouncy Castle (generating certificates) • JXTA Security Library (secure local data personal security environment
JXTA Security TLS sits beneath the Endpoint Service enabling TLS transport to multiplex multiple pipe connections between two peers.
Pipe
Endpoint
TLS from “Secure Peer-to-Peer Networking: The JXTA Example” William Jeager and Joseph Williams csdl.computer.org/comp/mags/it/2002/02/f2053abs.htm
TLS Overview
TLS in JXTA
•
Privacy using symmetric cryptography
•
Reliability MAC
Handshake Protocol •
Negotiate cipher suite - symmetric cipher to use, key exchange method, message digest function
HANDSHAKE
Record Protocol provides A Client Hello
B
Server Hello Certificate X509v3 Contains Public Key (RSA) (Optional certificate request) Encrypted pre-master secret
Calculate master secret master secret
•
Establish/share master secret
•
Authenticate (optional)
Messages encrypted using symmetric 3DES or RC4 using master secret. Message integrity is checked using MAC (SHA-1or MD5 hashing).
TLS in JXTA (cont’d) • • • •
TLS requires a reliable transport JXTA Message length is independent of TLS record size TLS Records may be dropped or arrive out of order JXTA uses a simple ACK / RTO mechanism to maintain stream reliability
TLS Requirements • X509.V3 certificate generation Root Certification Service certificates signed by root • Root Certificate distribution • What are the options in a P2P environment?
TLS Requirements • X509.V3 certificate generation Root Certification Service certificates signed by root • Root Certificate distribution • What are the options in a P2P environment?
TLS Requirements • SSL/TLS require certificates to be signed by trusted CAs • Creators of JXTA did not want to require every peer to pay for a CA signed certificate to implement TLS • $0 certificates? • Self-signed certificates, 3rd party peer signed, CA? • Trust spectrum • Peers assign levels of trust or PeerConfidence to other peers • Poblano Distributed Trust Model for P2P Networks
Poblano • •
Distributed Trust Model by Rita Chen and Bill Yeager Not part of JXTA spec or reference implementation
Ideas behind Poblano • Parameters of trust are personal and decentralized by nature • Users in P2P network can see where their information comes from and can communicate their opinions on both the information received and its source • Opinions can also be used as guidelines for searching for information and recommending sources - “Web of Trust”
Trust in Poblano Definition Codat: contains information - either data or code - that is shared in a JXTA peergroup.
Trust = prior experience with codat (current peer or others) + trust in the source (peer) of the codat
Uses for Poblano: • Reputation guided searching • Recommendation system for security purposes
Trust in Poblano Table on each peer stores codat/keyword information along with confidence values. Codat confidence table:
Table on each peer stores peer/keyword information along with confidence values. Peer confidence table:
Trust in Poblano Keyword based lookup of codat in local codat confidence table.. If not found, look in peer confidence table for peers highly associated with keyword. Forward codat query to that peer. Repeat if codat information is not on the remote peer.
Requestor
Provider
Codat query Codat query
Trust in Poblano
Trust in Poblano
Calculated codat confidence is a function of codat confidence returned by remote peer and peer confidence of all peers along the path to codat confidence information discovery.
Provider increases the popularity value of the codat it sends. The requestor calculates its confidence in the received codat and saves the information in its tables.
Codat request/reply Requestor
Provider
Codat query Codat query
Codat info
Poblano
New confidence information is propigated to all contributing peers (including the provider). All peers can update their codat confidence tables based on the new information. Over time a “Web of trust” is built in the network Codat confidence
Provider
Codat confidence
Codat query
Codat info
Trust in Poblano
Codat confidence
Provider
Codat query
Codat info
Codat info
Requestor
Requestor
Certificates are Codats, so a peergroup can share confidence in a particular certificate/public key using Poblano.
Peek at some code // create a service with the name "www.nike.laborpolicy" on virtual port 100 java.net.ServerSocket server = new P2PServerSocket("www.nike.laborpolicy", 100); // port 100 // wait for a client to connect java.net.Socket client = server.accept();
Peek at some code // connect to a peer offering the service at "www.nike.laborpolicy" on virtual port 100 java.net.Socket socket = new P2PSocket("www.nike.laborpolicy", 100); // now start communicating InputStream in = socket.getInputStream(); OutputStream out = socket.getOutputStream();
// client accepted; now communicate InputStream in = client.getInputStream(); OutputStream out = client.getOutputStream(); // do application specific work now
*from P2P Sockets Project Home Page
*from P2P Sockets Project Home Page
http://p2psockets.jxta.org/
http://p2psockets.jxta.org/
References:
References:
“Secure Peer-to-Peer Networking: The JXTA Example” Yeager/Williams csdl.computer.org/comp/mags/it/2002/02/f2053abs.htm
“Project JXTA Virtual Network” Traversat, Abdelaziz, Duiou, Hugly, Pouyoul, Yeager www.jxta.org/project/www/docs/JXTA2.0protocols1.pdf
“Self-Constructing P2P Networks on the Project JXTA Platform” -Bill Yeager netseminar.stanford.edu/sessions/yeager-small.pdf
“Poblano A Distributed Trust Model for Peer to Peer Networks” Rita Chen, William Yeager www.jxta.org/docs/trust.pdf
“JXTA In A Nutshell” Oaks, Traversat & Gong O’Reilly 2002
“The JXTA TLS Implementation” Bill Yeager www.jxta.org/community/SunDevCon-20011213/ Section4/4-HQBriefing(Security).PDF
“Project JXTA 2.0 Super-Peer Virtual Network” Traversat, Arora, Abdelaziz, Duigou, Haywood, Hugly, Pouyoul, Yeager www.jxta.org/project/www/docs/JXTA2.0protocols1.pdf “Pastry: Scalable, decentralized object location and routing for large-scale peer-to-peer systems” Rowstron and Druschel research.microsoft.com/~antr/PAST/pastry.pdf
“PKI Security for JXTA Overlay Networks” Jeffery Eric Altman www.jxta.org/docs/pki-security-for-jxta.pdf “An Introduction to SSL/TLS and Certificates” Fredrick J. Hirsch home.earthlink.net/~fjhirsch/SSL/tlstut_html/sld001.htm
References: P2P Sockets Project Home Page: http://p2psockets.jxta.org/ “JXTA Performance Study” Halepovic, Deters bosna.usask.ca/ pub/PacRim_Halepovic_JXTAPerformanceStudy.pdf
