Application layer protocols

The application layer is used by programs for network communication. Data is passed from the program in an application-specific format, then encapsulated into a transport layer protocol. Since the IP stack has no layers between the application and transport layers, the application layer must include any protocols that act like the OSI's presentation and session layer protocols. This is usually done through libraries.

Data sent over the network is passed into the application layer where it is encapsulated into the application layer protocol. From there, the data is passed down into the lower layer protocol of the transport layer. The two most common lower layer protocols are TCP and UDP. Common services have specific ports assigned to them (HTTP has port 80; FTP has port 21; etc.) while clients use ephemeral ports.

Routers and switches do not utilize this layer but bandwidth throttling applications do.

Figure 5.17. Application layer protocols

Application layer protocols

Telnet and FTP

TELNET (TELetype NETwork) is a network protocol used on the Internet or local area network (LAN) connections. It was developed in 1969 beginning with RFC#0015 and standardized as IETF STD 8, one of the first Internet standards. It has limitations that are considered to be security risks. The term telnet also refers to software which implements the client part of the protocol. TELNET clients have been available on most Unix systems for many years and are available for virtually all platforms. Most network equipment and OSs with a TCP/IP stack support some kind of TELNET service server for their remote configuration (including ones based on Windows NT). Recently, SSH has begun to dominate remote access for Unix-based machines.

"To telnet" is also used as a verb meaning to establish or use a TELNET or other interactive TCP connection, as in, "To change your password, telnet to the server and run the passwd command". A user will be telneting to a unix-like server system or a simple network device such as a switch. For example, a user might telnet in from home to check his mail at school. In doing so, he would be using a telnet client to connect from his computer to one of his servers. Once the connection is established, he would then log in with his account information and execute operating system commands remotely on that computer.

On many systems, the client may also be used to make interactive raw-TCP sessions. TELNET is a client-server protocol, based on a reliable connection-oriented transport. Typically this is on TCP port 23.

Experts in computer security recommend that the use of TELNET for remote logins should be discontinued under all normal circumstances, for the following reasons:

  • TELNET, by default, does not encrypt any data sent over the connection (including passwords), and so it is trivial to eavesdrop on the communications and use the password later for malicious purposes; anybody who has access to a router, switch, or gateway located on the network between the two hosts where TELNET is being used can intercept the packets passing by and easily obtain login and password information (and whatever else is typed) with any of several common utilities like tcpdump and Wireshark.

  • Most implementations of TELNET lack an authentication scheme that makes it possible to ensure that communication is carried out between the two desired hosts, and not intercepted in the middle.

  • Commonly used TELNET daemons have several vulnerabilities discovered over the years.

These security-related shortcomings have seen the usage of the TELNET protocol drop rapidly, especially on the public Internet, in favor of a more secure and functional protocol called SSH, first released in 1995. SSH provides all functionality of telnet, with the addition of strong encryption to prevent sensitive data such as passwords from being intercepted, and public key authentication, to ensure that the remote computer is actually who it claims to be.

FTP is a commonly used protocol for exchanging files over any network that supports the TCP/IP protocol. There are two computers involved in an FTP transfer: a server and a client. The FTP server, running FTP server software, listens on the network for connection requests from other computers. The client computer, running FTP client software, initiates a connection to the server. Once connected, the client can do a number of file manipulation operations such as uploading files to the server, download files from the server, rename or delete files on the server and so on. Virtually every computer platform supports the FTP protocol. This allows any computer connected to a TCP/IP based network to manipulate files on another computer on that network regardless of which operating systems are involved (if the computers permit FTP access). There are many existing FTP client and server programs.

FTP runs exclusively over TCP. FTP servers by default listen on port 21 for incoming connections from FTP clients. A connection to this port from the FTP Client forms the control stream on which commands are passed to the FTP server from the FTP client and on occasion from the FTP server to the FTP client. For the actual file transfer to take place, a different connection is required which is called the data stream. Depending on the transfer mode, the process of setting up the data stream is different. In active mode, the FTP client opens a random port (> 1023), sends the FTP server the random port number on which it is listening over the control stream and waits for a connection from the FTP server. When the FTP server initiates the data connection to the FTP client it binds the source port to port 20 on the FTP server.In passive mode, the FTP server opens a random port (> 1023), sends the FTP client the server's IP address to connect to and the port on which it is listening (a 16 bit value broken into a high and low byte) over the control stream and waits for a connection from the FTP client. In this case the FTP client binds the source port of the connection to a random port greater than 1023

The FTP specification is an inherently insecure method of transferring files because there is no method specified for transferring data in an encrypted fashion. This means that under most network configurations, user names, passwords, FTP commands and transferred files can be "sniffed" or viewed by anyone on the same network using a packet sniffer. This is a problem common to many Internet protocol specifications written prior to the creation of SSL such as HTTP, SMTP and Telnet. The common solution to this problem is to use either SFTP (SSH File Transfer Protocol), or FTPS (FTP over SSL), which adds SSL or TLS encryption to FTP as specified in RFC 4217.

SMTP/POP3/IMAP4

E-mail messages are generally sent to an e-mail server that stores received messages in the recipient's e-mail mailbox. The user later retrieves these messages with either a web browser or an e-mail client that uses one of a number of e-mail retrieval protocols. While some clients and servers preferentially use vendor specific, typically proprietary protocols, most support the Internet standard protocols SMTP for sending e-mail and POP3 and IMAP4 for retrieving e-mail, allowing interoperability with other servers and clients. For example, Microsoft's Outlook client typically uses a proprietary protocol to communicate with an Exchange server as does IBM's Notes client when communicating with a Domino server, but all of these products also support SMTP, POP3, and IMAP4. Support for the Internet standard protocols allows many other e-mail clients such as Qualcomm's Eudora or Mozilla Thunderbird to access these servers and similarly allows the clients to be used with other servers.

E-mail clients can generally be configured to use either POP3 or IMAP4 to retrieve e-mail and in both cases use SMTP for sending. Most e-mail programs can also use Lightweight Directory Access Protocol (LDAP) for directory services. Nearly all subscribers to individual Internet service provider e-mail accounts access their e-mail with client software that uses POP3. IMAP is often used in large networks; for example, a college campus mail system. IMAP allows users to access new messages instantly on their computers, since the mail is stored on the server. With POP3, users either download the e-mail to their computer or access it via the web.

Simple Mail Transfer Protocol (SMTP) is the de facto standard for e-mail transmissions across the Internet. The protocol used today is also known as ESMTP and defined in RFC 2821. SMTP is a relatively simple, text-based protocol, where one or more recipients of a message are specified (and in most cases verified to exist) and then the message text is transferred. SMTP uses TCP port 25. To determine the SMTP server for a given domain name, the MX (Mail eXchange) DNS record is typically used, falling back to a simple A record in the case of no MX (not all MTAs (Mail Transfer Agents) support fallback. SMTP is a "push" protocol that does not allow one to "pull" messages from a remote server on demand. To do this a mail client must use POP3 or IMAP.

SMTP started becoming widely used in the early 1980s. At the time, it was a complement to UUCP (Unix to Unix CoPy)and used a store and forward mechanism; an examples of push technology. Sendmail was one of the first Mail Transfer Agent (MTA) to implement SMTP. As of 2001 there are at least 50 programs that implement SMTP as a client (sender of messages) or a server (receiver of messages). Some other popular SMTP server programs include Postfix, qmail, Novell GroupWise, Exim, Novell NetMail and Microsoft Exchange Server.

Since this protocol started out as purely ASCII text-based, it did not deal well with binary files. Standards such as Multipurpose Internet Mail Extensions (MIME) were developed to encode binary files for transfer through SMTP. MTAs developed after sendmail also tended to be implemented 8-bit-clean, so that the alternate "just send eight" strategy could be used to transmit arbitrary data via SMTP. Non-8-bit-clean MTAs today tend to support the 8BITMIME extension, permitting binary files to be transmitted almost as easily as plain text.

One of the limitations of the original SMTP is that it has no facility for authentication of senders. Therefore the SMTP-AUTH extension was defined. However, the impracticalities of widespread SMTP-AUTH implementation and management means that E-mail spamming is not and cannot be addressed by it.

E-mail clients use the Post Office Protocol version 3 (POP3), an application-layer Internet standard protocol, to retrieve e-mail from a remote server over a TCP/IP connection. Nearly all subscribers to individual Internet service provider e-mail accounts access their e-mail with client software that uses POP3.

The design of POP3 and its procedures supports end-users with intermittent connections (such as dial-up connections), allowing these users to retrieve e-mail when connected and then to view and manipulate the retrieved messages without needing to stay connected. Although most clients have an option to leave mail on server, e-mail clients using POP3 generally connect, retrieve all messages, store them on the user's PC as new messages, delete them from the server, and then disconnect. In contrast, the newer, more capable Internet Message Access Protocol (IMAP) supports both connected and disconnected modes of operation. E-mail clients using IMAP generally leave messages on the server until the user explicitly deletes them. This and other facets of IMAP operation allow multiple clients to access the same mailbox. Most e-mail clients support either POP3 or IMAP to retrieve messages; however, fewer Internet Service Providers (ISPs) support IMAP. The fundamental difference between POP3 and IMAP4 is that POP3 offers access to a mail drop; the mail exists on the server until it is collected by the client. Even if the client leaves some or all messages on the server, the client's message store is considered authoritative. In contrast, IMAP4 offers access to the mail store; the client may store local copies of the messages, but these are considered to be a temporary cache; the server's store is authoritative.

MIME serves as the standard for attachments and non-ASCII text in e-mail. Although neither POP3 nor SMTP require MIME-formatted e-mail, essentially all Internet e-mail comes MIME-formatted, so POP clients must also understand and use MIME. IMAP, by design, assumes MIME-formatted e-mail.

Like many other older Internet protocols, POP3 originally supported only an unencrypted login mechanism. Although plain text transmission of passwords in POP3 still commonly occurs, POP3 currently supports several authentication methods to provide varying levels of protection against illegitimate access to a user's e-mail. One such method, APOP, uses the MD5 hash function in an attempt to avoid replay attacks and disclosure of a shared secret. Clients implementing APOP include Mozilla, Thunderbird, Opera, Eudora, KMail and Novell Evolution. POP3 clients can also support SASL authentication methods via the AUTH extension.

POP3 works over a TCP/IP connection using TCP on network port 110. E-mail clients can encrypt POP3 traffic using TLS or SSL. A TLS or SSL connection is negotiated using the STLS command. Some clients and servers, like Google Gmail, instead use the deprecated alternate-port method, which uses TCP port 995.

The current version of IMAP since 1996, IMAP version 4 revision 1 (IMAP4rev1), is defined by RFC 3501. Unlike many older Internet protocols, IMAP4 natively supports encrypted login mechanisms. Plain-text transmission of passwords in IMAP4 is also possible. Because the encryption mechanism to be used must be agreed between the server and client, plain-text passwords are used in some combinations of clients and servers (typically Microsoft Windows clients and non-Windows servers). It is also possible to encrypt IMAP4 traffic using SSL, either by tunneling IMAP4 communications over SSL on port 993, or by issuing STARTTLS within an established IMAP4 session (see RFC 2595).

IMAP4 works over a TCP/IP connection using network port 143.

Advantages of using IMAP4 over POP3:

  • When using POP3, clients typically connect to the e-mail server briefly, only as long as it takes to download new messages. When using IMAP4, clients often stay connected as long as the user interface is active and download message content on demand. For users with many or large messages, this IMAP4 usage pattern can result in faster response times.

  • The POP3 protocol requires the currently connected client to be the only client connected to the mailbox. In contrast, the IMAP protocol specifically allows simultaneous access by multiple clients and provides mechanisms for clients to detect changes made to the mailbox by other, concurrently connected, clients.

  • Nearly all internet e-mail is transmitted in MIME format, allowing messages to have a tree structure where the leaf nodes are any of a variety of single part content types and the non-leaf nodes are any of a variety of multipart types. The IMAP4 protocol allows clients to separately retrieve any of the individual MIME parts and also to retrieve portions of either individual parts or the entire message. These mechanisms allow clients to retrieve the text portion of a message without retrieving attached files or to stream content as it is being fetched.

  • Through the use of flags defined in the IMAP4 protocol clients can keep track of message state, for example whether or not the message has been read, replied to, or deleted. These flags are stored on the server, so different clients accessing the same mailbox at different times can detect state changes made by other clients. POP3 provides no mechanism for clients to store such state information on the server so if a single user accesses a mailbox with two different POP3 clients state information, such as whether a message has been accessed, cannot be synchronized between the clients. The IMAP4 protocol supports both pre-defined system flags and client defined keywords, System flags indicate state information such as whether a message has been read. Keywords, which are not supported by all IMAP servers, allow messages to be given one or more tags whose meaning is up to the client. Adding user created tags to messages is an operation supported by some Webmail services, such as Gmail, although generally not using IMAP.

  • IMAP4 clients can create, rename, and/or delete mailboxes (usually presented to the user as folders) on the server, and move messages between mailboxes. Multiple mailbox support also allows servers to provide access to shared and public folders.

  • IMAP4 provides a mechanism for a client to ask the server to search for messages meeting a variety of criteria. This mechanism avoids requiring clients to download every message in the mailbox in order to perform these searches.

  • Reflecting the experience of earlier Internet protocols, IMAP4 defines an explicit mechanism by which it may be extended. Many extensions to the base protocol have been proposed and are in common use. IMAP2bis did not have an extension mechanism, and POP3 now has one defined by RFC 2449.

HTTP

Hypertext Transfer Protocol (HTTP) is a method used to transfer or convey information on the World Wide Web. Its original purpose was to provide a way to publish and retrieve HTML pages. Development of HTTP was coordinated by the World Wide Web Consortium and the Internet Engineering Task Force, culminating in the publication of a series of RFCs, most notably RFC 2616 (1999), which defines HTTP/1.1, the version of HTTP in common use today.

HTTP is a request/response protocol between clients and servers. The originating client, such as a web browser, spider, or other end-user tool, is referred to as the user agent. The destination server, which stores or creates resources such as HTML files and images, is called the origin server. In between the user agent and origin server may be several intermediaries, such as proxies, gateways, and tunnels. An HTTP client initiates a request by establishing a Transmission Control Protocol (TCP) connection to a particular port on a remote host (port 80 by default). An HTTP server listening on that port waits for the client to send a request message. Upon receiving the request, the server sends back a status line, such as "HTTP/1.1 200 OK", and a message of its own, the body of which is perhaps the requested file, an error message, or some other information.

Resources to be accessed by HTTP are identified using Uniform Resource Identifiers (URIs) (or, more specifically, URLs) using the http: or https URI schemes.

The request message consists of the following:

  • Request line, such as GET /images/logo.gif HTTP/1.1, which requests the file logo.gif from the /images directory

  • Headers, such as Accept-Language: en

  • An empty line

  • An optional message body

The request line and headers must all end with CRLF (i.e. a carriage return followed by a line feed).

HTTP defines eight methods (sometimes referred to as "verbs") indicating the desired action to be performed on the identified resource.

  • HEAD

    Asks for the response identical to the one that would correspond to a GET request, but without the response body. This is useful for retrieving meta-information written in response headers, without having to transport the entire content.

  • GET

    Requests a representation of the specified resource. By far the most common method used on the Web today. Should not be used for operations that cause side-effects (using it for actions in web applications is a common misuse). See 'safe methods' below.

  • POST

    Submits data to be processed (e.g. from an HTML form) to the identified resource. The data is included in the body of the request. This may result in the creation of a new resource or the updates of existing resources or both.

  • PUT

    Uploads a representation of the specified resource.

  • DELETE

    Deletes the specified resource.

  • TRACE

    Echoes back the received request, so that a client can see what intermediate servers are adding or changing in the request.

  • OPTIONS

    Returns the HTTP methods that the server supports. This can be used to check the functionality of a web server.

  • CONNECT

    Converts the request connection to a transparent TCP/IP tunnel, usually to facilitate SSL-encrypted communication (HTTPS) through an unencrypted HTTP proxy.

Methods HEAD and GET are defined as safe, which means they are intended only for information retrieval and should not change the state of the server (in other words, they should not have side effects). Unsafe methods (such as POST, PUT and DELETE) should be displayed to the user in a special way, typically as buttons rather than links, thus making the user aware of possible obligations (such as a button that causes a financial transaction).

Despite the required safety of GET requests, in practice they can cause changes on the server. For example, a Web server may use the retrieval through a simple hyperlink to initiate deletion of a domain database record, thus causing a change of the server's state as a side-effect of a GET request. This is discouraged, because it can cause problems for Web caching, search engines and other automated agents, which can make unintended changes on the server. Another case is that a GET request may cause the server to create a cache space.

Methods POST, PUT, and DELETE are intended to cause "real-world" effects are defined as obligated' because the client that initiates the request is responsible for such effects. Methods GET, HEAD, PUT and DELETE are defined to be idempotent, meaning that multiple identical requests should have the same effect as a single request.

In HTTP/1.0 and since, the first line of the HTTP response is called the status line and includes a numeric status code (such as "404") and a textual reason phrase (such as "Not Found"). The way the user agent handles the response primarily depends on the code and secondarily on the response headers.

HTTP is stateless. The advantage of a stateless protocol is that hosts do not need to retain information about users between requests, but this forces the use of alternative methods for maintaining users' state, for example, when a host would like to customize content for a user who has visited before. The common method for solving this problem involves the use of sending and requesting cookies. Other methods include server side sessions, hidden variables (when current page is a form), and URL encoded parameters (such as index.php?userid=3).

There are currently two methods of establishing a secure HTTP connection: the https URI scheme and the HTTP 1.1 Upgrade header. The https URI scheme has been deprecated by RFC 2817, which introduced the Upgrade header; however, as browser support for the Upgrade header is nearly non-existent, the https URI scheme is still the dominant method of establishing a secure HTTP connection.

https: is a URI scheme syntactically identical to the http: scheme used for normal HTTP connections, but which signals the browser to use an added encryption layer of SSL/TLS to protect the traffic. SSL is especially suited for HTTP since it can provide some protection even if only one side of the communication is authenticated. In the case of HTTP transactions over the Internet, typically only the server side is authenticated.

HTTP 1.1 introduced support for the Upgrade header. In the exchange, the client begins by making a clear-text request, which is later upgraded to TLS. Either the client or the server may request (or demand) that the connection be upgraded to TLS.

DHCP

DHCP is a set of rules used by communications devices such as a computer, router or network adapter to allow the device to request and obtain an IP address from a server which has a list of addresses available for assignment. DHCP is a protocol used by networked computers (clients) to obtain IP addresses and other parameters such as the default gateway, subnet mask, and IP addresses of DNS servers from a DHCP server. It facilitates access to a network because these settings would otherwise have to be made manually for the client to participate in the network.

The DHCP server ensures that all IP addresses are unique, e.g., no IP address is assigned to a second client while the first client's assignment is valid (its lease has not expired). Thus IP address pool management is done by the server and not by a human network administrator. As of 2006, RFC 2131 provides the latest (dated March 1997) DHCP definition. DHCP functionally became a successor to the older BOOTP protocol, whose leases were given for infinite time and did not support options.

The Dynamic Host Configuration Protocol (DHCP) automates the assignment of IP addresses, subnet masks, default gateway, and other IP parameters. The assignment occurs when the DHCP-configured machine boots up or regains connectivity to a network. The DHCP client sends out a query requesting a response from a DHCP server on the locally attached network. The query is typically initiated immediately after booting up and before the client initiates any IP based communication with other hosts. The DHCP server then replies to the client with its assigned IP address, subnet mask, DNS server and default gateway information.

The assignment of the IP address generally expires after a predetermined period of time, before which the DHCP client and server renegotiate a new IP address from the server's predefined pool of addresses. Typical intervals range from one hour to several months, and can, if desired, be set to infinite (never expire). The length of time the address is available to the device it was assigned to is called a lease, and is determined by the server.

Configuring firewall rules to accommodate access from machines who receive their IP addresses via DHCP is therefore more difficult because the remote IP address will vary from time to time. Administrators must usually allow access to the entire remote DHCP subnet for a particular TCP/UDP port. Most home routers and firewalls are configured in the factory to be DHCP servers for a home network. ISPs generally use DHCP to assign clients individual IP addresses. DHCP is a broadcast-based protocol. As with other types of broadcast traffic, it does not cross a router unless specifically configured to do so.

Since DHCP servers provide IP addresses and thus network connectivity to anyone who has physical network access, DHCP simplifies network intrusion. If DHCP is used on an unprotected wireless LAN, anyone within range has access to the network, including use of internet connectivity and potentially access to data not otherwise protected.

Depending on implementation, the DHCP server has three methods of allocating IP-addresses:

  • manual allocation, where the DHCP server performs the allocation based on a table with MAC address - IP address pairs manually filled by the server administrator. Only requesting clients with a MAC address listed in this table get the IP address according to the table.

  • automatic allocation, where the DHCP server permanently assigns to a requesting client a free IP-address from a range given by the administrator.

  • dynamic allocation, the only method which provides dynamic re-use of IP addresses. A network administrator assigns a range of IP addresses to DHCP, and each client computer on the LAN has its TCP/IP software configured to request an IP address from the DHCP server when that client computer's network interface card starts up. The request-and-grant process uses a lease concept with a controllable time period. This eases the network installation procedure on the client computer side considerably.

Some DHCP server implementations can update the DNS name associated with the client hosts to reflect the new IP address. They make use of the DNS update protocol established with RFC 2136.

Firewalls usually have to permit DHCP traffic explicitly. Specification of the DHCP client-server protocol describes several cases when packets must have the source address of 0x00000000 or the destination address of 0xffffffff. Anti-spoofing policy rules and tight inclusive firewalls often stop such packets. To allow DHCP, network administrators need to allow several types of packets through the server-side firewall. All DHCP packets travel as UDP datagrams; all client-sent packets have source port 68 and destination port 67; all server-sent packets have source port 67 and destination port 68.

DHCP uses the same two IANA assigned ports as BOOTP: 67/udp for the server side, and 68/udp for the client side. DHCP operations fall into four basic phases. These phases are IP lease request, IP lease offer, IP lease selection, and IP lease acknowledgement. After the client obtained an IP address, the client may start an address resolution query to prevent IP conflicts caused by address poll overlapping of DHCP servers.

  • DHCP discovery

    The client broadcasts on the local physical subnet to find available servers. This client-implementation creates a UDP packet with the broadcast destination of 255.255.255.255 or subnet broadcast address. A client can also request its last-known IP address (in the example below, 192.168.1.100). If the client is still in a network where this IP is valid, the server might grant the request. Otherwise, it depends whether the server is set up as authoritative or not. An authoritative server will deny the request, making the client ask for a new IP immediately. A non-authoritative server simply ignores the request, leading to an implementation dependent time out for the client to give up on the request and ask for a new IP.

  • DHCP offers

    When a DHCP server receives an IP lease request from a client, it extends an IP lease offer. This is done by reserving an IP address for the client and sending a DHCPOFFER message across the network to the client. This message contains the client's MAC address, followed by the IP address that the server is offering, the subnet mask, the lease duration, and the IP address of the DHCP server making the offer.

  • DHCP requests

    When the client PC receives an IP lease offer, it must tell all the other DHCP servers that it has accepted an offer. To do this, the client broadcasts a DHCPREQUEST message containing the IP address of the server that made the offer. When the other DHCP servers receive this message, they withdraw any offers that they might have made to the client. They then return the address that they had reserved for the client back to the pool of valid addresses that they can offer to another computer. Any number of DHCP servers can respond to an IP lease request, but the client can only accept one offer per network interface card.

  • DHCP acknowledgement

    When the DHCP server receives the DHCPREQUEST message from the client, it initiates the final phase of the configuration process. This acknowledgement phase involves sending a DHCPACK packet to the client. This packet includes the lease duration and any other configuration information that the client might have requested. At this point, the TCP/IP configuration process is complete. The server acknowledges the request and sends the acknowledgement to the client. The system as a whole expects the client to configure its network interface with the supplied options.

  • DHCP information

    The client sends a request to the DHCP server: either to request more information than the server sent with the original DHCPACK; or to repeat data for a particular application - for example, browsers use DHCP Inform to obtain web proxy settings via WPAD. Such queries do not cause the DHCP server to refresh the IP expiry time in its database.

  • DHCP releasing

    The client sends a request to the DHCP server to release the DHCP and the client unconfigures its IP address. As clients usually do not know when users may unplug them from the network, the protocol does not define the sending of DHCP Release as mandatory.

DNS

The Domain Name System (DNS) stores and associates many types of information with domain names; most importantly, it serves as the phone book for the internet, translating human-friendly domain names and computer hostnames, e.g. en.wikipedia.org, into computer-friendly IP addresses. It also lists mail exchange servers accepting e-mail for each domain. In providing a worldwide keyword-based redirection service, DNS is an essential component of contemporary Internet use.

The most basic use of DNS is to translate hostnames to IP addresses. It is in very simple terms like a phone book. For example, if you want to know the internet address of en.wikipedia.org, DNS can be used to tell you it's 66.230.200.100. DNS also has other important uses. Pre-eminently, the DNS makes it possible to assign Internet destinations to the human organization or concern they represent, independently of the physical routing hierarchy represented by the numerical IP address. Because of this, hyperlinks and Internet contact information can remain the same, whatever the current IP routing arrangements may be, and can take a human-readable form (such as "wikipedia.org") which is rather easier to remember than an IP address (such as 66.230.200.100). The DNS also distributes the responsibility for assigning domain names and mapping them to IP networks by allowing an authoritative server for each domain to keep track of its own changes, avoiding the need for a central registrar to be continually consulted and updated.

The DNS uses TCP and UDP on port 53 to serve requests. Almost all DNS queries consist of a single UDP request from the client followed by a single UDP reply from the server. TCP typically comes into play only when the response data size exceeds 512 bytes, or for such tasks as zone transfer.

The practice of using a name as a more human-legible abstraction of a machine's numerical address on the network goes all the way to the ARPAnet era. With the older system, each computer on the network retrieved a file called HOSTS.TXT from a central computer. The HOSTS.TXT file mapped numerical addresses to names. A hosts file still exists on most modern operating systems, either by default or through configuration, and allows users to specify an IP address (eg. 192.0.34.166) to use for a hostname (eg. www.example.net) without checking the DNS. Systems based on a hosts file have inherent limitations, because of the obvious requirement that every time a given computer's address changed, every computer that seeks to communicate with it would need an update to its hosts file.

In 1984 Berkeley students wrote the first UNIX implementation of DNS. In 1985 Kevin Dunlap of DEC significantly re-wrote the DNS implementation and renamed it BIND. BIND was ported to the Windows NT platform in the early 1990s. Due to its long history of security issues, several alternative nameserver/resolver programs have been written and distributed in recent years.

The domain name space consists of a tree of domain names. Each node or leaf in the tree has one or more resource records, which hold information associated with the domain name. The tree sub-divides into zones. A zone consists of a collection of connected nodes authoritatively served by an authoritative DNS nameserver. (Note that a single nameserver can host several zones.) When a system administrator wants to let another administrator control a part of the domain name space within his or her zone of authority, he or she can delegate control to the other administrator. This splits a part of the old zone off into a new zone, which comes under the authority of the second administrator's nameservers. The old zone becomes no longer authoritative for what comes under the authority of the new zone.

A resolver looks up the information associated with nodes. A resolver knows how to communicate with name servers by sending DNS requests, and heeding DNS responses. Resolving usually entails iterating through several name servers to find the needed information. Some resolvers function simplistically and can only communicate with a single name server. These simple resolvers rely on a recursing name server to perform the work of finding information for them.

Figure 5.18. Domain names

Domain names

Domain names, arranged in a tree, cut into zones, each served by a nameserver.

A domain name usually consists of two or more parts, separated by dots. For example wikipedia.org.

  • The rightmost label conveys the top-level domain (for example, the address en.wikipedia.org has the top-level domain org).

  • Each label to the left specifies a subdivision or subdomain of the domain above it. Note that "subdomain" expresses relative dependence, not absolute dependence: for example, wikipedia.org comprises a subdomain of the org domain, and en.wikipedia.org comprises a subdomain of the domain wikipedia.org. In theory, this subdivision can go down to 127 levels deep, and each label can contain up to 63 characters,[1] as long as the whole domain name does not exceed a total length of 255 characters. But in practice some domain registries have shorter limits than that.

  • A hostname refers to a domain name that has one or more associated IP addresses. For example, the en.wikipedia.org and wikipedia.org domains are both hostnames, but the org domain is not.

The DNS consists of a hierarchical set of DNS servers. Each domain or subdomain has one or more authoritative DNS servers that publish information about that domain and the name servers of any domains "beneath" it. The hierarchy of authoritative DNS servers matches the hierarchy of domains. At the top of the hierarchy stand the root nameservers: the servers to query when looking up (resolving) a top-level domain name (TLD).

In theory a full host name may have several name segments, (e.g ahost.ofasubnet.ofabiggernet.inadomain.example). In practice, in the experience of the majority of public users of Internet services, full host names will frequently consist of just three segments (ahost.inadomain.example, and most often www.inadomain.example). For querying purposes, software interprets the name segment by segment, from right to left, using an iterative search procedure. At each step along the way, the program queries a corresponding DNS server to provide a pointer to the next server which it should consult.

As originally envisaged, the process was as simple as:

  • the local system is pre-configured with the known addresses of the root servers in a file of root hints, which need to be updated periodically by the local administrator from a reliable source to be kept up to date with the changes which occur over time.

  • query one of the root servers to find the server authoritative for the next level down (so in the case of our simple hostname, a root server would be asked for the address of a server with detailed knowledge of the example top level domain).

  • querying this second server for the address of a DNS server with detailed knowledge of the second-level domain (inadomain.example in our example).

  • repeating the previous step to progress down the name, until the final step which would, rather than generating the address of the next DNS server, return the final address sought.

The diagram illustrates this process for the real host www.wikipedia.org.

Figure 5.19. Domain resolution

Domain resolution

A DNS recurser consults three nameservers to resolve the address www.wikipedia.org.

The mechanism in this simple form has a difficulty: it places a huge operating burden on the root servers, with each and every search for an address starting by querying one of them. Being as critical as they are to the overall function of the system such heavy use would create an insurmountable bottleneck for trillions of queries placed every day.

When an application (such as a web browser) tries to find the IP address of a domain name, it doesn't necessarily follow all of the steps outlined in the section above. The DNS resolution process allows for caching (ie. the local recording and subsequent consultation of the results of a DNS query) for a given period of time after a successful answer. How long a resolver caches a DNS response (ie. how long a DNS response remains valid) is determined by a value called the time to live (TTL). The TTL is set by the administrator of the DNS server handing out the response. The period of validity may vary from just seconds to days or even weeks. If a DNS server is frequently queried for many addresses, there is an increased probability that a DNS response to at least part of a DNS query will already exist in that server's cache and will not yet have exceeded its time to live. Most domestic and small-business clients "hand off" address resolution to their ISP's DNS servers to perform the look-up process, thus allowing for the greatest benefit from those same ISPs having busy local caches serving a wide variety of queries and a large number of users. Generally, the Start of Authority (SOA) record specifies the time to live.

Users generally do not communicate directly with a DNS resolver. Instead DNS resolution takes place transparently in client applications such as web browsers, mail clients, and other Internet applications. When a request is made which necessitates a DNS lookup, such programs send a resolution request to the local DNS resolver in the operating system which in turn handles the communications required. The DNS resolver will almost invariably have a cache containing recent lookups. If the cache can provide the answer to the request, the resolver will return the value in the cache to the program that made the request. If the cache does not contain the answer, the resolver will send the request to a designated DNS server or servers. In the case of most home users, the Internet service provider to which the machine connects will usually supply this DNS server: such a user will either have configured that server's address manually or allowed DHCP to set it. The name server thus queried will follow the process outlined above, until it either successfully finds a result or does not. It then returns its results to the DNS resolver; assuming it has found a result, the resolver duly caches that result for future use, and hands the result back to the software which initiated the request.

The DNS includes several other functions:

  • Hostnames and IP addresses do not necessarily match on a one-to-one basis. Many hostnames may correspond to a single IP address: combined with virtual hosting, this allows a single machine to serve many web sites. Alternatively a single hostname may correspond to many IP addresses: this can facilitate fault tolerance and load distribution, and also allows a site to move physical location seamlessly.

  • There are many uses of DNS besides translating names to IP addresses. For instance, Mail transfer agents use DNS to find out where to deliver e-mail for a particular address. The domain to mail exchanger mapping provided by MX records accommodates another layer of fault tolerance and load distribution on top of the name to IP address mapping.

  • To provide resilience in the event of computer failure, multiple DNS servers provide coverage of each domain. In particular, more than thirteen root servers exist worldwide. DNS programs or operating systems have the IP addresses of these servers built in.

Important categories of data stored in the DNS include the following:

  • An A record or address record maps a hostname to a 32-bit IPv4 address.

  • An AAAA record or IPv6 address record maps a hostname to a 128-bit IPv6 address.

  • A CNAME record or canonical name record is an alias of one name to another. The A record that the alias is pointing to can be either local or remote - on a foreign name server. This is useful when running multiple services (like an FTP and a webserver) from a single IP address. Each service can then have its own entry in DNS (like ftp.example.com and www.example.com.)

  • An MX record or mail exchange record maps a domain name to a list of mail exchange servers for that domain.

  • A PTR record or pointer record maps an IPv4 address to the canonical name for that host. Setting up a PTR record for a hostname in the in-addr.arpa domain that corresponds to an IP address implements reverse DNS lookup for that address. For example (at the time of writing), www.icann.net has the IP address 192.0.34.164, but a PTR record maps 164.34.0.192.in-addr.arpa to its canonical name, referrals.icann.org.

  • An NS record or name server record maps a domain name to a list of DNS servers authoritative for that domain. Delegations depend on NS records.

  • An SOA record or start of authority record specifies the DNS server providing authoritative information about an Internet domain, the email of the domain administrator, the domain serial number, and several timers relating to refreshing the zone.

RIP/OSPF

The Routing Information Protocol (RIP) is one of the most commonly used interior gateway protocol (IGP) routing protocols on internal networks (and to a lesser extent, networks connected to the Internet), which helps routers dynamically adapt to changes of network connections by communicating information about which networks each router can reach and how far away those networks are. Although RIP is still actively used, it is generally considered to have been made obsolete by routing protocols such as OSPF.

RIP is a distance-vector routing protocol, which employs the hop count as a routing metric. The maximum number of hops allowed with RIP is 15, and the hold down time is 180 seconds. Each RIP router transmits full updates every 30 seconds by default, generating large amounts of network traffic in lower bandwidth networks. It runs at the network layer of the Internet protocol suite. RIP prevents routing loops from continuing indefinitely by implementing a limit on the number of hops allowed in a path from the source to a destination. RIP implements the split horizon and holddown mechanisms to prevent incorrect routing information from being propagated. These are some of the stability features of RIP. In many current networking environments RIP would not be the first choice for routing as its convergence times and scalability are poor it is however easier to configure because, using minimal settings for any routing protocols, RIP does not require any parameter on a router whereas all the other protocols require at least one or more parameters.

There are three versions of RIP, RIPv1, RIPv2, and RIPng -RIPv2 being the most often used.

The Open Shortest Path First (OSPF) protocol is a link-state, hierarchical interior gateway protocol (IGP) for network routing. Dijkstra's algorithm is used to calculate the shortest path tree. It uses path cost as its routing metric. Path cost is determined generally by the speed (aka bandwidth) of the interface addressing the given route. A link state database (LSDB) is constructed as a tree-image of the network topology, and identical copies of the LSDB are periodically updated on all routers in each OSPF-aware area. By convention, area 0 represents the core or "backbone" region of an OSPF-enabled network, and other OSPF area numbers may be designated to serve other regions of alarge network--however every additional OSPF area must have a direct connection to the backbone or 0 OSPF area. OSPF is perhaps the most widely-used IGP in large networks. The most widely-used exterior gateway protocol (EGP) is BGP. OSPF Protocol can operate securely, using MD5 to authenticate peers before forming adjacencies, and before accepting link-state advertisements. A natural successor to the Routing Information Protocol (RIP), it was VLSM-capable or classless from its inception. A newer version of OSPF (OSPFv3) now supports IPv6 as well.

An OSPF network can be broken up into smaller networks. A special area called the backbone area forms the core of the network, and other areas are connected to it. Inter-area routing goes via the backbone. All areas must connect to the backbone; if no direct connection is possible, a virtual link may be established.

Routers in the same broadcast domain or at each end of a point-to-point telecommunications link form adjacencies when they have detected each other. This detection occurs when a router "sees" itself in a hello packet. This is called a two way state and is the most basic relationship. The routers elect a designated router (DR) and a backup designated router (BDR) which act as a hub to reduce traffic between routers. OSPF uses both unicast and multicast to send "hello packets" and link state updates. Multicast addresses 224.0.0.5 and 224.0.0.6 are reserved for OSPF. In contrast to the Routing Information Protocol (RIP) or the Border Gateway Protocol (BGP), OSPF does not use TCP or UDP but uses IP directly, via IP protocol 89.

SNMP

The simple network management protocol (SNMP) forms part of the internet protocol is used by network management systems to monitor network-attached devices for conditions that warrant administrative attention. It consists of a set of standards for network management, including an Application Layer protocol, a database schema, and a set of data objects. SNMP exposes management data in the form of variables on the managed systems, which describe the system configuration. These variables can then be queried (and sometimes set) by managing applications.

In typical SNMP usage, there are generally a number of systems to be managed, and one or more systems managing them. A software component called an agent (see below) runs on each managed system and reports information via SNMP to the managing systems. Essentially, SNMP agents expose management data on the managed systems as variables (such as "free memory", "system name", "number of running processes", "default route"). Configuration and control operations are used only when changes are needed to the network infrastructure and the monitoring operations are frequently performed on a regular basis.

The variables accessible via SNMP are organized in hierarchies. These hierarchies, and other metadata, are described by Management Information Bases (MIBs).

An SNMP-managed network consists of three key components:


   1. Managed devices
   2. Agents
   3. Network-management systems (NMSs)

A managed device is a network node that contains an SNMP agent and that resides on a managed network. Managed devices collect and store management information and make this information available to NMSs using SNMP. Managed devices, sometimes called network elements, can be routers and access servers, switches and bridges, hubs, computer hosts, or printers.

An agent is a network-management software module that resides in a managed device. An agent has local knowledge of management information and translates that information into a form compatible with SNMP. An NMS executes applications that monitor and control managed devices. NMSs provide the bulk of the processing and memory resources required for network management. One or more NMSs must exist on any managed network. It functions as the equivalent of a client in the client-server architecture. It issues requests for management operations on behalf of an administrator or application and receives traps from agents as well.

The IETF recognizes Simple Network Management Protocol version 3 as defined by RFC 3411 - RFC 3418 (also known as STD0062) as the current standard version of SNMP as of 2004. The IETF considers earlier versions as obsolete. In practice, SNMP implementations often support multiple versions: typically SNMPv1, SNMPv2c, and SNMPv3. See RFC 3584 "Coexistence between Version 1, Version 2, and Version 3 of the Internet-standard Network Management Framework".

SNMPv3 provides three important services: authentication, privacy and access control.

SNMP is typically used for the following tasks:


    * Monitoring device uptimes (sysUpTimeInstance)
    * Inventory of OS versions (sysDescr)
    * Collect interface information (ifName, ifDescr, ifSpeed, ifType, ifPhysAddr)
    * Measuring network interface throughput (ifInOctets, ifOutOctets)
    * Querying a remote ARP cache (ipNetToMedia)

SNMP security implications:

  • Because they do not implement encryption, versions 1 and 2c are subject to packet sniffing of the clear text community string from the network traffic.

  • Because they do not implement a challenge-response handshake, all versions are subject to bruteforce and dictionary attacks for guessing the community strings/authentication strings.

  • Although SNMP works over TCP and other protocols, it is most commonly used over UDP which is connectionless and vulnerable to IP spoofing attacks. Thus, all versions are subject to bypassing access lists that might have been implemented to restrict snmp access.

  • Providing configuration/write features that are mostly overlooked in favor of the read/reporting features, the protocol is often forgotten and/or misconfigured and can be used to cause much damage.

  • SNMP tops the list of the SANS Institute's Common Default Configuration Issues with the issue of default SNMP community strings set to ‘public’ and ‘private’ and is number ten on the SANS The Top 10 Most Critical Internet Security Threats for the year 2000.

CORBA and DCOM

Distributed computing is a method of computer processing in which different parts of a program run simultaneously on two or more computers that are communicating with each other over a network. Distributed programming typically falls into one of several basic architectures or categories: Client-server, 3-tier architecture, N-tier architecture, Distributed objects, loose coupling, or tight coupling. There are numerous technologies and standards used to construct distributed computations, including some which are specially designed and optimized for that purpose, such as Remote Procedure Calls (RPC) or chacRemote Method Invocation (RMI) or .NET Remoting.

The composite parts of a distributed application interacts between themselves using the seven-level OSI model, where TCP/IP sockets used on lower levels and remote access tools like DCOM, CORBA, RMI on higher levels.

The Common Object Request Broker Architecture (CORBA) is a standard defined by the Object Management Group (OMG) that enables software components written in multiple computer languages and running on multiple computers to work together. The GIOP is an abstract protocol by which Object request brokers (ORBs) communicate. Standards associated with the protocol are maintained by the Object Management Group (OMG.). The GIOP architecture provides several concrete protocols:

  • Internet InterORB Protocol (IIOP) - The Internet Inter-Orb Protocol, is a protocol for communication between CORBA ORBs that has been published by the Object Management Group. IIOP is an implementation of the GIOP for use over an internet, and provides a mapping between GIOP messages and the TCP/IP layer.

  • SSL InterORB Protocol (SSLIOP) - SSLIOP is IIOP over SSL, providing encryption and authentication.

  • HyperText InterORB Protocol (HTIOP) - HTIOP is IIOP over HTTP, providing transparent proxy bypassing.

Figure 5.20. CORBA architecture

CORBA architecture

RMI-IIOP (read RMI over IIOP) denotes the Java RMI interface over the CORBA system.The EJB specification requires that Enterprise Java Beans (EJB) containers support accessing the EJBs using RMI-IIOP. EJBs may be accessed from any CORBA application or provide Web Services.

Distributed Component Object Model (DCOM) is a Microsoft proprietary technology for software components distributed across several networked computers to communicate with each other. DCOM, which originally was called "Network OLE", extends Microsoft's Component Object Model (COM), and provides the communication substrate under Microsoft's COM+ application server infrastructure. It has been deprecated in favor of Microsoft .NET. The addition of the "D" to COM was due to extensive use of DCE/RPC - more specifically Microsoft's enhanced version, known as MSRPC.

In terms of the extensions it added to COM, DCOM had to solve the problems of :

  • Marshalling - serializing and deserializing the arguments and return values of method calls "over the wire".

  • Distributed garbage collection - ensuring that references held by clients of interfaces are released when, for example, the client process crashed, or the network connection was lost.

One of the key factors in solving these problems is the use of DCE/RPC as the underlying RPC mechanism behind DCOM. DCE/RPC has strictly defined rules regarding marshalling and who is responsible for freeing memory.DCE/RPC stands for "Distributed Computing Environment / Remote Procedure Calls". DCE/RPC is a Remote Procedure Call system that allows software to work across multiple computers, as if it was all working on the same computer. This system allows programmers to write distributed software without having to worry about the underlying network code. DCE/RPC is frequently noted for its complexity, however this complexity is often a result of features that target large distributed systems and which are often unmatched by more modern RPC implementations such as SOAP.

DCOM was a major competitor to CORBA. Proponents of both of these technologies saw them as one day becoming the model for code and service-reuse over the Internet. However, the difficulties involved in getting either of these technologies to work over Internet firewalls, and on unknown and insecure machines, meant that normal HTTP requests in combination with web services won out over both of them.

DCOM is the platform of the Ole for Process Control (OPC) protocol which has been adopted as a de facto standard in the automation industry.