From nmon for Linux

Site: Nweb

nweb Tiny Webserver Example


nweb Tiny Safe Webserver

nweb is a 200 lines of C code example of how a webserver actually works at its core.
It is a static page only server (no fancy live content) and only specific files types.
It actually works too and a few 100's people use it on the Internet for for their own projects.

Details of running nweb are at the bottom of this page.

Note: This code is a simple code sample and so NOT under any license or Copyrighted. Use it any way you like.

Downloads!

Download FileComments
nweb25.cCurrent version 25 - 24th July 2016
Added SIGCHLD support for operating systems that don't support the System V signal SIGCLD
nweb24.cVersion 24 - 23rd July 2016
Added chroot() for security
nweb24_extra.zipCurrent version 24 client.c and a miniature example of a
website for testing purposes only And the README for changes and versions
- Read the README online nmon24_README
nweb24_Ubuntu_RaspberryPi3_ARMVersion 24 compiled for Raspberry Pi 3 ARM running
Ubuntu 16.04
nweb_AIX6_POWERVersion 24 compiled for AIX 6 (or above) on POWER

nweb - A Tiny Safe Web Server with source code!

Have you ever:

  1. Wondered how a web server actually works via a simple example?
  2. Wanted to run a tiny safe web server (but were worried about using a fully blown web server may cause security issues)?
  3. Wondered how to write a program that accepts incoming messages via a network socket?
  4. Just wanted your own web server to experiment, extend and learn with.

Well, look no further = nweb is what you need. This is a simple web server to understand (only 200 lines of C source code), so you work out how it functions to server out files for a web browser and make changes. Add to this the fact it runs just as a regular user (if you use a high port number), it can not run any server side scripts or programs, so it canít open up any special privileges or security holes. If you want it to run with the standard web server port of 80 you typically need to start it as the superuser root.

nweb only transmits to the browser the following types of files

If your favourite static file type is not in this list, you can simply add it in the source code and recompile to allow it - a few seconds work

The file supplied includes the UNIX source code in C and a precompiled AIX and Linux on Intel versions. The source will compile with the IBM VisualAge C compiler or the GNU C compiler and should run unchanged on AIX, Linux or any other UNIX version for which you care to compile it. On UNIX (assuming the compiler is in your path) just type:

Note: very new compiler versions in 2016 spew out eight "Warnings" about not checking the return value of the write() system call even though the code explicitly tells the compile we know we want to do that. You can safely, ignore those warnings.

Run as follows as the root user:

Summary of the functions within nweb!

There are only four functions in the source code and these are explained below:

stop nweb completely. nweb breaks from the terminal session so it can't output onto your screen any progress of good operations, nor any errors so these are saved in the nweb.log file. In some cases the failure can be reported to the user's web browser as a fake web page describing the issue for example if the browser can't find or access the requested file. In other cases like a fatal problem, the details written to the log and nweb stops - so if nweb seems to disappear - check the log. It is called from the main function and also from the web function if an error canít be reported back to the web server user.

This function is called in a child process, one for each web request and so allows the main web server process to continue waiting for more connections. Checks are made to ensure the request is safe and can be completed, it then transmits the requested static file to the browser/user and exits.

After checking the command parameters, it creates a socket for incoming browser requests and then sits in a loop accepting requests and starting child processes to handle them. It should never end.

Pseudo code of the program!

Below is the pseudo code for the ~200 lines of source code. It should help you to understand the flow of the program.

web()  - this function returns the requested data back to the browser
{
	read from the socket the request
	check itís a simple GET command
	check no parent directory requested to escape the web servers home directory
	if no file name given assume index.html
	check the file extension is valid and supported
	check the file is readable by opening it
	transmit the HTTP header to the browser
	transmit the file contents to the browser
	stop
}

main()
{
	check the arguments supplied are valid
	check the directory supplied is sensible and not a security risk
	create a socket, bind it to a port number and start listening for requests
	become a daemon process so it continues if the user logs out
	ignore child programs (to avoid zombies on their death)
	forever {
		    accept incoming socket connection
		    fork a child process
		    if the child process
		    then call the web function
		    else close socket 
		}
}

System Calls!

As you may not have come across some of these system calls before, these are explained in more detail below and in particular how they all fit together. Although, you can always look them up in the manual or on the web. It can hard from just the code or the manual to see what they do and how they together make up a web server. Following a worked example is the best way to learn.

In UNIX and Linux most I/O is performed by the read and write system calls. Instead of reading and writing a file or a pipe, in this case, of process to process communication, we read and write a socket between the processes.

Telephone Analogue!

socket()Order your line and telephone!
bind()Have a telephone number assigned
listen()Tell your telecomms provider you are ready for calls
accept()Wait for a call and answer the phone
fork() a sub-process to
return the requested file
Hand call to someone else to deal with

The socket(), bind(), listen() and accept() network system calls, all work together to create a web server process. Combined they set up a socket ready for use to communicate over a network. A socket is:

The socket() function creates the socket and returns the file descriptor which can be used with any function that uses file descriptors like read, write, close. The arguments to socket() tell the operating system what type of socket and communication you need. There are dozens of permutations and combinations available. The arguments used in the nweb program are very typical for a regular general purpose socket using IP and other options are rare in the writerís experience.

The bind() function attaches a particular port number to the socket. When a remote client process is trying to contact your server is will use the IP address (often found by using a DNS service to convert a hostname to an IP address) and the port number. The port number tells the operating system inetd process, which service you want on the server. The port details on most UNIX machines are listed in the /etc/services files. Included in here will be standard port numbers for services like FTP, telnet and web servers (usually port 80 or 8080). You should check the port numbers in this /etc/services file to make sure you donít try one which is already in use. Although, if you try you should get an error message as it is normally impossible for two servers to use the same port number But in this case, the nweb program bind() function lets inetd know it wants to handle the particular port number you told nweb to use.

The listen() function call tells the operating system you are now ready to accept incoming requests. This is the final switch that makes the socket available to local and remote programs over the network.

The accept() function actually stops your nweb program until there is a socket connection request to this IP address and port number. Once the accept function call returns it means that the socket file descriptor it returns is live. If you read bytes from the socket you get characters written from the client end and if you write bytes to the socket they get transmitted to the client process. But it is not normal to simply read or write a single socket. Normally, you want to allow multiple clients to access your servers service and if this process does a read or write operation it could block until there are characters to be read or the written characters can be transmitted. in other words this server process should be running the accept function again to allow a new connection to start. The way to handle this is for the server process to start a child process that does the "talking" to the client and any work it needs done and for the main server process to close the socket and rerun the accept function and so await the next client connection. Note: When the child process is started it inherits the parents open sockets and so the child keeps the socket alive. Note: The alternative to forking sub-processes is using multi-threaded functions but that is more complicated so we avoided in this webserver example.

So it is normal in network servers for you to find code that looks like this (in pseudo code form):

	socket() - create the resources
	bind()   - to a port
	listen() - ready to handle data
	forever {
		accept() - the connection
		Start a child to handle the work
}

Other system call functions used!

process. Within the nweb.c code, the main server wants to ignore the death of a child signal. Without this the main server process would have to run the wait system call for each child process or they would be forever stuck in the "zombie" state waiting to the parent to call the wait() function. Eventually, there would be to many 1000's zombie processes and the users environment would hang as they could not create further processes.

These are the regular C library system call functions but they are used for the reading of the sockets and files sent to the client web browsers. To a UNIX or Linux process the socket and files behave teh same way and use a file descriptor to access them. Note: the socket is "opened" with the accept() function - the open() in the code is for opening the requested file.

Now lets look at the actual code !

The nweb.c code only shows you the server side of the socket. For the record here is the code that is required for the client side. Fortunately, the client side is much simpler are you only need to supply the IP address and the port number to make the connection to the server. In the below the servers IP address is 192.168.0.42 and the server port is 8181.

	int sockfd;
	static struct sockaddr_in serv_addr;

	if((sockfd = socket(AF_INET, SOCK_STREAM,0)) <0)
		pexit("Network socket system call");

	serv_addr.sin_family = AF_INET;
	serv_addr.sin_addr.s_addr = inet_addr("196.168.0.42");
	serv_addr.sin_port = htons(8181);

	if(connect(connectfd, (struct sockaddr *)&serv_addr, sizeof(serv_addr)) <0)
		pexit("Network connect system call");

	/* now the sockfd can be used to communicate to the server */

Warning: the bind() and connect() functions take a structure as the argument that must be zero filled before the setting particular options. In the nweb.c the structure is made a static C variable to ensure a it is zero filled when the program is started. If this structure was on the stack as a variable within the function or in the heap of the process it could be filled with old data and so its content not be zero filled. In which case you need to ensure it is zero filled using the bzero() function can be used to do this.

Over that socket between the client browser and the webserver we are going to use HTTP!

HTTP is short for Hypertext Transfer Protocol. This is used by your browser to "talk to the webserver to do two things:

  1. GET operations - make requests for files to be sent from the webserver and
  2. PUT operation - for sending the webserver data that is wants from the user of a browser application.

To understand a little about this HTTP we are going to look at the nweb log file nweb.log as it records the incoming message and what it sent (but not the contents of the file):

 INFO: request:GET /temperature.html HTTP/1.1
**Host:pi.aixncc.uk.ibm.com**Connection: keep-alive
**Cache-Control: max-age=0**Upgrade-Insecure-Requests: 1
**User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36
**Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
**Accept-Encoding: gzip, deflate, sdch
**Accept-Language: en-GB,en-US;q=0.8,en;q=0.6****

 INFO: SEND:temperature.html
 INFO: Header:HTTP/1.1 200 OK
Server: nweb/24.0
Content-Length: 389934
Connection: close
Content-Type: text/html

This is an extract of the nweb.log file while a single browser requests a single web page. We have the

  1. incoming GET, the file is wants and then loads of optional information that the browser thinks might be useful
  2. followed by the SENDing of the file including a header with the protocol version "1.1", the "200 OK" - which tells the browser the GET was handled OK and some other details we talk about below.

This is a very simple one file webpage but the browser will automatically also make a 2nd requests a "favicon.ico" file. This is the tiny picture that the browser places on the tab for this webpage. This means two GET file operations to the webserver.

Normally, even a simple webpage included dozens of picture files, artwork graphic files to make the page look pretty and gif images. Then there could be more text and images for adverts. Each of those will generate a further GET file operations to the webserver. The extra files that a webpage needs is actually encoded in the webpage we returned to the browswer. Then browser goes through that original webpage and look for the instructions to fetch an extra file. They look like this:

     <img src='http:/docs/nweb_diagram1.gif' /> 

This reads:

Note: nweb only responds to the simple GET file operations. The more complex POST operations is simply ignored as this is a static webserver and we are not handling more interactive parts of the HTTP protocol - these are used to send information to the webserver for server side application code to deal with. For example logging in or sending screen field contents to the application.

 INFO: request:GET /temperature.html HTTP/1.1 **Host:pi.aixncc.uk.ibm.com
  • Connection: keep-alive**Cache-Control: max-age=0**Upgrade-Insecure-Requests: 1**User-Agent:

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36**Accept:

 text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8**Accept-Encoding: gzip, deflate, sdch
  • Accept-Language: en-GB,en-US;q=0.8,en;q=0.6****
 INFO: SEND:temperature.html
 INFO: Header:HTTP/1.1 200 OK

Server: nweb/24.0 Content-Length: 389934 Connection: close Content-Type: text/html

Note:

the browser wants sent back to it. nweb have to check the file exists, is readable and sent it back

The response section:

This is the sending of the data (not including the contents of the file) but that has a HTTP header send first and this is the lines following %blueServer: nweb/24.0 Content-Length: 389934 Connection: close Content-Type: text/html

Notes:

space ready

Text is the just words and image is a clue that the data is going to have to be displayed as a picture.

Then there is the GET for the favicon.ico (I hope you can workout the various parts of the GET and response SENT

 INFO: request:GET /favicon.ico HTTP/1.1**Host: pi.aixncc.uk.ibm.com**Connection: keep-alive**Pragma: 
no-cache**Cache-Control: no-cache**User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 
(KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36**Accept: */***Referer: 
http://pi.aixncc.uk.ibm.com/temperature.html**Accept-Encoding: gzip, deflate, sdch**Accept-Language: 
en-GB,en-US;q=0.8,en;q=0.6****
 INFO: SEND:favicon.ico:23
 INFO: Header:HTTP/1.1 200 OK
Server: nweb/24.0
Content-Length: 1334
Connection: close
Content-Type: image/ico

nweb client - probably the worst but working browser in the world!

We have focused on the webserver side but what does a browser do to request a file from the webserver.

The code is in the nweb_extra.zip downloadable file but you can see it online here: client.c

In just 50 lines of C code we look at the basic system call it needs to make to connect to the socket of the webserver, sent the GET operation request and gt the file contents back.

To make this really simple we have hard coded most things. If you want to use a different port, website or GET a different file you will need to change these lines of code and recompile before running the client again.

#define PORT        8181		/* Port number as an integer - web server default is 80 */
#define IP_ADDRESS "192.168.0.8"	/* IP Address as a string */
char *command = "GET /index.html HTTP/1.0 \r\n\r\n" ; /* Note: spaces are delimiters and VERY important */

Ignoring the pexit() function which is used to report an error and exit.

In the main function we use the socket() function call to create a socket like we did in the webserver. But next we do something vert different to the webserver.

  1. We use the connect() function to request to join our client's socket to an existing socket on the webserver's vis the IP Address and the webserver's port number.
  2. This is the clever bit - the connect() reaches over the network to the webserver and to the port number and to the inetd process on the webserver.
  3. The inetd process recognises the event and passes it to the program that made an accept() function.
  4. This sets up the by-directional streams of the two inter-connected network sockets so the two programs can communicate.
  5. If the connect() is successful the client can make the GET request before which we send "Send-bytes=" and the size in bytes of the GET request string. the size allows the webserver to prepare a buffer and read the request in one read() - GET might include lots of cookie information but PUT requests might send lots of data but so be huge.
  6. Be careful in this request format.
  7. nweb is not too fussy in the format.
  8. A regular webserver is not forgiving and will expect the "HTTP", version, spaces, carriage return and line feed characters exactly as above or will assume the browser is duff and ignore ill formed requests. But they will not have problems if the browser does not send lots of the other "stuff" like operating system, browser name/version and cookies as they are optional.
  9. Once the GET request is send this client program just reads the socket until it closes and dumps the contents on the screen - don't ask for a binary file or it can muck up your terminal session.

It then exits.

Hands-on - setting and using nweb!

Assuming a machine with hostname and domain of nigel.acme.com and a user called debbie

Compiling It is just a single 200 line C program and assuming your C compiler is installed and called cc:

'''Setting up a directory for the tiny sample webpage supplied in nweb_extra.zip

$ mkdir /home/debbie/webpages
$ cd /home/debbie/webpages
$ cp /tmp/nweb_extra.zip .
$ unzip nweb_extra.zip
$ ls
index.html
nigel.jpg
favicon.ico
client.c

Make sure these files have read permission.

The example index.html file has this contents:

<HTML>
<TITLE>nweb
</TITLE>
<BODY BGCOLOR="lightblue">
<H1>nweb Test page</H1>
<IMG SRC="nigel.jpg">
<p>
Not pretty but it should prove that nweb works :-)
<p>
Feedback is welcome to Nigel Griffiths nag@uk.ibm.com
</table>
</BODY>
</HTML>

A more complex webpage index.html file

<HTML>
<HEAD>
nweb test
</HEAD>
<BODY>
This is a test page.
<H1>Header level 1</H1>
<H2>Header level 2</H2>
<UL>
<LI>This is an un-ordered list.
<LI>Which means bullet points.
</UL>
<OL>
<LI>This is an ordered list.
<LI>Which means each line is numbered.
</OL>
This is a sample gif image
<IMG SRC="nweb.gif">

<P> start a new paragraph

A line break<BR>forcing a new line

<A HREF="\otherfile.html">A further website page, if you click on the text here </a>


<B>The End in bold</B>
</BODY>
</HTML>

Running nweb

Assuming the user of the directory is call debbie. Note: high port numbers can be used by regular users - only root can use port numbers in a lower number - this number range is dependant on your operating system.

$ cd /home/debbie/webpages
# make the nweb binary executable
$ chmod ugo+x nweb

# make the files readable
$ chmod ugo+r *.html *.jpg *.ico

$ nweb 8181 /home/debbie/webpages
$ hostname
nigel.acme.com

Start your browser and access the website:

If you have root access then you can run nweb with the classic default port number of 80

If you have some other programming already using port 80 this will fail - check the nweb.log

$ nweb 80 /home/debbie/webpages

Start your browser and access the website - note the missing port number as 80 is the default:

- - - The End - - -

Retrieved from http://nmon.sourceforge.net/pmwiki.php?n=Site.Nweb
Page last modified on August 10, 2016, at 03:43 PM