Close
0%
0%

micro HTTP server in C

Communicate with a computer from your browser, using a minimalist HTTP compliant server written in POSIX/C

Similar projects worth following
This is a compact, single-threaded, embedded server that supports blocking and polled modes, IPv4 and IPv6, CORS, and a few other features that could boost your "IoT" project.
You can use it to serve static files, receive basic commands, or both.

Licence: AGPLv3

Some of the goals of this project are:
* support HTTP 1.1 (persistent connections are not working yet)
* implement most of the HTTaP features
* remain as compact as possible
* serve some files (not functional yet)
* split the header and the actual response data
* implement proper timeouts

If you need a multithreaded web server, look at Apache (with CGI) or https://www.gnu.org/software/libmicrohttpd/


Here I resurrect the source code of the HTTP server that was published in "Comment contrôler les GPIO du Raspberry Pi par HTTP en C" (OpenSilicium n°6, march-may 2013)

Ideally this project supports the design of the #HTTaP protocol (as explained in "GLMF n°173 "HTTaP un protocole de contrôle basé sur HTTP")

The original source code was hosted there : http://ygdes.com/sources/

The code might use the GPIO library that I maintain in another project : #C GPIO library for Raspberry Pi (it was created and forked from the same article in OpenSilicium n°6)

TCP/IP socket setup : covered in "Un mini serveur HTTP pour dialoguer avec des applications interactives : les sockets réseau" (GNU/Linux Magazine n° 141 , sept 2011) (where I explain how it supports both IPv4 and IPv6)

Environment variables and user/group/chmod management were explained in "L'environnement POSIX du mini serveur embarqué en C" (GNU/Linux Magazine n° 177 , dec. 2014)

At this moment (20170225) the code is undergoing rework to serve files and better support HTTP1.1, including persistent connections.


Logs:
1. Timeout
2. MIME type handling
3. Licensing
4. New version
5. Security and sandboxing
6. Overview of the code

.


Since this is a software project, it's hard to create a diagram/illustration but the diagram showing #HTTaP works well because it shows how this server is intended to be used. It's the "HTTaP server" boxes:

The server is not a competitor to Apache and others, but a piece of code that is embedded in other programs to make them web-enabled. This explains why it is single-threaded : only one user is expected at a given time.

Here is an example of use : in 2017/03, the server runs on the Pi to drive the "Remote Controlled Car" extension board, for the workshop at IESA. You wouldn't want somebody else to "drive your car" at the same time as you, right ?


.

HTTaP_src.20170327.tgz

New revision : english comments and variable names, split files, example file... Timeout still not implemented.

x-compressed-tar - 4.61 kB - 03/27/2017 at 09:53

Download

test_XHR.html

tests serv_11_CORS.c (change the URLs in the source code to point to your own server)

HyperText Markup Language (HTML) - 1.58 kB - 02/26/2017 at 07:05

Download

serv_11_CORS.c

Includes CORS-compliant headers

text/x-csrc - 11.29 kB - 02/26/2017 at 07:05

Download

serv_10_timer.c

incomplete implementation of the timeout.

x-csrc - 11.09 kB - 02/26/2017 at 05:08

Download

serv_9_blocant.c

can change on-the-fly between polling and blocking mode

x-csrc - 11.03 kB - 02/25/2017 at 23:08

Download

serv_8_change-user.c

Change the running user and drop access rights (safety if you need to serve files)

x-csrc - 10.68 kB - 02/25/2017 at 23:08

Download

x-csrc - 8.31 kB - 02/25/2017 at 23:08

Download

serv_6_1ou0.c

a dumb server that understands the addresses 1 and 0

x-csrc - 7.36 kB - 02/25/2017 at 23:08

Download

serv_6_dualmode.c

can be compiled for either polled or blocking mode

x-csrc - 7.15 kB - 02/25/2017 at 23:05

Download

x-csrc - 7.00 kB - 02/25/2017 at 22:23

Download

View all 14 files

  • Overview of the code

    Yann Guidon / YGDES15 hours ago 0 comments

    I have chosen to avoid using a run-time configuration file. Most parameters are defined in a .h file and some can be overridden through the environment variables. Here is the list of the parameters:

    • $HTTAP_TCPPORT : sets the listening port of the server.
    • $HTTAP_KEEPALIVE : number of seconds for the HTTP timeout (bound between 3 and 200, default is 15).

    If you enable the "file server" (by compiling with the -DHTTAP_SERVE_FILES option), more parameters appear:

    • $HTTAP_STATICPATH : when set, sets the working directory where files are stored. If running as root, chroot() to it.
    • $HTTAP_ROOTPAGE : specifies the file that is handed when the resource path "/" is requested.
    • $HTTAP_USER : if running as root, drop admin rights and change to this user (hopefully the system is configured to give this user minimal rights)
    • $HTTAP_GROUP : if $HTTAP_USER is set, also change the current group ID

    Environment variables are both simpler than a configuration file and does not interfere with command line parameters, which are often critical to the main/host application. Furthermore, it's a bit more script-friendly and allows automation. This is excellent when you run several programs simultaneously, just increase some variables in a loop and you're done.

    You could configure the server (before running it) with the following script:

    # change the default TCP port and keepalive values
    export HTTAP_TCPPORT=61234
    export HTTAP_KEEPALIVE=123
    # use the current directory to serve the files
    unset HTTAP_STATICPATH
    # use the default root page
    unset HTTAP_ROOTPAGE
    # create an empty root page
    touch index.html
    # don't forget to run useradd and groupadd
    export HTTAP_USER=httap_sandbox
    export HTTAP_GROUP=httap_sandbox
    # start the server's program
    ./HTTaP
    

    This will (one day) serve an empty page for the root in the current directory.

    Inside the code, you can switch from polled mode to blocking mode (and vice versa) with the following functions:

    void HTTaP_polled()
    void HTTaP_blocking()

    .

  • Security and sandboxing

    Yann Guidon / YGDES15 hours ago 0 comments

    The server is designed to run almost like a simplified file server in a POSIX machine. This implies a significant attack surface and a lot of potential for abuse.

    (D)DOS attacks are pretty easy to create : the server is single-threaded and uses persistent connections but no authentication. This is not a security hazard, at least in a lab environment where there should be only one user.

    There are many kinds of known, unknown and potential risks that require careful coding and safe development practices. The attack surface is reduced by limiting the system to its core functionality and keeping things as simple as possible (KISS).

    But what about the unknown bugs ?

    One solution is to use sandboxing techniques and implement inherent UNIX protection mechanisms. This way, if the system ever goes bad, the potential damage is contained to a portion of the system. Two methods protect this server:

    • If the server is started with root privileges, and if the necessary informations are provided, the current user and group ID are changed to a non-privileged user, with limited rights.
    • If the server is started with root privileges, the program is chroot()ed to the current working directory (or another one, if provided) so it can't access the rest of the system.

    These are "last lines of defence" methods, which are complemented by many routine checks :

    • The server (if correctly configured) will not serve files or directories that it doesn't own, which prevent accidental exfiltration of data.
    • URIs must be filtered, rejecting any path containing two consecutive dots.

    Of course, a user can always configure the server badly...

  • New version

    Yann Guidon / YGDESa day ago 0 comments

    The server has been reworked and uploaded there :

    https://cdn.hackaday.io/files/20042857476928/HTTaP_src.20170327.tgz

    I have revised many things, cleaned up the variable names, split the large file in several sub-file, provided a minimal working example...

    Timeouts and persistent connections are not yet implemented but this should appear soon.

  • Licensing

    Yann Guidon / YGDES3 days ago 0 comments

    The project's source code started as examples in a magazine. They were pretty "public domain" and practical applications of man pages so I didn't bother with licensing.

    However the project has grown beyond mere examples and is a full system and requires protection under Copyright laws. So I have chosen to use the same license as most of my projects :



  • MIME type handling

    Yann Guidon / YGDES7 days ago 0 comments

    There's a "little detail" that I have to examine right now.

    For now, the response headers are hardwired so I can specify the MIME type inside a single, long characters string. Because there are only 2 or 3 responses.

    When the server will process files from the local filesystem, the response headers must also contain the MIME type, which is unknown. Any type file could be served. And I don't want to be bothered with this.

    Usually, classic servers like Apache manage a (long) list of file types, recognised by the filename's extension. This means : for each request, isolate the extension (don't get caught in traps such as PHP files with options) then look up in the (configurable) list. How inconvenient.

    For the sake of simplicity, I don't want to handle the type myself. I want the developer/maintainer to explicitly do it, which saves some efforts from my side. The metadata must then reside on the filesystem...


    My first idea was to add one small metadata file for each data file, using the same filename but with the added ".type" suffix.

    That's ugly. Apple's Mac do that (with the ._ prefix) and it's horrible, AND the maintainer must create these, one file for each chunk of data...

    These tiny MIME type files might not have to reside in the same directory as their data. The data can go in a /data directory and the corresponding types in the /type directory... then a script would populate /type once for all.

    The type files can be simple symbolic links to a few actual files, or even contain a broken symbolic link which is the actual metadata/type. But that's still one additional file for each contents file.


    The number of files can be reduced if the filename contains the metadata as well. However this greatly complexifies the procedure from the server's point of view. All I want to do is call stat() to see if the file already exists, then read() it. If I don't know the file's name end in advance, I have to manually scan the whole filesystem for a filename starting with the given string. Meh.

    On top of that, it's not really practical when the files are deployed because their names must be changed.


    Where can we put the metadata then ? Let's look at what stat() tells us :

    struct stat {
        dev_t     st_dev;     /* ID of device containing file */
        ino_t     st_ino;     /* inode number */
        mode_t    st_mode;    /* protection */
        nlink_t   st_nlink;   /* number of hard links */
        uid_t     st_uid;     /* user ID of owner */
        gid_t     st_gid;     /* group ID of owner */
        dev_t     st_rdev;    /* device ID (if special file) */
        off_t     st_size;    /* total size, in bytes */
        blksize_t st_blksize; /* blocksize for file system I/O */
        blkcnt_t  st_blocks;  /* number of 512B blocks allocated */
        time_t    st_atime;   /* time of last access */
        time_t    st_mtime;   /* time of last modification */
        time_t    st_ctime;   /* time of last status change */
    };

    One dirty, dirty way would be to put it in the group ID.

    OOooohhh that's naughty and the setup would be sordid. Additionally, the user/group names can't contain the dash and slash characters !


    Yet another approach uses symbolic links.

    All the site's files would be links to actual files, located each in one directory per MIME type. The server just follows the symlink, gets the MIME type from the directory name, then reads/serves the file. This looks pretty easy for the server but administration (and site development) would be terrible, as this breaks the directory structures...


    ________________________________________________________________________________
    The constrains are increasing:
    • The directory structure can't be changed
    • The filenames can't be changed
    • The file contents can't be changed
    • The system configuration can't be changed (can't add users or groups)
    • The directory structure can't be littered by additional files

    Ideally, no change must be required to make a directory ready to serve.

    The metadata must be looked up once (during site installation), probably...

    Read more »

  • Timeout

    Yann Guidon / YGDES03/19/2017 at 00:46 0 comments

    The server works rather well but there are flaws. In particular, there is no timeout, a client can block the server simply by opening a TCP connection...

    I had started the implementation of a timeout system but it is far from operational, as I tried to solve inter-programs communication issues...


    The server function is called periodically, at an unknown frequency. The caller/main function must call the server with one parameter that is set, every second, so the server can update its internal counters.

View all 6 project logs

  • 1

    Create your user program. Like an Arduino sketch, it must contain an infinite loop. This loop must run at least once per second. Inside the loop, call the server's routine.

    Here is an example :

    int main(int argc, char *argv[]) {
      const char spinner[4]="-\\|/";
      int phase=0;
      while (1) {
        // do something, dance, sing, whatever
        printf("%c%c", spinner[(phase++)&3],0xd);
        fflush(NULL);
        // run the server:
        serveur();
      }
    }
    

  • 2

    The server is a FSM (Finite States Machine) that contains all the necessary initialisation code. You don't even have to call a "init()" function, the FSM takes care of it.

    However if you use the HTTaP protocol, you must modify the "request method" parsing routine and add your own "words", "commands" and their effects.

    The code in question is there :

        case ETAT2_attente_donnee_entrante :
          /* receive the request */
          .....
          /* analyse de la requête */
          if ((recv_len >= 7) && (strncmp("GET /1 ", buffer, 7) == 0)) {
             // do something here because we received the URI "/1"
          }
    
    You should be careful with the reply buffer (the pointer b), don't forget to update it to prevent from sending garbage.
  • 3

    Update all the #defines you can. For example, which features are allowed (is it blocking, polled or both? is file-serving enabled ?)

    There are several predefined values and I have to move them to a separate file...

View all 4 instructions

Enjoy this project?

Share      

Discussions

Stuart Longland wrote 02/25/2017 at 22:03 point

What environment is this http server intended for?  Embedded MCU?  General purpose Unix/Linux?

  Are you sure? yes | no

Yann Guidon / YGDES wrote 02/25/2017 at 22:10 point

It's a POSIX-compliant C code so Linux and such. Its supports IPv6 and v4.

I've made a version that controls a LED on a Raspberry Pi.

  Are you sure? yes | no

Stuart Longland wrote 02/25/2017 at 22:14 point

Nice.  Been looking around at a nice small asynchronous HTTP server for an API… all the cool kids are doing Python/NodeJS/Go/etc web services, but I still have a soft spot for things done in plain C. :-)

  Are you sure? yes | no

Yann Guidon / YGDES wrote 02/25/2017 at 22:15 point

then keep watching this project, as I'm trying to solve the CORS issue these days :-)

  Are you sure? yes | no

Does this project spark your interest?

Become a member to follow this project and never miss any updates