Close
0%
0%

micro HTTP server in C

Connect your browser to your smart devices, using a minimalist HTTP compliant server written in POSIX/C

Similar projects worth following
This is a compact, single-threaded, embedded server that supports blocking and polled modes, IPv4 and IPv6, CORS, MIME types, and a few other features that could boost your "IoT" project.
You can use it to serve static files, receive basic commands, or both.

Licence: AGPLv3

Some of the goals of this project are:
* support HTTP 1.1 (20170331: persistent connections are finally working)
* implement most of the HTTaP features (such as loopback)
* remain as compact as possible (so keep the feature set minimal)
* serve some files (20170401: done)
* split the header and the actual response data (ok)
* implement proper timeouts (20170331: done)

If you need a multithreaded web server to host data for everyone on the web, look at Apache (with CGI) or https://www.gnu.org/software/libmicrohttpd/

However they are limited by the CGI's basic inability to preserve a context through several HTTP requests and implement efficient sessions : each request must analyse the headers again, check the cookies again, even though the connection with the server was not closed ! Actually you can't even know the order of requests with a multithreaded server, and race conditions are expected if you do something more than serve static files.

This project solves the problem of implicit sessions with a slower but inherently safer approach : a single-threaded server ensures that the request are received and replied in order. No risk of race conditions from that angle, no need of funky programming techniques to avoid them. A session is congruent with an open TCP/IP connection, which also helps with safey (despite dubious security but it's not the purpose).


The server's code can contain two modules:

  1. File server
  2. HTTaP manager

They can be individually disabled but they are usually integrated in the same server because HTTP/JS makes it much more difficult to connect to resources located on a different IP address or IP port.

The client first points the browser to IPaddress:IPport which provides the gateway to the application, with its code, images, text... It looks like a normal web server.

The code is downloaded by the client then JavaScript is executed. The server can focus on real-time, low-level operations while the client performs the high-level application logic, user interaction, computations and display.

This workflow is ideal when you want to control, configure and interact with embedded devices, over wire or radio link if you need. Your client can be any brand or model as long as it abides to simple web standards. You can code once in JS and run everywhere !


Here I resurrect the source code of the HTTP server that was published in "Comment contrôler les GPIO du Raspberry Pi par HTTP en C" (OpenSilicium n°6, march-may 2013)

Ideally this project supports the design of the #HTTaP protocol (as explained in "GLMF n°173 "HTTaP un protocole de contrôle basé sur HTTP")

The original source code was hosted there : http://ygdes.com/sources/

The code might use the GPIO library that I maintain in another project : #C GPIO library for Raspberry Pi (it was created and forked from the same article in OpenSilicium n°6)

TCP/IP socket setup : covered in "Un mini serveur HTTP pour dialoguer avec des applications interactives : les sockets réseau" (GNU/Linux Magazine n° 141 , sept 2011) (where I explain how it supports both IPv4 and IPv6)

Environment variables and user/group/chmod management were explained in "L'environnement POSIX du mini serveur embarqué en C" (GNU/Linux Magazine n° 177 , dec. 2014)

At this moment (20170225) the code is undergoing rework to serve files and better support HTTP1.1, including persistent connections.


Logs:
1. Timeout
2. MIME type handling
3. Licensing
4. New version
5. Security and sandboxing
6. Overview of the code
7. Timeout and persistence
8. When to enable CORS


Since this is a software project, it's hard to create a diagram/illustration but the diagram showing #HTTaP works well because it shows how this server is intended to be used. It's the "HTTaP server" boxes:

The server is not a competitor to Apache and others, but a piece of code that is embedded in other programs to make them web-enabled and work with real time constraints. This is another reason why it is single-threaded : only one user is expected at a given time.

You can adapt the server to recognise SCPI commands if you are not a HTTaP fan.

Here is an example of use : in 2017/03, the server runs on the Pi to drive the "Remote Controlled Car" extension board, for the workshop at...

Read more »

HTTaP_src_20170401.tgz

Timeout, connection persistence, file serving, etc. are implemented, with crude MIME support through links

x-compressed-tar - 7.75 kB - 04/01/2017 at 08:09

Download

HTTaP_src.20170327.tgz

New revision : english comments and variable names, split files, example file... Timeout still not implemented.

x-compressed-tar - 4.61 kB - 03/27/2017 at 09:53

Download

test_XHR.html

tests serv_11_CORS.c (change the URLs in the source code to point to your own server)

HyperText Markup Language (HTML) - 1.58 kB - 02/26/2017 at 07:05

Download

serv_11_CORS.c

Includes CORS-compliant headers

text/x-csrc - 11.29 kB - 02/26/2017 at 07:05

Download

serv_10_timer.c

incomplete implementation of the timeout.

x-csrc - 11.09 kB - 02/26/2017 at 05:08

Download

serv_9_blocant.c

can change on-the-fly between polling and blocking mode

x-csrc - 11.03 kB - 02/25/2017 at 23:08

Download

serv_8_change-user.c

Change the running user and drop access rights (safety if you need to serve files)

x-csrc - 10.68 kB - 02/25/2017 at 23:08

Download

x-csrc - 8.31 kB - 02/25/2017 at 23:08

Download

serv_6_1ou0.c

a dumb server that understands the addresses 1 and 0

x-csrc - 7.36 kB - 02/25/2017 at 23:08

Download

serv_6_dualmode.c

can be compiled for either polled or blocking mode

x-csrc - 7.15 kB - 02/25/2017 at 23:05

Download

View all 15 files

  • Files are served

    Yann Guidon / YGDES04/01/2017 at 08:25 0 comments

    It's April 1st and it's not a joke !

    I have a seemingly working version of the server, with the crude MIME support outlined in MIME type handling

    As expected, a critical prerequisite of the code was to get the timeout and the connection persistence working well. Then a thorough parsing of request, and countless checks of boundaries and return values...

    The MIME type handling requires manual intervention but I will certainly write a script to automate the creation of the links.

    I have uploaded the latest archives at HTTaP_src_20170401.tgz

  • When to enable CORS

    Yann Guidon / YGDES03/31/2017 at 22:34 0 comments

    You will find these lines in the beginning of HTTaP.h :

    #ifndef  ACCESS_CONTROL_ALLOW_ORIGIN
      #ifndef HTTAP_SERVE_FILES
        #define ACCESS_CONTROL_ALLOW_ORIGIN "Access-Control-Allow-Origin: *\x0d\x0a"
      #else
        #define ACCESS_CONTROL_ALLOW_ORIGIN
      #endif
    #endif

    This code covers two usual situations :

    • #define HTTAP_SERVE_FILE :
      The server includes the static file server, the browser can fetch files and send HTTaP commands to the same IP address and port, everything is great. No need to declare CORS.
    • #undef HTTAP_SERVE_FILE ;
      The server doesn't implement the file server : the browser gets the HTML pages from a different IP/port pair and the "Same Origins Policy" enforced by web clients will block HTTaP traffic. You must enable CORS so the HTML/JS files that are served by a different server (maybe Apache, on a different port and IP address) can use this server.

    There are other corner cases and you are free to combine the parameters, but you must examine the risks and benefits of each configuration. If you want to override the default behaviour, just define ACCESS_CONTROL_ALLOW_ORIGIN yourself on the compiler's command line for example.

  • Timeout and persistence

    Yann Guidon / YGDES03/31/2017 at 08:54 0 comments

    I have finally implemented the connection's persistence along with the timeout mechanism !

    I'm also including the basic features of the HTTaP protocol, which I test using telnet:

    The first request is the typical "GET /?" command, which returns the JSON-formatted list of features and informations. The format of this list will evolve quickly.

    Any invalid or malformed request closes the connection, which happened when I hit "Enter" (resulting in nothing written on my side).

    I still have to implement a lot of things !

  • Overview of the code

    Yann Guidon / YGDES03/28/2017 at 03:05 2 comments

    I have chosen to avoid using a run-time configuration file. Most parameters are defined in a .h file and some can be overridden through the environment variables. Here is the list of the parameters:

    • $HTTAP_TCPPORT : sets the listening port of the server. Similar to Apache's Listen in ports.conf.
    • $HTTAP_KEEPALIVE : number of seconds for the HTTP timeout (bound between 3 and 200, default is 15). This is equivalent to KeepAliveTimeout in Apache's configuration file.

    If you enable the "file server" (by compiling with the -DHTTAP_SERVE_FILES option), more parameters appear:

    • $HTTAP_STATICPATH : when set, sets the working directory where files are stored. If running as root, chroot() to it. Similar to Apache's DocumentRoot
    • $HTTAP_ROOTPAGE : specifies the file that is handed when the resource path "/" is requested.
    • $HTTAP_USER : if running as root, drop admin rights and change to this user (hopefully the system is configured to give this user minimal rights) Similar to APACHE_RUN_USER
    • $HTTAP_GROUP : if $HTTAP_USER is set, also change the current group ID (similar to APACHE_RUN_GROUP)

    Environment variables are both simpler than a configuration file and does not interfere with command line parameters, which are often critical to the main/host application. Furthermore, it's a bit more script-friendly and allows automation. This is excellent when you run several programs simultaneously, just increase some variables in a loop and you're done.

    You could configure the server (before running it) with the following script:

    # change the default TCP port and keepalive values
    export HTTAP_TCPPORT=61234
    export HTTAP_KEEPALIVE=123
    # use the current directory to serve the files
    unset HTTAP_STATICPATH
    # use the default root page
    unset HTTAP_ROOTPAGE
    # create an empty root page
    touch index.html
    # don't forget to run useradd and groupadd
    export HTTAP_USER=httap_sandbox
    export HTTAP_GROUP=httap_sandbox
    # start the server's program
    ./HTTaP
    

    This will (one day) serve an empty page for the root in the current directory.


    The call graph is :

    MyApplication.c       // contains your main() and infinite loop
       |__ HTTaP.h        // user configurable values
             |__ HTTaP.c  // the server's boring posixy code
                   |___ your code to process dynamic addresses 
    You'll want to look at HTTaP.h which contains the user-modifiable default parameters, though most of them are preferably updated by the above environment variables.

    It also contains compile-time flags that enable certain features:

    • HTTAP_VERBOSE : printf() some useful information (good for debugging)
    • SHOW_CLIENT : add more verbosity
    • ENABLE_LOOPBACK : defined if a Loopback server is implemented.
    • HTTAP_SERVE_FILES : this is the most important switch. This enables the static files server.

    Inside your code, you can switch from polled mode to blocking mode (and vice versa) with the following functions:

    void HTTaP_polled()
    void HTTaP_blocking()

    The server starts in polled mode, which requires repetitive calls to the HTTaP_server() function. If your program has finished working and waits for more commands, you can save CPU cycles (and battery ?) by switching to the blocking mode. You can revert to polled when a valid command is received.

  • Security and sandboxing

    Yann Guidon / YGDES03/28/2017 at 02:56 0 comments

    The server is designed to run almost like a simplified file server in a POSIX machine. This implies a significant attack surface and a lot of potential for abuse.

    (D)DOS attacks are pretty easy to create : the server is single-threaded and uses persistent connections but no authentication. This is not a security hazard, at least in a lab environment where there should be only one user.

    There are many kinds of known, unknown and potential risks that require careful coding and safe development practices. The attack surface is reduced by limiting the system to its core functionality and keeping things as simple as possible (KISS).

    But what about the unknown bugs ?

    One solution is to use sandboxing techniques and implement inherent UNIX protection mechanisms. This way, if the system ever goes bad, the potential damage is contained to a portion of the system. Two methods protect this server:

    • If the server is started with root privileges, and if the necessary informations are provided, the current user and group ID are changed to a non-privileged user, with limited rights.
    • If the server is started with root privileges, the program is chroot()ed to the current working directory (or another one, if provided) so it can't access the rest of the system.

    These are "last lines of defence" methods, which are complemented by many routine checks :

    • The server (if correctly configured) will not serve files or directories that it doesn't own, which prevent accidental exfiltration of data.
    • URIs must be filtered, rejecting any path containing two consecutive dots.
    • No open directories. This reduce coding efforts and bugs, as well as removes a potential exfiltration method.

    Of course, a user can always configure the server badly...


    Many security considerations are covered in « The Tangled Web - AGuide to Securing Modern Web Applications » by Zalewski, Michał

  • New version

    Yann Guidon / YGDES03/27/2017 at 09:59 0 comments

    The server has been reworked and uploaded there :

    https://cdn.hackaday.io/files/20042857476928/HTTaP_src.20170327.tgz

    I have revised many things, cleaned up the variable names, split the large file in several sub-file, provided a minimal working example...

    Timeouts and persistent connections are not yet implemented but this should appear soon.

  • Licensing

    Yann Guidon / YGDES03/26/2017 at 03:37 0 comments

    The project's source code started as examples in a magazine. They were pretty "public domain" and practical applications of man pages so I didn't bother with licensing.

    However the project has grown beyond mere examples and is a full system and requires protection under Copyright laws. So I have chosen to use the same license as most of my projects :



  • MIME type handling

    Yann Guidon / YGDES03/21/2017 at 23:37 0 comments

    There's a "little detail" that I have to examine right now.

    For now, the response headers are hardwired so I can specify the MIME type inside a single, long characters string. Because there are only 2 or 3 responses.

    When the server will process files from the local filesystem, the response headers must also contain the MIME type, which is unknown. Any type file could be served. And I don't want to be bothered with this.

    Usually, classic servers like Apache manage a (long) list of file types, recognised by the filename's extension. This means : for each request, isolate the extension (don't get caught in traps such as PHP files with options) then look up in the (configurable) list. How inconvenient.

    For the sake of simplicity, I don't want to handle the type myself. I want the developer/maintainer to explicitly do it, which saves some efforts from my side. The metadata must then reside on the filesystem...


    My first idea was to add one small metadata file for each data file, using the same filename but with the added ".type" suffix.

    That's ugly. Apple's Mac do that (with the ._ prefix) and it's horrible, AND the maintainer must create these, one file for each chunk of data...

    These tiny MIME type files might not have to reside in the same directory as their data. The data can go in a /data directory and the corresponding types in the /type directory... then a script would populate /type once for all.

    The type files can be simple symbolic links to a few actual files, or even contain a broken symbolic link which is the actual metadata/type. But that's still one additional file for each contents file.


    The number of files can be reduced if the filename contains the metadata as well. However this greatly complexifies the procedure from the server's point of view. All I want to do is call stat() to see if the file already exists, then read() it. If I don't know the file's name end in advance, I have to manually scan the whole filesystem for a filename starting with the given string. Meh.

    On top of that, it's not really practical when the files are deployed because their names must be changed.


    Where can we put the metadata then ? Let's look at what stat() tells us :

    struct stat {
        dev_t     st_dev;     /* ID of device containing file */
        ino_t     st_ino;     /* inode number */
        mode_t    st_mode;    /* protection */
        nlink_t   st_nlink;   /* number of hard links */
        uid_t     st_uid;     /* user ID of owner */
        gid_t     st_gid;     /* group ID of owner */
        dev_t     st_rdev;    /* device ID (if special file) */
        off_t     st_size;    /* total size, in bytes */
        blksize_t st_blksize; /* blocksize for file system I/O */
        blkcnt_t  st_blocks;  /* number of 512B blocks allocated */
        time_t    st_atime;   /* time of last access */
        time_t    st_mtime;   /* time of last modification */
        time_t    st_ctime;   /* time of last status change */
    };

    One dirty, dirty way would be to put it in the group ID.

    OOooohhh that's naughty and the setup would be sordid. Additionally, the user/group names can't contain the dash and slash characters !


    Yet another approach uses symbolic links.

    All the site's files would be links to actual files, located each in one directory per MIME type. The server just follows the symlink, gets the MIME type from the directory name, then reads/serves the file. This looks pretty easy for the server but administration (and site development) would be terrible, as this breaks the directory structures...


    ________________________________________________________________________________
    The constrains are increasing:
    • The directory structure can't be changed
    • The filenames can't be changed
    • The file contents can't be changed
    • The system configuration can't be changed (can't add users or groups)
    • The directory structure can't be littered by additional files

    Ideally, no change must be required to make a directory ready to serve.

    The metadata must be looked up once (during site installation), probably...

    Read more »

  • Timeout

    Yann Guidon / YGDES03/19/2017 at 00:46 0 comments

    The server works rather well but there are flaws. In particular, there is no timeout, a client can block the server simply by opening a TCP connection...

    I had started the implementation of a timeout system but it is far from operational, as I tried to solve inter-programs communication issues...


    The server function is called periodically, at an unknown frequency. The caller/main function must call the server with one parameter that is set, every second, so the server can update its internal counters.

View all 9 project logs

  • 1

    Create your user program. Include HTTaP.h at the beginning : since it contains many inclusions, you don't need to include them as well.

    Like an Arduino sketch, it must contain an infinite loop. This loop must run at least once per second. Inside the loop, call the server's routine.

    Here is an example :

    #include "HTTaP.h"
    
    int main(int argc, char *argv[]) {
      const char spinner[4]="-\\|/";
      int phase=0;
      while (1) {
        // do something, dance, sing, whatever,
        // but set timeout to 1 once in a while.
        printf("%c%c", spinner[(phase++)&3],0xd);
        fflush(NULL);
        // run the server:
        HTTaP_server(timeout);
      }
    }
    

    Note : the server requires to be called with an integer number, that is set to 1 every second, to increment its internal counter.

  • 2

    The server is a FSM (Finite States Machine) that contains all the necessary initialisation code. You don't even have to call a "init()" function, the FSM takes care of it.

    However if you use the HTTaP protocol, you must modify the "request method" parsing routine and add your own "words", "commands" and their effects.

    The code in question is there :

        case ETAT2_attente_donnee_entrante :
          /* receive the request */
          .....
          /* analyse the request */
          if ((recv_len >= 7) && (strncmp("GET /1 ", buffer, 7) == 0)) {
             // do something here because we received the URI "/1"
          }
    
    You should be careful with the reply buffer (the pointer b), don't forget to update it to prevent from sending garbage.
  • 3

    Update all the #defines you can. For example, which features are allowed (is it blocking, polled or both? is file-serving enabled ?)

    There are several predefined values in HTTaP.h

View all 4 instructions

Enjoy this project?

Share

Discussions

Tor wrote 04/19/2017 at 06:30 point

Out of curiosity, what is the main difference between the micro HTTP server and say, Mongoose? (https://github.com/cesanta/mongoose)

  Are you sure? yes | no

rockerajay wrote 03/30/2017 at 07:20 point

Whats the difference between your previous project and this one?

https://hackaday.io/project/20170-httap
Just asking..

  Are you sure? yes | no

Yann Guidon / YGDES wrote 03/31/2017 at 06:17 point

Hello,

This project is the implementation of the protocol, so I here discuss coding considerations.

The HTTaP page discusses definitions, interoperability, protocols... so it doesn't provide source code.

I believe it's best to separate definition and implementation, just like a computer is split into an "architecture" (say, x86_32) and its implementation (the i486 for example).

  Are you sure? yes | no

Stuart Longland wrote 02/25/2017 at 22:03 point

What environment is this http server intended for?  Embedded MCU?  General purpose Unix/Linux?

  Are you sure? yes | no

Yann Guidon / YGDES wrote 02/25/2017 at 22:10 point

It's a POSIX-compliant C code so Linux and such. Its supports IPv6 and v4.

I've made a version that controls a LED on a Raspberry Pi.

  Are you sure? yes | no

Stuart Longland wrote 02/25/2017 at 22:14 point

Nice.  Been looking around at a nice small asynchronous HTTP server for an API… all the cool kids are doing Python/NodeJS/Go/etc web services, but I still have a soft spot for things done in plain C. :-)

  Are you sure? yes | no

Yann Guidon / YGDES wrote 02/25/2017 at 22:15 point

then keep watching this project, as I'm trying to solve the CORS issue these days :-)

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates