Getting a Web Server running on my hobby OS on OSHub

by

After a long break from working on my hobby operating system, I finally got back into it and accomplished a very important milestone: a working web server.Full Github repo: https://github.com/joexbayer/RetrOS-32

retros web

Web browser accessing HTTP server on my OS

Networking has always been an integral part of my hobby projects. The first goal was to implement the basic networking stack: Ethernet, IP, ARP, UDP, TCP, DHCP, and DNS. Apart from TCP it was fairly straightforward, but things broke when moving to HTTP.

Screenshot%202023 04 06%20205404

For the first time, TCP is being put to work.


networking stack code

This gave me my first break from the project, but it also left me with a nagging thought of wanting to work on it. I finally sat down and started debugging.

After hours of dissecting my own code I finally found the culprit, the problem was a broken implementation of the terminal buffer, overwriting the lock in another process… funny. Additionally, the E1000 network driver did not handle incoming packets correctly, which I eventually worked around by handling bursts of packets.

I started seeing a lot of performance errors and hangs from TCP after returning an HTML page from the web engine, mainly because quickly refreshing the browser resulted in a spam of RST packets that were not handled correctly.

After a few hours of tinkering I finally got RST packets working and the network stack is now able to handle packet spam from the browser.

The next step was to actually implement an HTTP engine, which parsed user requests. Before this engine I would simply return a static HTTP response, no matter what the actual request was.

In keeping with the spirit of this hobby OS I wanted to write everything from scratch, luckily I had already implemented a full HTTP parser for my other project c-web-module. So I extracted the HTTP parser as a standalone library and ported it to my OS.

After the HTTP engine was done I moved on to the web engine, and focused on something small rather than big and fancy. Mainly it was important to add routing and route handlers. Allowing the user to specify a route, method, and Lambda function handler.
/* Simple routing */
engine.get("https://oshub.org/", [](const http::Request& req, http::Response& res) {
    (void)req;
    res.setBody("hello world!");
    ....
});
It’s a small example, but it reflects how a lot of modern C++ and web frameworks think about routing: match a path + method, call a handler, create a response.
#lang:plaintext
[ Browser ]
    |
    v
[ Web Server (userspace):
  WebEngine | HTTPEngine | FileRepository ]
    |
    v
[ Network stack:
  TCP/UDP | IP | ARP | DHCP | DNS | Ethernet(E1000) ]
The final step was to update the userspace program with the new HTTP and web engines. Finally I added a way to present files using a FileRepository that supports caching. Now I can edit files inside the operating system and then serve them with a web server.
#lang:cpp
WebEngine webEngine(80, 16);
web::FileRepository fileRepo;

/* Simple static pages */
webEngine.get("/home", [&fileRepo](const http::Request& req, http::Response& res) {
    (void)req;
    res.sendFile(fileRepo, "/web/index.htm");
});

webEngine.get("/about", [&fileRepo](const http::Request& req, http::Response& res) {
    (void)req;
    res.sendFile(fileRepo, "/web/about.htm");
});

webEngine.get("/status", [&fileRepo](const http::Request& req, http::Response& res) {
    (void)req;
    res.sendFile(fileRepo, "/web/status.htm");
});

webEngine.run();

The next thing on the TODO list would be to add a more fancy UI to the webserver and a way to shut it down gracefully.

(Graceful shutdown is one of those “boring” features you don’t miss… unless you corrupt something the first time you exit.)

When this is over the biggest work yet will begin… Web browser.



<a href

Leave a Comment