Arduino Ethernet Shield under SIEGE

Today there are many single-board computers sufficient enough to create great projects for a low cost – an Arduino Electronic Pot for example. Although the Arduino Ethernet Shield is not meant to be a web server providing hosting services for many clients (more often to make automated requests), my curiosity about the capabilities was too strong…

The set-up

Preparing the tests was very simple, I already had the Arduino Leonardo main board and an Ethernet Shield attached to it. Hardware-wise – that’s it.

arduino ethernet shield

All tests were done by SIEGE benchmarking software (Joe Dog – creator; Man Pages). This application allowed me to make simultaneous GET requests to the root (/) of the web server. Additionally, for the created virtual clients, I could set the time between the requests.

For the Arduino I used the default example code of a web server. The only changes I made were configuration variables and the HTML section – instead of displaying the analog inputs, I created a sample website (using HTML-ipsum).

sample website

Siege

To get SIEGE:

sudo apt-get update
sudo apt-get install siege

If you want to know if you already have it:

siege --version

The structure of the command I used:

siege -v -c NUM -d TIME -r1 URL

Where:

  • NUM – number of server transactions (“virtual users”) – {5, 10, 15, 20, 25, 30, 35}
  • TIME – time delay between simulated users actions – {1, 2, 5}
  • URL – local IP address of the web server based on Arduino + Ethernet Shiled
  • -r1 – one repetition of transfer per virtual user

Results

I run the test three times for every instance and calculated the average values. My main focus was the response time, I was curious where was the boundary after which the website was “unworkable”.

According to the Nielsen Norman Group, we can differentiate 3 main response time actions:

  • 0.1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result.

  • 1.0 second is about the limit for the user’s flow of thought to stay uninterrupted, even though the user will notice the delay. Normally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 second, but the user does lose the feeling of operating directly on the data.

  • 10 seconds is about the limit for keeping the user’s attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect.

Based on the  above selections, none of my tests results would qualify to the first group. The lowest value was 0.78 of a second – for 5 virtual users with 5 seconds delayed actions.

responses

For tested delay values, only the groups with 5 and 10 virtual users fit under the 10 seconds mark. It is also the only group, where the Availability of the web server was 100% for all tests:

Virtual Users 5 10 15 20 25 30 35
With 1 second delay 100% 100% 98% 82%
With 2 second delay 100% 100% 98% 82%
With 3 second delay 100% 100% 87% 88%
With 5 second delay 100% 100% 100% 100% 100% 93% 90%

The group with the 5 seconds delay achieved also 100% of web server availability for 25 virtual users. Making sure, that the delay between connections with our web server is enough to handle the responses could increase the possible availability of our server. Of course this is at the expense of cutting out users. In addition, this test was based on only request – in real life situations the user stays on the page and interacts with it, this scenario was not taken under consideration in this tests.

responses5sec

Conclusion

As assumed at the beginning, the Arduino + Ethernet Shield can’t be a robust and reliable web server for website hosting. Although it handled the requests for a small group of virtual users, the sample page was minimalistic and probably no one would host a website with this amount of content. The results were acceptable only for 5 virtual users, where the main average response time was about 1.5 seconds.