Two weeks ago was my last day at Oracle (after 7 years!) and now I am starting a new adventure.

I joined Treasure Data Inc as an Open Source developer to contribute to the Fluentd open source project and it related tools. Definitely this is a very exciting movement, the project is widely used and is the defacto Log/Events Collector for Servers, Mobile & Embedded. There is a lot of fun things to do and I will be mostly focused on it Golang, Embedded Linux versions and it Community within others, more news will come soon...

If you are a Fluentd user let me know, I'd be happy to hear from feedback and use cases, otherwise let's talk anyways, I am sure it can solve your events collection needs 🙂

This is a very late post about the Google Summer of Code Reunion 2014. I will not share too many details, just a summary of the most relevant things (for me).

The big difference on this year, was that people were celebrating the 10th anniversary of Summer of Code. So many things were different compared to previous Mentors Summit:

  • Not a Mentors Summit, a Complete Reunion. A lot of students participated this time. Personally I met and I talked to many students, it's very good to get feedback and see how was their experience, face-to-face feedback makes a big difference.
  • The event was hosted in the San Diego Marriott hotel. In previous years most of the activities were done on the Google Plex, this time was different and I can conclude it's because the large amount of people attended between Organizations, Past-Organizations and Students.
  • Gala night event: I had to admit that was fun to see many people from the open source community dressed in casual mode, had the opportunity to see people from other countries using their native customs (I am sure there is a best word for that but I cannot find it). The big surprise of the night was to see Linus Torvalds who shared his experiences on stage. For the record: he was very nicely with everyone who wanted to talk to him, take a pic or drink a beer.
  • Google rented the Great America park just for US!, our first impression was: is this just for us!???.

On the other side, I had the opportunity to met one of the Monkey core developers Sonny Karlsson, very nice person!. Would be awesome if open source projects had the opportunity to met people more frequently.

Another surprise was to met face-to-face people that I used to collaborate with. Many years ago I was lucky enough to join the One Laptop Per Child organization through Google Summer of Code. On that moment I met Walter Bender, Bernie Innocenti and Martin Abente. Most of them are hardly working on Sugar and still helping the XO laptops on development countries, isn't wonderful ? 🙂








More pictures available on the following link:

The event was a real celebration, we also held many presentations and meetings, not only about GSoC but also about side projects within others.

In a few words, the event was great and very professional, I am always happy to share and listen about experiences from people that have years on Open Source or others that just joined, this environment really adds value and is very refreshing to have this kind of events.

Note: if you were interested into the Duda I/O presentation, here are the slides

- Attached a Slideshare link to the Duda I/O presentation given at the lightning talks session

Before to start playing with the commands suggested here let me clarify my context:

  • I had a Mac Book Pro with OSX Maverick with dual boot working with Ubuntu 14.04 through rEFInd boot loader.
  • Today I upgraded OSX to Yosemite and after everything was updated rEFInd stopped working.

So my goal was to make able to have rEFInd working again to switch between operating systems. My primary OS is Linux so if you are in a similar situation this may help:

On a Twitter search I found someone linked the following solution:

That worked for me, I got a few warnings and I did not need to run the 'mount' command. Good luck!

The next versions of Monkey HTTP Server and Duda I/O are very challenging, because both are very related and big changes are coming in terms of optimizations and general features.

As said many times, Monkey is the base HTTP stack for Duda I/O, for hence any improvement made on Monkey is inherit by Duda I/O, as well any extra feature is exposed too. Let's see what is expected for the following major versions:


The code base continue being optimized, we care a lot of performance on all environments, not just high-end production servers, also in Embedded based systems where we got a lot of traction.

The big news besides performance, is the new event loop (polling) mechanism implemented, so starting from v1.6 we support Linux and also OSX. Yes, I know...what about our speech about Monkey is just for Linux ?, here is the answer: As a HTTP server Monkey it self does not have too much things to offer on an operating system that is not widely deployed on Internet Servers, neither on Embedded systems, but we need to look beyond and think in use cases, the main reason of Monkey being supported on OSX is Duda I/O.

Duda I/O is a fast web services framework, in short words: it let you build scalable C web services with a very simple API. Nowadays OSX is a base platform for developers, so our move is to provide a stable and fully featured web services stack to the Apple world. Wouldn't be great to write your scalable web services in C on your favorite OSX system and deployment later easily on Linux servers or devices ?, yes it is. This is very exciting for us and continue involving a lot of work, but fun 🙂 .

Duda I/O

When I mention a web services stack I refer to a professional product, fully documented and that have been used by production systems on the last two years. So major versions do not happen too frequently, we focus on long term support and stability.

The next version DST-2 (Duda Stable 2) will be released with great changes that covers a bit of everything:

  • Co-routines support (dthreads)
  • New routing model, do you like URLs like /users/:id/about/:phone/ ?, there you go.
  • Re-designed Console and optional Dashboard UI
  • Streams
  • Docker Support
  • More Documentation
  • Duda QA: Quality Assurance.

I mentioned Duda QA, so let me explain. Starting from DST-2 there is a new project that is part of Duda ecosystem, it deliver a complete Test system for Duda. For every Object API exists an unit test, as many changes are underway we need to make sure to avoid regressions. That means that as of now there are around 200 tests under 1300 lines of code...and growing. So every time we do some changes, you as an end-user can trust that we are not breaking the API or anything similar. There is nothing more frustrating than a broken API/stack, so keep calm as we are really taking care of that. More details about the project and status here:

In general many things are coming, there are other features and improvements that I did not mention but I am sure I will once I take some time to write the Release Notes, by now the works continue.


I planned to have a quite Saturday, but after check my email in the morning, I found a possible security issue reported for Monkey (1.5 onwards), it may lead to a DDoS based on the new optional FDT feature, so it was time to research and release a fix as soon as possible.

So Monkey v1.5.3 was released which include the fix for the problem reported, i also take the opportunity to include other patchs sent by the reporter, so the goal was accomplished: act transparent and quick.

Duda I/O

Also during the week i have been working in a new look & feel for Duda I/O project, you cannot imagine how hard is to create an image for a Software that don't have a UI (yet). So the project just got 3 major things:

New logo

Duda I/O is a Stack, for hence from my perspective I think this represents it nature and simplicity:

not sure if people would love it but I am sure it will fit very well on Project T-shirts and Stickers 🙂

New Web Site

As the new logo arrived, was a good time to complete the new responsive web site:

Disclaimer: I bought a bootstrap template and did the adjustments, i am not a designer.

compared to the previous project site this is a major upgrade.

Google Summer of Code

We are in the pencils down phase of the Google Summer of Code program, that means most of our students are doing their final adjustments on their projects as the final evaluations may start this evening, all of them have done a very good job, they deserve the credit. More news about cool stuff that was made will be public in the incoming days.

It's a good jurney, life is moving a lot in a positive way, in all aspects...

Now it's official, Monkey is a participant organization of Yocto Project.

Participate in Yocto Project is not just about software recipes, it's about a compromise to contribute to make Embedded Linux environment compliant, and as an Open Source organization there is a huge responsibility into push forward stable and innovative technology, so for us this is really good news, make strong relationships is critical to align objectives and goals.

You can read more details about this in the Monkey Blog Post here.



I always believed that integration and focus are a success key for any kind of thing on this life. When talking about technology and servers specifically, integration may be a fundamental part to achieve different goals like good performance. Years ago was very valuable to have a software that can work on multiple Operating Systems, but there is a point that if you want to increase your performance limits you need to look forward for integration.

In the last years, Linux have become the best choice for Servers, it's well known that every big company trust in Linux and also commit to their development: Redhat, Oracle, Google, HP, IBM and many many others. So we can say that Linux is doing pretty well on professional Hardware, but what about user-space software ?.

If we look in the HTTP Servers (or Web Servers) context, we will find that most used solutions are made to support different platforms and on specific build-time for a specific Operating System, different approaches are used to work better, in servers performance and resources consumption are critical, and when moving those solution to Embedded even more, so every taken decision may sacrifice something, we do in OS portability.

Monkey HTTP Server and Duda I/O are written with focus on the Linux Kernel, every Unix operating system despite their POSIX API, provides extra system calls that aims to extend functionality and offer features. Of course this kind of integration have a cost which is basically: do not be portable to other Operating System, so depending of your human resources you need to decide how to invest better: we do in focus and integration, and it works great.

As a reference i would like to mention which specific features we use and i hope people who write servers or any kind of networking application would take this in count. Of course  these are dependent of specific Linux Kernel versions:

Linux 3.14

  • TCP AutoCorking: let Kernel do smart buffering when sending outgoing data, the old fashion way is to do manual Cork over sockets (more).

Linux 3.9

  • SO_REUSEPORT: be able to reuse port-addresses, e.g: multiple threads under same process can bind the same port and let the Kernel distribute load across them (more).

Linux 3.7

  • TCP FastOpen: Reduce the number of round-trip required in a TCP conversation (more).

Also other very helpful interfaces exists such as accept4(2), sendfile(2), epoll(7) and splice(2), these are not the only ones but that should be considered when writing a Server for Linux.

We are in good times where performance matters and make a good design to make distribute load across multi-core architectures is fun and challenging. Again, if you write with focus and integration will be a win:win relationship between your network server and the HW behind it.

since some time ago when using a specific Duda I/O feature to connect backend servers using a custom web service, some inconsistency existed if those backends closed the connection, for some reason this ended up into the web service missing the connection close event.

This problem was detected months ago but it was not easy to determinate the root cause, it affected MariaDB/MySQL package and custom web services that use the stack event loop to be notified on sockets connected to third party servers. Last night doing some example code for a different thing the problem could be reproduced, and the steps were very simple:

  • create a TCP socket
  • make it asynchronous
  • run a connect(2) to a specific backend
  • register the socket file descriptor into the events loop interface
  • shutdown the backend server

and when a connection close notification should arrive, that never happened. This ended up in an inconsistency of a zombie TCP socket without events notifications on it. Looking around and tracing Monkey HTTP Server found the following difference:

  • when registering a new socket in the events interface, the following flags are always associated to a socket: EPOLLERR, EPOLLHUP and EPOLLRDHUP.
  • when someone change the socket direction (read to write, write to read), all flags are overwritten but on this time just EPOLLERR and EPOLLHUP were used, EPOLLRDHUP was missing!

Googling around i found this relevant comment:

A socket listening for epoll events will typically receive an EPOLLRDHUP (in addition to EPOLLIN) event flag upon the remote peer calling close or shutdown(SHUT_WR). This does not neccessarily mean the socket is dead. Subsequent calls to recv() will return any unread data on the socket and eventually "0" will be returned to indicate EOF. It may even be possible to send data back if the remote peer only did a half-close of its socket.

The one notable exception is if the remote peer is using the SO_LINGER option enabled on its socket with a linger value of "0". The result of closing such a socket may result in a TCP RST getting sent instead of a FIN. From what I've read, a connection reset event will generate either a EPOLLHUP or EPOLLERR. (I haven't had time to confirm, but it makes sense).

There is some documentation to suggest there are older Linux implementations that don't support EPOLLRDHUP, as such EPOLLHUP gets generated instead.

After apply the following fix, the issue have gone:

Now wondering when (and who) introduced the problem:

$ git blame src/mk_epoll.c -L362,+1
  10c3eb27 (Eduardo Silva 2011-09-15...) = EPOLLERR | EPOLLHUP;


Git blame me and lesson learned... never miss EPOLLRDHUP again.

Note: the fix have been backported to Duda-Stable-Branch 1 (DST-1), so people developing services only need to update their stable stack version.

Duda I/O web services framework project was started almost two years ago, initially as a Monkey HTTP Server extension to create scalable web services in C, nowadays both components together are a complete stack to develop any kind of event-driven web backend application that relies on HTTP or Websockets.  As mentioned in early post, the web is not just about images, CSS or any JS scripts, its about also about fast data transfer and scalable services that aim to optimize resources usage with low overhead.

Sincerely i would say that Duda I/O and Monkey projects are the Mozilla of the backend side, of course we are far of their commitment but i would say the mission is the same: promote openness, innovation & opportunity on the Web.

Scale a C project at a community level is hard, due to the target market, the number of early adopters is very low compared to any  GUI or Web (front end) project, but once it get's in and it works.. things changes in a positive way. At the moment there are a couple of Duda I/O instances running in production in private companies from US, Sweden and Germany, and i can conclude by their feedback the reasons of the adoption are pretty much the same:

  • High performance: all implemented web services requires some kind of real-time processing data coming from HTTP requests.
  • Open source stack: proof of good security handling, no back-doors. What really matters is that "It is open" (not talking about money).
  • Proof of years of development and commitment, the project will not disappear because of a one man project.
  • Good support: on early stages of evaluating the stack, their always receive good support through the mailing list or private emails.

Also much of the latest changes on the stack, comes from end-users feedback on topics such as documentation, development environment, API changes, etc. So this year have been a big step forward on terms of adoption and features improvements, next year will be even more fun and of course more hard work on:

  • Native deployment, native packaging for different Linux distributions.
  • Co-routines support to make easier to defer async work from callbacks.
  • SPDY & HTTP/2.0 .
  • New QA environment, stability and avoid regressions are high priority.
  • Clustering mode.
  • Sandboxing.
  • ...others.

Personally i am just happy with all goals achieved during this period and very thankful to be part of a small but great community.