THIS WEEK’S ACCOMPLISHMENTS

The last week was a bit sparse on accomplishments. I had a lot of preparation to do to prep for the first week of school, and most problems have been resolved, but the following changes were made:

• Application checking was reworked to verify that Web.Config also exists. As it stands, the application manager checks for both Web.Config and Bin when adding an application. Web.Config doesn’t necessarily mean something is an application, but the lack of it means that it isn’t. (I’ve been told this is a IIS rule.)
• Requests that are directories but don’t end in a slash are redirected to the one with a slash.
• Case insensitive checking is used for “Web.Config”, “Bin”, and index pages to improve compatibility.
• Log levels can be set in the command line.
• Log messages can be sent to the standard output.
• ConfigurationManager.xml has been reorganized to group commands more logically.

WHERE IT STANDS NOW

At this point, the FastCGI Mono Server works completely with Lighttpd and Abyss, and when used with Apache and mod_fcgid, behaves like the default config supplied with mod_mono.

Cherokee has some problems when multiple requests are sent at the same time, but this may be a problem on either end. (In most cases, the mistake has been on my end.)

PLANS FOR THE FUTURE…

src/Mono.WebServer: Much of my work on Mono.WebServer is to be refactored and worked into Mono.WebServer (XSP package). This may mean automatic mapping support in mod_mono as well in the future.

src/Mono.WebServer.FastCgi: Most of the Mono.WebServer.FastCgi stuff will be restructured to and made more generic (think Mono.WebServer.CgiRequestWorker and Mono.WebServer.CgiApplicationHost) which will be reusable for things like SCGI support. This means fastcgi-mono-server may appear in XSP as well.

src/Mono.FastCgi: Support still needs to be finished for the Authorizer and Filter roles, and some renaming/FxCop/Gendarme needs to be applied. In the near future, I recommend bundling it with your application if you’re using it, but in the distant future, I expect it to be API stable/GAC-able.

Unfortunately, I will be taking 19 credit hours this semester, to accelerate my transition from Chemical to Computer Systems Engineering. This means I’ll have much less time to contribute, but I will continue to work on this (and my other) project.

SPECIAL THANKS

Special thanks to Google, the Mono SoC administrators, Marek, Daniel, Dick, Miguel, and everyone who commented or contributed in some way to my project! I appreciate all the support and the opportunity not to totally waste my summer.

Sincerely,
Brian Nickel

PS. Trac seems to have my IP address confused with a spammer, so I can’t report bugs on sites that don’t provide public registration. Does anyone know how I can fix this?

THIS WEEK’S ACCOMPLISHMENTS

The Changelog grew by 116 lines this week. Key features are as outlined.

• “Blocking” property was removed from Mono.FastCgi.Socket. It was only used to disable blocking on a socket that already communicated with asynchronous calls and was causing a problem with Abyss and unmanaged sockets.
• All headers are now sent in a single batch rather than one at a time. This should reduce the amount of network traffic and prevents Nginx from occasionally losing a request.
• Added documentation for Abyss X1 and Nginx.
• Fixed .in files not to hardcode mono location.
• Provide a friendly response 500 error page if no application could be found for a request.
• Added directory root detection with code borrowed from XspWorkerRequest.
• Found bug in Mono.WebServer where MonoWorkerRequest.SendFile was double closing a handle.
• Removed check for file existence when creating applications. This way proper applications can still be created when the user mistypes the path.
• Added documentation index page with important information on using paths vs. extensions, what are ASP.NET applications, how applications are handled, and running fastcgi-mono-server on a different machine than the web server. Plus brief information on how a request is actually processed, from Firefox to ASP.Default_aspx:Render(). See http://mono-soc-2007.googlecode.com/svn/trunk/brian/FastCgi/doc/index.html
• Added svn:eol-style=native to .html and .cs files.
• Added svn:mime-type to HTML and PNG files so they will show up in a web browser.
• Fixed the status line. It was being sent in the HTTP: “HTTP/1.1 200 OK” format instead of the CGI “Status: 200 OK” format. This was causing some breakage where status 200 was always being sent from web servers and Apache wasn’t working at all.
• Better output for “/help”
• Added some docs for using paths instead of directory mapping.
• Cleaned up UnmanagedSocket with proper SocketExceptions and a wait handle for EndAccept.

and

After discussion with Daniel Nauck changed the recommended method of handling ASP.NET to sending all requests to fastcgi-mono-server. The benefits are outlined at http://mono-soc-2007.googlecode.com/svn/trunk/brian/FastCgi/doc/index.html#info1 but in short they are improved security and enhanced features with application specific configuration.

PLANS FOR THE COMING WEEK

1. Continue documenting.
2. Alpha release.
3. Clean up API.
4. Adding open source tools: mailing list, bugzilla, wiki page???

CHALLENGES I’M FACING

Mostly just a million different web server nuances.

RESOURCES USED THIS WEEK

• MSDN: WaitHandle, System.Web.UI.Page, IAsyncResult
• Netcraft: Server statistics.
• Apache API reference: Trying to understand cmd_parms.
• FastCGI.com: Performance statistics.
• CGI specification: Review.
• http://svnbook.red-bean.com: Information on properties.

Sincerely,
Brian Nickel

<IfModule fcgid_module>
 <FilesMatch ".(aspx|asmx|ashx|asax|ascx|soap|rem|axd|cs|config|Config|dll)$>
    SetHandler fcgid-script
    FCGIWrapper /usr/bin/fastcgi-mono-server .aspx
    FCGIWrapper /usr/bin/fastcgi-mono-server .asmx
    FCGIWrapper /usr/bin/fastcgi-mono-server .ashx
    FCGIWrapper /usr/bin/fastcgi-mono-server .asax
    FCGIWrapper /usr/bin/fastcgi-mono-server .ascx
    FCGIWrapper /usr/bin/fastcgi-mono-server .soap
    FCGIWrapper /usr/bin/fastcgi-mono-server .rem
    FCGIWrapper /usr/bin/fastcgi-mono-server .axd
    FCGIWrapper /usr/bin/fastcgi-mono-server .cs
    FCGIWrapper /usr/bin/fastcgi-mono-server .config
    FCGIWrapper /usr/bin/fastcgi-mono-server .Config
    FCGIWrapper /usr/bin/fastcgi-mono-server .dll
    Options +ExecCGI
 </FilesMatch>

 DirectoryIndex index.aspx
 DirectoryIndex Default.aspx
 DirectoryIndex default.aspx
</IfModule>

There are two important shortcomings:

1. Just wrapping extensions is not a recommended way of working with ASP.NET, as it lacks certain security and management features. It is better to just send the whole directory to the Mono server. (More on this, Friday.) As far as I understand it, this will require a change in mod_fcgid.
2. WebResource.axd shows up as a 404 error rather than the expected resource, this can be overcome hackishly by creating an empty WebResource.axd where the other one is expected. Does anyone who knows Apache know how to overcome this?

Sincerely,
Brian Nickel

Important changes that make this possible:

• Status 200 is sent again. Abyss was sending its own status (I presume 404) if the file didn’t exist and fastcgi-mono-server wasn’t sending in information.
• Blocking is no longer set for sockets. It didn’t appear to provide any essential features, as the server uses asynchronous calls anyway, but it was causing the unmanaged socket to stop listening on Abyss.
• Response headers are now stored in a string builder and sent only once the server has finished outputting them. The fast onslaught of tiny requests was causing occasional failure in Nginx.

If you use either of these server, please test them out and let me know of any issues you might encounter.

Sincerely,
Brian Nickel

THIS WEEK’S ACCOMPLISHMENTS

This week saw an explosion of development activity. Daily commits yielded a Changelog 234 lines (or 64%) larger than the week before. Key features are as outlined.

• Support was added for serving on a new Unix socket, as opposed to a piped socket or TCP socket. This can be used in several web servers in place of TCP sockets.
• Started work on documenting how to configure servers. Lighttpd and Cherokee are completely documented, Abyss is documented except for a bug appearing in ASP.NET 2.0, and the current problems with Apache2 are documented.
• Support was added for setting all command line options via environment variables, due to limitations in some servers. (MONO_FCGI_SOCKET=tcp is the same as setting /socket=tcp)
• Directories containing a “bin” directory are automatically treated as new applications. This is not a perfect way of checking for applications, as it can yield false positives, and not all applications use DLL’s, but it is pretty good for now.
• Support was added for writing to a log file.
• UnmanagedSocket now appropriately flags a closed connection.
• Sending and receiving records was redone, partially because I didn’t fully understand how reading and writing sockets worked and the request would die if Receive didn’t yield the exact number of bytes requested.
• When targetting .NET 2.0, Mono.FastCgi uses strictly generic collections. This reduces the size of the heap and should speed things up slightly.
• Records are now sent at no more than half the maximum size to avoid problems with some web servers. This behavior is similar to that of Cherokee.
• My patch to System.Net.Sockets.Socket was accepted. This patch is important because it removes the limitation of only supporting Unix and TCP sockets. (And may save some resources by not loading Mono.Unix if it isn’t needed.)
• Tons of bug fixes.

and

FastCGI Mono Server can handle full blown applications, like mojoPortal:

This is not hosted on a virtual path, but automatically recognized because the application contains a “bin” directory.

PLANS FOR THE COMING WEEK

1. I’m going to continue my work documenting servers. Each server presents different problems and test cases I may have overlooked and extends the reach of Mono.FastCgi.
2. Work with my mentor on what to do next.

CHALLENGES I’M FACING

I’m not sure how I want to proceed with the socket issue. It turns out all the problems I was having stemmed from bad implementation on my part and problems in mod_fcgid. Now that they’re overcome, the unmanaged socket is working fine. Not adding a mono specific feature to System.Net.Sockets.Socket would allow me to keep the version dependency down, make things easier if building it against MS .NET in the future, and give me an abstraction for using Named Pipes on Windows.

Another challenge is how to proceed with Apache. Admittedly, there are two existing open source implementations, both with their flaws, but both lacking things like Bugzilla, and both with highly cryptic source code. It may be best to develop a new FastCGI module under the Mono Project umbrella, to protect the project’s interests. It could be based off code from mod_mono and design attributes from my FastCGI library.

RESOURCES USED THIS WEEK

• MojoPortal forums.
• Abyss documentation.
• Wrapper scripts for other applications.

Sincerely,
Brian Nickel

Last Friday, I made it possible to compile and install your own copy of “fastcgi-mono-server” using standard methods. Now I have some in depth information on how to configure your server and what to look out for.

lighttpd

Lighty works fantastic. With about 5 minutes of configuration, you can get ASP.NET and all its glory running. Simply follow the instructions located here or in the “doc/linux/” directory of the source package.

cherokee

Cherokee, up to and including Cherokee 0.5.6, has a bug where it fails to send an empty Stdin record if no input data is present. I could write a hack to get around this problem, but I’m holding out for now. The Cherokee 0.6 series, on the other hand, works very well with fastcgi-mono-server. If you’re ready to move to the 0.6 series, follow the instructions located here or in the “doc/linux/” directory of the source package.

apache2

Apache’s FastCGI situation is problematic. There are currently 2 implementations of FastCGI that I’m aware of: mod_fastcgi and mod_fcgid, and neither one of them is working just right. Mod_fastcgi was designed by the authors of FastCGI to be a replacement for CGI (hence the name), and is strongly tied to the paradigm of each file being its own application. It does have some tricks which would let a single server be used, but it sends some very unusual and sometimes junk parameters back to the server. It would require some major hacks to transform its parameters back into values that the server could understand.

Mod_fcgid, on the other hand, has the advantage of being younger, hipper, and more in tune with the concept of script servers. It is almost perfect except for the fact it sends the entire contents of the response to the server rather than stripping off the status line and sending it as r->status_line. This causes Apache to reject the response and display an error message. fastcgi-mono-server doesn’t send a status line of the status is 200 (Ok), so it’ll only work for pages that don’t do anything fancy or crash.

THIS WEEK’S ACCOMPLISHMENTS

The first accomplishment is the incorporation of autotools. You can now install the server using the standard methods. It does not yet install assemblies into the GAC, but I plan on adding that by next week.

svn co http://mono-soc-2007.googlecode.com/svn/trunk/brian/FastCgi/ fastcgi-mono-server
cd fastcgi-mono-server
./autogen.sh && make && sudo make install

Once installed, you can use either of the following, for the 1.1 or 2.0 framework respectively:

fastcgi-mono-server /socket=tcp:1234
- or -
fastcgi-mono-server2 /socket=tcp:1234

Then use the methods discussed in previous reports to add support to the webserver. (Lighttpd, Cherokee)

The second accomplishment is a patch to the mono class library for System.Net.Sockets.Socket <http://mono-soc-2007.googlecode.com/svn/trunk/brian/patches/System.Net.Sockets.Socket-endpoint.diff>. It is a precursor to a constructor that will make a wrapper from the unmanaged to the managed socket.

PLANS FOR THE COMING WEEK

I’m going to continue working on what I’ve been working on but have not completed this week:

1. I’m going to work with my mentor to develop a plan for a control panel and additional auto-configuration.
2. I’m going to work with the mono developers on “the socket issue”.
3. I’m going to contact developers of various FastCGI implementations on servers to overcome various issues I’ve encountered.

CHALLENGES I’M FACING

Nothing major at the moment.

RESOURCES USED THIS WEEK

• MSDN and SharpDevelop (Windows) – Studying and testing out the Windows implementation of System.Net.Sockets.Socket.
• http://www.koders.com – Searching for information on WinSock2
• various source packages – How to do things with autotools.

Sincerely,
Brian Nickel

THIS WEEK’S ACCOMPLISHMENTS

This week I worked at re-prioritizing project goals, putting automatic configuration above a configuration application.

I implemented automatic application detection. Now you can simply call:

fastcgi-mono-server /socket=tcp:1234 /automappaths

That way you can just drop your ASPX files in the standard htdocs or public_html directories and an appropriate application is registered. This should overcome a lot of confusion in configuration and for automatic registration when creating new sub-domains or users.

Beyond that, I got the server working perfectly on Lighttpd:

In /etc/lighttpd/conf.d/fastcgi.conf:

#######################################################################
##
##  FastCGI Module
## ---------------
##
## http://www.lighttpd.net/documentation/fastcgi.html
##

server.modules += ( "mod_fastcgi" )

fastcgi.server = (
        ".aspx"   => (( "host" => "127.0.0.1", "port" => 1234 )),
        ".asmx"   => (( "host" => "127.0.0.1", "port" => 1234 )),
        ".ashx"   => (( "host" => "127.0.0.1", "port" => 1234 )),
        ".asax"   => (( "host" => "127.0.0.1", "port" => 1234 )),
        ".ascx"   => (( "host" => "127.0.0.1", "port" => 1234 )),
        ".soap"   => (( "host" => "127.0.0.1", "port" => 1234 )),
        ".rem"    => (( "host" => "127.0.0.1", "port" => 1234 )),
        ".axd"    => (( "host" => "127.0.0.1", "port" => 1234 )),
        ".cs"     => (( "host" => "127.0.0.1", "port" => 1234 )),
        ".config" => (( "host" => "127.0.0.1", "port" => 1234 )),
        ".dll"    => (( "host" => "127.0.0.1", "port" => 1234 ))
)

In /etc/lighttpd/modules.conf:

include "conf.d/fastcgi.conf"

In /etc/lighttpd/lighttpd.conf:

index-file.names = (
  "index.xhtml", "index.html", "index.htm", "default.htm", "index.php", "default.aspx", "index.aspx"
)

PLANS FOR THE COMING WEEK

1. I’m going to work with my mentor to develop a plan for a control panel and additional auto-configuration.
2. I’m going to work with the mono developers on “the socket issue”.
3. I’m going to contact developers of various FastCGI implementations on servers to overcome various issues I’ve encountered.
4. I’m going to start reorganizing the source for autotools.
5. I’m going to continue work on documentation.

CHALLENGES I’M FACING

I’ve had trouble getting enough time in due to family obligations and coursework. I’ve also had a lack of motivation due to a lack of clarity over how to address the control panel. Both of these problems should be resolved.

Sincerely,
Brian Nickel

THIS WEEK’S ACCOMPLISHMENTS

One goal of the project is to make it easy to configure the server. This week I created the backing for a configuration system, which allows users to specify server options through either command line options or an XML file (that will be editable by a configuration ASP.NET application).

Features:

1. Settings can be done written a number of ways that are both Linux and Windows friendly:

Via the command line:

 /arg=VALUE
--arg=VALUE
 -arg=VALUE
 /arg VALUE
--arg VALUE
 -arg VALUE

Via XML:

<Setting Name="arg" Value="VALUE" />
<Setting>
    <Name>arg</Name>
    <Value>VALUE</Value>
</Setting>

2. All configuration options are stored in an XML definition file making it easy to add new arguments:

<Setting Name="port" AppSetting="MonoServerPort" Type="UInt16"
    ConsoleVisible="True">
    <Description>
        <para>Specifies the TCP port number to listen on.<br />
        To use this argument, "socket" must be set to "tcp".</para>
    </Description>
</Setting>

3. The configuration manager does the basic checks and conversions:

try {
   port = (int) configmanager["port"];
} catch (ApplicationException e) {
   Console.WriteLine (e.Message);
   return 1;
}

Reads the port argument and tries to convert it to a UInt16, and fails with:

Error in argument "port". "xxx" cannot be converted to an integer.

This also does checks for file and directory existence.

4. Boolean options that default to true can be set to false in the command line:

/multiplex=False

5. All option descriptions are stored in an XML format that can be used by both the web and the console, with whitespace being gracefully ignored:

    <Setting Name="nonstop" Type="Bool" ConsoleVisible="True">
        <Description>
            <para>Specifies not to stop the server by if "Enter" is pressed. This must be used when the server has no controlling terminal.</para>
        </Description>
    </Setting>

and

    <Setting Name="nonstop" Type="Bool" ConsoleVisible="True">
        <Description>
            <para>Specifies not to stop
            the server by if "Enter" is
            pressed. This must be used
            when the server has no
            controlling terminal.</para>
        </Description>
    </Setting>

both yield the following when –help is called:

  /nonstop[=True|=False]     Specifies not to stop the server by if "Enter" is
                             pressed. This must be used when the server has no
                             controlling terminal.

6. Options can be set via the the application with all the same conversions:

configmanager ["port"] = "1234";
ushort port = (ushort) configmanager ["port"];

7. Order of precedence:

When reading a setting, the command line is checked first, then the XML file (if loaded), then the AppSettings (stored in myexecutable.exe.config)

Hopefully, this class will be useful in other applications as well.

In other news:

Instead of just not working at all on Apache, it now fails badly. Needless to say, that’s very exciting news. It was very last minute that this change occurred, so there will be much work to do next week.

PLANS FOR THE COMING WEEK

Next week will be continuing with the work of this week. I will continue to work the Apache angle and hopefully get things working, and I’ll finish up the ConfigurationManager and add a custom ApplicationHost that can bond with it. This week I got through the harder problem of “Where do I even begin?” so next week should feature a lot of progress.

CHALLENGES I’M FACING

It was really hard to get grounded on an initial idea for the ConfigurationManager. There were a lot of issues like how to organize things, what the XML should look like, etc. It really slowed me down and I went through many iterations I didn’t really like before I reached my final solution.

RESOURCES USED THIS WEEK

• Pencil and Paper – Organizing my thoughts on paper is usually the last thing I want to do, but it really helps clarify a lot of things and I think makes your brain operate on a different level than usual.
• MSDN – Looking up NameValueCollection.
• Google – Searching for information on the ConfigurationManager.

Sincerely,
Brian Nickel

THIS WEEK’S ACCOMPLISHMENTS

• I completely documented the Mono.FastCgi namespace using XML documentation.
• I submitted 6 patches to XSP, 3 code changes and documentation on 4 files.
• I finished the implementation of the FastCGI protocol except for the unmanaged socket.
• I implemented buffers for the sending and receiving of data and buffer management on the server to greatly reduce the amount of memory allocation performed.
• I started work on an unmanaged socket wrapper.
• I researched the configuration manager and how to store the configuration.
• I submitted a bug report on Cherokee’s FastCGI implementation.

PLANS FOR THE COMING WEEK

I plan to finish my work on configuration which should allow me to simplify and clean up server.cs and implement configuration files for things that don’t necessarily need to be done in the command line. Additionally, I will test the work I’ve done with the socket wrapper to try and get Apache working. Then I’ll start writing an ASP.NET configuration page for the server.

CHALLENGES I’M FACING

Nothing major at the moment.

RESOURCES USED THIS WEEK

• http://www.koders.com/ – Searching Mono’s System.Net.Sockets library.

I apologize of the lateness of this report. I had to go to El Centro for my grandmother’s 75 birthday and didn’t have it done by the time I had to leave.

Sincerely,
Brian Nickel