Brian Nickel’s Online Journal

If you don’t C# you’ll B♭

Archive for June, 2007

FastCGI ASP.NET Server Status Report 5

Posted by Brian on June 29, 2007

This is a status report for the week of June 25 - June 29. This week started out rather slow, but I managed to get a large part of the configuration system completed.

THIS WEEK’S ACCOMPLISHMENTS

One goal of the project is to make it easy to configure the server. This week I created the backing for a configuration system, which allows users to specify server options through either command line options or an XML file (that will be editable by a configuration ASP.NET application).

Features:

1. Settings can be done written a number of ways that are both Linux and Windows friendly:

Via the command line:

 /arg=VALUE
--arg=VALUE
 -arg=VALUE
 /arg VALUE
--arg VALUE
 -arg VALUE

Via XML:

<Setting Name="arg" Value="VALUE" />
<Setting>
    <Name>arg</Name>
    <Value>VALUE</Value>
</Setting>

2. All configuration options are stored in an XML definition file making it easy to add new arguments:

<Setting Name="port" AppSetting="MonoServerPort" Type="UInt16"
    ConsoleVisible="True">
    <Description>
        <para>Specifies the TCP port number to listen on.<br />
        To use this argument, "socket" must be set to "tcp".</para>
    </Description>
</Setting>

3. The configuration manager does the basic checks and conversions:

try {
   port = (int) configmanager["port"];
} catch (ApplicationException e) {
   Console.WriteLine (e.Message);
   return 1;
}

Reads the port argument and tries to convert it to a UInt16, and fails with:

Error in argument "port". "xxx" cannot be converted to an integer.

This also does checks for file and directory existence.

4. Boolean options that default to true can be set to false in the command line:

/multiplex=False

5. All option descriptions are stored in an XML format that can be used by both the web and the console, with whitespace being gracefully ignored:

    <Setting Name="nonstop" Type="Bool" ConsoleVisible="True">
        <Description>
            <para>Specifies not to stop the server by if "Enter" is pressed. This must be used when the server has no controlling terminal.</para>
        </Description>
    </Setting>

and

    <Setting Name="nonstop" Type="Bool" ConsoleVisible="True">
        <Description>
            <para>Specifies not to stop
            the server by if "Enter" is
            pressed. This must be used
            when the server has no
            controlling terminal.</para>
        </Description>
    </Setting>

both yield the following when –help is called:

  /nonstop[=True|=False]     Specifies not to stop the server by if "Enter" is
                             pressed. This must be used when the server has no
                             controlling terminal.

6. Options can be set via the the application with all the same conversions:

configmanager ["port"] = "1234";
ushort port = (ushort) configmanager ["port"];

7. Order of precedence:

When reading a setting, the command line is checked first, then the XML file (if loaded), then the AppSettings (stored in myexecutable.exe.config)

Hopefully, this class will be useful in other applications as well.

In other news:

Instead of just not working at all on Apache, it now fails badly. Needless to say, that’s very exciting news. It was very last minute that this change occurred, so there will be much work to do next week.

PLANS FOR THE COMING WEEK

Next week will be continuing with the work of this week. I will continue to work the Apache angle and hopefully get things working, and I’ll finish up the ConfigurationManager and add a custom ApplicationHost that can bond with it. This week I got through the harder problem of “Where do I even begin?” so next week should feature a lot of progress.

CHALLENGES I’M FACING

It was really hard to get grounded on an initial idea for the ConfigurationManager. There were a lot of issues like how to organize things, what the XML should look like, etc. It really slowed me down and I went through many iterations I didn’t really like before I reached my final solution.

RESOURCES USED THIS WEEK

  • Pencil and Paper - Organizing my thoughts on paper is usually the last thing I want to do, but it really helps clarify a lot of things and I think makes your brain operate on a different level than usual.
  • MSDN - Looking up NameValueCollection.
  • Google - Searching for information on the ConfigurationManager.

Sincerely,
Brian Nickel

Posted in Programming, Summer of Code 2007 | No Comments »

FastCGI ASP.NET Server Status Report 4

Posted by Brian on June 24, 2007

This is the status report for June 18 - June 22. This week was mostly spent doing cleanup, documenting, and applying the final parts of the FastCGI specification.

THIS WEEK’S ACCOMPLISHMENTS

  • I completely documented the Mono.FastCgi namespace using XML documentation.
  • I submitted 6 patches to XSP, 3 code changes and documentation on 4 files.
  • I finished the implementation of the FastCGI protocol except for the unmanaged socket.
  • I implemented buffers for the sending and receiving of data and buffer management on the server to greatly reduce the amount of memory allocation performed.
  • I started work on an unmanaged socket wrapper.
  • I researched the configuration manager and how to store the configuration.
  • I submitted a bug report on Cherokee’s FastCGI implementation.

PLANS FOR THE COMING WEEK

I plan to finish my work on configuration which should allow me to simplify and clean up server.cs and implement configuration files for things that don’t necessarily need to be done in the command line. Additionally, I will test the work I’ve done with the socket wrapper to try and get Apache working. Then I’ll start writing an ASP.NET configuration page for the server.

CHALLENGES I’M FACING

Nothing major at the moment.

RESOURCES USED THIS WEEK

I apologize of the lateness of this report. I had to go to El Centro for my grandmother’s 75 birthday and didn’t have it done by the time I had to leave.

Sincerely,
Brian Nickel

Posted in Programming, Summer of Code 2007 | No Comments »

FastCGI ASP.NET Server Status Report 3

Posted by Brian on June 15, 2007

This is a status report for the week of June 11 - June 15. This week has been incredibly productive, culminating in a working ASP.NET server.

THIS WEEK’S ACCOMPLISHMENTS

This week was an intense coding week:

  • I repackaged all the FastCGI specific code into its own library (Mono.FastCgi.dll)
  • I created Mono.FastCgi.Socket to wrap around System.Net.Sockets.Socket until the socket situation is fully resolved.
  • I copied Mono.WebServer into my project and have modified components to better work with the FastCGI server. (It isn’t necessary to modify the library to get the FastCGI server to work, but the changes allow the FastCGI server code to be cleaner.)
  • I created Mono.WebServer.FastCgi using code from XSP to produce a fully working ASP.NET server.
  • Lots of debugging.
  • Started documenting with /doc.

Session Test

Try it out for yourself:

  1. Check out the latest version from Subversion: <http://mono-soc-2007.googlecode.com/svn/trunk/brian/FastCgi>
  2. Open the project in MonoDevelop.
  3. Build Mono.WebServer.FastCgi
  4. Enter FastCgi/Mono.WebServer.FastCgi/bin/Debug in the console.
  5. Run MONO_PATH=. mono fastcgi-mono-server.exe –port {port_number} –applications /:{path}
    where {port_number} is equal to the port you wish to run on. In my previous example, this was 1234
    and {path} is the path to the server root. In the case of OpenSuSE, this is “/srv/www/htdocs/”.
  6. Configure your web server to use FastCGI on {port_number} for the “aspx” extension.
  7. Enjoy.

PLANS FOR THE COMING WEEK

With the first goal close to complete, I’m going to examine optimization a bit. (The current implementation loads the entire post data into memory.) In addition, I plan to get a working resolution to the socket issue that has been barring me from using Apache. Once these issues are resolved, I’ll begin work on the second goal: “A configuration page (aspx) and a web service to adjust or reset the server’s configuration remotely.”

CHALLENGES I’M FACING

The biggest challenge I ran into this week was in dealing with AppDomains, which I didn’t know existed until 1AM on Tuesday after a series of confusing errors. That issue has since been resolved and things have been running smoothly since.

A frustrating but workable problem I’m having is that the server runs in quite a few threads and sometimes when an exception occurs it doesn’t print to the standard output. It just hangs. Once you figure out the method which its hanging in, via –trace, you can wrap it in a try/catch and get the message.

RESOURCES USED THIS WEEK

Sincerely,
Brian Nickel

Posted in Programming, Summer of Code 2007 | No Comments »

Make any .NET app a Web Server

Posted by Brian on June 12, 2007

While researching of my SOC project (FastCGI ASP.NET server), I stumbled across a neat little trick. In 4 lines of code, you can run an ASP.NET webserver from any .NET application:

XSPWebSource source = new XSPWebSource (System.Net.IPAddress.Any, 8080);
server = new ApplicationServer (source);
server.AddApplicationsFromCommandLine ("/:/path/to/asp.net/application");
server.Start (true);

Presto, you’ve got an ASP.NET server running on your computer. The only caveat is that the pages are running in a different AppDomain (aka a different universe) so you can’t access your static classes from an ASP.NET page and have to do DBUS style transactions to get any information.

Fortunately, there are two internal object types capable of crossing the boundary between between this world to the next: ISerializable and MarshalByRefObject. ISerializable being a traveler, moving from one domain to the other, but remembering nothing except what it contains within itself, and MarshalByRefObject having a foot in both worlds, sending ISerializable messages from one to the other.

The gateway between domains is someDomain.SetData(string name, object data) and someDomain.GetData (string name).

An example of ISerializable:

someDomain.SetData ("cowSay", Cow.Say ());

...

someDomain.GetData ("cowSay") == "Moo!"

MarshalByRefObject on the other hand is a bit more useful:

public class AnimalNoises : MarshalByRefObject {
   public string CowSay () {return Cow.Say ();}
}

someDomain.SetData ("animalNoises", new AnimalNoises ());

...

(someDomain.GetData ("animalNoises") as AnimalNoises).CowSay () == "Moo!"

So, now it’s possible to send information from one domain to another. To get the domain for the ASP.NET pages, you call:

AppDomain myDomain = server.GetApplicationForPath (null, -1, "/", true).AppHost.Domain;

You access the object in the ASP.NET page via:

AnimalNoises noises = AppDomain.CurrentDomain.GetData ("animalNoises") as AnimalNoises;

Finally, to avoid countless errors, you need to A) include the path to the calling application in MONO_PATH so it can be used to create the AppDomain, and B) create a link to the DLL declaring AnimalNoises in the application root’s bin directory so the ASP.NET pages can be compiled against it.

Putting it all together, you can get something like this:

Banshee XSP Plugin

Instructions:

  1. Download XspPlugin.cs to ~/.config/banshee/plugins/
  2. Modify it so it points to your banshee config directory.
  3. gmcs XspPlugin.cs -target:library -r:Mono.WebServer2 -pkg:banshee
  4. mkdir ~/.config/banshee/plugins/xsp
  5. Download index.aspx to ~/.config/banshee/plugins/xsp/
  6. mkdir ~/.config/banshee/plugins/xsp/bin
  7. ln -s XspPlugin.dll xsp/bin/XspPlugin.dll
  8. MONO_PATH=/usr/lib/banshee banshee
  9. Visit http://localhost:8080

Now, combine with avahi-sharp, a good sound daemon, and several more sophistocated ASP.NET pages, and you’ve turned banshee into a household music server.

The same could be done for an F-Spot photo browser, a BitTorrent client, etc.

Enjoy,
Brian

P.S. There is probably a much easier way of doing this, but this worked for me.

Posted in Programming, Summer of Code 2007 | 3 Comments »

FastCGI ASP.NET Server Status Report 2

Posted by Brian on June 8, 2007

This is a status report for the week of June 4 - June 8. This week is best categorized as a research and planning week.

THIS WEEK’S ACCOMPLISHMENTS

On the coding front, there hasn’t been too much progress. The main things that changed were a general tidying of the API, addition of more FastCGI features, and general stabilization. The complete list is in the ChangeLog <http://mono-soc-2007.googlecode.com/svn/trunk/brian/FastCgi/ChangeLog>.

Outside of coding, I spent most of my time rummaging through the XSP source code trying to make heads and tales of what everything does. It has been a pretty slow process but I’ve been storing my results in MonoDoc format <http://mono-soc-2007.googlecode.com/svn/trunk/brian/docs>, so it should be helpful to other developers down the line.

Another thing that occupied my time was deciding what to do to bridge my FastCGI implementation with the current ApplicationServer. Below is my plan.

PLANS FOR THE COMING WEEK

This week, my #1 goal is to get an actual working ASP.NET implementation on top of my FastCGI library. To do this, I plan to separate the FastCGI implementation into its own library and implement three classes on top of it: FastCgiApplicationHost, FastCgiWorkerRequest, and FastCgiWebSource. ApplicationServer may or may not be used as the server, depending on whether my AbstractSocket (below) works out. If ApplicationServer is not used as the server, it will still be used to manage application paths. After all that, a driver class similar to XSP’s server.cs should be all that I need.

As for the socket issue I faced last week, I intend to create a base class AbstractSocket with two subclasses, ManagedSocket which will wrap around System.Net.Sockets.Socket, and UnmanagedSocket which will work with the existing socket by P/Invoking libc. All operations will then be done via AbstractSocket. This will require research on how to accomplish a similar effect on other operating systems.

CHALLENGES I’M FACING

The biggest challenge this week has been sorting through everything. There currently isn’t any documentation on the library and there are very few source code comments.

RESOURCES USED THIS WEEK

  • MSDN - Looking up information on various classes and methods.
  • FastCGI Specification - Obvious reasons.
  • XSP Source Code - To see what everything does.
  • Running XSP with tracing enabled - This was extremely helpful because it not only described the path of operation but showed what arguments were passed. This was very helpful for cryptic identifiers. (For example, seeing that ‘verb’ == “GET” really clarified things.)

Sincerely,
Brian Nickel

Posted in Programming, Summer of Code 2007 | No Comments »

My Monodoc shame script…

Posted by Brian on June 6, 2007

Possibly the single most important thing in ensuring the use of your library and the happiness of those using it is documentation. It is also a great way to ensure your API is up to snuff before you declare it “stable” ’cause you have to review every class and member.

As such, I’ve written a little shell script to pinpoint where you have work to do: shaminator.bash.

Simply run “shaminator.bash path/to/docs” and watch the fun results:

~/Programming/taglib-sharp> shaminator.bash docs/en/
1       ./TagLib.Mpeg/VideoHeader.xml
2       ./TagLib.Mpeg/XingHeader.xml
1       ./TagLib.Mpeg/Marker.xml
...
1       ./TagLib.Flac/BlockType.xml
10      ./TagLib.Flac/Metadata.xml
6       ./TagLib.Flac/BlockHeader.xml

Namespaces:            14
Visible Types:         153
   Classes:            89
   Abstract Classes:   9
   Static Classes:     7
   Structures:         18
   Interfaces:         5
   Enumerations:       20
   Delegates:          3
Visible Members:       1273
   Constructors:       266
   Methods:            398
   Properties:         427
   Fields:             182

Missing Documentation: 700

And once you’re finished, you’ll have some sweet statistics to show off.

Disclaimers: This program does not account for blank fields, AKA fields that have been cleared, and bases its assessment on the number of occurrences of “To be added.” in the text.

- Brian

Posted in Documentation, Programming | No Comments »

FastCGI ASP.NET Server Status Report 1

Posted by Brian on June 1, 2007

This is a status report for the week of May 28 - June 1. A lot of great things have been accomplished this week and one major roadblock has been hit.

THIS WEEK’S ACCOMPLISHMENTS

The greatest accomplishment has been the creation of a basic FastCGI server that can communicate with the Cherokee Web Sever via TCP sockets. A test page outputs a HTML document with the page’s FastCGI parameters (equivalent to the environment variables passed to a standard CGI document) and the POST data received.

You can test this for yourself by:

  1. Downloading and installing the Cherokee Web Server <http://www.cherokee-project.com/downloads>
  2. Configuring <file:///etc/cherokee/cherokee.conf> to include the following lines:
    vserver!default!extensions!aspx!handler = fcgi
    vserver!default!extensions!aspx!handler!balancer = round_robin
    vserver!default!extensions!aspx!handler!balancer!type = interpreter
    vserver!default!extensions!aspx!handler!balancer!local1!host = localhost:1234
    vserver!default!extensions!aspx!priority = 99999
  3. Check out my project via svn: <http://mono-soc-2007.googlecode.com/svn/trunk/brian/FastCgi>
  4. Use monodevelop <http://www.monodevelop.com/Main_Page> to the project.
  5. Run “mono FastCgi/bin/Debug/server.exe”
  6. Run “cherokee” as root.
  7. View <http://localhost/any-page.aspx> Feel free to make a page post to it to see the raw results.
  8. Press “Enter” to kill the FastCGI server and Ctrl-C to kill the web server.

Status Report 1

PLANS FOR THE COMING WEEK

This week I plan to finish the last bits of the FastCGI specification by handling FCGI_GET_VALUES, adding support for server options (max requests, max connections, multiplexing), and supporting the multiplexing of responses by threading the Responder class. Additionally, I plan on bonding my classes to Mono.WebServer classes. At current inspection, it appears I can modify FastCgi.Server to inherit Mono.WebServer.WebSource and FastCgi.Connection to Mono.WebServer.Worker. And of course, there’s cleaning and commenting my code.

ONE MAJOR OBSTACLE

You may have noticed that I opted for Cherokee rather than Apache in my example. In default configuration, which is the only one I can figure out for Apache’s mod_fcgid, the server creates a socket at STDIN_FILENO which the FastCGI server is supposed to call accept() on. Unfortunately, I cannot find a mechanism within the Mono framework for accessing an unmanaged socket. The main problem is that the socket is already bound and the Apache server is waiting for it to accept() when the application begins, so the only action available would be to deal strictly with unmanaged sockets.

The .NET 2.0 framework, does however provide a structure called SocketInformation for transferring a socket from one process to another. It is not yet implemented in Mono (I think because Linux might not have a DuplicateAndClose option for sockets), but it may provide a workable solution.

As it stands, this problem is standing in the way of support for the Apache server and will need to be resolved in some way.

RESOURCES USED THIS WEEK

  • MSDN - Looking up information on various classes and methods.
  • FastCGI Specification - Obvious reasons.
  • In Dealing with the socket problem:
    • Manpages - Looking up information on socket access.
    • FastCGI C++ and Java Implementations - To study other methods of dealing with the socket.
    • <sys/socket.h> - Getting information on the socketaddr structure.
    • http://blogs.msdn.com/malarch/archive/2005/12/26/507461.aspx - Transferring sockets between managed and unmanaged processes.

Sincerely,
Brian Nickel

Posted in Programming, Summer of Code 2007 | 1 Comment »