Screaming Penguin - Source Does Matter

I wanted to really see request and response data as I browsed around a site I had recently worked with. It was a java based site with JSP and servlets, but thats really irrelevant to the resulting "weight" of the pages.

So I got the REQUEST information very easily using Java, in fact I just used my old JWebUtil from a few years back. Its crude, and it could be vastly improved, but it works. Using that I could see all the reqest data, headers, parameters, etc and I could also see everything is the session.

From there I wanted to see all of the RESPONSE data. That gets trickier. Specifically what I wanted was the time the request was done and the response was started, and the time the response was done and sent to the client. And I wanted the resulting size of the response. I didnt want to just see the size of the web page at the client end, but the size of the response (in theory this should be the same thing, or very close, but thats why I wanted to see and compare it). I tried using a few things to print the response data and never really got what I wanted. Never got the timing data right, etc.

I decided the only way to REALLY do this was to use a client that could do it (I really wish GUI browsers such as Mozilla/IE had a view response headers feature, the way Lynx -mime_headers does). Then I wanted more so I decided I would write a proxy server that would get the request, print whatever data about it I wanted, then send the request to the actual dest server while keeping track of timing, then get the response back and print whatever data I wanted about it and finish the timing stuff and then forward the response on to the actual client.

Sounded good. I actually got a proxy server written using Java that listened on a specified port and received and forwarded on the request using HttpClient, but getting the response back and forwarding it on looked like it was going to take more time that I had available.

Hence Google and the resultant WebDebug to the rescue.

Of course someone has already done this, WebDebug is just that, a proxy server that prints all the data you could ever want to know about the requests and responses you make with your normal browser (once you set it to use the proxy server).

The formatted output is fantastic and it does everything I wanted it to do. WebDebug is something EVERY web developer, front or back end or whatever, should have. It allows you to see real timing information that can help with making efficient front and back end pages.

It runs Python, but dont panic if you dont have Python, its painless to install and its a beautiful high level language because its so simple yet so powerful. Just hit the python site and install it (even winders executables) and then download WebDebug and simply click on the webdebug.py file. Then your running. To see what its all about check out the ouput of Yahoos front page in WebDebug format.

Also note that I found a few other great Java proxy servers in my searching. The best IMHO was Muffin. It has some debug info but is not really directed at doing debug but rather its a full featured proxy that allows you to "filter" whatever you want out. Filter pop ups, filter ad servers, filter cookies, etc. You can write custom filters to do whatever you want and they plug right in, good stuff.