Reply to comment

Navigation

User login

GWT in Practice at Manning
GWT in Practice at Amazon
GWT Resources

Quote

Edward Murrow

We must remember always that accusation is not proof, and that conviction depends upon evidence and due process of law. We will not walk in fear - one, of another. We will not be driven by fear into an age of un-reason, if we dig deep in our history and our doctrine, and remember that we are not descended from fearful men; Not from men who feared to write, to speak, to associate, and to defend causes that were - for the moment - unpopular.

Lucene log4j appender

Fri, 04/21/2006 - 12:00 — charlie.collins

I wrote a little side project the other day which does various things around log4j and Lucene. It creates indexes from existing log files via a utility, allows projects to log to a lucene index via a custom appender for said task - and it has a web interface for searching the index (which uses the idiom of two steps where step one you search with a query and get a lits of "hits" back and step two you give it a time frame and style around a particular hit and it returns a snapshot of the log based on that criteria).

The point of this was to be able to search logfiles from a web application interface (for use by those that do not have access to the actual log, and or dont care to use grep/sed/awk, etc).

I used Chainsaw and a few other things (which are great for what they do) but wanted something a bit different (something that can access all the history of multiple logs over a specified time period without sitting there listening on its own port, etc) so I wrote this stuff.

The aspect that I think might be most interesting to others would be the appender itself, hence here it is (no merchantability of fitness, warranty, etc, use at own risk, may completely suck - blah blah blah). This should be viewed as an EXAMPLE of how it might be done and where to get started (this has not yet been extensively tested).

LuceneIndexAppender

UPDATE
I have not yet updated the code provided here, but I real life I ended up spinning the flush buffer off into another thread, and creating DAILY indexes - both to improve performance. (When I get a chance I will update the example here, the concept though is the important part, flush in another thread and make daily indexes.)