OSGi: YAGNI

Tagged:
So Eric Burke and I have been (somewhat) bashing OSGi on Twitter today. I have picked this fight with the OSGi people about a hundred times now, but I feel compelled to actually rant about it. There are generally 3 models for OSGi on the server side: First, you bundle your WAR file effectively as an OSGi module, you install it into a running OSGi container, then start Pax, or something else that has a servlet container in it. There are better versions of this tact, like using Spring DM, which makes some of the day to day stuff easier. This is all well and good, unless you want support from your Operations group. This amount of black magic and non-"This is my server" weirdness generally pisses off your IT group. Moreover, using embedded servlet containers, you lose a lot of the "real" server configuration you get with a real app server. Second, and this is what I have been working on, is you have your servlet containers that are standardized and you want to run OSGi underneath a plain webapp context. This is doable, but results in much gnashing of teeth, and honestly, I am not sure how much it buys you. Looking at what I have now, a GWT RPC layer that bootstraps as an OSGi web app, then proxies calls into another OSGi module that goes to the "Enterprise" services. This lets me, let's say, upgrade the Enterprise service layer without altering the GWT webapp. OK. Let's put bookmark here and come back to it. Third, you can use something like Glassfish which now has built in OSGi support -- as I understand it, never used it. This sounds like it might be a pretty good way to do this, but, aah. At that point, who really cares? What level of "modularity" am I getting from OSGi that a full EE server isn't already giving me? If someone can tell me why an OSGi bundle is so much better than good old EAR files, I am willing to listen. Or you could use something like Spring DM which is basically a whole new container. Now you have just replaced EE with OSGi. So, aah. Yeah. it is an OSGi app instead of an EE app. Now, let's talk about reality: First, OSGi hot deployment works every time, 60% of the time. It is WAY too often that multi-module dependencies get a class reference that won't die and you end up having to bounce the whole server anyway to get them to refresh. Oh, and did I mention you can't actually tell that this has happened until it fails? EE, or heaven forbid, something web service-y work a lot more reliably for this stuff. And again, to be an ass about it, if OSGi hot deploy works so damned well, why does Eclipse tell me to restart every time I install a new plug in? Secondly, baring a minor bug fix, any real change to the Enterprise service layer is likely going to warrant a change to the webapp module anyway. At this point, I might as well just build a new WAR and redeploy it, because I have to dump the webapp anyway. Thirdly, and this is still the one that gets me every time, WARs and EARs have very clear, mostly monolithic artifacts. They can be moved from build to QA to stage to prod in one fell swoop, carrying with them all they need. Doing what I would call "fractional" deployment to multiple servers seems like a great way to introduce unexpected errors, or misconfigurations. None of the OSGi containers (that I have seen) have a concept of a transactional deployment. I said this on Twitter and I will say it now: OSGi is EJB 2.1. If you really need it, it can do some things really well. For the most part, though, it just adds complexity that is mostly needless to 95% of projects. Maybe JAM/Jigsaw/Whatever will be the EJB 3 of OSGis. I don't know. I haven't followed it that closely, but for right now, OSGi is solving problems I don't have.

Comments

Some misconceptions ...

Eclipse asks you to restart not because the OSGi layer can't handle it, but because plugins from pre 3.0 days weren't dynamic. And thanks to backward compatibility, some of those can still run (which is what the story aimed for). In addition, there is undoubtably code which e.g. caches the extensions instead of iterating through them each time; case in point, the CVS team provider. So Eclipse gives you the option, defaulting to the safest way. Calling OSGi EJB 2.1 shows a heap of ignorance, though. EJB 2.x forces you to depend on code, both interfaces and classes, in your domain logic. You don't need to depend on any OSGi APIs in your code if you don't want to. EJB servers are about as heavyweight as they come; meanwhile, the Concierge OSGi engine is circa 80k in its entirity. Lastly, the spring dm server allows you to deploy sets of bundles transactionally (called a plan) and OSGi 4.2 has some provisional work on "composite bundles" which are similar to your beloved EAR. Alex

OSGi reality check

The reality is operations will always apply a change by restarting the server process. They have very little trust in the software and most importantly they want to be sure that they server process will actually restart in the event of a crash (following the deployment). No one wants to find out you can restart the process when a server has crashed because you simply did a hot deployment. What is also worth nothing is that it is impossible to test such an OSGi beast as you would need to mimic every hot-deployment that was made to production in your test environment to be truly sure you have done some degree of testing of the software software and its actual deployment. Remember OSGi only attempts (and fails in my opinion with complexity) to solve the code aspect but not necessarily the transient state (object, system, process, files) dependencies that exist in any real-world application outside of a "Hello OSGi" example. William

Hey - no one is forcing you to use it

If you don't like OSGi, stay away from it. No problem. If you don't have a problem that it solves, then don't use it. This sounds like common sense. Yet it seems that what you have said is that you did not have a problem that OSGi solves and decided to use it anyway. As has been pointed out, it's important not to confuse Eclipse with OSGi. Although it's built on it, using Eclipse is not the same as using OSGi directly (for several reasons). And yes, the whole application lifecycle issue hasn't been addressed - at least not yet. We are working on it. If Java EE and EJB3 is working fine for you, no one is telling you to change. Please understand that. OSGi like any technology solves a certain class of problems. If you (or William) has a problem that OSGi isn't suited for, please don't use OSGi. And especially don't use it for something it isn't good for and then say it doesn't work! Eric

ps Glassfish has done a good job with OSGi

From what I heard from the GlassFish folks presenting at EclipseCon last month anyway, GlassFish uses OSGi extensively, and has done a good job of implementing the OSGi service oriented programming model. This is BTW what's most important about OSGi. But if you want to use the Java EE APIs instead for developing your applications, that's of course fine, but don't complain about OSGi because it doesn't work the same way - that's the point.

Actually, a little work

Actually, a little work later, I am torn between Spring DM and Glassfish. ORA.com might make that decision for me soon. Actually, my point here, though, is buzzword compliance. I am dealing with Day/Apache Sling(and therefor Felix) and trying to come up with a reasonable deployment model. The problem is OSGi doesn't have a deployment model that an operations team wants to deal with in its current form. I am spending more time with Spring DM because it seems closest to what we really want, however, this is driven *entirely* by Sling/Felix not what we actually need to deploy. Even with Spring DM we still aren't in the range of what I would want to see, but maybe closer. The problem is Sling. It is OSGi based, but only behind the webapp level. If it was shipped as a collection of OSGi modules, rather than a Web App with Felix behind it, that would be one thing. However, there is no good deployment model for that yet (see above), and now I have to deal with a mishmosh of other OSGi stuff, and WebWork stuff that I have to integrate with it. I can shoehorn the other stuff into OSGi, but it becomes more, not less complicated.

This article seems to be from another world

So, OSGI: YAGNI?... lets study the arguments: -You IT group gets pissed off because they do not get OSGi:
well, get them to learn, o get a new IT group, unless they have clear objective reasons to dislike OSGi other than feeling it is "black magic".
-What level of "modularity" am I getting from OSGi that a full EE server isn't already giving me?
Well, for example, during development, I hate to wait for everthing else and their family to reload because I made a change to my code. If I only changed my code, only my code should reload, not all those .jars in WEB-INF\lib... if you do not think that wastes time, please do share with me how do you solve that... do you use JavaRebel?
-If someone can tell me why an OSGi bundle is so much better than good old EAR files, I am willing to listen:
Well, EARs are monolithic artifacts, they are not agile, they are specially cumbersome during development, because you can not modify them partially, you have to wait for you gigantic EAR file to reload everything that is on it, even if you only made modification to you application code. And dont tell me you solve that by adding all those WEB-INF\lib .jars in to the "share lib" of you application server, because chances are that your app server fails to do that correctly if you have many applications with different versions installed thanks to classpath hell.
EE, or heaven forbid, something web service-y work a lot more reliably for this stuff:
It is more like "the JEE spec does not really have hot deployment". But maybe I am wrong, maybe you are using a EE server with non-Osgi, half-build, incomplete, Osgi-wannabe module management, good for you, but I would prefer it all applications server use the same standar ( OSGi or another that does the same thing, if you can convince those JavaRebel guys to give away their product for free, please do it) so that I can port my application between application servers without too much complication.
Secondly, baring a minor bug fix, any real change to the Enterprise service layer is likely going to warrant a change to the webapp module anyway:
Yes, build a new WAR and redeploy it, please continue to waste your time as much as you like to do that, I prefer to minimize time wasted by only reloading my changes, thank you.
They can be moved from build to QA to stage to prod in one fell swoop, carrying with them all they need. Doing what I would call "fractional" deployment to multiple servers seems like a great way to introduce unexpected errors, or misconfigurations:
Yes, monolithic deployment is great in theory, in pratice, it is a PITA, wanna guess why those PHP or Ruby (or even ASP.NET) guys thing JEE is not really agile? because when they need to fix something fast, they just drag & drop the fix and it is instantly available to the user, but use JEE guys no, we love having to reload everything, just to fix a little tiny thing. Well, I for one am just tired of that. If a change is small, it should be easy to upload in to production. I agree that monolithic deployment has its advantages, but a lot of time it just unnecessarily complicates things.
I haven't followed it that closely, but for right now, OSGi is solving problems I don't have:
So you do not suffer with classpath hell, you are happy with monolithic modules, love to have static updates with full application reboots, etc... well, I sure like to work doing the kind of simple things you do that are not affected by all this problems, but out here, in the real world, OSGi looks like it is going to make things a lot easier for those of us that actually build complex stuff.

Other world?

I don't really have a dog in this fight, but I think the heart of this matter, for me is that I don't want a partial update/change in my production environment because I don't trust it. For that alone I would have to agree with at least part of Cooper's rant. I want to deploy and install EXACTLY what the build produces, and what gets tested, etc. Hot deploying one component is not what I want. Dynamic business rules, or content, or such, sure, but that can be handled 1000 other ways - the runtime CODE, the logic, etc, I don't want you monkeying with unless the entire artifact and system has been tested together - don't deploy part of it, no thanks. I would also say that that is the same reason any good IT sysadmin/dept doesn't want that. It's not that they can't understand it, or need to "learn." It's that they are smart and have had to troubleshoot complex systems before and they understand more moving parts complicates the hell out of it. Partial deployment might be convenient for developers (+1 for JavaRebel and OSGi on that front, but I wish my damn Eclipse updater would actually work, I digress), but that's exactly NOT what I want at runtime in a production system. Replacing the entire artifact isn't a waste of time, it's a smart way to make sure the entire build and test cycle is reproducible and predictable so as to make it "nice for the people," it's not about what's convenient for developers.

Is it better to reign in ClasspathHell, than to serve in OSGi ?

I don't really have a dog in this fight, but I think the heart of this matter, for me is that I don't want a partial update/change in my production environment because I don't trust it. For that alone I would have to agree with at least part of Cooper's rant. I want to deploy and install EXACTLY what the build produces, and what gets tested, etc. Hot deploying one component is not what I want. Dynamic business rules, or content, or such, sure, but that can be handled 1000 other ways - the runtime CODE, the logic, etc, I don't want you monkeying with unless the entire artifact and system has been tested together - don't deploy part of it, no thanks.
Yes, I also know the nice warm feeling of "I just uploaded my huge EAR with everything inside, and I have a tag in my svn that exactly matches what it is in production. I just feels right... until you realize that your EAR weights 80 Mbytes because of all those nice .jar dependencies that made your work so easy because you didn't have to reinvent the way to generate PDF reports and Excel files and whateverelse. Wanting to upload all the EAR is an indicator of the very bad state of affairs that Java has forced in to us. For example: Why it is no standard to have something like a Diff engine in our build management tools (Ant-Ivy or Maven2) that makes it possible to "Diff" and EAR with the last one we uploaded in to production and only upload the changes? perhaps as some kind of binary patch? (a .PAR file perhaps?) that would save network bandwidth, time, space, etc. Databases have had incremental backup restoring for quite a long time, why are we forced to upload everything again if we only made a tiny little change?
I would also say that that is the same reason any good IT sysadmin/dept doesn't want that. It's not that they can't understand it, or need to "learn." It's that they are smart and have had to troubleshoot complex systems before and they understand more moving parts complicates the hell out of it.
Again this is because are low standards and suboptimal tools have made the monolithic way the only way, that does not mean it is the best and we should strive to keep it as the only way to do things. And even with the current inefficient deployment tools, not all applications require the same level of strictness on deployment, not all of them a mission critical... why force everybody to do monolithic deployment if it is not always needed? Why if it is so easy to fix a typo in a JSP file I have upload an 80Mbyte file to see my changes in production!
Partial deployment might be convenient for developers (+1 for JavaRebel and OSGi on that front, but I wish my damn Eclipse updater would actually work, I digress), but that's exactly NOT what I want at runtime in a production system.
Might be convenient? Common! go an tell any PHP or Ruby developer that you have to wait between 8 (Tomcat) and 30 (JBoss) and some times even more for you changes to be visible during development (while they at most need to wait 1 sec) and they will laugh at you. Really we Java guys are so used to things being super-slow that we don't realize that there is a world out there that works way faster than us.
Replacing the entire artifact isn't a waste of time, it's a smart way to make sure the entire build and test cycle is reproducible and predictable so as to make it "nice for the people," it's not about what's convenient for developers.
I agree with you, I really do, I like the fact that something like EARs exist, and that it is possible to do "single shot update it all to the same compatible version", but that should be OPTIONAL, I should be possible for me to choose to use it when I require to do so, not forced in to me even if I do not need it. And what about all those other problems OSGi solves? like the classpath hell? Are people from your world immune to them? Or have you just learn to love them? Is it really better to reign in ClasspathHell, than to serve in OSGi ? If so, you we really live in very different worlds

Fair enough, but I still am on the other side

I basically have a response to just about all of that, but I will just cut it back to one thing due to time (I am recompiling my 80MB EAR you see, and don't have time to respond to blog posts ;)): "Really we Java guys are so used to things being super-slow that we don't realize that there is a world out there that works way faster than us." No, Java isn't "slow" in this regard, it's COMPILED. Having a STATICALLY typed language actually has many advantages in my book (and several computer science axioms would agree). Sure, there are other advantages to dynamic and interpreted languages, but that is apples and oranges, it's not that Java is "slow" or PHP is "fast" its that they are entirely different things.

PHP 6 Ruby can be compiled: Can Java be interpreted?

I basically have a response to just about all of that, but I will just cut it back to one thing due to time (I am recompiling my 80MB EAR you see, and don't have time to respond to blog posts ;)):
LOL! Do as I do, I to respond to blog posts while I wait for my 80MB EAR to recompile. You will have enough time to even contribute an article or two for Wikipedia. ;-)
No, Java isn't "slow" in this regard, it's COMPILED. Having a STATICALLY typed language actually has many advantages in my book (and several computer science axioms would agree). Sure, there are other advantages to dynamic and interpreted languages, but that is apples and oranges, it's not that Java is "slow" or PHP is "fast" its that they are entirely different things.
Of course Java isn't slow, WE are slow (thanks to Java). Java being Compiled and statically typed language has NOTHING to do with it. Have you read about JavaRebel?. JavaRebel is the way Java should have been implemented right from the start. Java Rebel does not remove the compiled and statically typed nature of java, but it makes it as agila as if it were interpreted. IMO Java should come right-out-of-the-box with a "dynamic mode" providing exactly what JavaRebel provides and a "static mode" for optmized excution (when and only if needed). After all, PHP & Ruby can be compiled you know. (I do not like either of them, but their users have a key advantages, they do enjoy of interpretation (dynamic mode) and compilation (static mode) while we Java developers do not. Yes, there is Groovy, but I don want to have to switch all my code no another language, and sadly BeanShell is now an abandoned project (I guess that is because we Java developers do not realize that other plataforms are intrinsically way more agile, and we could be that agile too, but we blind ourselves by believing that "compiled and statically typed language" has to mean "inflexible time wasting language").

Interpreted Java

Interpreted Java is called Groovy ;). It's a layer on top of the statically typed compiled language, which is some regard is what other languages are really doing anyway. I didn't mean one is better than the other, I just meant they are different things. There are advantages and disadvantages to each, even in terms of making "us" slow.

I just read the end of your

I just read the end of your comment about Groovy AFTER I posted my other response. I was running in "dynamic" mode and hadn't parsed the entire comment before I responded ;). If we can have the best of both worlds, then sure, that would be great. That is a little of what we get from JavaRebel I suppose (I haven't read up on it, but have used it a few times - honestly I just don't know that I have that many "slow" spots when building something - I usually have things compiled into separate modules/points and am working on one focused thing at a time, and I publish the artifacts separately and they all refer to each other fine and roll along - maybe I am still in the stone ages, but it hasn't been a pain point for me - whereas dynamically applied patches and updates and such on servers have been).

Sad misconceptions between static and dynamic

What I find sad is that most people, believe is right for java to be cumbersome because it is "compiled" and "statically typed" like that mean that it was impossible to build a "statically typed" interpreter. Maybe the fact that most interpreters are dynamically typed, and most compilers are statically typed is the root cause of this myth... But as developers we should know better, and ask for tools that make our jobs comfortable. The slow development cycle of Java wasn't much of an issue to me either (I had only coded in Delphi, C#, Java and Objective-C back then, all of them "compiled-slow-to-deploy-languages"). But then I got in contact with PHP and Ruby... and while I disliked both of them, I was surprised by how easy and fast was for people using them to fix bugs or create new features, they are used to wait 1 or 2 seconds to see their changes, while I had to wait 8 to 30 seconds. I guess i do not even have to say that they disliked my slow development ways as much as I disliked the messy architecture of PHP and the lack of static typing in Ruby. I also have met other Java developers that couldnt understand why PHP guys think Java is not agile, in those cases, it was beause they coded without following MVC pattern, all their code was inside their.jsp files (not a single .java file in their projects, and maybe 1 or 2 jars in WEB-INF/lib for JDBC connection). At first they liked the idea of using MVC, and separating logic from presentation, until they realized that mean a much slower development cycle (they were also used to wait 1 or 2 secs to see their changes). So they reversed to their .jsp mess after a few weeks of suffering with the sluggish MVC pattern. (The sad thing is, again that it is not that MVC is slow, and it is not that java is slow, it is just that the way it is currently used makes it work slow)

Static v Dynamic misconceptions

There are indeed a million misconceptions about static v dynamic. Some of them sad, some of them understandable (it's complicated). Most people debate "static vs dynamic" (as I have done here, in general, just because of laziness and haste) when really they need to be more specific. Static types vs dynamic types, weak types vs strong types, compile typing vs runtime typing, and so on. Then you get to "interpreted" vs "compiled" and you have another set of overlapping and blurring boundaries with modern languages. It's a blurry landscape overall, but there are many good articles out there about it, still I wouldn't call it sad so much as confusing. I agree with you overall though, in the future I think more research might meld the worlds more. Yegge had a talk about this a while back that was very good: http://steve-yegge.blogspot.com/2008/05/dynamic-languages-strike-back.html. (Linked from this article, also very good - http://notes-on-haskell.blogspot.com/2008/05/static-vs-dynamic-languages....) In all I think this axiom from those links is more important than which language you are using, to be honest: "Google has some of the most brilliant people and language designers in the industry, but they use C++, Java, Python and JavaScript, and none of the esoteric languages the smart kids love. This is because it's more important to standardize on a set of technologies and get stuff done than let the geeks run wild and free. But you probably knew that already. ;-)" The key there for me though is "set of languages." Not one. Not "set of tools," I could digress into another debate here, but I will shut up for now ;).

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.