2009 brought a major change to the forums at JavaRanch: we mooooved our old UBB forums to fresh new JForum software. Check out what Jeanne Boyarsky, Ulf Dittmer, Ernest Friedman-Hill, Bear Bibeault and Amit Ghorpade reveal about behind-the-scenes action on moving day and during the whole year-long (!) process.
JavaRanch: Jeanne, of all the people involved in this effort, you're probably the one who's seen the inner workings of JForum most up close and personal. What's your impression? Has anything made you scratch your head and wonder why they did it that way?
Jeanne: I'm happy with JForum. It was a stable base that we could build on. There's always going to be moments when you look at someone else's code and wonder "what were they thinking." For me, that came from the size of some of the methods. For example, the "insert save" method for posts was 301 lines originally. Not to say that I refactored it to make things any better. I tried to change "as little as possible." And that the unit tests that did come out of the box didn't all pass. The JForum forums say this is known so we deleted the tests altogether in our copy.
JR: Was there anything in the code that made you say, oh cool, I like the way they did that!
Jeanne: I really like that all the SQL is in one property file with overrides for database specific ones.
JR: Ooh, we like it when you talk like that... Other cool things about JForum? Or other things that got y'all scratching your heads?
Bear: I really like the way that code blocks are styled, with the line numbers and all. And the fact that you can actually enter "onclick" without resorting to HTML entities.
With regard to head scratching, I wonder why they resorted to Freemarker templates over pure JSP. Nothing against Freemarker (and other templating engines) but with the advent of JSP 2.0, where JSPs are easy to use as pure templates, I just don't think they're necessary anymore.
Ernest: Overall the design is awesome and clean. As far as the choice of Freemarker, I understand that JForum 3, now under development, is moving to JSPs, so they've seen the light there. In any case, Freemarker doesn't let you embed Java scriptlets, so there is still a nice separation between presentation and business logic.
Bear: True, dat.
JR: Speaking of presentation, Ernest, you dove in and did a lot of serious modifications to JForum's look and feel. Are you a touchy feely kind of guy?
Ernest: Hmmm. Maybe. I really did it because I thought that the sense many of us have of the Saloon being a tangible place is strongly tied to its look and feel. I wanted to make sure that when we launched, it would seem like the same place. For those following along at home, you need to know that we used JForum to host our "Moderator's Only" forum for a month or two before we went live with the rest of the Saloon. At that time, it looked a lot more like JForum's own native look and feel. My sense was that the staff didn't feel at ease; it felt like we were all sitting around in the lobby of a strange hotel, rather than our old familiar digs. I was worried that the abrupt change would alienate the Ranchers at large. So I did what I could to make our software look like UBB. Some people thought I was nuts, but I think it has worked out for the best.
As time goes on, I think we'll move away from this look and try to bring things into the 21st century, but for the moment, I am happy that it feels like the same old place.
JR: Bear, you did a bit of sprucing up around the forum too, in lovely chocolate tones. Can you tell us more about those yummy chocolate buttons? As a self-respecting chocolatier, you probably mixed up a quick batch of 70% Ecuadorian, yes? Do you reveal your recipes?
Bear: Whoa, whoa! One question at a time! I have a short attention span.
Bear: Oh, yeah. Sorry about that. The chocolate-y buttons actually have a fairly rich history.
Those who have been around the Ranch for some time may remember that way back when the buttons were oval, and aqua, and used a serif font that didn't match anything else on the rest of the site.
At one point, someone mentioned that it'd be nice to replace them with something that matched the rest of the site better. Paul nixed the idea, but before I saw that, I had whipped up some buttons to replace the aqua ones. Naturally, I chose browns to match the Ranch-y colors, and used a sans serif font that didn't clash with the text of the site.
When I saw that Paul wasn't keen on replacing the existing buttons, I almost discarded them. But, I went ahead and posted them in a reply just to show what I had come up with.
When Paul saw the new buttons, he did an about face and said something along the lines of "I want those!". I've always thought of that as a supreme compliment.
When the time came to create buttons for the JForum version, I figured I could use the same theme and volunteered. The new buttons are a cross between the default JForum buttons, and the brown, oval buttons I had created for the "old Ranch".
JR: Were they hard to make?
Bear: Not really, but the process would probably be of little interest to anyone but Photoshop geeks. An alpha channel here, some layer blending there, and liberal use of layers, and there you have them!
Now that the "recipes" are set up, it's pretty easy to generate new buttons as needed.
JR: Ooooh recipes! Do you share?
Bear: When it comes to real recipes, I share pretty liberally. I'm actually not super-big on sweets, so dark chocolate appeals to me more than something sweeter like milk chocolate. Add some bitter orange flavor, or better yet, chiles, and I'm a happy bear.
Bear: I like it hot! Want my recipe for 7-Chile Jelly?
JR: Hmmm, think I prefer my jelly sans chiles, thanks anyway. Ulf, just about a year ago, you actually did the first installation of JForum. Assuming you weren't under the influence of 7-Chile Jelly, what possessed you to do that?
Ulf: A few months earlier I had downloaded and installed JForum for a project related to my day job. So when the JR moderators discussed what to do about the forum software -improve UBB, switch to something else or write our own- JForum was mentioned along with questions like "what can it do", "what's its source code like" etc. Since I knew what was involved in installing it -quite easy really: run a DB script and get a web app running-, I put it on a test server, so that people could play around with it. Not for a moment did I imagine that for two of us -Jeanne and Pauline- this would be the starting point for pushing the migration project forward; it was meant as a useful contribution to a discussion, nothing more.
JR: There have been numerous valiant attempts to update our old UBB forums; this is the only one that really made it through. When did you really, really believe that this time javaranch really would move to a new forum? Why?
Ulf: I can't put a precise date on that, so let's see ... the initial install was in January, and then some quick progress was made. I think by June it seemed to have fizzled out. There were also some bad vibes amongst moderators around that time; those certainly dampened my enthusiasm. On top of that, it was summer, so I tried to spend more time away from computers... So I lost track of progress for a while, and was pleasantly surprised to discover that Jeanne had soldiered on, and completed a good deal of the ToDo list items. So when she started talking about what might be involved in actually switching to JForum, I figured I had better get cracking on the items I had signed up for. According to the Subversion history I started committing code in earnest in early November; so I'd say by that time I believed it would happen.
Given the time frame, maybe we should have had a tag line like "JavaRanch on JForum - Change you can believe in" :-)
JR: Yes we can!! Sorry... couldn't help myself...
Jeanne: My article in this newsletter 'The Great Forum Migration Project' sketches out the timeline for the entire project. There were two milestones that really jumped out at me. The first was in March 2008. We did enough proof of concept work to feel comfortable that we could extend JForum and that it was stable. This was when we got buy-in from Paul Wheaton - the site owner and trailboss - that this was going to be THE attempt that we were going to support and that we wouldn't let scope creep mess things up. We all agreed that the primary goal was to get JavaRanch up on JForum without losing data and everything else was secondary. If that meant going without certain features for a while, so be it. To me, that meant the project would eventually succeed and it was just a question of when. The second milestone was November 29, 2008 when we cut over the Moderators Only forum. While we were ready for some time before, that day represented the "point of no return." It was the first time we couldn't go back to UBB without losing any data. The feeling that weekend was awesome because it felt real!
JR: So people were busy testing an installation of JForum for quite a while. Amit, you were one of the first official testers. In fact, it is rumored that you had very, very close contact with Jeanne during the migration from UBB to JForum. Can you tell us a little about that (without getting either of you in trouble)? ;-)
Amit: Maybe yes, for two reasons: First Jeanne's patience, where others put me to their ignore list for reasons below, Jeanne showed the courage to reply to my idiocy each and every time. Second I learned somehow that Jeanne will be one of those getting more perks for the new software, so I thought she would recommend my name. Okay can you replace the second reason with something that safeguards my honesty, like she was my reporting person.
JR: OK, we'll put it on the wish list. What was your favorite bug report?
Amit: Actually I thought all of my bugs would turn out to be nightmares for the implementors, but sadly none did. Still my most favorite bug was the "Service Unavailable bug" which I reported when server maintainance was going on.
JR: Is it true that you reported duplicate bugs to increase your post count?
Amit: That is supposed to be a trade secret, how come you know this? (Note to self : Don't let out your trade secrets, already lost two.)
JR: Forget the bugs, let's talk features! What's your favorite JForum feature?
Amit: Extended bartender's powers! Err, it's the preview button.
Jeanne: Sticky thread for book promos, quick reply, code tags
JR: OK Jeanne, since it's you, we'll give you three favorites. ;-)
Ernest: How easy it is to modify and customize. What else would an inveterate hacker say?
Ulf: My favorite feature is its code base. I really want to give credit to Rafael Steil (JForum's main developer) for this. It's well-organized, has useful abstractions with a well-implemented MVC separation, and many parameters and switches one can tinker with. And it's written in Java, so all of us can contribute: looking at the repository history, 10 moderators have worked on it so far!
Thinking back to UBB (which is written in Perl) where any code modification but the most trivial was a gamble, and not even those moderators who did dare tinker with it (primarily Jim and Ernest) could always predict what might happen as the result of a code change, it's a difference like night and day.
JR: Night and day, indeed. And how does this night time migration project differ from your day job?
Ulf: Not so much, actually. I've worked on web-based Java projects for more than 10 years, so I'm quite familiar with the issues that can arise. Of course, here we're all volunteers working in our spare time, so sometimes people can't contribute as much as they thought they'd be able to, or have to push things back because real life interferes. So any kind of deadline generally doesn't work out well, which is why we have always responded vaguely to any public requests for more features in the forum software, until the very minute we were ready to announce it.
But I enjoy doing the occasional actual programming. In my day job I'm in leadership positions where I don't get to do much of that any more.
Ernest: Unlike most folks, I mostly do desktop software development at work, so working on Web technologies is a nice change of pace for me. Hacking on the Ranch is a great way for me to keep those skills sharp, and the Ranch staff are a great bunch of teachers.
Jeanne: Where to begin. I work for a bank in New York City. There's a lot more structure and bureaucracy. In some ways, it's nice to be able to deploy to production without getting five approvals and sign off from a bunch of people. In other ways, it's dangerous that a change can get into production without being properly checked. It's also a different dynamic because almost everyone here at the ranch is a techie. All the roles had to be filled on this project. I must say that I really appreciate what our project managers, operations/deployment/server folks and test group do! Another aspect was having less confidence that things would go smoothly. I took January 5th off from work in case there were problems on our first business day on the new software. And I was sincerely worried that something big would go wrong. Luckily that didn't happen. By contrast, I'm never worried about this at my day job. There's enough process in place to KNOW rather than HOPE that things work as they should.
JR: New York City? Do they have little moose bags in Central Park, you know the kind you use when you take your moose on a walk?
Jeanne: Unlike the guy with a pet tiger, I recognize keeping a real moose in a New York City apartment wouldn't go well. So mine are stuffed. I have a girl stuffed animal moose on my desk at home and a boy one at work. (Build a Bear says it a Moose and not a reindeer.) And my moose are easy to clean up after!
JR: You certainly did a clean job on the migration. Given the long process, you must have faced plenty of decisions. What was the toughest decision you had to make along the way?
Jeanne: When we deviated too much to upgrade when JForum itself does. That was a big deal because it means that we now own our fork of JForum. Upgrading again would be another monumental effort and from what history shows is unlikely to happen. On the bright side, we now have the skills and expertise to add what we need to our installation.
JR: On January 3rd, 2009 when most people were still nursing leftover holiday hangovers, Ernest, Ulf and Jeanne were busy doing the The Big Move. Can you guys give us an idea of what that day was like for you?
Ernest: We did the migration on a Saturday morning and my kids were home. Despite repeatedly telling them "Shh, Daddy's working!" they wouldn't really leave me alone, so at one point I was hacking Perl code with a preschooler on my lap. I guess he probably could have taken out the whole Ranch with a few spectacularly mistimed keyboard bashes. Didn't happen, though.
I might be the only team member to feel a little sad and wistful about getting rid of UBB. It was ultimately up to me to "throw the switch" that directed all our traffic to the new site, and immediately after I did it, I felt an instant of "What have I done!" panic. And then I couldn't believe we had finally gotten to that point.
It was hard work but a lot of fun. It's always a great experience to tackle a project with smart, talented people who enjoy what they're doing, and that was definitely the case here. There were a couple of speed bumps, but due to some impressive speed-debugging, we got over them very quickly. Jeanne and Ulf really know the code well.
Jeanne: The migration was LONG. We predicted 3 hours and it went 5. I felt bad that it got to 8pm Ulf's time. It was also more intense than I expected because we were concentrating pretty much the whole time.
It was cool working on a project where everyone is a techie. Ernest and Ulf were awesome to work with!
Ulf: This was by far the longest chat session I've ever been engaged in. It was also a bit different from previous ones since I have never met Jeanne or Ernest, nor talked with them on the phone, so I have no voices to connect with them.
In a way it's amazing how far the internet has come. You're looking at a web page in the browser, looking at the server that serves that page in an SSH session, talk about what you're seeing via chat, and trade emails back and forth about it all at the same time. This all works in near real time nowadays. None of us were within 2000 kilometers of the servers, and in my case it was more like 8000+ km.
It did run longer than we hoped for, but it's been my experience that initial rollouts of sizeable applications generally proceed less smoothly than predicted. Although our test system ran well for a while, it had been set up and tuned over a period of months. It can easily happen that some required step doesn't get recorded for the "live" rollout, or that things work slightly different on the live system than they do on the test server. So I wasn't surprised by that.
The next week was a bit of a scramble, though. The software had a number of bugs that we hadn't encountered during testing - having a large member community sure helps to trigger those quickly! But by now we're at a point where the log files are pretty clean of unhandled exceptions, so we can start thinking more long-term.
Jeanne: Want a behind-the-scenes peek at how much fun we had? Check out these live chats from different points along the way:
While Ulf was troubleshooting a problem during our Moderators Only migration Thanksgiving weekend:
Jeanne: aww. gmail handles smilies cute! [they are animated]
Ernest: Hey, guys, I've been making waffles this whole time. The waffles are done now. Pull up a chair!
Jeanne: mmm. imaginary waffles!
Ernest: OK, I will be eating wafles. I'll check back in a few minutes...
Kicking off the migration:
Jeanne: It seems like we should say something ranch appropriate before
kicking this off. I'm not a westerner, so I'll go with something NYC
based (Macy's parade) - 3! 2! 1! Let's have a migration!
Jeanne: well that's western :)
Ulf: I'll let the experts decide :-)
Ernest: Wait, I am going to get my hat...
Ernest: OK, seriously, I am wearing a cowboy hat [see Ernest's profile for picture]
Near the end:
Ernest: The saloon is dead; long live the saloon!