jeffcoughlin.com

Running ColdFusion Solr on a 64-bit JVM

One of my clients had the need for a lot of search collections and even more data to put in them (especially with massive archives of PDF files, xls, word docs, etc) for a very large intranet. So I tried to plan ahead by installing Solr on a separate server (Windows 2008 R2 64-bit with a decent chunk of RAM). You can find the separate installer on Adobe's website. For our initial tests it worked out great, but when we starting throwing more data at it, it quickly ran out of memory.

The first thing I did before I even created the first collection was to increase the memory for the JVM (to do this, edit the solr.lax file and change the default setting of -Xmx256m to your preferred max memory setting. I also prefer to set the min setting as well to the same value (see below)). What I quickly found was the the default version of the Solr install comes with a 32-bit version of Jetty. The 32-bit JVM maxes out at 1.5GB of RAM use. So even though we had enough RAM to use and were running on a 64-bit OS, the JVM wasn't being allowed to use the extra memory.

The solution ended up being very simple (even for a non-java person like myself). I downloaded a 64-bit JVM (in my case I just downloaded Sun's 64-bit JDK to test. I suggest the JRE though) and installed it on the box. I edited the solr.lax file and commented out the single line where Jetty calls javaw.exe (for Windows its that file, for other OS's it will be sloght different with a different path format) and instead pointed it to the new 64-bit JVM's javaw.exe file.

I then set the min and max memory settings to 3GB (-Xms3072m -Xmx3072m) and started Solr up. No more out of memory errors (no more java heap errors).

---

I didn't figure it out all on my own though. Several people helped me come to the final result that I'd like to thank (in order of me bugging them): Mark Mandel, Matt Woodward, and Sean Coyne

Comments (Comment Moderation is enabled. Your comment will not appear until approved.)
Julian Halliwell's Gravatar Jeff, you should check out Sami Hoda's blog where you'll find details of other useful Solr customisations and tips, including upgrading to the latest version, which is surprisingly easy.

http://www.bytestopshere.com/post.cfm/lessons-lear...
#1 by Julian Halliwell | 10/14/10 3:07 AM
Jeff Coughlin's Gravatar Thanks Julian. That's pretty cool to know (that I can easily upgrade Solr and possibly get better results and bug fixes from it).

I might just do that.
#2 by Jeff Coughlin | 10/14/10 3:41 AM
Sami Hoda's Gravatar Jeff,

I strongly suggest you read my series on upgrading. It came out to be an almost MUST for me.

Julian - thanks for plugging me!
#3 by Sami Hoda | 10/17/10 7:21 PM
Jeff Coughlin's Gravatar Yes. After Julian posted a link to the series I went through it. Thanks so much for the work (and for anyone who hasn't seen it, please check it out).

Sami, I hope you don't mind if I ask the Adobe engineers to take a look at your blog post. They are working very hard to transition Solr into ColdFusion and still make it very easy for people to use. Although most people probably don't realize it, CF 9.0.1 had massive fixes and tweaks behind the scenes for better Solr integration. So if/when they don't get something right, don't think its not because they aren't trying. It's just a new change after using Verity all these years.

I've seen you posting CF9 bugs in the past (during the beta), so please keep them up if you have suggestions for thing like Solr :)
#4 by Jeff Coughlin | 10/18/10 9:48 AM
Sami Hoda's Gravatar Jeff,

Adobe Engineers already know about my blog posts. In fact, as the series mentions, a lot of the work was done in coordination with folks in their teams. But they need reminders, so feel free to ping them again.
#5 by Sami Hoda | 10/18/10 11:51 AM
Jeff Coughlin's Gravatar Ahh cool. Yeah, maybe we can convince them to use the Solr 1.5 pre-release rather than the 1.4.1 pre-release :)
#6 by Jeff Coughlin | 10/18/10 12:21 PM



BlogCFC 5.9.8.007 by Raymond Camden | RSS | Contact Blog Owner