Running Solr 6.1 as a Windows Service

Goal

Get Solr 6.1 running as a Windows Service and have it recover on OutOfMemory Exceptions.

Install Solr 6.1 as a Service

1. Download NSSM
2. Extract nssm.exe somewhere
3. Create solr_start_6.1.0.cmd (this is a Windows Command file that does all of my configuration--yours will definitely be different):
   C:\apache\solr-6.1.0\bin\solr start -f -h node3 -c -p 8983 -z "zk1:2181,zk2:2181,zk3:2181" -m 128m -s C:\apache\solr-6-cores
   NOTE: The -f to run the script in the foreground.  I set the JVM heap size to 128 MB (we want this thing to crash and burn to test out OutOfMemoryException restart!).
4. Test your script to make sure it starts solr and you can access your Solr Admin UI in a web browser
5. Open a command window and navigate to your nssm.exe directory
6. nssm.exe install "Apache - Solr 6.1"
1. Application Tab
1. Path: Select your solr_start_6.1.0.cmd from earlier
2. Startup Directory: set it to the directory containing your script (should populate by default)
2. Details Tab
1. Display Name: Apache - Solr 6.1
2. Startup type: Automatic
3. Log on Tab
1. Make sure you specify an account that has administrator-level permissions (Use your account if you're stuck here--but make sure to set it to something production-worthy later)
4. I/O Tab
1. I/O Redirection
1. Output (stdout): Set this to something like path\to\my\solr\cmd\script\dir\solr-6.1.0-out.txt
2. Error (stderr): path\to\my\solr\cmd\script\dir\solr-6.1.0-error.txt
2. File rotation
1. Check Rotate files
2. Check Rotate while service is running
3. Restrict rotation to files bigger than: (use common sense here, I did 5 MB, so 5242880 went into the box)
5. Click Install Service
7. Open Component Services and select Apache - Solr 6.1
8. Start the service
9. Validate that it came up by going to your Admin UI webpage

Make Solr Service respond to Out Of Memory Exceptions

1. Navigate to this JIRA ticket
2. Download oom_win.cmd and place it in your solr\bin directory next to solr.cmd
3. Open solr.cmd in a text editor
4. Find all the places where the script starts the server:
1. Search for /solr_gc.log
5. Immediately after /solr_gc.log, paste the following:
   -XX:OnOutOfMemoryError="%SOLR_SERVER_DIR%\..\bin\oom_win.cmd %SOLR_PORT% !SOLR_LOGS_DIR!"
1. I had to replace two lines.  NOTE that this is just the manual way of applying the patch file associated with the JIRA ticket above.  If you want, apply it however you want.
6. Now that we've made our changes, go ahead and restart our new Solr 6.1 service so it knows to kill the process on OutOfMemory errors.
7. To force an OutOfMemoryError, query *:* and return 1000000 rows
1. If you have a decent amount of content, this should force an OutOfMemory exception.  If you don't have a lot of content, do whatever you can to make it do a lot of memory-intensive work.  Perhaps consider lowering the JVM memory, too.
2. You should see the web server go offline temporarily and then come back online
8. Now that you've seen it restart and come back online, let's give the JVM a good amount of RAM so that it doesn't run out of memory every other request.  Just edit your solr_start_6.1.0.cmd file and change the -m 128m to -m 4g (128 MB to 4 GB)
9. Save and restart the service
10. Confirm that you have the new amount of RAM for the JVM by visiting the Dashboard tab in the Admin UI

Logs

When the OutOfMemory Killer runs, it generates a log file in the normal log directory.  Navigate to that directory and you should see a file that looks something like: solr_oom_killer-9000-2016-07-06_13_59_39.  Now you can know when this script runs and hopefully anticipate it in the future or make changes to not get it.

Final step (important!)

Do a happy dance!