Wednesday, May 18, 2016

Compiling and Running Apache Solr 6 from Source on Windows

To get started, I'm going to quickly outline a bunch of little, specific steps to get us where we want to be.
  1. Download and install the Oracle Java SE JDK
    1. For Solr 6.0.0, you need at least Java SE 8
    2. I chose to install the Windows x64 product
    3. Make sure that java is accessible from the command line
      1. Open a new command window
      2. type: java -version
      3. You should see something like:
        java version "1.8.0_91"
        Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
        Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
  2. Download and install Apache Ant
    1. Download page
    2. I downloaded the .zip archive
    3. I chose to extract it to C:\Apache\ant-1.9.7
    4. Add Apache Ant's bin directory to your PATH environment variable (C:\Apache\ant-1.9.7\bin)
  3. Download Apache Solr source (6.0.0 in this case)
    I used this link to download the tgz source file (they didn't have a ZIP version of the source at this time)
  4. Since I'm on Windows, I needed to download 7-Zip (I used the 64-bit version)
  5. Extract the source (I will use D:\solr, which should generate a D:\solr\solr-6.0.0 folder)
    1. Open this folder in Windows Explorer
    2. Shift + Right-Click in the whitespace inside the folder and click Open command window here (or open a command window and cd to your new solr-6.0.0 folder)
    3. cd solr
    4. ant server
      At this point, Solr should be compiling everything that's necessary to run the server.
      You should see something like this:
      BUILD SUCCESSFUL
      Total time: 4 minutes 3 seconds

Now that we have something to work with, let's talk a bit about it.
We now have something that we can run--let's do that now.

Solr Cloud

Since most people are going crazy over Solr Cloud, let's use that example.
From the same command prompt you used to compile Solr:
bin\solr.cmd start -e cloud -noprompt
You should see something like:

Welcome to the SolrCloud example!
Starting up 2 Solr nodes for your example SolrCloud cluster.
Solr home directory D:\solr\solr-6.0.0\solr\example\cloud\node1\solr already exists.D:\solr\solr-6.0.0\solr\example\cloud\node2 already exists.
Starting up Solr on port 8983 using command:D:\solr\solr-6.0.0\solr\bin\solr.cmd start -cloud -p 8983 -s "D:\solr\solr-6.0.0\solr\example\cloud\node1\solr"
Waiting up to 30 to see Solr running on port 8983Started Solr server on port 8983. Happy searching!
Starting up Solr on port 7574 using command:D:\solr\solr-6.0.0\solr\bin\solr.cmd start -cloud -p 7574 -s "D:\solr\solr-6.0.0\solr\example\cloud\node2\solr" -z localhost:9983
Waiting up to 30 to see Solr running on port 7574Started Solr server on port 7574. Happy searching!
Connecting to ZooKeeper at localhost:9983 ...Uploading D:\solr\solr-6.0.0\solr\server\solr\configsets\data_driven_schema_configs\conf for config gettingstarted to ZooKeeper at localhost:9983
Creating new collection 'gettingstarted' using command:http://localhost:8983/solr/admin/collections?action=CREATE&name=gettingstarted&numShards=2&replicationFactor=2&maxShardsPerNode=2&collection.configName=gettingstarted
{  "responseHeader":{    "status":0,    "QTime":14712},  "success":{    "192.168.28.56:7574_solr":{      "responseHeader":{        "status":0,        "QTime":4758},      "core":"gettingstarted_shard1_replica1"},    "192.168.28.56:8983_solr":{      "responseHeader":{        "status":0,        "QTime":5443},      "core":"gettingstarted_shard1_replica2"}}}
Enabling auto soft-commits with maxTime 3 secs using the Config API
POSTing request to Config API: http://localhost:8983/solr/gettingstarted/config{"set-property":{"updateHandler.autoSoftCommit.maxTime":"3000"}}Successfully set-property updateHandler.autoSoftCommit.maxTime to 3000

SolrCloud example running, please visit: http://localhost:8983/solr
Hallelujah!  Let's check it out!  Copy the URL from the last line in the output and paste it in the browser.

Beautiful!

Now, we need to import some content--and for that, let's go back to our command prompt.
Solr ships with a bunch of test documents, so let's throw those guys at the server and see what happens.

java -Dc=gettingstarted -Dauto=yes -Drecursive=yes -jar example\exampledocs\post.jar example\exampledocs

You should see something like:
SimplePostTool version 5.0.0Posting files to [base] url http://localhost:8983/solr/gettingstarted/update...Entering auto mode. File endings considered are xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,logEntering recursive mode, max depth=999, delay=0sIndexing directory example\exampledocs (19 files, depth=0)POSTing file books.csv (text/csv) to [base]POSTing file books.json (application/json) to [base]/json/docsPOSTing file gb18030-example.xml (application/xml) to [base]POSTing file hd.xml (application/xml) to [base]POSTing file ipod_other.xml (application/xml) to [base]POSTing file ipod_video.xml (application/xml) to [base]POSTing file manufacturers.xml (application/xml) to [base]POSTing file mem.xml (application/xml) to [base]POSTing file money.xml (application/xml) to [base]POSTing file monitor.xml (application/xml) to [base]POSTing file monitor2.xml (application/xml) to [base]POSTing file more_books.jsonl (application/json) to [base]/json/docsPOSTing file mp500.xml (application/xml) to [base]POSTing file sample.html (text/html) to [base]/extractPOSTing file sd500.xml (application/xml) to [base]POSTing file solr-word.pdf (application/pdf) to [base]/extractPOSTing file solr.xml (application/xml) to [base]POSTing file utf8-example.xml (application/xml) to [base]POSTing file vidcard.xml (application/xml) to [base]19 files indexed.COMMITting Solr index changes to http://localhost:8983/solr/gettingstarted/update...Time spent: 0:00:10.135
Now that we have something to search for, let's search!
Go back to the Admin UI (BTW: the UI was redone for 6.0) in the web browser.
Click on the Collection Selector and select gettingstarted.
Now click on the Query tab.
There is a "match all" query already written out for you, so all you need to do is click "Execute Query" at the bottom and notice the search results come back.
Let's change the query a bit.
Change *:* to inStock:false, press the tab key and press enter (This is the short-cut so you don't have to scroll down and click "Execute Query")

Conclusion

There you have it!  From source to server in probably less than ten minutes.  I know for myself, this was a long path to success as I had to figure out how to do things a bit differently on Windows (since most of the time they assume you're using Linux).

Most Importantly

I know for myself, I went through life with people not sharing the most important truth of life and I don't want that to happen to you.  As I'm sure you know, we as people are not good--we do terrible, immoral things--all of us.  God, the Creator of the universe, is going to judge us and we are going to be found deserving punishment--in the Lake of Fire.  The Good News is that He knew we would be depraved and he loved us so much that he sent his Son, Jesus Christ (Y'shua in Hebrew), to provide a way that God can be just and loving.  His Son, Jesus, was sent from heaven (where God lives), was born as a baby, grew up and didn't sin (break God's laws) even once, told people the truth about their condition, was brutally beaten, mocked, and murdered in the most humiliating and torturous way.  This was all planned from the beginning--before God created the world.  He knew going into it all that his Son was going to go on this rescue mission and die at the very hands of those he sought to save.  This is the first story I've heard where the hero dies for the villain.  However, three days after Jesus was murdered, God raised him from the dead, validating everything Jesus spoke.
Here's the overview:  You've broken God's law and the penalty for that is being thrown into the Lake of Fire (Revelation 20).  It doesn't have to be that way, but it will be if you don't obey what Jesus said:
“For this is how God loved the world: He gave his one and only Son, so that everyone who believes in him will not perish but have eternal life. God sent his Son into the world not to judge the world, but to save the world through him.
“There is no judgment against anyone who believes in him. But anyone who does not believe in him has already been judged for not believing in God’s one and only Son. And the judgment is based on this fact: God’s light came into the world, but people loved the darkness more than the light, for their actions were evil.  All who do evil hate the light and refuse to go near it for fear their sins will be exposed. But those who do what is right come to the light so others can see that they are doing what God wants.” 
John 3:16-21 NLT
Here's how Paul puts it:
If you openly declare that Jesus is Lord and believe in your heart that God raised him from the dead, you will be saved.  For it is by believing in your heart that you are made right with God, and it is by openly declaring your faith that you are saved. As the Scriptures tell us, “Anyone who trusts in him will never be disgraced.” Jew and Gentile are the same in this respect. They have the same Lord, who gives generously to all who call on him. For “Everyone who calls on the name of the Lord will be saved.”
Romans 10:9-13 NLT
How about you? You can change your mind about your sins, confess them to God, and trust in Jesus to save you from the wrath of God right now.  The only way we can know anything for sure is because of God's Word, so the more you study and apply what it teaches, the more spiritually blessed you will be.  I recommend starting out reading in the Gospel of John, whose purpose in writing it was to present the facts so that you can believe and be saved.  Learn more.

While you're reading along in the Gospel of John, watch this free movie called The Gospel of John on YouTube that follows along quite nicely.

May the Lord Jesus Christ bless you with grace and truth and life and life in abundance!

6 comments:

  1. Hi Brandon,

    I have follow your guide and able to run solr. However I will like to use java to test on solr with eclipse, how can I proceed here? Do you any guide?

    ReplyDelete
    Replies
    1. Hi Alex Poh.

      install run-jetty-run plugin from eclipse marketplace and run with debug.

      Delete
    2. Alex Poh,
      I was able to run unit tests in Eclipse by Right-clicking a JUnit test file, package, etc. (as long as it has a JUnit test in there somewhere), click Debug As > JUnit Test.

      That being said, many of the unit tests fail outright with something like this:
      Assertions mismatch: -ea was not specified but -Dtests.asserts=true
      Dec 13, 2016 10:08:48 AM com.carrotsearch.randomizedtesting.RandomizedRunner runSuite
      SEVERE: Panic: RunListener hook shouldn't throw exceptions.
      java.lang.NullPointerException
      at org.apache.lucene.util.RunListenerPrintReproduceInfo.printDebuggingInformation(RunListenerPrintReproduceInfo.java:131)
      at org.apache.lucene.util.RunListenerPrintReproduceInfo.testRunFinished(RunListenerPrintReproduceInfo.java:118)
      at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:706)
      at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
      at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:591)

      When you get that, simply go to Run > Debug Configurations...
      Under JUnit, select the name of the file you tried to debug. On the tabbed section, click Arguments.
      Paste the following into VM Arguments: -ea
      Then click Run.

      Now, every time you try to run that particular group of tests (even by right-clicking > Debug, etc.), it will use that mandatory -ea VM argument and likely work.

      If anyone knows of a way to get this argument set for any JUnit test, please, please tell me how! I have a debug configuration for every branch I try to run. Sometimes I just run a whole branch even when I work on a single test, simply because I don't want to create another debug configuration for that one test.

      I hope that helps!

      Shalom

      Delete
  2. bin/solr start -e cloud -noprompt is not work in unix.

    ReplyDelete
    Replies
    1. That is correct. This post is about running it on Windows. It's unfortunate that there is a difference in the interface between the two environments.

      Delete
  3. I really appreciate information shared above. It’s of great help. If someone want to learn Online (Virtual) instructor led live training in Apache Solr, kindly contact us http://www.maxmunus.com/contact
    MaxMunus Offer World Class Virtual Instructor led training on Apache Solr. We have industry expert trainer. We provide Training Material and Software Support. MaxMunus has successfully conducted 100000+ trainings in India, USA, UK, Australlia, Switzerland, Qatar, Saudi Arabia, Bangladesh, Bahrain and UAE etc.
    For Demo Contact us.
    Nitesh Kumar
    MaxMunus
    E-mail: nitesh@maxmunus.com
    Skype id: nitesh_maxmunus
    Ph:(+91) 8553912023
    http://www.maxmunus.com/


    ReplyDelete