Re: [docbook] More memory for Lucene

[Stefan Hinz <stefan.hinz@oracle.com>]

Hi David,

I'll investigate how our catalogs are set up on the documentation server.

As for "timestamps", I can always see the time it took Lucene to build the index files. For example, on my local machine, it looks like this:

java \
        -DhtmlDir=ndbapi.webhelp-tmp \
        -DindexerLanguage=en \
        -DhtmlExtension=html \
        -DdoStem=true \
        -DindexerExcludedFiles=ix01.html \
        -Dorg.xml.sax.driver=org.ccil.cowan.tagsoup.Parser \
        -Djavax.xml.parsers.SAXParserFactory=org.ccil.cowan.tagsoup.jaxp.SAXFactoryImpl \
        -classpath ../../mysqldoc-toolset/xsl.d/docbook-xsl-1.78.0/extensions/webhelpindexer.jar:../../mysqldoc-toolset/xsl.d/docbook-xsl-1.78.0/extensions/tagsoup-1.2.1.jar:../../mysqldoc-toolset/xsl.d/docbook-xsl-1.78.0/extensions/lucene-analyzers-3.0.0.jar:../../mysqldoc-toolset/xsl.d/docbook-xsl-1.78.0/extensions/lucene-core-3.0.0.jar \
        com.nexwave.nquindexer.IndexerMain
Stemming enabled
The created index files are located in ndbapi.webhelp-tmp/search
Indexed the contents in 18 seconds

On the documentation server, for the same document, I see this:

Indexed the contents in 3116 seconds

That's 51 minutes!

Cheers,

Stefan


On 27.01.2013 20:08, David Cramer wrote:

On 01/27/2013 12:29 PM, Stefan Hinz wrote:

But how would I do that, i.e. allocate more than the default amount of
memory (whatever it is) to Lucene? Any pointers appreciated!

Output some timestamps from your Makefile into your log to confirm that
it's indexing that's taking a long time and not some other part of the
build (that might well be the indexing, but it's good to check first).
Another thing that can stall the build for a while is if you are
processing DocBook 4.x without catalog files setup and the parser grabs
the DTD off the net every time. So if you had catalog files on your
local system but not on the build box, that could make a difference.

David



---------------------------------------------------------------------
To unsubscribe, e-mail: docbook-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: docbook-help@lists.oasis-open.org

-- 
Cheers,

Stefan Hinz <stefan.hinz@oracle.com>, MySQL Documentation Manager

Phone: +49-30-82702940, Fax: +49-30-82702941, http://dev.mysql.com/doc

ORACLE Deutschland B.V.&  Co. KG
Registered Office: Riesstr. 25, 80992 Muenchen, Germany
Commercial Register: Local Court Of Munich, HRA 95603
Managing Director: Jürgen Kunz

General Partner: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Register Of Chamber Of Commerce: Midden-Niederlande, No. 30143697
Managing Directors: Alexander van der Ven, Astrid Kepper, Val Maher