Re: [docbook] More memory for Lucene

[Stefan Hinz <stefan.hinz@oracle.com>]

Hi Thomas,

> > I'll investigate how our catalogs are set up on the documentation
> > server.
> >
> > [...]
> >
> > On the documentation server, for the same document, I see this:
> >
> > Indexed the contents in 3116 seconds
> >
> > That's 51 minutes!
>
> Could it be that the you miss some JAR files in your classpath? Can you
> check if both are the same?

I use the exact same tools both on my local computer and on the 
documentation server.

The big Java command, including the configuration for that command, is a 
copy from the sample Makefile provided with the 1.78.0 style sheets 
(provided as webhelp/Makefile.sample). Here is the configuration:

DOCBOOK_EXTENSIONS_DIR = $(DOCBOOK_DIST)/extensions
INDEXER_JAR   := $(DOCBOOK_EXTENSIONS_DIR)/webhelpindexer.jar
TAGSOUP_JAR   := $(DOCBOOK_EXTENSIONS_DIR)/tagsoup-1.2.1.jar
LUCENE_ANALYZER_JAR   := 
$(DOCBOOK_EXTENSIONS_DIR)/lucene-analyzers-3.0.0.jar
LUCENE_CORE_JAR   := $(DOCBOOK_EXTENSIONS_DIR)/lucene-core-3.0.0.jar
classpath := 
$(INDEXER_JAR):$(TAGSOUP_JAR):$(LUCENE_ANALYZER_JAR):$(LUCENE_CORE_JAR)

If there's something missing, it's missing in the 1.78.0 style sheets, too.

> This line makes me suspicious:
>
> > -classpath ../../mysqldoc-toolset/xsl.d/docbook-xsl-1.78.0/extensions/webhelpindexer.jar:../../mysqldoc-toolset/xsl.d/docbook-xsl-1.78.0/extensions/tagsoup-1.2.1.jar:../../mysqldoc-toolset/xsl.d/docbook-xsl-1.78.0/extensions/lucene-analyzers-3.0.0.jar:../../mysqldoc-toolset/xsl.d/docbook-xsl-1.78.0/extensions/lucene-core-3.0.0.jar
>
> I would have expected to see the xml-commons-resolver jar. It seems
> it's not there. This jar is responsible for XML catalog resolution
> and if you don't have that, your tools will load them from the
> Internet.

That's interesting. Our documentation server runs behind a firewall, and 
while it does have access to the internet, I suspect it's low-bandwidth. 
So, indeed, this could be responsible for the huge index creation time 
we get on that machine.

As I said initially, we're using the Java tools shipped with the 1.78.0 
style sheets in the extensions directory. That directory contains these 
JAR files (in other words, it doesn't contain xml-commons-resolver.jar):

xsl.d/docbook-xsl-1.78.0/extensions> ls -1 *jar
lucene-analyzers-3.0.0.jar
lucene-core-3.0.0.jar
saxon65.jar
tagsoup-1.2.1.jar
webhelpindexer.jar
xalan27.jar

Regarding xml-commons-resolver.jar, it's installed (with xml-commons) on 
my computer and on the documentation server. Could it be possible that 
it's (implicitly) used on my computer, but not on the documentation server?

--
Cheers,

Stefan Hinz <stefan.hinz@oracle.com>, MySQL Documentation Manager

Phone: +49-30-82702940, Fax: +49-30-82702941, http://dev.mysql.com/doc

ORACLE Deutschland B.V.&  Co. KG
Registered Office: Riesstr. 25, 80992 Muenchen, Germany
Commercial Register: Local Court Of Munich, HRA 95603
Managing Director: Jürgen Kunz

General Partner: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Register Of Chamber Of Commerce: Midden-Niederlande, No. 30143697
Managing Directors: Alexander van der Ven, Astrid Kepper, Val Maher