Mailing List Archive

»Ø¸´£º Why does this query slow down Lucene?
The query has been stuck for more than an hour. The total size is less than 1G, and the number of docs is around 100,000. Hardware is ok as it works well with other much more demanding projects.








------------------ ԭʼÓʼþ ------------------
·¢¼þÈË: "Li Li"<fancyerii@gmail.com>;
·¢ËÍʱ¼ä: 2012Äê8ÔÂ16ÈÕ(ÐÇÆÚËÄ) ÉÏÎç9:59
ÊÕ¼þÈË: "java-user"<java-user@lucene.apache.org>;

Ö÷Ìâ: Re: Why does this query slow down Lucene?



how slow is it£¿ are all your searches slow or only that query slow£¿ how
many docs are indexed and the size of the indexes£¿ whats the hardware
configuration£¿
you should describe it clearly to get help.
ÔÚ 2012-8-16 ÉÏÎç9:28£¬"zhoucheng2008" <zhoucheng2008@gmail.com>дµÀ£º

> Hi,
>
>
> I have the string "$21 a Day Once a Month" to search on a large index.
>
>
> I escape the $ sign, and the query string looks like:
>
>
> +level:0 +(title:21 title:a title:day title:once title:a title:month)
> release_year:1941 code:movieverycd126087
>
>
> This slows down Lucene. Any idea?
>
>
> Thanks,
> Cheng
Re: »Ø¸´£º Why does this query slow down Lucene? [ In reply to ]
use jstack pid to check any deadlock.

On Thu, Aug 16, 2012 at 10:09 AM, zhoucheng2008 <zhoucheng2008@gmail.com> wrote:
> The query has been stuck for more than an hour. The total size is less than 1G, and the number of docs is around 100,000. Hardware is ok as it works well with other much more demanding projects.
>
>
>
>
>
>
>
>
> ------------------ ԭʼÓʼþ ------------------
> ·¢¼þÈË: "Li Li"<fancyerii@gmail.com>;
> ·¢ËÍʱ¼ä: 2012Äê8ÔÂ16ÈÕ(ÐÇÆÚËÄ) ÉÏÎç9:59
> ÊÕ¼þÈË: "java-user"<java-user@lucene.apache.org>;
>
> Ö÷Ìâ: Re: Why does this query slow down Lucene?
>
>
>
> how slow is it£¿ are all your searches slow or only that query slow£¿ how
> many docs are indexed and the size of the indexes£¿ whats the hardware
> configuration£¿
> you should describe it clearly to get help.
> ÔÚ 2012-8-16 ÉÏÎç9:28£¬"zhoucheng2008" <zhoucheng2008@gmail.com>дµÀ£º
>
>> Hi,
>>
>>
>> I have the string "$21 a Day Once a Month" to search on a large index.
>>
>>
>> I escape the $ sign, and the query string looks like:
>>
>>
>> +level:0 +(title:21 title:a title:day title:once title:a title:month)
>> release_year:1941 code:movieverycd126087
>>
>>
>> This slows down Lucene. Any idea?
>>
>>
>> Thanks,
>> Cheng

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: »Ø¸´£º Why does this query slow down Lucene? [ In reply to ]
and also try jmap -heap pid to check whether it runs out of memory
or jstat -gcutil pid 1000

On Thu, Aug 16, 2012 at 10:09 AM, zhoucheng2008 <zhoucheng2008@gmail.com> wrote:
> The query has been stuck for more than an hour. The total size is less than 1G, and the number of docs is around 100,000. Hardware is ok as it works well with other much more demanding projects.
>
>
>
>
>
>
>
>
> ------------------ ԭʼÓʼþ ------------------
> ·¢¼þÈË: "Li Li"<fancyerii@gmail.com>;
> ·¢ËÍʱ¼ä: 2012Äê8ÔÂ16ÈÕ(ÐÇÆÚËÄ) ÉÏÎç9:59
> ÊÕ¼þÈË: "java-user"<java-user@lucene.apache.org>;
>
> Ö÷Ìâ: Re: Why does this query slow down Lucene?
>
>
>
> how slow is it£¿ are all your searches slow or only that query slow£¿ how
> many docs are indexed and the size of the indexes£¿ whats the hardware
> configuration£¿
> you should describe it clearly to get help.
> ÔÚ 2012-8-16 ÉÏÎç9:28£¬"zhoucheng2008" <zhoucheng2008@gmail.com>дµÀ£º
>
>> Hi,
>>
>>
>> I have the string "$21 a Day Once a Month" to search on a large index.
>>
>>
>> I escape the $ sign, and the query string looks like:
>>
>>
>> +level:0 +(title:21 title:a title:day title:once title:a title:month)
>> release_year:1941 code:movieverycd126087
>>
>>
>> This slows down Lucene. Any idea?
>>
>>
>> Thanks,
>> Cheng

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
»Ø¸´£º Why does this query slow down Lucene? [ In reply to ]
Yes. The query is bit long, which is the major reason.




------------------ ԭʼÓʼþ ------------------
·¢¼þÈË: "Trejkaz"<trejkaz@trypticon.org>;
·¢ËÍʱ¼ä: 2012Äê8ÔÂ16ÈÕ(ÐÇÆÚËÄ) ÖÐÎç11:09
ÊÕ¼þÈË: "java-user"<java-user@lucene.apache.org>;

Ö÷Ìâ: Re: Why does this query slow down Lucene?



On Thu, Aug 16, 2012 at 11:27 AM, zhoucheng2008 <zhoucheng2008@gmail.com> wrote:
>
> +(title:21 title:a title:day title:once title:a title:month)

Looks like you have a fairly big boolean query going on here, and some
of the terms you're using are really common ones like "a".

Are you using AND or OR for the default operator? I think AND was
significantly faster.

I would also consider culling stop words to get things like "a" out of
the picture.

I also find it interesting that Lucene doesn't normalise title:a
title:a to title:a^2 or something. :)

TX

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org