Mailing List Archive

DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32965>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965


paul.elschot@xs4all.nl changed:

What |Removed |Added
----------------------------------------------------------------------------
Summary|Use filter bits for next() |[PATCH] Use filter bits for
|and skipTo() in |next() and skipTo() in
|FilteredQuery |FilteredQuery




------- Additional Comments From paul.elschot@xs4all.nl 2005-01-08 17:21 -------
This 5 Jan 2005 version of FilteredQuery.java could be simplified
by deriving an intIterator directly from a BitSet.
In the useIterFilter() method this is done with a SortedVIntList
in between, but it could easily be done directly.

Perhaps the intIterator attachment should be renamed to DocNrIter
or DocNrIterator.

With the current Filter only a java.lang.BitSet
can be used as the underlying data structure for the filter.
IterFilter allows the use of other data structures,
eg. an int array of document numbers, or a SortedVIntList.

Regards,
Paul Elschot


--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32965>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965





------- Additional Comments From paul.elschot@xs4all.nl 2005-01-08 17:24 -------
(In reply to comment #6)
> This 5 Jan 2005 version of FilteredQuery.java could be simplified

That should be 8 Jan 2005.


--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32965>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965


paul.elschot@xs4all.nl changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #13940|0 |1
is obsolete| |




------- Additional Comments From paul.elschot@xs4all.nl 2005-01-08 22:40 -------
Created an attachment (id=13944)
--> (http://issues.apache.org/bugzilla/attachment.cgi?id=13944&action=view)
Refactoring of FilteredQuery, 2nd version of 8 Jan 2005.

FilteredScorer now has separate methods for next() and skipTo()
for the filter (abstract filterNext() and abstract filterSkipTo())
and for the query scorer (scorerNext() and scorerSkipTo()).
The next() and skipTo() methods work like ConjunctionScorer,
but the filter and query scorer are separately called.
The filter is always used before the query scorer to minimize I/O.

The inline subclasses of FilteredScorer for Filter and IterFilter
only differ for the filterNext() and filterSkipTo() methods and on a helper
method for explain(), so they now have a readable size.

Regards,
Paul Elschot


--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32965>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965


paul.elschot@xs4all.nl changed:

What |Removed |Added
----------------------------------------------------------------------------
BugsThisDependsOn| |33019




--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32965>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965


paul.elschot@xs4all.nl changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #13939|0 |1
is obsolete| |




------- Additional Comments From paul.elschot@xs4all.nl 2005-01-16 18:11 -------
Created an attachment (id=14023)
--> (http://issues.apache.org/bugzilla/attachment.cgi?id=14023&action=view)
SkipFilter.java

A query filter in the form of a DocNrSkipper from an IndexReader.

--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32965>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965


paul.elschot@xs4all.nl changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #13944|0 |1
is obsolete| |




------- Additional Comments From paul.elschot@xs4all.nl 2005-01-16 18:16 -------
Created an attachment (id=14024)
--> (http://issues.apache.org/bugzilla/attachment.cgi?id=14024&action=view)
SkipFilter.java, 16 Jan 2005

FilteredQuery.java using skipTo() on the query, and extended
with a SkipFilter constructor.

--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32965>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965





------- Additional Comments From paul.elschot@xs4all.nl 2005-01-16 18:22 -------
The patch in IndexSearcher is independent of the rest of these
attachments.

SkipFilter.java also contains a method to replace the Filter by
a SkipFilter. This method is for testing only.

With and without the use of this method, and with the patch to
IndexSearcher, all current tests pass.
This could indicate that the combination of BooleanQuery and Filter
is is missing from the current tests.

FilteredQuery is also considerably simplified wrt. to the version
of 8 Jan 2005.

Regards,
Paul Elschot


--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32965>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965


paul.elschot@xs4all.nl changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #14024|SkipFilter.java, 16 Jan 2005|FilteredQuery.java, 16 Jan
description| |2005




--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
Erik Hatcher wrote:
> Oh, and one other thing.... Paul's code relies on JDK 1.4's assert

Erhm.. you meant 1.5 (five), right?

--
Best regards,
Andrzej Bialecki
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
On Apr 4, 2005, at 8:18 AM, Andrzej Bialecki wrote:
> Erik Hatcher wrote:
>> Oh, and one other thing.... Paul's code relies on JDK 1.4's assert
>
> Erhm.. you meant 1.5 (five), right?

No, 1.4. Assert's were added in JDK 1.4. I live in a Mac-centric
world and 1.5 (errr, 5.0!), and I've yet to run 1.5 since its not
freely available for Mac OS X yet.

Erik


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
Erik Hatcher writes (4/4/2005 2:36 AM):

> Oh, and one other thing.... Paul's code relies on JDK 1.4's assert
> keyword. It seems this is an unnecessary reason to jump to 1.4
> dependence.

As a 1.5 user, I'd love to see Lucene at least at 1.4. Assert's are a
good thing.

Chuck


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
Erik Hatcher wrote:

> Oh, and one other thing.... Paul's code relies on JDK 1.4's assert
> keyword. It seems this is an unnecessary reason to jump to 1.4
> dependence.
>
> What do folks think about JDK 1.4 as a minimum Lucene requirement?

I'm not a fan of outdated software or historical systems. So i think the
best would be to keep lucene still backward compatible with version 1.9
and perform the switch to JDK 1.4 with lucene 2.0.

Bernhard

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
Erik Hatcher wrote:

> Oh, and one other thing.... Paul's code relies on JDK 1.4's assert
> keyword. It seems this is an unnecessary reason to jump to 1.4
> dependence.
>
> What do folks think about JDK 1.4 as a minimum Lucene requirement?
>
> Erik

I'm not a fan of outdated software or historical systems. So i think the
best would be to keep lucene still backward compatible with version 1.9
and perform the switch to JDK 1.4 with lucene 2.0.

Bernhard

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
On Monday 04 April 2005 11:34, Erik Hatcher wrote:
> I added Paul's SkipFilter and overwrote my FilteredQuery class from the
> code here. However, your FilteredQuery depends on two additional
> classes: DocNrSkipper and SortedVIntList that are not provided.
> Please attach those classes to this issue.

They are in one of the bugs the filter depends on:
http://issues.apache.org/bugzilla/show_bug.cgi?id=32921

>
> I'm not personally fond of the abbreviation of "Number" to "Nr" - any
> objections to spelling out "Number" entirely?

Not at all. My tradeoff between programming line length and
identifier readability is just slightly different. For me, number is
general enough to abbreviate in an identifier.

> Any others have objections to Paul's patches here? I'll commit them
> once I have everything working locally, unless I hear otherwise.

Please note the other dependency: this filter only works correctly when
documents are always scored in document number order.
This means that the bucket table of the 1.4.3 BooleanScorer can not
be used anymore, except when sorting is added.
See also the last comments I posted at that bug:
http://issues.apache.org/bugzilla/show_bug.cgi?id=33019
The (non ordered) bucket scoring mechanism the fastest one that is
available, anything else is slower, alhough not by much.

As for the assert statements: could you comment them out in case
the dependency on java 1.4 is not acceptable? That would
leave them useable for later.

Regards,
Paul Elschot


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [ In reply to ]
On Monday 04 April 2005 11:36, Erik Hatcher wrote:
> Oh, and one other thing.... Paul's code relies on JDK 1.4's assert
> keyword. It seems this is an unnecessary reason to jump to 1.4
> dependence.
>
> What do folks think about JDK 1.4 as a minimum Lucene requirement?

The FilteredQuery as posted there requires jdk 1.4
because it uses BitSet.nextSetBit():

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965#c2

Regards,
Paul Elschot



---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org