Skip to content

[BUG] NOT IN includes null/missing rows due to missing exists filter in pushdown #5165

@penghuo

Description

@penghuo

Query Information

PPL Command/Query:

source=bounty-types | where int_field NOT IN (42, -1, 0) | fields int_field

Expected Result:
NOT IN should exclude documents where int_field is null/missing.

Actual Result:
Result includes both 2147483647 and null rows.

Dataset Information

Dataset/Schema Type

  • OpenTelemetry (OTEL)
  • Simple Schema for Observability (SS4O)
  • Open Cybersecurity Schema Framework (OCSF)
  • Custom (details below)

Index Mapping

{
  "mappings": {
    "properties": {
      "int_field": { "type": "integer" }
    }
  }
}

Sample Data

[
  { "int_field": 42 },
  { "int_field": -1 },
  { "int_field": 0 },
  { "int_field": 2147483647 },
  { "int_field": null }
]

Bug Description

Issue Summary:
NOT IN incorrectly matches null/missing field rows.

Steps to Reproduce:

  1. Create index with int_field.
  2. Insert sample rows including one null value.
  3. Run the query above.
  4. Observe null row in output.

Impact:
Incorrect filter semantics and wrong result sets in production queries.

Metadata

Metadata

Assignees

No one assigned

    Labels

    PPLPiped processing languagebugSomething isn't working

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions