Skip to content

QL: update tree sitter grammar to version 0.26 - #22695

Open
owen-mc wants to merge 5 commits into
github:mainfrom
owen-mc:ql/update-tree-sitter-0.26
Open

owen-mc wants to merge 5 commits into
github:mainfrom
owen-mc:ql/update-tree-sitter-0.26

Conversation

@owen-mc

@owen-mc owen-mc commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

I was getting FPs, which turned out to be because the QL extractor couldn't parse overlay[local] module;. The first commit adds tests demonstrating the FP. The second commit updates the tree-sitter grammar version we are using, which fixes the tests.

This was done by copilot. I'm not very familiar with QL-for-QL. I've reviewed it as best I can. It seems low risk as it's just a dependency update.

I assume this doesn't need a change note.

owen-mc and others added 2 commits September 29, 2026 10:59
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@owen-mc
owen-mc requested a review from a team as a code owner September 29, 2026 10:10
Copilot AI balanced review requested due to automatic review settings September 29, 2026 10:10
@owen-mc owen-mc added the no-change-note-required This PR does not need a change note label Sep 29, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The checked-in schema statistics must be regenerated to match the updated schema.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Updates QL-for-QL parsing to support file-level module; declarations and prevent related dead-code false positives.

Changes:

  • Upgrades Tree-sitter and the QL grammar revision.
  • Regenerates the QL schema and AST bindings.
  • Adds regression coverage for parameterized modules.
File Description
ql/​extractor/​Cargo.toml Updates parser dependencies.
ql/​Cargo.lock Locks updated dependencies.
ql/​ql/​src/​ql.dbscheme Reflects the revised module grammar.
ql/​ql/​src/​codeql_ql/​ast/​internal/​TreeSitter.qll Updates generated module accessors.
ql/​ql/​test/​queries/​style/​DeadCode/​Foo.qll Adds regression scenarios.
ql/​ql/​test/​queries/​style/​DeadCode/​Parameterized.qll Adds parameterized-module fixture.
ql/​ql/​test/​queries/​style/​DeadCode/​DeadCode.expected Updates expected locations.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread ql/extractor/Cargo.toml
hvitved
hvitved previously approved these changes Sep 29, 2026

@hvitved hvitved left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

Many languages clear this and fix any bad join orders that arise,
to avoid having to update it with every dbscheme change.

@hvitved hvitved left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests are failing.

Give anonymous module declarations a source location so file-level overlay annotations apply to their declarations. Exclude valid top-level anonymous modules from the missing-name consistency check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@owen-mc

owen-mc commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

I've just pushed a commit which should fix them.

@hvitved hvitved left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The removal of DB stats likely introduced a bad join:

[2/65 eval 2.3s] Evaluation done; writing results to codeql/ql/queries/diagnostics/SuccessfullyExtractedFiles.bqrs.
[3/65 eval 7.7s] Evaluation done; writing results to codeql/ql/queries/reports/OutdatedDeprecations.bqrs.
[4/65 eval 27.6s] Evaluation done; writing results to codeql/ql/queries/style/QlRefInlineExpectations.bqrs.
[5/65 eval 2m7s] Evaluation done; writing results to codeql/ql/queries/bugs/MissingSanitizerGuardCase.bqrs.
[6/65 eval 2m7s] Evaluation done; writing results to codeql/ql/queries/bugs/OrderByConst.bqrs.
[7/65 eval 2m7s] Evaluation done; writing results to codeql/ql/queries/bugs/PathProblemQuery.bqrs.
[8/65 eval 2m7s] Evaluation done; writing results to codeql/ql/queries/bugs/SumWithoutDomain.bqrs.
[9/65 eval 2m7s] Evaluation done; writing results to codeql/ql/queries/performance/DontUseGetAQlClass.bqrs.
[10/65 eval 2m7s] Evaluation done; writing results to codeql/ql/queries/diagnostics/EmptyConsistencies.bqrs.
[11/65 eval 2m42s] Evaluation done; writing results to codeql/ql/queries/performance/MissingNoinline.bqrs.
[12/65 eval 2m42s] Evaluation done; writing results to codeql/ql/queries/performance/NonInitialStdLibImport.bqrs.
[13/65 eval 2m42s] Evaluation done; writing results to codeql/ql/queries/style/AcronymsShouldBeCamelCase.bqrs.
[14/65 eval 2m43s] Evaluation done; writing results to codeql/ql/queries/style/AndroidIdPrefix.bqrs.
[15/65 eval 2m43s] Evaluation done; writing results to codeql/ql/queries/style/ConsistentAlertMessage.bqrs.
[16/65 eval 2m43s] Evaluation done; writing results to codeql/ql/queries/style/ConsistentCasing.bqrs.
[17/65 eval 2m43s] Evaluation done; writing results to codeql/ql/queries/style/CountingToZero.bqrs.
[18/65 eval 2m43s] Evaluation done; writing results to codeql/ql/queries/style/DataFlowConfigModuleNaming.bqrs.
[19/65 eval 2m43s] Evaluation done; writing results to codeql/ql/queries/style/DBTypeInNonLib.bqrs.
[20/65 eval 2m44s] Evaluation done; writing results to codeql/ql/queries/style/GetAPrimaryQlClassConsistency.bqrs.
[21/65 eval 2m44s] Evaluation done; writing results to codeql/ql/queries/style/MissingQualityMetadata.bqrs.
[22/65 eval 2m44s] Evaluation done; writing results to codeql/ql/queries/style/LibraryAnnotation.bqrs.
[23/65 eval 2m44s] Evaluation done; writing results to codeql/ql/queries/style/IfWithElseNone.bqrs.
[24/65 eval 2m44s] Evaluation done; writing results to codeql/ql/queries/style/MissingSecurityMetadata.bqrs.
[25/65 eval 2m44s] Evaluation done; writing results to codeql/ql/queries/style/Misspelling.bqrs.
[26/65 eval 2m44s] Evaluation done; writing results to codeql/ql/queries/style/NameCasing.bqrs.
[27/65 eval 2m44s] Evaluation done; writing results to codeql/ql/queries/style/NonDocBlock.bqrs.
[28/65 eval 2m44s] Evaluation done; writing results to codeql/ql/queries/style/RankOne.bqrs.
[29/65 eval 2m44s] Evaluation done; writing results to codeql/ql/queries/style/RegexpInsteadOfPattern.bqrs.
[30/65 eval 2m44s] Evaluation done; writing results to codeql/ql/queries/style/RepeatedWord.bqrs.
[31/65 eval 2m44s] Evaluation done; writing results to codeql/ql/queries/style/SingletonSetLiteral.bqrs.
[32/65 eval 2m45s] Evaluation done; writing results to codeql/ql/queries/style/RedundantImport.bqrs.
[33/65 eval 2m49s] Evaluation done; writing results to codeql/ql/queries/bugs/InconsistentDeprecation.bqrs.
[34/65 eval 2m49s] Evaluation done; writing results to codeql/ql/queries/bugs/NameClashInSummarizedCallable.bqrs.
[35/65 eval 2m51s] Evaluation done; writing results to codeql/ql/queries/performance/UnusedField.bqrs.
[36/65 eval 2m55s] Evaluation done; writing results to codeql/ql/queries/performance/AbstractClassImport.bqrs.
[37/65 eval 3m14s] Evaluation done; writing results to codeql/ql/queries/style/SwappedParameterNames.bqrs.
[38/65 eval 3m21s] Evaluation done; writing results to codeql/ql/queries/style/OverrideAny.bqrs.

Match builtin predicate calls by name before computing their arity, and calculate the arity directly from builtin parameters. This avoids a broad virtual getArity dispatch that produced tens of billions of intermediate tuples without dbscheme statistics.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-change-note-required This PR does not need a change note QL-for-QL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants