Yeast: some fixes#21924
Open
asgerf wants to merge 8 commits into
Open
Conversation
When a {..expr} splice in an output template is empty (e.g. from an
optional capture that did not match), drop the field entirely rather
than emitting an empty named field. This lets a single rule with
optional captures replace what used to be two near-identical rules.
Also re-renders the corpus to drop the now-suppressed empty fields.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add FieldCardinality to Schema to track required/multiple per field, populated from the ast_types.yml suffixes (bare = required single, ? = optional single, + = required multiple, * = optional multiple). dump_ast_with_type_errors now emits: <-- ERROR: missing required field 'name' for any node in the output AST whose declared schema requires a field that is absent from the actual node. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
Author
Rerun has been triggered: 2 restarted 🚀 |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR applies a set of Yeast and unified extractor test-harness fixes that came up while working on Swift mapping rules, including more robust corpus parsing, improved schema/type diagnostics, and query matching behavior aligned with tree-sitter semantics.
Changes:
- Make corpus parsing tolerate missing
---separators by stopping at the next case header and treating missing sections as empty. - Align Yeast positional query matching with tree-sitter semantics by skipping
extras(e.g. comments), and add a regression test. - Extend Yeast schema loading to record field cardinality (
multiple/required) and surface missing required fields in typed dumps; also adjust OneShot capture mapping to support 0/Many rewrites.
Show a summary per file
| File | Description |
|---|---|
| unified/extractor/tests/corpus_tests.rs | Makes corpus parsing stop at next header and handle missing separators by treating remaining sections as empty. |
| shared/yeast/tests/test.rs | Adds regression test ensuring positional wildcards skip tree-sitter extras (comments). |
| shared/yeast/src/schema.rs | Introduces FieldCardinality, stores per-field multiplicity/requiredness, and exposes required fields per kind. |
| shared/yeast/src/node_types_yaml.rs | Populates FieldCardinality from node-types.yml specs while building schema. |
| shared/yeast/src/query.rs | Skips extra nodes during positional matching to mirror tree-sitter query semantics. |
| shared/yeast/src/lib.rs | Adds Node::field_children helper and updates OneShot capture recursion to allow 0/Many results. |
| shared/yeast/src/dump.rs | Emits errors for missing required fields during typed AST dumping. |
| shared/yeast/src/captures.rs | Generalizes try_map_all_captures to map each id to 0/Many ids (splicing/deletion). |
| shared/yeast-macros/src/parse.rs | Adjusts template parsing for raw identifiers and avoids emitting empty named fields; treats empty () as empty sequence. |
| shared/tree-sitter-extractor/src/generator/mod.rs | Improves panic message for union types referencing unknown member node types. |
Copilot's findings
- Files reviewed: 10/10 changed files
- Comments generated: 2
Comment on lines
413
to
415
| let field_name = expect_ident(tokens, "expected field name")?; | ||
| let field_str = field_name.to_string(); | ||
| let field_str = field_name.to_string().strip_prefix("r#").unwrap_or(&field_name.to_string()).to_string(); | ||
| expect_punct(tokens, ':', "expected `:` after field name")?; |
Comment on lines
+276
to
+284
| // Check for required fields that are absent | ||
| if let Some((schema, _, _)) = type_check { | ||
| for (field_id, field_name) in schema.required_fields_for_kind(node.kind_name()) { | ||
| if !node.fields.contains_key(&field_id) { | ||
| let name = field_name.unwrap_or("child"); | ||
| writeln!(out, "{prefix} <-- ERROR: missing required field '{name}'").unwrap(); | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Some fixes that came along with working on Swift mapping rules