Repository navigation
Conversation
xymus
changed the title
Test Results 67 files 337 suites 16m 56s ⏱️ Results for commit 2da2ef6. ♻️ This comment has been updated with latest results. |
| var t = scan_token | ||
| while t == null do t = scan_token |
There was a problem hiding this comment.
Can't this end up being an infinite loop ?
There was a problem hiding this comment.
It shouldn't, scan_token returns null only on comments and some whitespaces. It will always return an EOF token for the end of file.
There was a problem hiding this comment.
The doc on scan_token and get_token was barebone, I updated it to be clearer on the expected tokens and the redef logic.
|
CI fails on the curl issue fixed in #2867. |
Intro `scan_token` as the service actually retrieving the next token from the source file, and keep `get_token` as the service returning the token to the clients. `get_token` can then act as a refinement point to filter or alter the token stream. Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
The dependencies listed by `nitls -M` dropped any module with a parsing error. A broken file would then be missing from the dependencies of a Makefile, and make could see the binary as up to date. Without the rebuild the error is fully hidden from the user. Let's make this less likely by parsing only the part nitls actually needs to list dependencies. This way a syntax error after the import list doesn't affect nitls' behavior. Intro the module loader_header_only, imported by nitls only, which refines the Lexer to return EOF at the first token of the body. As a bonus, the reduced work speeds up nitls significantly, `nitls -M src/nitc.nit` is about 9x faster. Since it can run many time on every `make` invocation, it becomes noticeably more snappy in large projects. Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
With `nitls -M` a module that failed to parse was not listed in the output. This could have make think no rebuild was necessery which fully silenced the error. Let's list the files with the parse error to address this issue. To do so we keep a list of the parsing errors. An alternative would be to keep a list of the AModule parsed or the AError for the failed ones. Then `nitls` could list those in for the `-M` output instead of the type-checked MModules. Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
privat
left a comment
privat
left a comment
There was a problem hiding this comment.
My only concern is that we might forget the hack when updating the grammar (but we should be fine as it never happens)
| # Blanks | ||
|
|
||
| redef class TEol | ||
| redef fun is_blank do return true |
There was a problem hiding this comment.
I suggest lifting up the is_blank method (in src/parser/parser_nodes.nit I suppose).
The blankness is not a property specific to the header only.
I think it's the same for is_prefix. start_body should stay here.
There was a problem hiding this comment.
I moved is_blank up as can_be_loose, the term blank means something else in the parser and is_loose was closer to what we needed. While is_loose is computed at parsing, can_be_loose is static per token kind and available before parsing.
Same for is_prefix, it was moved up and renamed to the more precise is_definition_prefix.
|
|
||
| if opt_make.value then | ||
| # The modules that could not be parsed didn't result in a mmodule that | ||
| # could be listed above. List them here from the collected errors. |
There was a problem hiding this comment.
I assume the idea here is to genuinely inject buggy modules as dependency and let the rest of the make compilation chain deal with that.
There was a problem hiding this comment.
Yes exactly, any error reported by nitls when building the dependency list for make would be lost. So we inject back known unparsable files so the real call to nitc reports the errors to the user.
At this point we print two lists, files linked to a type-checked MModule and the ones with parsing errors that didn't make it to a MModule. We could instead just keep the list of files the compiler reached tried to parse and it would give us the same information.
Extract and rename `is_blank` and `is_prefix` out of the header only loader logic up to `parser_nodes.nit`, renaming them to `can_be_loose` and `is_definition_prefix`. The new names are more precise for public services. The term blank of `is_blank` means something else in the parser, the preexisting `is_loose` is a better match but computed at parsing while we need this information before hand. We can still match the naming with `can_be_loose`. And `is_definition_prefix` makes it clear what kind of prefix we're looking for over `is_prefix`. Signed-off-by: Alexis Laferrière <alexis.laf@xymus.net>
Improve the dependency listing mode
-Mof nitls to be more resilient to parsing errors. A parsing error in a source file used to make nitlis skip listing that file in its output and not parse its imports. Skipping a file in the dependency list could havemakenot trigger rebuilds of existing binaries as expected. Without a rebuild, the original parsing errors are simply not reported to the user and themakecommand succeeds on broken code without doing any work.Let's make
nitlsmuch more resilient to parsing errors with two new behaviors:In nitls, parse only the header of each source file: the module declaration and imports. Ignore all class and method definitions as well as top-level code. This prevents parsing errors later in the file to break the whole file.
The header information is enough for nitls’ use cases. As a bonus, the reduced work speeds up nitls significantly,
nitls -M src/nitc.nitis about 9x faster. Since it can run many time on everymakeinvocation, it becomes noticeably more snappy in large projects.In the case of a parse error in the header, we now correctly list that file in the
nitls -Moutput. It used to be skipped as we listed only type-checked MModules. Now we keep track of the parsing errors and use that information to list problematic files as well.