Skip to content

Normalize CRLF before highlighting source code - #1261

Open
sjh9714 wants to merge 3 commits into
ruby:masterfrom
sjh9714:codex/20261010-1037-crlf-source-highlighting
Open

sjh9714 wants to merge 3 commits into
ruby:masterfrom
sjh9714:codex/20261010-1037-crlf-source-highlighting

Conversation

@sjh9714

@sjh9714 sjh9714 commented Oct 10, 2026

Copy link
Copy Markdown

Summary

Refs #1037.

CRLF still appears as ^M inside highlighted comments and multiline strings, including in show_source and the source shown around a binding. Normalize CRLF before parsing colored source so those line endings render normally. Standalone carriage returns remain visible, and uncolored output is unchanged.

Changes

  • Normalize CRLF in the shared source highlighter.
  • Cover CRLF comments, multiline strings and method definitions, plus standalone CR handling.

Testing

  • bundle exec rake test TEST=test/irb/test_color.rb
  • bundle exec rake test (410 tests, 0 failures/errors, 2 omissions)
  • bundle exec rubocop
  • bundle exec rake build

Verified on macOS with Ruby 4.0.6. The Windows/Linux and other Ruby version matrices were not run locally.

Comment thread lib/irb/color.rb Outdated
def colorize_code(code, complete: true, ignore_error: false, colorable: colorable?, local_variables: [])
return code unless colorable

code = code.gsub("\r\n", "\n")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normalizing CRLF should be done AFTER Prism.parse_lex, not before. The original source code should be passed to the parser.
Replacing \r\n to \n will change the meaning of the code in some case, such as:

eval "%\r\r\na\r"
#=> "\na"
eval "%\r\r\na\r".gsub("\r\n", "\n")
#=> SyntaxError

eval "%\ra\r\r\n"
#=> "a"
eval "%\ra\r\r\n".gsub("\r\n", "\n")
#=> SyntaxError

Reline::Unicode.escape_for_print(code) is the one changing "\r" to "^M".
We can replace Reline::Unicode.escape_for_print(s) to a method call, calling a method defined in IRB::Color that escapes control characters except "\r\n".

@sjh9714

sjh9714 commented Oct 14, 2026

Copy link
Copy Markdown
Author

Moved the normalization after parsing, as you suggested, and added both examples to the tests. I also fixed the failure with the latest Prism, which splits CRLF at the end of a comment across two tokens. The full suite passes locally with both released and development Prism.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants