fix: correct multibyte character handling in get_selected_text() - #14
Open
visitorise wants to merge 2 commits into
Open
fix: correct multibyte character handling in get_selected_text()#14visitorise wants to merge 2 commits into
visitorise wants to merge 2 commits into
Conversation
The get_selected_text() method was using character indices from tui-textarea as byte offsets for string slicing. This caused incorrect text selection and potential panics when selecting Korean, Japanese, or other multibyte UTF-8 text. The fix uses the existing char_col_to_byte_offset() helper to properly convert character indices to byte offsets before slicing. Added tests for both English ASCII and Korean multibyte text selection.
…CJK text Fixed byte-offset to char-index conversion in visual line computation to properly position the terminal cursor for IME candidate windows in CJK environments. The cursor position was incorrectly calculated for multi-byte text, causing the IME window to appear at the wrong location. - Fix byte_offset to char_index conversion in VisualLine computation - Add set_terminal_cursor_position to position IME candidate window correctly for CJK text with proper Unicode width calculation - Add tests for CJK cursor positioning and English fallback
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix: CJK IME candidate window positioning and multibyte selection bugs
Problem
Two issues related to CJK (Korean/Japanese/Chinese) input handling in crabcode:
IME candidate windows appeared at wrong position: The
Inputcomponent'srendermethod rendered the textarea text but never calledframe.set_cursor_position()to tell ratatui where the physical terminal cursor should be positioned. This is critical for CJK input methods, which rely on the terminal knowing the exact cursor cell to position their candidate windows (the popup that shows character candidates as you type). Withoutset_cursor_positionbeing called, IME candidate windows appeared at the top-left of the screen (0,0), making CJK text input essentially unusable.Text selection crashed with multibyte characters: The
get_selected_text()method used character indices (fromtui-textarea) directly as byte offsets when slicing the underlyingString. For Korean text like"안녕하세요"(5 chars, 15 bytes — each character is 3 bytes in UTF-8), slicing at byte position 2 would land in the middle of the first character, producing garbage or panic.Root Causes
Issue 1: Missing terminal cursor position
TextAreawidget fromtui-textareaonly draws a visual cursor in the bufferratatui'sFrameAPI providesset_cursor_position()which moves the physical terminal cursor after renderingset_terminal_cursor_positionwas never called, ratatui left the terminal cursor hidden (default behavior when no cursor position is set during render)Issue 2: Character indices vs byte offsets
tui-textareastores cursor positions as character indices (Unicode scalar values), not byte indices"안녕하세요": selecting chars 2-3 should return"하세", but slicing at bytes 2-3 hits the middle of the first characterFix
Issue 1: IME cursor positioning (
src/ui/components/input.rs)Added
set_terminal_cursor_position()method that:self.textarea.cursor()UnicodeWidthChar) — Korean/Japanese chars are width-2, so column offset differs from character countframe.set_cursor_position()so ratatui moves the physical terminal cursor after renderingIssue 2: Character-to-byte offset conversion (
src/ui/components/input.rs)Used the existing
char_col_to_byte_offset()helper to convert character indices to byte offsets before slicing:This is consistent with how other parts of the codebase (e.g.,
flat_cursor_offset,flat_offset_for_position,line_char_slice) already handle the char-to-byte conversion.Files Changed
src/ui/components/input.rsget_selected_text()method — usechar_col_to_byte_offset()for proper char-to-byte conversionset_terminal_cursor_position()method for IME cursor placementset_terminal_cursor_positionafter rendering the textareatest_get_selected_text_english_asciitest_get_selected_text_korean_multibytetest_cursor_position_for_ime_cjktest_cursor_position_for_ime_englishTesting
All 1068 tests pass (1066 existing + 2 new IME tests). The 4 failing tests are pre-existing failures unrelated to this change (Ollama CLI tests and one diff rendering test).
Impact