Adventures with UITextInteraction

Last year at WWDC19, Apple introduced UITextInteraction saying, “it’s only three lines of code to add all of the system text selection gestures to your app.”

That got my attention. Storyist for iOS first shipped on iOS 4–before Text Kit came to the platform—and to this day has a lot of custom code to handle text layout and selection. Over the years, Storyist has tracked Apple’s changes to the text selection UI. It has been a necessary but tedious task, and one I’d happily leave to Apple. I’ve been to enough WWDCs to take “three lines of code” with a very large grain of salt, but I was eager to put the API through it’s paces and adopt it if possible.

Unfortunately, early tests were disappointing, especially with respect to keyboard shortcut support, and I shelved the project last fall. But with the introduction of iOS 13.4 and the prospect of having to add custom pointer support to the custom selection code, I decided to give UITextInteraction another go.

Here’s what I found.

Under the Hood

Most components of UTextInteraction have been part of the platform for a while as private classes used by UITextView and UITextField.

The “three lines of code” are:

let selectionInteraction = UITextInteraction(for: .editable)
selectionInteraction.textInput = textView
textView.addInteraction(selectionInteraction)

Adding the interaction does several things. First, it installs a view hierarchy beneath your custom text view to represent the selection controls. When the insertion point is set, the view hierarchy is as follows:

CustomTextView
    UITextSelectionView
        UIView <-- Cursor

Second, it adds gesture recognizers to handle the selection.

UITextLoupePanGestureRecognizer
UITextMultiTapRecognizer
UIVariableDelayLoupeGesture
UITextMultiTapRecognizer
UITapAndAHalfRecognizer
UITapGestureRecognizer
_UISecondaryClickDriverGestureRecognizer
_UITouchDurationObservingGestureRecognizer
_UITouchDurationObservingGestureRecognizer
_UIRelationshipGestureRecognizer
_UIRelationshipGestureRecognizer
_UIPointerInteractionHoverGestureRecognizer
_UIPointerInteractionPressGestureRecognizer

When a range of text is selected, the hierarchy becomes:

CustomTextView
    UITextSelectionView
        UITextRangeView
            UIView
                UIView <-- Selection rects
            UISelectionGrabber
            UISelectionGrabber
    UISelectionGrabberDot
    UISelectionGrabberDot

and the custom text view gains the following recognizers:

UIPanGestureRecognizer
UITapGestureRecognizer

UITextInteraction also adds the following recognizers to UITextRangeView:

UITextRangeAdjustmentGestureRecognizer
_UIPointerInteractionHoverGestureRecognizer
_UIPointerInteractionPressGestureRecognizer

Challenges Implementing Autoscroll

The first challenge in adopting UITextInteraction is deciding how to implement auto-scroll. UITextInteraction doesn’t support it explicitly, and while the gesture recognizer that tracks cursor movement is available in -[UITextInteraction gesturesForFailureRequirements], the recognizer that tracks the range controls (UITextRangeAdjustmentGestureRecognizer) is not.

Fortunately, you can obtain a reference to it from -[UIGestureRecognizer gestureRecognizer:shouldRecognizeSimultaneouslyWithGestureRecognizer:] without using private API. This is obviously an opportunity for enhancement. (FB7637400)

Once you have a reference to the recognizers, you can track the gestures and scroll the text appropriately.

Implementing Zoom

Storyist, like Pages, supports pinch-to-zoom. Unfortunately, the UITextInteraction UI doesn’t track the zoom or adjust the selection when zoom finishes. One solution is to hide the selection UI when zooming begins and then update the UI when it ends. UITextInteraction doesn’t have API for doing this, so you have to either remove the interaction when the zoom begins and add it back when it ends, or hide/show the private view installed by the interaction. Neither option is very elegant. (FB7637377)

6/16/20 Update: As @knutknatter points out, the selection handles do scale during the pinch. The issue is with what happens when you release the pinch at the new scale. To keep the text and UI crisp, you need to transform the text container (probably a subview of your custom text view) and reset the scroll view zoomScale. UITextInteraction should be part of this process so that the handles and highlight are drawn at the correct size and location.

Implementing Floating Cursor Support

In iOS 9, Apple added a feature to iPad that allowed you to move the cursor by tapping the virtual keyboard with two fingers and dragging to a new location. To support this in a custom text view, you need to implement beginFloatingCursorAtPoint:, updateFloatingCursorAtPoint:, and endFloatingCursor and do two things:

1. Return a view object to represent the floating cursor.
2. Dim the real cursor that tracks that actual position while the floating cursor is visible.

UITextInteraction does not provide an API to access the cursor view, so it’s challenging to implement step 2. You could walk the view hierarchy and find the cursor view (it is a grandchild of your custom text view), but that is not a maintainable solution.

Problems with Keyboard Shortcuts

Support for keyboard shortcuts has improved since last fall, but some shortcuts are still missing. Those that use arrow keys plus modifiers largely work. However, as of 13.4, many “Emacs-style” shortcuts are not implemented by UITextInteraction or leave the selection UI in an incorrect state. (FB7655372)

Shortcut	Command
⌥⌫	Delete Word Backward
⌘⌫	Delete to Beginning of Line
⌃P	Move Up
⌃N	Move Down
⌃F	Move Forward
⌃B	Move Backward
⌥⌃B	Move Word Backward
⌥⌃F	Move Word Forward
⇧⌃B	Move Backward and Modify Selection
⇧⌃F	Move Forward and Modify Selection
⇧⌃P	Move Up and Modify Selection
⇧⌃N	Move Down and Modify Selection
⌃A	Move to Beginning of Paragraph
⌃E	Move to End of Paragraph
⇧⌃A	Move to Beginning of Paragraph and Modify Selection
⇧⌃E	Move to End of Paragraph and Modify Selection

These shortcuts have been supported on iOS for some time, and advanced users expect them to be available. You’ll need to implement them in the app if you need them.

Control Tint Color

Both UITextView and UITextField display the cursor and selection handles using the effective tint color of the view. They accomplish this using the private insertionPointColor, selectionBarColor, and selectionHighlightColor properties of UITextInputTraits. UITextInteraction does not respect the custom text view’s tint color (FB7657103), so for a custom text view to support the iOS conventions, you need to override these private properties to return the appropriate color.

Spell Checking and UITextReplacement

UITextInteraction respects the spellCheckingType property of UITextInputTraits. If the value is UITextSpellCheckingTypeDefault or UISpellCheckingTypeYes, spell checking occurs as expected. Unfortunately, UITextInteraction uses the private UITextReplacement class to handle spell checking. When it encounters a misspelled word, it correctly presents a menu with the spelling options. However, when you tap the correction, it attempts to call the private replace: method with a sender of private class UITextReplacement. The replacementText property of the sender contains the misspelling. I can report that using this private method and property earns you a rejection from app review, so it isn’t possible to use the spell checking features of UITextInteraction.

UITextInteraction implements other private menu actions as well, for example _lookup:, _define: and _share: to name a few. It’s currently not possible to remove them.

Miscellaneous Bugs

The UITextInteraction support in Storyist is still in beta, and there are several selection-related bugs I’ve not been able to track down yet. UITextInteraction is necessarily dependent on the underlying UITextInput and UITextInputTokenizer implementations, so it’s possible these are Storyist bugs, not UITextInteraction bugs. With that said:

• If you press the down arrow key several times in a row, then tap or click elsewhere, then press the down arrow key again, the selection ends up one line down from the previous down arrow, not one line down from the tap. Down arrow support in UITextInteraction is implemented by the -[UITextInteractionSelectableInputDelegate _moveDown:withHistory:] method. I’m guessing that the tap/click handler does not properly clear the history.

• If autocapitalization is enabled and you tap to position the cursor in the middle of a sentence, the letter typed at that location is occasionally capitalized incorrectly.

Next Steps

WWDC20 is right around the corner, and hopefully, many of these issues will be addressed in iOS 14. I’ll update this post if there is anything to report.

If you’re a developer working on an UITextInteraction implementation, get in touch. I’d be happy to share tips and swap promo codes.

And if you’re lucky enough to work at the mother ship, I’d appreciate it if you could flag the Feedback Assistant reports to the appropriate people. They’re all currently marked as Open with no similar reports.