Google’s mobile keyboard app, Gboard, uses statistical decoding to counteract the inherent inaccuracies of touch input on small screens, often referred to as the ‘fat finger’ problem. To assist users, Gboard has several features covering word completion, next-word predictions, active auto-correction and active key correction. However, these models do struggle with more complex errors which need longer-span contexts and users have to systematically repair words using grammar and spell checkers.
To overcome these issues, Google has developed a feature called Proofread. This innovation addresses the most prevalent problems for rapid typers, improving their productivity by offering repairs at sentence and paragraph levels at a single touch. Consequently, users find it much simpler to rectify errors in their text.
Proofread falls under the broad heading of Grammatical Error Correction (GEC) and has been influenced by research into rule-based solutions, statistical methods and neural network models. Large Language Models have a vast capacity for expansion, providing the opportunity for refining sentence-level grammar corrections.
Data production, metrics design, model tweaking and model serving comprise the backbone of the Proofread feature. They ensure the feature works effectively by ensuring that the data distribution matches the Gboard context as closely as possible. Model evaluation uses several measures to guarantee quality, such as grammar mistake existence checks and ‘same meaning’ checks.
Followed by reinforcement learning, the LLM dedicated to proofreading applies the InstructGPT method using supervised fine-tuning. It was found that the model’s proofreading performance was significantly boosted due to the method of reinforcing learning and tailoring rewrites. Proofread is built on top of a medium-sized LLM PaLM2-XS, which can be run on a single TPU v5 to reduce serving costs.
Results have shown that segmentation, speculative decoding and bucket keys can further reduce latency. The proposed model has already gone live, benefiting tens of thousands of Pixel 8 users. Further research is anticipated to get the most out of LLMs, including exploring aspects such as adaptation to different languages, personalisation, privacy protection on devices and bringing the technology to real-user data.
