Teaching children to read in the digital age with Scribble Stadium
Technology is taking over the world, and every day we find new ways to integrate the cloud into our lives. Sometimes, this can result in asinine apps and widgets, but other times the technology we have developed can prove itself useful in a number of ways. The team at Scribble Stadium has identified one of the ways screen time can be turned into a positive habit — encouraging children to write and read. Scribble Stadium is an app created by sixth-grade teacher Graig Peterson and data scientist Darwin Johnson that encourages children to be more creative by turning children who may not favor reading into authors through an immersive, multi-player game.
Handwriting as an identifier
I worked with a remote, collaborative team of front-end web developers and data scientist in order to create a feature for the game that may not be seen by many people — child identifying language processing. My team was tasked with creating an algorithm that put each character a child has written into a dictionary of the variations of that letter. The basic concept was that a child would copy sentences from an onboarding form, and our code would extract the sentences the children wrote, breaking them down into singular characters, and then assigning them to the correct key value of the dictionary.
As we were creating the MVP, I noticed our code would also take the digital text on the page, the pre-written guide sentences, and try to identify those sentences to be parsed as well. We had completely overlooked the fact that standard OCR does not discriminate between handwritten and digital text, so our code was trying to identify everything on the page. Through some investigation as a team, we were able to identify a process using OpenCV that would help us identify the contours, or outline, of each text box and extract only the handwriting from the inside.
Time and aspirations
While we managed to get a lot accomplished by turning the text into JPGs of boxes containing the handwritten sentences, and JPGs of the individual characters, due to time constraints our team has not yet completed the task of assigning these characters to their rightful values in the dictionary.
Despite the work that is yet to be done, we consider the creation of the onboarding form, and subsequent extraction of the handwriting to be great progress. In the future we hope for the feature to be completed with a dictionary for each individual child, as well as generated reports and visualizations of the child’s progress in handwriting, and storytelling through transcribed entries made by the children. This feature is the groundwork for many more features to come in terms of personalizing Scribble Stadium for each child, and therefore encouraging the creativity that can be generated with proper support and care.
Given the anticipated size of the user base, I foresee scalability as a technical challenge we may encounter, but I believe with proper attention our code can be optimized. Throughout the course of this project we have received extremely helpful feedback regarding the onboarding form and our contour finding algorithm. Constructive criticism really pushed our team, and myself especially, to develop better habits in how problems are approached and tested, such as when we discovered the bug with the text identifier.
This project has really opened my eyes to the possibilities of OCR, and Tesseract specifically, as my career goals are centered around business analysis and marketing. Learning and practicing extracting data from handwritten forms, while possibly obsolete in the near pandemic-ridden future, has helped me really understand how OCRs like OpenCV and Tesseract operate and how to adapt them to my needs and wants. These are two new tools added to my tool belt, and I’m excited to learn about more in the future.
