ArticulateMe
Audio Visual Speech Therapy for DHH Children
A
j
r
B
k
s
C
l
t
D
e
m
f
v
g
o
w
h
p
x
i
y
z
/ 22
0
Sound Articulate
Role
Human- Computer - AI - Interaction Designer
+ UX Researcher
Duration
6 Months
Institution
IIT Bombay
Year
B.Des Thesis
Most DHH children get 30 minutes of precise speech therapy in a clinic and then 10,000 minutes at home where everyone is guessing.
In the clinic, a trained therapist models the sound, shapes the lips, corrects gently, and tracks progress. At home, tired parents hold up flashcards at 9 PM and hope they're doing it right. Practice collapses after 90 seconds. A week later, the therapist spends half the session undoing wrong habits instead of moving forward.
I grew up Deaf in a hearing family. My parents did this dance for years without any clear feedback. ArticulateMe is my attempt to fix that gap.
01
What I Set Out to Fix
Objective
I didn't want to build "yet another speech app". I wanted to fix one specific failure in the system:
Design a home practice companion that gives DHH children real-time, visual feedback on every attempt, and gives parents confidence that they're helping, not harming.
To do that, the solution had to:
1
Work for three people at once : child, parent, therapist.
2
Turn a phone camera into a kind of therapist mirror — not a toy.
3
Produce reliable data that therapists can actually use to plan care.
If a family could realistically do 5 minutes of practice a day, feel "this is working", and therapists could see the progress in clinic, the project would have succeeded.
02
Inside the World of DHH Speech Therapy
Domain
Deaf and Hard of Hearing children rely heavily on early, consistent intervention to develop spoken language. Speech therapy for them is not just about hearing sounds; it's about learning how to shape sounds with lips, tongue, and airflow — a very physical, visual process.
Today, that world looks like this:
Clinics and hospitals
Structured sessions, professional techniques, heavy time pressure on therapists.
Special schools and resource rooms
Group activities, mixed abilities, limited individual attention.
Homes and neighborhoods
Parents and children trying to repeat clinic exercises in messy, real environments.
Tools
Mirrors, paper worksheets, articulation charts, generic educational apps, sometimes video calls.
What's missing is not content. It's guided, precise feedback at home — especially tailored to DHH children's reliance on visual cues and repetition.
03
My Role in This System
About the Project
ArticulateMe began as my 6‑month B.Des final-year project at IIT Bombay, but it was never "just" an academic brief. It sat directly on top of my own lived experience as a Deaf student and child of hearing parents.
Project Setup
Type
B.Des Final Year Project (Thesis)
Institute
IIT Bombay, Industrial Design Centre
Timeline
6 months
Team
Solo designer, with guidance from faculty, audiologists, and speech therapists
My roles
Primary & Secondary Research
Service/System Mapping
Interaction & UI Design
Prototyping & AI-behavior Specification
I set myself two constraints:
Constraint 01
Real world first
Insights had to come from actual clinics, families, and therapists — not just papers.
Constraint 02
Feasible stack
The solution had to run on commodity phones using a camera + mic; no special hardware, no lab-only tech.
The output at this stage is a tested, high-fidelity prototype and a complete interaction model, not a fully deployed product.
04
Seeing the Whole System, Not Just the Screen
Brainstorming & Mappings
Before drawing interfaces, I mapped everything: the objects, spaces, people, and interactions that already exist in DHH speech therapy.
Artifacts: The Things Everyone Touches
These are the objects that carry the current system:
Context
Artifact
How it's used today
Hidden problem
Clinic
Mirror
Child watches their lips as therapist models sounds
Only available in clinic; no home version
Clinic
Articulation charts & worksheets
Given as "homework"
No feedback loop once sent home
Clinic
Session notes / reports
Stored in files, sometimes shared with parents
Too technical, not actionable daily
Home
Flashcards / notebooks
Parents improvise practice routines
Children lose interest in 1–2 minutes
Home
YouTube videos
Used as visual references
Passive; no checking of the child's attempt
Home
Phone camera
Sometimes used as a mirror
No intelligence, no guidance
Digital
WhatsApp / calls
Parents send questions or videos to therapists
Asynchronous, irregular, hard to track
What if the phone — already present in most homes — could become an intelligent, therapist-style mirror instead of a useless camera?
Spaces: Where the Story Actually Plays Out
The same child moves through different spaces that completely change how practice feels.
Clinic room
Controlled, quiet, one-on-one. Therapist's full focus, specialized tools, high cognitive load but also high trust.
School classroom
Noisy, social, full of comparison. Child may avoid speaking to escape embarrassment.
Home
Kitchen table, sofa, parent's lap, or on a bus. Siblings around, TV on, chores happening. Time slices of 5–10 minutes at best.
Online space
WhatsApp voice notes, short videos, occasional teletherapy calls.
The design couldn't assume "perfect clinic conditions". It had to survive kitchen light, background noise, and a child who may have just come back from school.
Stakeholders: The People Holding Different Pieces
I mapped the main people in the system and their goals.
Stakeholder
What they want
What blocks them
DHH Child
To be understood and participate confidently
Boring drills, fear of being "wrong" in front of others
Parent / Caregiver
To help, not harm, their child's progress
No clear guidance; guilt and exhaustion
Speech Therapist
To improve articulation efficiently
Zero visibility into home practice; time pressure
Teacher
To support in class without singling out
Little insight into therapy activities
This made it clear: ArticulateMe isn't a "kid app". It's a shared tool for a small, high-stakes team.
Where the Current System Breaks
Once the artifacts, spaces, and people were visible, the failure points became obvious.
The clinic session ends with a stack of paper and a vague: "Practice this at home."
At home, practice depends entirely on parent energy and memory of how the therapist did it.
Children receive corrective feedback once a week , long after wrong habits are formed.
Therapists plan the next session with zero structured data from those 10,000 minutes.
Everyone feels they are working hard, yet feels they are failing.
The system is not failing from lack of care. It is failing from lack of continuous, precise feedback between sessions.
Interactions: Today vs What They Should Be
To capture this, I compared critical moments before and after the envisioned intervention.
Situation
Today (Before ArticulateMe)
With ArticulateMe (Desired Interaction)
Child practices /P/ at home
Parent holds flashcard, imitates therapist from memory; child repeats a few times, gets bored or upset
Phone shows animated lips, watches child's attempt, gives gentle, specific feedback
Parent wonders "Did we do enough?"
No clear answer; maybe messages therapist rarely
Sees a simple summary: attempts, accuracy trend, and a one-line suggestion for tomorrow
Therapist plans next session
Asks "Did you practice?"; hears vague "yes" or "no"
Opens dashboard with per-sound accuracy and home practice history
Child talks about practice
Associates practice with "drills" and parental stress
Experiences practice as a short, visual "game" with structured improvement
These mappings turned a vague intention ("support practice at home") into a concrete interaction brief
.
05
From 200 Notes to One Clear Problem
Affinity Mapping & Scales
I ended up with pages of quotes, screenshots, and observations. Affinity mapping turned that chaos into a focused problem.
Clustering What People Said and Did
All research notes were grouped by emotional tone and systemic failure, not just topic.
Cluster / Theme
Example insight or quote
Feedback delay
"I only know something went wrong a week later." — Speech therapist
Parent guilt & anxiety
"I'm scared I'll teach her wrong, so some days I just don't start." — Parent
Visual vs audio learning
Children locked onto lips and faces more than spoken words
Boredom & resistance
Child hides flashcards after 1–2 minutes; responds longer to screens/games
Data gap for therapists
"I don't know what they did all week at home." — Speech therapist
Time & energy limits
Parents working long hours, practicing late at night, often exhausted
Instead of seeing 200 "findings", I saw 6 forces pulling the system away from success.
What Mattered Most (Affinity "Scale")
Not every problem could be solved in one thesis project, so I prioritised by impact and frequency
.
High impact · High frequency (must solve)
●
Feedback delay between sessions
●
Lack of real-time, visual feedback for the child
●
Parent confidence / fear of doing harm
High impact · Lower frequency (future work)
●
Poor connectivity in some areas
●
Multi-child, multi-language households
Lower impact · High frequency (polish)
●
Minor UI text confusions
●
Preference for specific reward styles
Low / Low (parking lot)
●
Deep clinic EMR integrations
●
Printer-based materials
This led to a single, sharp problem statement:
The home speech practice loop is blind.
DHH children repeat errors, parents guess, and therapists only discover issues a week too late.
ArticulateMe focuses entirely on closing that loop with visual, AI-assisted feedback at home and a shared sense of progress for child, parent, and therapist.
06
Secondary Research
Understanding the Landscape Before Stepping Into It
Before visiting clinics or talking to families, I needed to understand what the literature and existing solutions already knew about DHH speech development.
What I Looked At
I reviewed:
Clinical and public health reports on childhood hearing loss, early intervention, and the importance of consistent speech practice for DHH children.
Global burden and trend studies
showing how many children live with disabling hearing loss and how that number has changed over time.
Guidelines from organizations and research groups on how families should support speech and language at home (roles of parents, frequency of practice, importance of repetition).
Existing tools and apps: AAC systems, generic speech therapy apps, educational games, and communication aids used by DHH children.
Key Things I Learned
1. Early, frequent practice is critical — but hard to sustain.
Research consistently emphasizes that DHH children need frequent, structured practice to develop intelligible speech, not just occasional clinic sessions.
2. Visual information is central for DHH learners.
Studies highlight that DHH children rely heavily on visual cues (lips, facial expressions, gestures) to understand and produce spoken language, which reinforces the need for visual-first tools.
3. Families are expected to be co-therapists, but rarely trained.
Guidelines often assume parents can carry out daily practice, yet offer limited support on
how
to give correct feedback or build a sustainable routine.
4. Most digital tools are not built specifically for DHH speech articulation.
Many existing apps focus on vocabulary, general language, or generic articulation, with limited adaptation to DHH-specific needs (e.g., lip pattern emphasis, hearing device use, noise, home environments).
How Secondary Research Shaped the Design Brief
It confirmed that
the problem is systemic, not local
: DHH children worldwide face similar practice challenges.
It sharpened the requirement that my solution must be
visual-first
,
repeatable in short bursts
, and
realistic for busy families
.
It highlighted a gap: very few tools close the loop between
clinic guidance
,
home practice
, and
therapist visibility
.
Secondary research gave me the confidence that I wasn't solving a "nice-to-have" problem, but a recognised gap that existing tools only partially address.
07
Primary Research
Listening to the People Living This Every Day
Once I understood the landscape on paper, I went into clinics, schools, and homes to see how speech practice actually happens.
Who I Spoke To and Observed
I used a mix of interviews, shadowing, and contextual observation.
Participant group
Count / scope
What I focused on
DHH children (5–10 yrs)
12 children
Reactions to practice, boredom, engagement
Parents / caregivers
10 parents
Daily routines, emotional journey, home practice
Speech therapists
5 professionals
Techniques, constraints, blind spots
Clinics & special schools
3 institutions, 15+ sessions observed
Real therapy flows vs. homework given
*Numbers are rounded to protect identities; individual details are anonymised.
What I Did (Methods)
Clinic & school shadowing
Sat in therapy sessions (with consent), watched how therapists:
• Used mirrors, touch, and exaggerated lip movements
• Broke sounds into components
• Adjusted for each child\'s level and device (hearing aids, implants)
Contextual inquiry at home
Listened to parents describe and demonstrate how they tried to practice:
• Where they sat, how long sessions lasted
• What tools they used (flashcards, notebooks, phones)
• Moments where practice turned into conflict or guilt
Semi-structured interviews
• With parents: routines, fears, motivators, what "good practice" meant to them
• With therapists: caseloads, planning time, what they wish they could see from home practice
Quick concept probes
Showed simple sketches (e.g., phone mirror, progress screen) to test whether ideas felt intuitive or overwhelming.
What I Heard and Saw
From children:
• Strong attraction to screens and to seeing their own faces.
• Quick boredom with paper-based drills; attention moved from flashcards to anything else in 1–2 minutes.
• Pride when they knew they "got it right", but also visible frustration when corrected repeatedly without clear reasons.
From parents:
"I'm scared I'll teach it wrong, so sometimes I just don't start."
"After work, I don't have the energy for a 30‑minute session, but I could do 5 minutes."
"I wish I knew if we're actually improving between visits."
From therapists:
"I only know something went wrong when the child comes back a week later."
"I give detailed home instructions, but I have no idea if they\'re followed or how."
"If I had simple per-sound data from home, I could plan much more targeted sessions."
Primary Research Insights That Directly Shaped the Product
1. The clinic is not the bottleneck; the home loop is.
Therapists are trained and structured. The breakdown happens at home, where there is
no reliable feedback
and practice is emotionally loaded.
2. Visual, live feedback is more powerful than post-hoc correction.
Children responded strongly when they could see immediately that "green = good attempt". Parents found it easier to encourage when the system acted as a neutral judge.
3. Parents need emotional support, not just instructions.
A design that reduces guilt ("I'm ruining my child") and replaces it with achievable micro-goals is as important as the child\'s UI.
4. Therapists want data, but only if it doesn't add work.
Any therapist-facing view had to transform existing home behavior into insight, not ask them to manage a second system.
Primary research turned the abstract brief into a vivid picture of evenings at the kitchen table, clinic rooms under time pressure, and children who wanted to be understood but were tired of being drilled. ArticulateMe grows directly out of those scenes.
08
Competitive Analysis
Why Existing Tools Weren't Enough
Before defining my solution, I reviewed existing products and tools that DHH children and their families already use or could use.
What I Looked At
• AAC and communication tools for non-verbal and DHH users
• Generic speech therapy / articulation apps
• Educational games and language apps for children
• Clinic-provided worksheets, printouts, and mirror-based exercises
Summary Table
Tool type
Strengths
Gaps for DHH home speech practice
AAC / communication apps
Great for basic expression and vocabulary
Not focused on articulation practice or lip shapes
Generic speech apps
Lots of content, some audio feedback
Limited visual-lip focus and DHH-specific tuning
Educational games
High engagement, kid-friendly
Learn "words" but not how to physically form sounds
Clinic worksheets & mirrors
Clinically grounded, familiar
Don't travel well into home life; no feedback loop
Competitive Insight
Most tools solved
pieces
of the puzzle — vocabulary, motivation, or clinic instruction — but none provided:
• Real-time
visual + acoustic feedback
on each attempt at home
• A shared, trusted data view for
child, parent, and therapist
together
This validated that ArticulateMe should focus not on "more content", but on
better feedback and shared understanding
.
09
Affinity Scales
From 200 Notes to a Single Problem
After primary and secondary research, I had a wall of quotes and observations. Affinity mapping and prioritisation turned them into a clear direction.
Core Themes That Emerged
Theme
Example signal
Feedback delay
Therapists only see mistakes a week later in clinic.
Parent guilt
Parents avoid practice out of fear of teaching it wrong.
Visual learning bias
Children focus on lips and faces more than on sounds alone.
Motivation & boredom
Children quit drills quickly but stay longer with interactive media.
Data gap
Therapists have no structured view of what happened at home.
Time & energy constraints
Parents realistically have 5–10 minutes, not 30+ minute blocks.
Prioritisation (Affinity "Scale")
I rated each theme by
impact on outcomes
and
frequency across participants
:
High impact · High frequency → Focus now
• Feedback delay between clinic and home
• Parent confidence and emotional safety
• Need for visual, real-time feedback
High impact · Lower frequency → Plan for later
• Connectivity issues in some homes
• Multiple children / languages in one household
Lower impact → Local UI/UX decisions
• Reward styles, screen microcopy, aesthetic preferences
Resulting Problem Statement
The at-home speech practice loop is blind. DHH children repeat errors without knowing it, parents guess and feel guilty, and therapists only discover problems when it's too late to correct them quickly.
ArticulateMe is designed as a direct response to this blind loop.
10
Identifying Problem Areas
Where the System Actually Breaks
To go deeper than one statement, I broke the experience into
problem areas
:
1
Moment of Practice at Home
• No live feedback on whether an attempt is correct.
• Practice stops quickly because it feels like failing, not progressing.
2
Parent's Decision to Initiate Practice
• Emotional load: "What if I reinforce bad habits?"
• Competes with work fatigue, chores, and other children.
3
Therapist's Planning Process
• No reliable record of which sounds were practiced or how often.
• Session time used to diagnose what happened instead of building on it.
4
Child's Relationship With Practice
• Practice often associated with tension and pressure.
• No clear, visual sense of improvement or mastery.
These problem areas created a structured target for the intervention: fix the home moment, support the parent decision, feed therapists useful data, and make practice emotionally safer for the child.
11
Scoping the Project
What This Thesis Would and Would Not Do
Given limited time and a solo design role, I scoped the project deliberately.
In Scope
✓
Designing the child practice flow (Learn → Mirror Mode → Feedback).
✓
Designing the parent-facing summary (daily view + trend).
✓
Designing a lightweight therapist dashboard (who needs attention, which sounds).
✓
Specifying AI behaviors (what is checked, what feedback states exist).
✓
Testing with DHH children and families to validate the interaction model.
Out of Scope (for now)
○
Full clinic EMR integration.
○
Multi-language, large-scale deployment.
○
Formal clinical trials and regulatory classification.
○
Deep teletherapy workflows beyond basic sharing.
The scope for this phase:
prove that a phone-based, visual + AI feedback loop at home is usable, emotionally safe, and valuable to all three stakeholders.
12
Proposed Intervention
Turning the Phone Into a Therapist-Style Mirror
The proposed intervention is a three-sided system
:
👦
Child-facing app
Visual lip animations + Mirror Mode with live feedback.
👨👩👦
Parent summary view
Simple, encouraging guidance after each session.
👩⚕️
Therapist dashboard
Clear per-sound progress and practice consistency.
Instead of a worksheet and "Please practice at home", the therapist now prescribes ArticulateMe. At home, the child opens Mirror Mode, sees how to pronounce through lip movement sound should look, says it into the camera, and gets instant, visual confirmation. The parent gets a simple "Today you did X and here's what to focus on next". The therapist, next week, sees who practiced what and where to intervene.
The intervention lives inside the routines families already have — phones, short bursts of time — but changes the quality of feedback and the sense of shared progress
.
13
User Personas
Understanding Our Three Key Stakeholders
👦
Aarav
Age 7, DHH Child
Profile
• Moderate to severe hearing loss
• Uses hearing aids
• Attends mainstream school
• Loves mobile games
Behaviors
• Bored quickly with worksheets
• Engages with visual feedback
• Avoids speaking publicly
Needs
• Visual models of sounds
• Immediate feedback
• Progress, not punishment
👩
Priya
Age 34, Working Parent
Profile
• Full-time job, long commute
• One DHH child + sibling
• Committed but exhausted
Goals
• Improve Aarav's confidence
• Not "undo" therapy work
• Fit practice into busy day
Frustrations
• "Scared to teach wrong"
• Guilt when skipping
• Technical therapy reports
Needs
• Simple, clear guidance
• Daily reassurance
• Proof of effort to therapist
👩⚕️
Dr. Sharma
Age 42, Speech Therapist
Profile
• Works at city clinic
• Manages 15–20 DHH children
• Skilled but time-pressed
Goals
• Improve articulation
• Maximize session efficiency
• Support confident families
Frustrations
• "No idea what happened"
• Re-diagnosing vs building
• No time to train parents
Needs
• Quick progress overview
• Trustworthy data
• No extra admin work
15
Concept Mapping
Connecting People, Places, and Feedback Loops
To align all three personas, I created a concept map of how information, actions, and emotions flow through the system.
System Flow Concept
Current State (Broken Loop)
CLINIC
Gives worksheet
→
PARENT
Tries to help
→
CHILD
Practices blindly
→
NO FEEDBACK
Week passes
→
THERAPIST
Discovers issues too late
New State (ArticulateMe)
CLINIC
Prescribes ArticulateMe
→
CHILD
Uses Mirror Mode
(live AI feedback)
→
PARENT
Sees summary
→
THERAPIST
Sees practice data
→
NEXT SESSION
Targeted intervention
Key Improvements
•
Live feedback loop between child and device at home
•
Shared truth (data) that parent and therapist can both see
•
Reduced emotional load on parents by letting the system handle correctness
Conceptually, the map shows how ArticulateMe adds a live feedback loop between child and device at home, creates a shared truth (data) that parent and therapist can both see, and reduces emotional load on parents by letting the system handle correctness, so they can focus on encouragement.
16
Why This Solution? Why AI?
The Strategic Rationale
Why This Solution?
This solution was chosen because it aligns directly with:
The most critical problem
Lack of real-time, precise feedback at home.
The most constrained actors
Parents with limited time and high emotional load.
The most underused asset
Phones with cameras already in many homes.
It doesn't try to replace therapists or redesign clinical workflows. It extends them into the home in a realistic, everyday form.
Why AI?
AI (specifically computer vision + audio analysis) is used because:
Therapists can't be present at home.
An automated system can approximate "Is this attempt close enough?" when a clinician is not in the room.
Children need instant, attempt-level feedback.
AI enables per-attempt evaluation of lip position and sound quality, turning each repetition into a learning moment.
Therapists need structured data, not anecdotes.
AI-generated metrics (e.g., attempts, estimated accuracy per sound) give a quick, consistent snapshot of home practice they can trust.
Crucially, AI here is not a gimmick; it is the only scalable way to bring therapist-style, per-attempt feedback into ordinary homes without overloading human experts.
17
Information Architecture
Making Sure Every Screen Has a Clear Job
Before drawing detailed UI, I defined how information should flow between child, parent, and therapist views. The goal was to keep each surface focused and lightweight , not overloaded.
High-Level Structure
At a high level, the system splits into three worlds:
Child app (Practice world)
• Home
• Learn (sound library)
• Practice (Mirror Mode)
• Progress (simple, child-friendly overview)
Parent app / mode (Support world)
• Today's session summary
• Weekly trends
• Tips & guidance
• Share with therapist
Therapist dashboard (Clinic world)
• Patient list
• Per-child overview
• Per-sound breakdown
• Notes & plan adjustment
IA Table (Child Side)
Screen
Purpose
Key elements
Child Home
Launchpad for practice
Today goal, streak, Start Practice button
Sound Library
Choose what to practice
Grid of sounds with simple status (locked / in progress / done)
Learn Sound
Understand correct visually
Big animated lips, slow/loop controls, simple description
Mirror Mode
Real-time practice & feedback
Camera view, lip box, speak cue, feedback messages
Progress (child)
Celebrate and reinforce progress
Stars, levels, simple badges, streak visualization
IA Table (Parent Side)
Screen
Purpose
Key elements
Parent Home
Answer How did we do today
Attempts, time spent, today focus sounds
Weekly View
See consistency over time
Calendar or bar chart of sessions, small trend arrows
Sound Detail
Understand strengths & challenges
Accuracy per sound, simple explanations
Guidance
Know what to do next
1-2 line suggestions, FAQs, When to ask therapist cues
IA Table (Therapist Side)
Screen
Purpose
Key elements
Dashboard
Quickly triage who needs attention
Child list, status chips (On-track / Needs attention / Inactive)
Child Detail
Prepare for next session
Accuracy per sound, practice frequency, short notes
Sound Breakdown
See patterns across children
Table or chart of sounds vs. average accuracy
Notes & Plan
Adjust goals and instructions
Editable plan, recommendations synced back to parent app
This Information Architecture ensured each persona has just enough information to act confidently, without being overwhelmed.
18
Mind Mapping & Flow Mapping
Visualising the Journey From 'Open App' to 'Feel Progress'
To translate the IA into concrete experiences, I built mind maps and user flows for key journeys.
Mind Map: Child Journey
At the center of the mind map : "Child wants to practice (or is asked to)"
.
Child wants to practice
Motivation
• Start button feels like "play", not "test"
• Small, reachable goal visible up front (e.g., "3 sounds today")
Understanding
• Tap sound → see animation first
• Option to replay until they feel ready
Action
• Enter Mirror Mode → speak when cued
• See box color + tip immediately
Reward
• Star, small celebration, clear "Next"
• Progress toward streak or level
The mind map helped ensure no step was "floating"; each had emotional and functional support around it.
Flow Diagram: Parent Journey
Starting node: "Parent thinks: Should we practice now?"
STEP 1
Opens app → sees Today card (3 sounds · ~5 min)
→
STEP 2
Taps Start with [Child name] → guided mode
→
STEP 3
After session → sees summary with friendly feedback
→
STEP 4
Optionally shares summary with therapist
This flow ensured the parent's cognitive load stays low: one decision at a time, no deep menu diving.
Flow Diagram: Therapist Journey
Starting node: "Therapist preparing for today's sessions"
STEP 1
Opens dashboard → sees list with status chips
→
STEP 2
Taps child flagged Needs attention
→
STEP 3
Skims per-sound data and practice history
→
STEP 4
Jots quick note or adjusts focus sounds
The flows kept the time-to-insight for therapists very short.
19
Wireframes
Roughing Out the Experience Before Adding Polish
Once flows were solid, I sketched and then built low-fidelity wireframes to validate structure and content.
Child Wireframes
Key wireframes:
Child Home
• Minimal elements: avatar, today goal, big Start Practice button.
• Optional: small streak indicator to encourage continuity.
Learn Sound
• Top: large animated mouth placeholder.
• Middle: simple label (P) and hint (Lips together, then a small burst).
• Bottom: buttons for Slow, Replay, and Try in Mirror Mode.
Mirror Mode
• Full-screen video frame.
• Soft bounding box for lips.
• Progress bar or cue showing when to speak.
• Space for 1-line feedback message.
Parent Wireframes
Parent Home
• Today card: attempts, time spent, quick smiley or simple Great job today.
• Weekly graph: days practiced vs skipped.
Sound Insight
• List of sounds with simple labels (Strong, Improving, Needs attention).
• Tap to see a short explanation.
Therapist Wireframes
Dashboard
• Search by child name.
• Filter by Needs attention, Inactive (no practice), etc.
Child Detail
• Mini timeline of last week practice.
• Sound accuracy chart (bar or chip list).
• Space for quick notes.
Wireframes were tested early with a small number of parents and therapists to ensure they could answer basic questions like "Can you tell what to do next?" and "Would you know where to look for progress?"
20
Visual Design & Design System
Making It Feel Safe, Playful, and Trustworthy
After wireframes were validated, I developed a visual system tuned for DHH children and their families.
Design Principles
Gentle, not clinical
Avoided harsh medical aesthetics; used soft colors and rounded shapes.
Readable at a glance
Large touch targets, high contrast where it matters, minimal text on child screens.
Emotionally safe
No red crosses or failed messages; feedback focuses on almost there, try this tone.
Core Styles
Color palette
•
Primary: calm blues/teals for trust.
•
Accent: soft greens for success, never aggressive reds for correction.
•
Neutrals: light greys for backgrounds, ensuring focus on content.
Typography
• Friendly sans-serif font.
• Bigger sizes for children screens; clear hierarchy on parent/therapist screens.
Iconography & Mascot
• Simple, rounded icons, easily readable.
• Friendly guide character used sparingly to celebrate wins and gently encourage.
Components
Built a mini design system including:
• Buttons (primary, secondary, pill-style chips).
• Cards (for daily summaries, sound status).
• Status indicators (e.g., Locked / In progress / Strong).
• Graph styles (for parent & therapist views).
This system made the experience cohesive across child, parent, and therapist worlds.
21
Prototype
Bringing the Experience to Life
With the design system in place, I created an interactive prototype focusing on the core loop :
•
Child: Learn → Mirror Mode → Feedback → Celebration
•
Parent: View summary → Understand what's next
•
Therapist: See snapshot of home practice before a session
Prototype Scope
Depth, not breadth:
Rather than building every screen, I built the most critical flows end-to-end:
• Child picks one sound and completes a short practice session.
• Parent sees the summary and weekly view.
• Therapist sees that child recent data.
High-fidelity interactions:
• Mirror Mode transitions between idle, listening, and feedback states.
• Smooth animations on success (short, non-distracting).
• Tap-through paths that match real usage (no fake shortcuts).
Behaviours Modeled
Feedback timing and messages (e.g., no instant punishment after a single mispronunciation).
Session length (~5 minutes) and pacing (when to suggest ending vs one more try).
How summaries roll up small details into simple, reassuring messages.
The prototype was realistic enough that children and parents reacted to it as if it were a real app, which made the next step — testing — much more valuable.
22
Usability Testing With the Prototype
What Worked, What Didn't, and What Changed
With the interactive prototype ready, I ran multiple rounds of usability testing.
Testing Setup
Participants:
• DHH children already in speech therapy
• Their parents
• A few therapists observing or using the therapist view
Scenarios:
• Child completes a short practice session with one or two sounds.
• Parent explores what happened and decides what to do tomorrow.
• Therapist uses the dashboard to prepare for a hypothetical session.
Key Findings
For children:
Findings:
• Mirror Mode was intuitive; they understood green = good quickly.
• Long celebrations disrupted flow; short cheers worked better.
• Too many unlocked sounds confused them.
Changes made:
→ Shortened celebration animations.
→ Limited the number of sounds available at first; unlocked new ones gradually.
For parents:
Findings:
• Loved the Today summary, but some didn't understand all metrics at first.
• Wanted very clear language (Good progress, Keep focusing on P and B).
Changes made:
→ Simplified the language and used icons/smiley-style cues.
→ Added one-line Next step at the bottom of the summary.
For therapists:
Findings:
• Appreciated per-sound accuracy and last practiced indicators.
• Wanted a way to quickly see who hadn't practiced at all.
Changes made:
→ Added simple filters: No practice in 7 days, Dropping accuracy, etc.
→ Made per-sound table the default view for a child, not hidden behind a click.
Outcome
Usability testing proved that: Children understood and enjoyed the Learn → Practice → Feedback → Reward loop. Parents felt less anxious and more guided after a few sessions. Therapists could realistically use the dashboard in under a minute per child.
These results gave confidence that the core interaction model was solid and worth taking further, beyond the thesis phase.