Back to all Case Studies
Back to all Case Studies
Back to all Case Studies
Testing the New UPI Payment Flow: Navigating the Homepage, Understanding the Payment Process, and Enhancing App Usability with a Clearer Call-to-Action.
Testing the New UPI Payment Flow: Navigating the Homepage, Understanding the Payment Process, and Enhancing App Usability with a Clearer Call-to-Action.
Testing the New UPI Payment Flow: Navigating the Homepage, Understanding the Payment Process, and Enhancing App Usability with a Clearer Call-to-Action.
Usability . Qualitative . Unmoderated
Usability . Qualitative . Unmoderated
Usability . Qualitative . Unmoderated
About Company
About Company
About Company
Kotak Mahindra Bank Limited is an Indian banking and financial services company headquartered in Mumbai. It offers banking products and financial services for corporate and retail customers in the areas of personal finance, investment banking, life insurance, and wealth management. Under that comes Kotak811 which is a fully Digital Savings Account. Kotak811 Account holders can avail of free funds transfer online using NEFT IMPS or RTGS.
Kotak Mahindra Bank Limited is an Indian banking and financial services company headquartered in Mumbai. It offers banking products and financial services for corporate and retail customers in the areas of personal finance, investment banking, life insurance, and wealth management. Under that comes Kotak811 which is a fully Digital Savings Account. Kotak811 Account holders can avail of free funds transfer online using NEFT IMPS or RTGS.
Kotak Mahindra Bank Limited is an Indian banking and financial services company headquartered in Mumbai. It offers banking products and financial services for corporate and retail customers in the areas of personal finance, investment banking, life insurance, and wealth management. Under that comes Kotak811 which is a fully Digital Savings Account. Kotak811 Account holders can avail of free funds transfer online using NEFT IMPS or RTGS.
Duration
Duration
Duration
1 month
1 month
1 month
Role
Role
Role
Lead User Researcher
Lead User Researcher
Lead User Researcher
Team
Team
Team
1 Product Manager, 1 Note-taker Intern, 2 Client’s Team Designers
1 Product Manager, 1 Note-taker Intern, 2 Client’s Team Designers
1 Product Manager, 1 Note-taker Intern, 2 Client’s Team Designers
Key tools
Key tools
Key tools
Google Forms, Sheets, Docs & Meet, Cogno AI, UserStudy, Grain, Notion
Google Forms, Sheets, Docs & Meet, Cogno AI, UserStudy, Grain, Notion
Google Forms, Sheets, Docs & Meet, Cogno AI, UserStudy, Grain, Notion
Background
Background
Background
The Kotak811 team has designed new flows for their new banking “UPI”. According to the design, the team wants to test the 5 screens in the prototype. The screens are the homepage, scanner & payment, pay to contact, account details, and sharing transactions. They want to know the usability and navigation issues, along with the user’s comprehensiveness of the features.
The Kotak811 team has designed new flows for their new banking “UPI”. According to the design, the team wants to test the 5 screens in the prototype. The screens are the homepage, scanner & payment, pay to contact, account details, and sharing transactions. They want to know the usability and navigation issues, along with the user’s comprehensiveness of the features.
The Kotak811 team has designed new flows for their new banking “UPI”. According to the design, the team wants to test the 5 screens in the prototype. The screens are the homepage, scanner & payment, pay to contact, account details, and sharing transactions. They want to know the usability and navigation issues, along with the user’s comprehensiveness of the features.
Day 1 - 3
Day 1 - 3
Day 1 - 3
Research Planning & Interview Script
Research Planning & Interview Script
Research Planning & Interview Script
Research Objective
Research Objective
Research Objective
To understand users’ thoughts, opinions, and feedback on the different screens on the app without any guidance.
To understand users’ thoughts, opinions, and feedback on the different screens on the app without any guidance.
To determine the user’s understanding of the homepage and overall payment process of the app as the client wants to test their new UPI flow, designs, and usability
To determine the user’s understanding of the homepage and overall payment process of the app as the client wants to test their new UPI flow, designs, and usability
To determine the user’s understanding of the homepage and overall payment process of the app as the client wants to test their new UPI flow, designs, and usability
To access the discovery of the “account details” screen by the participants
To access the discovery of the “account details” screen by the participants
Research Methodology
Research Methodology
Research Methodology
Conducted unmoderated usability testing with a qualitative approach to code themes from the participant's videos.
Conducted unmoderated usability testing with a qualitative approach to code themes from the participant's videos.
Conducted unmoderated usability testing with a qualitative approach to code themes from the participant's videos.
The designed task/script was sent out to filtered participants to perform and talk aloud their thoughts and feedback
The designed task/script was sent out to filtered participants to perform and talk aloud their thoughts and feedback
The designed task/script was sent out to filtered participants to perform and talk aloud their thoughts and feedback
Once we received all the videos from the participants we chose color color-coding thematic analysis
Once we received all the videos from the participants we chose color color-coding thematic analysis
Once we received all the videos from the participants we chose color color-coding thematic analysis
Day 4 - 6
Day 4 - 6
Day 4 - 6
Participant Recruiting & Scheduling
Participant Recruiting & Scheduling
Participant Recruiting & Scheduling
Target participants
Target participants
Target participants
No. of participants: 20 users each with 15mins of tasks performing video
No. of participants: 20 users each with 15mins of tasks performing video
No. of participants: 20 users each with 15mins of tasks performing video
Age: 25-35
Age: 25-35
Age: 25-35
Gender: Male and Female
Gender: Male and Female
Gender: Male and Female
Location: Tier 1 & tier 2 cities of India
Location: Tier 1 & tier 2 cities of India
Location: Tier 1 & tier 2 cities of India
Occupation: Salaried above 25k
Occupation: Salaried above 25k
Occupation: Salaried above 25k
Language: English & Hindi
Language: English & Hindi
Language: English & Hindi
Behaviour: participants who are quite tech-savvy and have some knowledge of UPI payments or have UPI accounts
Behaviour: participants who are quite tech-savvy and have some knowledge of UPI payments or have UPI accounts
Behaviour: participants who are quite tech-savvy and have some knowledge of UPI payments or have UPI accounts
Participant Recruitment Process
Participant Recruitment Process
Participant Recruitment Process
Using our own participant panel or user base we first filtered out 50 participants who we thought were the right participants. To eliminate the drop rates we chose participants who had been a part of our previous studies that were both moderated and unmoderated.
Using our own participant panel or user base we first filtered out 50 participants who we thought were the right participants. To eliminate the drop rates we chose participants who had been a part of our previous studies that were both moderated and unmoderated.
Using our own participant panel or user base we first filtered out 50 participants who we thought were the right participants. To eliminate the drop rates we chose participants who had been a part of our previous studies that were both moderated and unmoderated.
Once the sorting is done, keeping all the criteria in mind, we now use cognoAI to send out a Google Form link to participants’ WhatsApp to fill out the form so that we can recruit more filtered participants. After the second round of vetting, we now launched the study, but with a link for the participants to perform.
Once the sorting is done, keeping all the criteria in mind, we now use cognoAI to send out a Google Form link to participants’ WhatsApp to fill out the form so that we can recruit more filtered participants. After the second round of vetting, we now launched the study, but with a link for the participants to perform.
Once the sorting is done, keeping all the criteria in mind, we now use cognoAI to send out a Google Form link to participants’ WhatsApp to fill out the form so that we can recruit more filtered participants. After the second round of vetting, we now launched the study, but with a link for the participants to perform.
Day 4 - 6
Day 4 - 6
Day 4 - 6
Interview Script
Interview Script
Interview Script
Contents of the interview script
Contents of the interview script
Contents of the interview script
Introduction by a welcome message and ending with a Thank You note
Introduction by a welcome message and ending with a Thank You note
Starting with asking participants about their initial understanding of the homepage
Starting with asking participants about their initial understanding of the homepage
Starting with asking participants about their initial understanding of the homepage
Follow up with performing tasks to scan and pay to known contacts
Follow up with performing tasks to scan and pay to known contacts
Follow up with performing tasks to scan and pay to known contacts
Moving on to discovering account details and sharing transaction details
Moving on to discovering account details and sharing transaction details
Moving on to discovering account details and sharing transaction details
Rating each of the designs and process out of 5 and why
Rating each of the designs and process out of 5 and why
Rating each of the designs and process out of 5 and why
Lastly, any improvements or suggestions they would like in the app
Lastly, any improvements or suggestions they would like in the app
Lastly, any improvements or suggestions they would like in the app
Day 4 - 6
Day 4 - 6
Day 4 - 6
Synthesis, Analysis & Insights
Synthesis, Analysis & Insights
Synthesis, Analysis & Insights
Analysis and Synthesis Process
Analysis and Synthesis Process
Analysis and Synthesis Process
Once all the videos were received, I asked my intern who was helping me with the notetaking to write down the notes in the Google sheet which was pre-made earlier
Once all the videos were received, I asked my intern who was helping me with the notetaking to write down the notes in the Google sheet which was pre-made earlier
Once all the videos were received, I asked my intern who was helping me with the notetaking to write down the notes in the Google sheet which was pre-made earlier
The Google sheet contains all the focus area categories that we had divided and decided to put all the transcription notes there. To lower the burden for my intern I took up 10 videos to transcribe and also to save time.
The Google sheet contains all the focus area categories that we had divided and decided to put all the transcription notes there. To lower the burden for my intern I took up 10 videos to transcribe and also to save time.
The Google sheet contains all the focus area categories that we had divided and decided to put all the transcription notes there. To lower the burden for my intern I took up 10 videos to transcribe and also to save time.
After that, we color-coded the themes and insights in 3 different colors: green for the users who could perform the task successfully, yellow for the users who could partially perform the task, and lastly, and red who could not perform the task at all.
After that, we color-coded the themes and insights in 3 different colors: green for the users who could perform the task successfully, yellow for the users who could partially perform the task, and lastly, and red who could not perform the task at all.
After that, we color-coded the themes and insights in 3 different colors: green for the users who could perform the task successfully, yellow for the users who could partially perform the task, and lastly, and red who could not perform the task at all.
Once the thematic color coding is done, insights are now grouped into different sections to create a summary and report for the client.
Once the thematic color coding is done, insights are now grouped into different sections to create a summary and report for the client.
Once the thematic color coding is done, insights are now grouped into different sections to create a summary and report for the client.
Outcomes/Key insights
Outcomes/Key insights
Outcomes/Key insights
+45% of users did not discover the “pay contact” button and also failed to complete the task
+45% of users did not discover the “pay contact” button and also failed to complete the task
+45% of users did not discover the “pay contact” button and also failed to complete the task
~30% of the users did open the transaction page but still could not locate the “share button”
~30% of the users did open the transaction page but still could not locate the “share button”
40% of the users felt that the positioning of the account details button was not obvious so the users overlooked it
40% of the users felt that the positioning of the account details button was not obvious so the users overlooked it
40% of the users felt that the positioning of the account details button was not obvious so the users overlooked it
20% of the users dropped the “scan and pay” task mid-way out of frustrations of not being able to figure out how to do it
20% of the users dropped the “scan and pay” task mid-way out of frustrations of not being able to figure out how to do it
20% of the users dropped the “scan and pay” task mid-way out of frustrations of not being able to figure out how to do it
Day 4 - 6
Day 4 - 6
Day 4 - 6
Report Submission, Correction and Presentation
Report Submission, Correction and Presentation
Report Submission, Correction and Presentation
Recommendations/Suggestions
Recommendations/Suggestions
Recommendations/Suggestions
Introducing various features is overwhelming for the users, to eliminate that, designers can create an automatic “take a tour” feature to introduce all the features to the users
Introducing various features is overwhelming for the users, to eliminate that, designers can create an automatic “take a tour” feature to introduce all the features to the users
Introducing various features is overwhelming for the users, to eliminate that, designers can create an automatic “take a tour” feature to introduce all the features to the users
Change the colors and shape of the icons to known ones that the users are already familiar with.
Change the colors and shape of the icons to known ones that the users are already familiar with.
Change the colors and shape of the icons to known ones that the users are already familiar with.
Add a “24/7 chat” button in the app for the users if they need any assistance mid-way through
Add a “24/7 chat” button in the app for the users if they need any assistance mid-way through
Add a “24/7 chat” button in the app for the users if they need any assistance mid-way through
Keep all the important features or buttons forefront of the user’s screen so that they do not waste their time finding it
Keep all the important features or buttons forefront of the user’s screen so that they do not waste their time finding it
Keep all the important features or buttons forefront of the user’s screen so that they do not waste their time finding it
Challenges faced and how we overcame
Challenges faced and how we overcame
Challenges faced and how we overcame
Out of the 50 participants we sent out to the study only 12 of the participant’s videos were proper with the insights we needed. Some of their videos had a lot of background noise, some dropped out of the study in the middle and there was some language barrier.
Out of the 50 participants we sent out to the study only 12 of the participant’s videos were proper with the insights we needed. Some of their videos had a lot of background noise, some dropped out of the study in the middle and there was some language barrier.
Out of the 50 participants we sent out to the study only 12 of the participant’s videos were proper with the insights we needed. Some of their videos had a lot of background noise, some dropped out of the study in the middle and there was some language barrier.
One of the solutions that we could come up with was to call the participants again who we thought was 90% making sense of their video while performing the task and talking aloud. We had asked them to redo the test to get their incentive before what we had promised. As a result, we did get all the videos within 2 days with amazing insights.
One of the solutions that we could come up with was to call the participants again who we thought was 90% making sense of their video while performing the task and talking aloud. We had asked them to redo the test to get their incentive before what we had promised. As a result, we did get all the videos within 2 days with amazing insights.
One of the solutions that we could come up with was to call the participants again who we thought was 90% making sense of their video while performing the task and talking aloud. We had asked them to redo the test to get their incentive before what we had promised. As a result, we did get all the videos within 2 days with amazing insights.
Deliverables
Deliverables
Deliverables
Research Plan
Research Plan
Research Plan
Full participants video
Full participants video
Full participants video
Note-taking sheet
Note-taking sheet
Note-taking sheet
Analysis and synthesis sheet
Analysis and synthesis sheet
Analysis and synthesis sheet
Insight and recommendation report
Insight and recommendation report
Insight and recommendation report
Task design script
Task design script
Task design script
Participants demographics
Participants demographics
Participants demographics
Video evidence of the insights
Video evidence of the insights
Video evidence of the insights
Stakeholders presentation
Stakeholders presentation
Stakeholders presentation