Home Syllabus Project Readings
Part 3
Table of contents
- Overview
- Part 3A – Paper Prototype
- Part 3B – Heuristic Evaluation
- Part 3C – Usability Testing Check-in
- Part 3D – Usability Review
- Part 3E – Digital Mockup
- Part 3F – Presentation
- Part 3G – Final Report
Overview
Now that you have chosen a design, it is time to iteratively refine that design. This group assignment, tackles the problem of getting the design right through paper prototyping, heuristic evaluation, and user testing. Design refinement culminates in a digital mockup.
This assignment is worth 20 of the 60 points of your course project. The following are the stages that you need to complete for this assignment:
| Part | Due Date |
|---|---|
| 3A Paper Prototype (2 points) | Mon., Mar. 25 @ 11:59 pm |
| 3B Heuristic Evaluation (2 points) | Wed., Mar. 27 @ 11:59 pm |
| 3C Usability Testing Check-in (2 points) | Sun., Mar. 31 @ 11:59 pm |
| 3D Usability Testing Review (2 points) | Sun., Apr. 7 @ 11:59 pm |
| 3E: Digital Mockup (2 points) | Mon., Apr. 8 @ 11:59 pm |
| 3F: Presentation (5 points) | Tue. Apr. 9 @ 1:00 pm |
| 3G: Final Report (5 points) | Mon., Apr. 15 @ 11:59 pm |
Project Description
In this assignment, you will iteratively improve your design. You will construct a paper prototype informed by your earlier work. You will then use inspection to identify and correct issues in the paper prototype. You will next conduct a series of three usability tests, further identifying and correcting issues in your paper prototype. You will create a digital mockup that captures the result of your design process. Finally, you will present your design process in a report and presentation.
Part 3A
Develop a paper prototype for your design. It should convey all critical aspects of your design, including support for testing your two primary tasks. This prototype has been informed by your prior sketching and storyboarding. But it also contains more details than your prior exploratory designs. If necessary or appropriate, revise your tasks or scenarios based on what you learn in prototyping.
Part 3B
Conduct a heuristic evaluation of your paper prototype. You will perform an evaluation in class, and will supplement this with another evaluation. Revise your paper prototype based on what you learn in heuristic evaluation. Your tasks and scenarios are likely stable, but revise them if your evaluation reveals a need.
Part 3C and 3D
Conduct usability tests with three participants, each completing your two primary tasks. Make a log of critical incidents during the test, including both positive and negative events. For example, a person might make a mistake or might comment on something they like. Write it down along with a description of the context of the incident. Later you will discuss and prioritize these events, assigning a severity rating to problems (ranging from 0 for no problem to 4 for usability catastrophe).
You should not use friends, members of the class, or people you know to have engineering backgrounds. You also should not use people who have been previously exposed to your project. Target an appropriate set of participants based on your design research and resulting design.
Be sure to divide your team into roles when conducting the usability test. There is simply too much for a single person to conduct a test. As discussed in class, these roles include the facilitator, the computer, and one or more observers. As part of your introduction, have your facilitator show the participant how to interact with your design. Do not show participants how to perform your tasks that you will later test. Instead show them how paper prototyping works, how your system generally works, and give an example of something specific that is sufficiently different from your tasks.
Consider recording your tests to allow you to see exactly what happened. If you do this, ensure your recording preserves the anonymity of the participant. Frame your shot such that you capture interaction with the prototype without unnecessarily capturing the participant.
Each participant should perform both tasks. Keep the data separate for each participant and each task. Keep participant names confidential, and preserve anonymity in your reporting. Remember participation must be voluntary, participants should be free to stop at any time, and you should emphasize that your focus is on identifying usability issues in the design.
Revise your paper prototype based on what you learn in usability testing. This can and should include fixing major issues as soon as they are detected, before additional tests. Your tasks and scenarios are likely stable, but revise them if your evaluation reveals a need.
Part 3E
After refining your paper prototype based on your usability testing, construct a digital mockup of your design.
Do not begin working in digital prototyping until you finish revising your paper prototype. It is much easier to iterate on your design in the paper prototyping phase. As you transition into digital prototyping, you will consider other aspects of the design. These details you may not have considered (e.g., visual design, layout, color, typography), but can also include constraints or mismatches introduced by the technology itself.
We have no strong preference to what digital tools you utilize to create your mockup. But it must be digital (e.g., you may not use pictures of a hand-drawn sketch).
Some recommended software include:
- Adobe Experience Design (XD)
- Adobe Illustrator
- Microsoft Power Apps
- Axure
- Balsamiq
- Figma
- Indigo Studio
- InkScape
- Microsoft Visio
- Omnigraffle
- Vectr
You will use this digital mockup to effectively communicate your design in your final report, website, and poster. Some tools support creation of interactive mockups (e.g., allow clicking active regions to move between screens). This would add additional capabilities to your website, and you are welcome to use such a tool. But interactive mockups are neither expected nor required. Implement your digital mockup to include key frames that illustrate your design. It should convey all critical aspects of your design, including walkthroughs for your two primary tasks.
Part 3F and 3G
Summarize your design process in a report and presentation.
Part 3A – Paper Prototype
Due Date: Mon., Mar. 25 @ 11:59 pm
The goal of this part is to develop an initial paper prototype that will begin to allow testing of your design.
Develop a paper prototype for your design. It should convey all critical aspects of your design, including support for testing your two primary tasks.
This prototype has been informed by your prior sketching and storyboarding. But it also contains more details than your prior exploratory designs. If necessary or appropriate, revise your tasks or scenarios based on what you learn in prototyping.
Deliverable
Capture and submit detailed images of your paper prototype. Include:
- An overview photo and paragraph that shows and explains your entire paper prototype.
- Detailed photos showing each component as you walk through your two primary tasks with tour paper prototype.
Contributions
Your submission must also include a team contribution statement as an additional page in your submission, just as in previous parts. This will be used to adjust your grades in case of unequal participation.
Submission
Collect your text, images, and contributions into a PDF.
You will submit through Blackboard.
Grading
This will be graded on a scale of 2 points:
- Overall Completeness and Appropriateness of Paper Prototype: (0.5 points)
- Completeness, Appropriateness, and Documentation of Each Task: (0.75 points per task)
Examples
Part 3B – Heuristic Evaluation
Due Date: Wed., Mar. 27 @ 11:59 pm
The goal of this milestone is to obtain a heuristic evaluation of your paper prototype according to the Nielsen heuristics. Recall that the evaluators decide on their own how they want to proceed with evaluating the interface – they are not necessarily using the prototype to perform a task, but will discover heuristic violations on their own.
You will receive two heuristic evaluations of your paper prototype. Make a list of what heuristic is violated and a severity rating (ranging from 0 for no problem to 4 for usability catastrophe). The fixability rating is not required. The list of Nielsen heuristics is here.
Plan to revise your paper prototype based on what you learn in heuristic evaluation. You will submit before and after photos of your revised prototype in Part 3C. Your tasks and scenarios are likely stable, but revise them if your evaluation reveals a need.
Deliverable
Please provide the following in your submission:
- the group number and the names of the classmates from which you received two heuristic evaluations
- the group numbers to which you provided heuristic evaluations to
- an overview image showing your paper prototype and the notes from your received evaluations
The details of your heuristic evaluations will be submitted in Part 3C as a table, so please make sure to save all your received feedback.
Contributions
Your submission must also include a team contribution statement as an additional page in your submission, just as in previous parts. This will be used to adjust your grades in case of unequal participation.
Submission
Records of your inspection-based methods will be submitted as part of later assignments.
Be sure to keep detailed records, including images of relevant portions of the prototype that you presented to the other groups.
You will submit through Blackboard.
Grading
This step will be graded on a scale of 2 points with the following breakdown:
- Heuristic Evaluation (1 point for each received evaluation)
Examples
Part 3B Examples from Fall 2021
Part 3C – Usability Testing Check-in
Due Date: Sun., Mar. 31 @ 11:59 pm
The goal of this part is to document changes resulting from your heuristic evaluation and to begin usability testing of your revised prototype with participants.
Complete one usability test with your paper prototype, examining your two primary tasks. Target an appropriate participant based on your design research and resulting design.
Be sure to divide your team into roles when conducting a usability test. There is simply too much required for a single person to effectively conduct a test. As discussed in class, roles include the facilitator, the computer, and one or more observers.
As part of your introduction, have your facilitator show the participant how to interact with your design. Do not show participants how to perform your tasks that you will later test. Instead show them how paper prototyping works, how your system generally works, and give an example of something specific that is sufficiently different from your tasks.
During the test, record a log of critical incidents, including both positive and negative events. For example, a person might comment on something they like or might make a mistake in a task. Record the incident along with a description of the context of the incident.
Consider recording your tests to allow you to see and show exactly what happened. You do not need to this, but if you do this, obtain permission from the participant and ensure your recording preserves the anonymity of the participant. Frame your shot such that you capture interaction with the prototype without unnecessarily capturing the participant.
Each participant should perform both of your primary tasks. Keep the data separate for each participant and each task. Remember participation must be voluntary, participants should be free to stop at any time, and you should emphasize that your focus is on identifying usability issues in the design.
Revise your paper prototype as you perform your tests using what you learn in heuristic evaluations and usability testing. This means fixing major issues as soon as they are detected, before additional tests are performed. Your tasks and scenarios are likely stable, but revise them if your evaluation reveals a need.
Deliverable
Present your iterative revisions and your plan going forward:
- Present a table or list of results from your heuristic evaluation. For each identified issue, include:
- The heuristic that is violated in the walkthrough.
- A description of the usability issue, including an image of the relevant portion of the prototype.
- A severity rating (ranging from 0 for no problem to 4 for usability catastrophe).
- An image and explanation of any revision implemented as a result.
Discuss your first usability test. Provide a description of the test, including the participant, the environment where you conducted the test, why you chose this participant and environment, the test protocol, and the roles of each team member who participated in the test. Discuss anything you learned about the testing process itself, or any revisions you decided you need to make to your testing process.
- Present a table or list of results from your first usability test. For each critical incident, include:
- An image of the relevant portion of the prototype.
- Description of the incident, remembering that these can be either positive or negative.
- For negative incidents, a severity rating (ranging from 0 for no problem to 4 for usability catastrophe).
- For negative incidents, an image and explanation of any revision implemented as a result.
- Submit detailed images of your revised paper prototype after both the heuristic evaluation and usability test. Include both:
- An overview image that shows your entire paper prototype.
- Detailed images showing each component as you walk through your two primary tasks.
- Present your plan for the remainder of your usability tests. Describe what goals you have for the additional tests and any new approaches you will employ. Describe your plans for recruiting an additional participant.
You may find that some changes feel minor relative to the overhead and work involved in documenting those changes, which is why you should focus on fixing the major usability issues first. Your overall goals in this part of the assignment are to:
- Show your work in the iterative design process, giving you enough insight to understand and convey your progress.
- Create a document for sharing as web content and for material to be used in a presentation.
You may choose to not fully document minor changes, but be sure to document enough of them to satisfy the two above goals.
Contributions
Your submission must also include a team contribution statement as an additional page in your submission, just as in previous parts. This will be used to adjust your grades in case of unequal participation.
Submission
Collect your text and contributions into a PDF. The amount of text should be sufficiently descriptive.
You will submit through Blackboard.
Grading
This step will be graded on a scale of 2 points with the following breakdown:
- Issues Identified from Heuristic Evaluation, Revisions Made in Response: (0.5 points)
- Discussion of Usability Test and Plan: (0.5 points)
- Issues Identified in Usability Test, Revisions Made in Response: (0.5 points)
- Updated Images of Paper Prototype: (0.5 points)
Examples
Part 3D – Usability Review
Due Date: Sun., Apr. 7 @ 11:59 pm
The goal of this part is to complete usability testing of your prototype with participants. Complete a total of three usability tests (including your first one from Part 3C) and finalize your paper prototype according to what you learn.
Deliverable
Present the results of your usability testing and your final paper prototype:
Discuss your three usability tests. Provide a description of each test, including the participant, the environment where you conducted the test, why you chose this participant and environment, the test protocol, and the roles of each team member who participated in the test. Briefly discuss revisions you made in your process and prototype over the course of your testing.
- Present a table or list of results from your three usability tests. For each critical incident, include:
- An image of the relevant portion of the prototype.
- Description of the incident, remembering that these can be either positive or negative.
- For negative incidents, a severity rating (ranging from 0 for no problem to 4 for usability catastrophe).
- For negative incidents, an image and explanation of any revision implemented as a result.
- Submit detailed images of your final paper prototype, as revised in this process. Include both:
- An overview image that shows your entire paper prototype.
- Detailed images showing each component as you walk through your two primary tasks.
- Discuss two to four revisions that you consider to be your most salient and important modifications based on issues identified in heuristic evaluation, usability testing. These may already be included in the above list, but discuss them in more detail and argue why detecting and fixing them is important to your design.
Again, you may find that some changes feel minor relative to the overhead and work involved in documenting those changes, which is why you should focus on fixing the major usability issues first. Your overall goals in this part of the assignment are to:
- Show your work in the iterative design process, giving you enough insight to understand and convey your progress.
- Create a document for sharing as web content and for material to be used in a presentation.
You may choose to not fully document minor changes, but be sure to document enough of them to satisfy the two above goals.
Contributions
Your submission must also include a team contribution statement as an additional page in your submission, just as in previous parts. This will be used to adjust your grades in case of unequal participation.
Submission
Collect your text and contributions into a PDF. The amount of text should be sufficiently descriptive.
You will submit through Blackboard.
Grading
This will be graded on a scale of 2 points:
- Discussion of Usability Tests and Plan: (0.5 points)
- Issues Identified in Usability Tests, Revisions Made in Response: (0.5 points)
- Updated Images of Paper Prototype: (0.5 points)
- Discussion of Key Revisions in Design Process: (0.5 points)
Examples
Part 3E – Digital Mockup
Due Date: Mon., Apr. 8 @ 11:59 pm
The goal of this part of the assignment is to transition your prototype into a digital mockup of your design.
Create your digital mockup based on your final paper prototype from Part 3D. Do not begin working in digital prototyping until you finish revising your paper prototype. It is much easier to iterate on your design in the paper prototyping phase.
As you transition into digital prototyping, you will need to consider other aspects of the design. This can include detail you may not have previously considered (e.g., visual design, layout, color, typography), but can also include constraints or mismatches introduced by the technology itself. Note your decisions or changes you need to make as you transition from paper to a digital representation.
The mockup should effectively communicate your design in a manner appropriate for your web content and presentation. Implement your digital mockup to include key frames that illustrate your design. It should convey all critical aspects of your design, including walkthroughs for your two primary tasks.
We have no strong preference regarding what digital tools you use to create your mockup. But it must be digital (e.g., you may not use pictures of a hand-drawn sketch). Some tools support creation of interactive mockups (e.g., allow clicking active regions to move between screens). You are welcome to use such a tool, and such interactivity may be helpful in your video. But interactive mockups are neither expected nor required.
Some recommended software include:
- Adobe Experience Design (XD)
- Adobe Illustrator
- Microsoft Power Apps
- Axure
- Balsamiq
- Figma
- Indigo Studio
- InkScape
- Microsoft Visio
- Omnigraffle
- Vectr
Deliverable
Present your digital mockup together with discussion of the decisions and changes you made:
- Submit detailed images of your digital mockup. Include both:
- Overview images that show your digital mockup.
- Detailed images showing each component as you walk through your two primary tasks.
- Discuss decisions and changes you made in implementing the digital mockup.
Contributions
Your submission must also include a team contribution statement as an additional page in your submission, just as in previous parts. This will be used to adjust your grades in case of unequal participation.
Submission
Collect your text and images and contributions into a PDF. The amount of text should be sufficiently descriptive.
You will submit through Blackboard.
Grading
This will be graded on a scale of 2 points:
- Overall Completeness and Appropriateness of Digital Mockup: (0.5 points)
- Completeness, Appropriateness, and Documentation of Each Task: (2 x 0.5 points)
- Discussion of Decisions and Changes in Implementation: (0.5 points)
Examples
Part 3F – Presentation
Due Date: Tue. Apr. 9 @ 1:00 pm
Prepare a presentation of your process in getting the design right. It should encompass all of your work in Part 3.
Deliverable
- Two members of your team should deliver the presentation, each speaking to relatively equal portions.
- A 7 minute time limit will be strictly enforced.
- Your presenters should be different from your presenters in Part 2H, such that everybody has presented at least once this quarter, unless your group has three members.
A suggested organization of this presentation is:
- Title: A short, creative, and marketable title capturing the key idea. Include team member names and roles.
- Overall Problem: Tell this as a story, instead of simply reading the slide. Motivate your audience to be interested in your problem.
- Initial Paper Prototype: Present your initial paper prototype. Convey the critical aspects of your design, including your two primary tasks.
- Testing Process and Results: Describe your testing process, what you learned from it, and how you refined your prototype as a result.
- Final Paper Prototype: Present your final paper prototype. Convey the critical aspects of your design, including your two primary tasks. This might be shorter than your presentation of the initial paper prototype, because you already presented changes and are about to present the digital mockup. Keep your presentation appropriately paced instead of getting bogged down here.
- Digital Mockup: Present your digital mockup. Convey the critical aspects of your design, including your two primary tasks.
- Summary: Summarize the lessons learned in your design process.
Contributions
Your submission must also include a team contribution statement as an additional page in your submission, just as in previous parts. This will be used to adjust your grades in case of unequal participation.
Submission
Your presentation may be in PPT, PPTX, PDF or Keynote format.
You will submit through Blackboard.
Grading
This will be graded on a scale of 5 points:
- Presentation shows appropriate preparation, with visual aids that are effective, properly prepared, and properly employed.
- Slides are legible, such that people in the back of the room can still see them.
- Presentation should not have an outline slide. It is short enough to be told as a story of your process.
- Problem is presented in a manner that is compelling and achievable.
- First paper prototype effectively demonstrates completion of a scenario.
- Testing process is appropriate for the design.
- Testing process is carried out in an appropriate manner.
- Testing results are illuminating in terms of the design.
- Final paper prototype has a strong connection to the results of the testing process.
- Digital mockup effectively implements the final paper prototype.
- Digital mockup is at the proper fidelity.
- Presentation covered the required scope within the allowed time period.
Examples
Part 3G – Final Report
Due Date: Mon., Apr. 15 @ 11:59 pm
Prepare a report documenting your process of getting the design right. Your report should follow the outline given in the Deliverable section below and will be graded using the grading guidelines. The provided page allocations are estimates, intended to help convey how to divide up the space.
If you completed all of your milestones above, you will have much of the content for this report. But it is critical that you revise and update that content. You have received extensive feedback throughout your design process, and evaluation of your report will include how you have addressed and incorporated that feedback to improve relative to your prior milestones.
Deliverable
Provide the following in your report:
- Title: A short, creative, and marketable title capturing the key idea.
- Each Team Member’s Name and Role
- Problem and Solution Overview: (1 paragraph) A concise statement of the problem you are tackling and a brief synopsis of your proposed solution. Consider updating this from your previous report to reflect the scope of your final design.
- Initial Paper Prototype: (1 page) Present your original paper prototype, as a baseline against which this report will present your iteration. Convey the critical aspects of your design, including your two primary tasks.
- Testing Process: (1 page) Present your testing process, including your method and your participants. This process probably evolved some in the course of your project. Include a retrospective on how you refined your process, separate from how you refined your design.
- Testing Results: (2 pages) Summarize the results of your paper prototype testing and refinement. Present what you learned about each version of your prototype and what changes you made as a result. This will include changes from your heuristic evaluation, usability testing, and design critique.
- Final Paper Prototype: (1 page) Present your final version of your paper prototype, as you did for your initial paper prototype above. Convey the critical aspects of your design, including your two primary tasks.
- Digital Mockup: (1 page) Present your digital mockup. Convey the critical aspects of your design, including your two primary tasks. Briefly discuss any changes you needed to make as you switched to your digital tools instead of paper. Briefly discuss any changes you made in response to critique. Include descriptions of how your design supports each of your primary tasks (e.g., one paragraph per task).
- Discussion: (1 page) Reflect upon and discuss your project and your results. For example:
- What did you learn from the process of iterative design?
- How did the process shape your final design?
- How have your tasks changed as a result of your usability tests?
- Do you think you could have used more, or fewer, iterations upon your design?
- Appendix: Attach copies of all materials involved in your testing. Includes any instructions or task descriptions you handed out or read aloud to your participants. Include identified critical incidents from your usability testing. The appendix materials and screenshots do not count in your page limit.
Contributions
Your submission must also include a team contribution statement as an additional page in your submission, just as in previous parts. This will be used to adjust your grades in case of unequal participation.
Submission
No more than eight pages of text in PDF format, following the above outline. Images and your appendix do not count against your page limit, and are therefore effectively free. You should embed images throughout your PDF, keeping them near the text that references them. The limit applies to the approximate amount of text you would have if all images were removed.
You will submit through Blackboard.
Grading
This will be graded on a scale of 5 points:
- Title: (0.2 points)
- Team Member Names and Roles: (0.2 points)
- Problem and Solution Overview: (0.4 points)
- Initial Paper Prototype: (0.6 points)
- Testing Process: (0.4 points)
- Testing Results: (0.6 points)
- Final Paper Prototype: (0.6 points)
- Digital Mockup: (0.6 points)
- Discussion: (0.6 points)
- Report Clarity and Presentation: (0.7 points)
- Appendix: (0.1 point)