On the Evaluation of Novel Sonification Techniques for Non Visual Shape Exploration

Sergio Mascetti, Andrea Gerino, Cristian Bernareggi and Lorenzo Picinali

Interaction examples from the Invisible Puzzle application.

AS MOST SONIFICATIONS FROM INVISIBLE PUZZLE MAKE USE OF SPATIALIZED AUDIO YOU SHOULD WEAR HEADPHONES WHILE WATCHING THE VIDEOS. PLEASE CHECK THE L/R AS INDICATED TO ENSURE PROPER USE.

• Sonification_1D_FF_Music.mov : this video shows two tasks sonified with the 1-D fixed frequency music sonification
• Sonification_1D_FF_Noise.mov : this video shows two tasks sonified with the 1-D fixed frequency noise sonification
• Sonification_1D_FF_Pure.mov : this video shows two tasks sonified with the 1-D fixed frequency pure sonification
• Sonification_1D_VF_Noise.mov : this video shows two tasks sonified with the 1-D variable frequency noise sonification
• Sonification_1D_VF_Pure.mov : this video shows two tasks sonified with the 1-D variable frequency pure sonification
• Sonification_2D.mov : this video shows two tasks sonified with the 2-D sonification
• Sonification_1D_NI_Pure.mov : this video shows two tasks sonified with the 1-D NI Pure sonification
• Sonification_2D_Pulse.mov : this video shows two tasks sonified with the 2-D Pulse sonification

• Interaction_1D.mov : this video shows how users are introduced to the 1-D exploration mode, the first learning task and an example of an exploration task
• Interaction_2D.mov : this video shows how users are introduced to the 2-D exploration mode, the first learning task and an example of an exploration task
• Interaction_NI.mov : this video shows how users are introduced to the 1-D NI exploration mode, the first learning task and an example of an exploration task

• Interaction_VO.mov : this video shows how subjects with visual impairments interact with the application using VoiceOver

Sonification source code

If you are interested in obtaining the software implementing the sonifications, please write us at sergio.mascetti@unimi.it

Tasks from the Invisible Puzzle application.

In the following we report, for each task in Invisible Puzzle, the set of possible answers. In the following figures the first image on the left is the correct answer. Clearly in Invisible Puzzle the answers are shuffled.

Each figure's caption is the description that is read to the user by the speech synthesizer.

(a) Three points placed top left, centre, bottom right

Fig. 1. Training task for the first task group.

 

(a) One point placed middle right (correct)

(b) One point placed bottom right

(c) One Point placed in the centre

(d) One point placed top left

Fig. 2. Objects included in the first task of the first task group.

 

(a) One point placed top left (correct)

(b) One point placed middle left

(c) One point placed middle right

(d) One point placed in the centre

Fig. 3. Objects included in the second task of the first task group.

 

(a) Two points: top left and bottom right (correct)

(b) Two points: middle left and top right

(c) Two points: top left and middle right

(d) Two points: centre and top right

Fig. 4. Objects included in the third task of the first task group.

 

(a) Two points: middle left and middle right (correct)

(b) Two points: centre and top right

(c) Two points: middle left and top right

(d) Two points: bottom left and top right

Fig. 5. Objects included in the fourth task of the first task group.

 

(a) One central vertical line

Fig. 6. Training task for the second task group.

 

(a) One vertical line on the left (correct)

(b) One central vertical line

(c) One diagonal line from top left to bottom right

(d) One vertical line on the right

Fig. 7. Objects included in the first task of the second task group.

 

(a) One central horizontal line (correct)

(b) One central vertical line

(c) One vertical line on the left

(d) One diagonal line from top left to bottom right

Fig. 8. Objects included in the second task of the second task group.

 

(a) One diagonal line from top left to bottom right (correct)

(b) One central vertical line

(c) One central horizontal line

(d) One diagonal line from bottom left to top right

Fig. 9. Objects included in the third task of the second task group.

 

(a) Reverse V shaped lines (correct)

(b) One vertical line on the left and one on the right

(c) X shaped lines

(d) V shaped lines

Fig. 10. Objects included in the fourth task of the second task group.

 

(a) Diamond

Fig. 11. Training task for the third task group.

 

(a) Triangle (correct)

(b) Diamond

(c) Circle

(d) Square

Fig. 12. Objects included in the first task of the third task group.

 

(a) Square (correct)

(b) Diamond

(c) Triangle

(d) Circle

Fig. 13. Objects included in the second task of the third task group.

 

(a) Triangle with one base vertically aligned on the left (correct)

(b) Triangle

(c) Centered square

(d) Square on the left

Fig. 14. Objects included in the third task of the third task group.

 

(a) Wide diamond (correct)

(b) Narrow diamond

(c) Square on the left

(d) Triangle with one base vertically aligned on the left

Fig. 15. Objects included in the fourth task of the third task group.

 

(a) Triangle (correct)

(b) Reverse V shaped lines

(c) V shaped lines

(d) Diamond

Fig. 16. Objects included in the first task of the fourth task group.

 

(a) Circle (correct)

(b) Central vertical line

(c) X shaped lines

(d) Centered square

Fig. 17. Objects included in the second task of the fourth task group.

 

(a) Octagon (correct)

(b) Diamond

(c) Centered square

(d) Circle

Fig. 18. Objects included in the third task of the fourth task group.

 

(a) Large circle (correct)

(b) Small circle

(c) Pentagon

(d) Octagon

Fig. 19. Objects included in the fourth task of the fourth task group.

 

(a) Star

Fig. 20. Training task for the fifth task group.

 

(a) Hearts (correct)

(b) Diamonds

(c) Clubs

(d) Spades

Fig. 21. Objects included in the first task of the fifth task group.

 

(a) Butterfly (correct)

(b) Octopus

(c) Fish

(d) Giraffe

Fig. 22. Objects included in the second task of the fifth task group.

 

(a) Car (correct)

(b) Motorbike

(c) Airplane

(d) Train

Fig. 23. Objects included in the third task of the fifth task group.

 

(a) Cherry (correct)

(b) Banana

(c) Pear

(d) Apple

Fig. 24. Objects included in the fourth task of the fifth task group.

 

(a) Uppercase Z

Fig. 25. Training task for the sixth task group.

 

(a) Uppercase G (correct)

(b) Uppercase X

(c) Uppercase K

(d) Uppercase O

Fig. 26. Objects included in the first task of the sixth task group.

 

(a) Lowercase a (correct)

(b) Lowercase i

(c) Lowercase b

(d) Lowercase o

Fig. 27. Objects included in the second task of the sixth task group.

 

(a) Number 8 (correct)

(b) Number 3

(c) Number 0

(d) Number 9

Fig. 28. Objects included in the third task of the sixth task group.

 

(a) Number 5 (correct)

(b) Number 3

(c) Number 9

(d) Number 4

Fig. 29. Objects included in the fourth task of the sixth task