INTERACTIVE TACTILE GRAPHICS FOR THE BLIND.

two living only 5% receive education. One of the largest reasons for this is unavailability of teachers with training in special education, or even basic knowledge of handling students with special needs. Education for blind children can be expensive. The expenses include printing Braille scripts, printing tactile graphics, costly technology, The project aims at reducing the cost of education of blind children and making the application interactive with the help of audio interaction. The project will detect finger/touch based input, process the input as per the module selected and give audio output.

There are over two million children in India living with vision loss, and only 5% receive education. One of the largest reasons for this is unavailability of teachers with training in special education, or even basic knowledge of handling students with special needs. Education for blind children can be expensive. The expenses include printing Braille scripts, printing tactile graphics, costly technology, etc. The project aims at reducing the cost of education of blind children and making the application interactive with the help of audio interaction. The project will detect finger/touch based input, process the input as per the module selected and give audio output.
Tactile graphics (TG) are images that are formed in relief on paper or plastic sheets to represent graphics in a tangible form [6]. As an alternative medium, it is an important part of inclusive design and has been used to make information accessible in schools, offices and public spaces. In this project, the use of interactive tactile graphics in primary school education has been addressed, although the framework described may be used in any situation where the basic equipment is available.
Considering the high amount of effort involved in the production of tactile graphics, their utility is rather limited [9]. This is not to say they are non-essential, but that there are many ways they could be enhanced as sources of information.

Existing System
Tactile graphics are used to embossed images on paper that is used to represent images and graphs. It is an important part of inclusive education and have been used to make information accessible in schools, offices and public spaces.
Braille textbooks and special equipment can also increase education costs considerably. Many government schools suffer from issues such as low funding, low teacher attendance and lack of amenities. A low cost device that can be used independently by early-school students could ease many of these challenges currently faced in education for blind children.

Proposed System
The proposed system will be a smartphone or a desktop app which will make the learning of the visually impaired simple and interactive. The application will use image processing techniques to recognize the tangible objects that are placed on the workspace and produce an audio output depending on the object placed.
If all of the NCERT educational content is designed in this medium using a fixed camera and inexpensively produced tactile graphics, cost of educational material for blind children could be brought down by simply reducing the amount of Braille printed textbooks to be purchased.
The next task to take this forward is the context creation interface for special educators. Massive amounts of content needs to be converted from visual to interactive-tactile form. Applications of this system extend beyond education and can be applied in public spaces, transit locations, offices, restaurants, etc. where information access is currently a challenge.

Interactive Tactile Graphics and Its Function:-
This design project uses an inexpensive webcam or a camera phone to make TGs interactive. A camera is mounted at a fixed height above the TG. As a user explores the TG with their hands, the finger motion is recognized as well as gestures are recognized, and is used to interact with the voice system.
Basic content primitives and interactions are designed based on the unique constraints, affordances and potential of this novel interactive medium, which serve as building blocks for more complex content. Usability of the medium is studied using simple tasks designed to demonstrate the possibilities. Many examples of assistive technology developed in the past have been found to useful for a broader audience as well. [10] Similarly, this medium of interaction can be used to enhance the learning experience for sighted children, and for children with learning disabilities [11].

User Interface
The application will give a voice based clue to the student who is currently playing the game. Based on the clue the student will sense the tactile graphic by touch and give a touch based input. Application will detect the input through the camera attached to the system. If the input is correct, the application will acknowledge the correct answer and proceed to the next clue.
For example, if the application gives a voice clue indicating a household animal who likes to eat fish and drink milk and the student spells "CAT", the application will say "Correct" and move to giving the next clue.
Similarly along with spelling, modules for mathematics and board games can be implemented as well. In mathematics module, concepts such as numbers, operations (addition and subtraction), and time on clock can be implemented. Board games can include multiple concepts such as counting, directions; general knowledge, etc. are implemented.

Implementation Methods:-Alphanumeric
Alphanumeric games can be developed to teach basic spelling as well as arithmetic methods for children. It can be used to improve the vocabulary of the user [4].

Counting
The counting game can be implemented using a set of coins. The app would give an audio output to place a specific amount of coins on the board. On placing the coins, the audio output would tell to add more coins or remove some coins thus teaching basic counting. Initially the same braille alphabet cards were used for counting game as were used in the spelling game. However, Vuforia could only detect the instances as one character i.e. if the user places 2 C characters on the board, it will only detect one C. Hence vumarks were implemented for counting game. VuMark is the next generation barcode. It allows the freedom for a brand-conscious and customized design while simultaneously encoding data and acting as an AR target. The designs of VuMark are completely customizable, so you can have a unique VuMark for every unique object.Thus counting game was implemented using VuMark.

Improvements in the Design:-
The application was initially implemented as an augmented reality application where the characters were detected by the AR camera and based on the correctness of the answer, voice based output was given [3][5].
A time limit was set for giving the input after which app proceeds to the next clue. However considering uncertainty of the student recognizing the clue and taking time, virtual button method was implemented. On detecting the image, the app assumes a button on a certain location which when pressed, confirms the input given by the student.
An experiment was conducted with a blind person in order to understand the usability of our product. From the obtained results, various problems were found in the functioning of our product. There are as follows: The Time Based level was too short and difficult for the blind user to search through the tactile graphics blocks and place the right tactile block on the workspace. The Virtual Button was not working as expected as the persons hand kept on pressing the virtual button unintentionally.
After considering the above problems, various improvements are elaborated below:

Gesture based input
If the user gives gestures to confirm his input such as drawing a circle with his finger, it becomes more efficient as there is less room for errors in comparison with time as well as virtual button based input. For gesture recognition, finger detection and tracking algorithms are implemented. This also makes the application more responsive and efficient [8]. Changes based on user experience enhancement can be easily implemented and tested again. Thus gesture based input is the solution to the problems faced by AR app and an improvement in the design.

Voice Based input
Using AI techniques and Natural Language Processing, the app can be made interactive using voice commands [2]. This can completely remove the gesture based inputs and make the app more fun and easy to use. The app can continuously listen to the user's voice input and perform a particular action based on the voice input [7]. Fig. 5:-The app being tested and feedback being provided of the user An interview was conducted with a visually impaired person in order to learn about the difficulties faced by him. Technologies available for aiding the blind, their advantages and disadvantages.

User Research:-
A usability test was performed in order to understand the functionalities provided by the available technologies and their shortcomings. The app was tested with various visually impaired children to find out the requirements and modifications which are needed in the app

Key Findings:-
There are very few technologies available for the visually impaired: 905 Braille Watch: The Braille Watch uses braille to identify the time of the day. However, the dial is too small to fit all braille characters of each number. This makes it difficult to distinguish the numbers and recognize the time.

Google/Apple Talkback:
It allows the user to use mobile using gestures. For example, if the user gets a call and wants to answer it, he needs to swipe up anywhere on the screen. However, not many people know about it and use it.

Be My Eyes:
It is an app that allows a visually impaired person to ask help from a sighted volunteer if he need help with some task like crossing the road. However, there are not many volunteers available at all time on the app. Vuforia is an augmented reality software that is used for image recognition. The images to be recognized are stored in the Vuforia database. The application is built in Unity and uses Vuforia to recognize the alphabet cards.
Compares it with the database and outputs whether the image is matched or not.

Conclusion:-
Thus interactive, cost effective and user friendly tactile graphics application is implemented using image processing. This can be a replacement for traditional methods for educating visually impaired students which is difficult and monotonous and teach them subjects like mathematics, Basic English, geography, general knowledge, etc. Simple tasks are designed using primitives designed for the project. Task completion and think out loud evaluation is conducted with the blind users.
The lighting conditions have an impact on the quality of detection and tracking. Calibration options are available, but it is found that when daylight is inconsistent, or if there are shadows then the performance becomes erratic. Steady and uniform light makes the detection and tracking more accurate. Size of the objects also contributes to the tracking quality. When the lighting conditions are inconsistent, a larger object compensates for the tracking errors. Ideally the size of the objects under right lighting conditions is found to be 10mm x 10mm, which is around the same size as a fingernail.
In the future, the applications of this system can be extended beyond education and can be applied in public spaces, transit locations, offices, restaurants, etc where information access is currently a challenge. Furthermore, AI can be