Future trends – Augmented Reality (AR) and Virtual Reality (VR) in Education

Yash Paul Sharma*

*Central Institute of Educational Technology, NCERT, New Delhi

Abstract

ICT changing society dynamically. ICT tools influencing lives. Its influence also felt more and more at educational institutions. Because ICT provides both students and teachers with more opportunities in adapting learning, teaching, and managing individual needs, society is forcing schools to respond to this innovation aptly.  Using ICT in education has become one of the most effective factors in school improvement. In today’s scenario, the ability to connect reality and digital content has been steadily improving, opening more options for teachers and students and we may find some excellent examples of augmented reality (AR) and virtual reality (VR) in education also.

Introduction

Classroom teaching and learning is undergoing unprecedented change with time. Various theories of education play their best role in the teaching/learning process, and Educational Technology provides an extra edge in implementing the same. Although we may not always realize but technology is an integral part of our daily lives. It influences how we communicate, socialize, connect, play and learn. Technology augments its presence in our daily lives; it would be imperative to ensure the presence of the same in the classroom.

Gradually, technology has been making its way to revolutionizing the methods of teaching and learning. Technology has made its way to the classroom, increasing the engaging and interactive elements; many students are benefitting from it, right from Audio, Video, Graphics, and Simulations. Learners, in today’s context, are already familiar with various technologies, which helps in modernizing the classroom with computerized tools and apps that make sense in a classroom setting.

As we are moving into the next generation of the media revolution, “immersive” technologies like Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) are becoming the center of discussion in educational technologies. You must have seen the sci-fi movies, where all data capture and interpretation is just happening in front of glasses or goggles. With the advancement of science, all these technologies are available now, although mostly for commercial purposes. But these technologies equally can change the field of education if employed meticulously. So, before we understand immersive technologies and their implication in education, let’s first discussed what these technologies are.

  1. Augmented Reality: Augmented reality (AR) is a view of the real and physical world in which users find elements enhanced by computer-generated simulations; by overlaying graphics, music, animations, GPS markers/logs, etc. to augment the user environment.

Literary, the word “Augment” refers to the action of adding to something to make it more substantial. Currently, Augmented Reality is being experienced by using a mobile/tablet device and/or AR glasses.

For example, pointing your device towards a building and all information popped out in front of you (building). You are in a place where you are unable to read the language and just by pointing your device, the entire content will be translated into your known language (lens). Or a dinosaur might be landing in front of you (dinosaur), etc.

So, with the help of software AR technology, add to the real environment to make it more enriched, helpful and user friendly. With advances in AR technology, all these examples might already be available in your smartphones through various apps.

“Augmented Reality may be defined as a system that performs three essential criteria: combining virtual and real-world, real-time interaction, and 3D effects of virtual and real objects. The projected sensory information may be additive to the natural world or destructive to the natural world. The AR experience is do intermingled with the physical environment so that it feels to be immersive as per the real world.”

There are many ways to augment the environment around us, and each has its strength and weakness. Generally, an AR experience requires a trigger to start and this trigger helps the AR application in deciding to place the AR content. Depending on the trigger, AR may be of the following types:

  1. Marker-based AR: Also called as Image recognition AR. In this case, an image act as a marker to trigger. The camera of the device scans the marker, which is different from the surrounding environment and triggers the application to place the content. Content may be in the form of images, Audio, Video, Animation etc. While rotating the marker camera, the content on the marker moves accordingly, but a deflection from marker may cause deviation in projecting the content. The Marker-based AR mostly handled through mobile apps. So, the users first have to download the app in their device to experience the AR. For example (NCERT AR App), see this demonstration. In this case, this image is used as marker. The camera of the mobile phone is scanning the image marker and projected a 3D simulation of this image. Now, the learner can interact with this 3D simulation on mobile screen and by orienting the camera in different corners, the shape of the cell in 3D can be seen from various angles. To interact with the different parts of the cell, click on the menu. Say mitochondria, now mitochondria from the cell is visible in 3D along with voice description for better understanding. So, this simple AR content gives us an idea that how 3D contents, with the help of software augment our knowledge about the cell. AR content is much better than the 2D image shown to students.
  2. Marker less AR: Also called location-based AR. This type of AR is more versatile and it doesn’t require any image to cue to deploy the AR content. To trigger the AR content, it relies on the data from various sensors like GPS, gyroscope, accelerometer, digital compass aligned within the camera environment. The data from all these sensors provide input to understand the 3D environment. This process is known as SLAM (Simultaneous Localization and Mapping) and the programming to develop marker less AR is complicated than marker-based AR. SLAM algorithms bring AR to new environment, mostly limited to flat surfaces. Marker less AR may further be divided into location-based AR, Projection-based AR or Superimposed AR depending on the technology used.

For example, the “Spacecraft AR” app developed by the Jet Propulsion Laboratory, NASA. The students can see and interact with various 3D models of spacecraft and to have a better understanding of probe even without visiting the actual laboratory.

 

  • Technology to deploy AR: According to Gartner (IT research and analysis company), 70% of companies will be experimenting with immersive technology for consumer and enterprise use, and 25% will have to deploy them to production by 2022.

Due to the increased traction of AR technologies, many leading enterprises are exploring novel ways to harness its potential. But to leverage any modern technology, it is crucial to understand its fundamentals and to work. So, let’s discuss the technology to deploy AR:

  1. Hardware: Hardware components for AR require a processor, sensors, display, input devices etc. Since modern smartphones contain all these elements like camera, GPS, accelerometer, digital compass etc., making them an excellent choice for AR platform.
  2. Eyeglasses: AR displays may be rendered on eyeglass-like devices. Versions include eyewear that uses cameras to intercept the view of the real environment and re-display its augmented view through the eyepieces and devices in which the AR imagery is projected through or reflected off the eyewear lens pieces surfaces.
  3. Tracking: Tracking based AR uses this technology where the position of orientation and of the user is very important. In fact, you must have seen the live AR emojis on your mobile phone. The camera of the device using one or more sensors like GPS, gyroscope, accelerometer, RFID, digital compass etc.
  4. Environment: This includes the end-user’s physical surrounding or spatial space. Following scenarios should be taken care of while designing:
  5. Private: When a user is experiencing AR on a wearable device.
  6. Intimate: when a user is sitting with a desktop and not moving.
  • Personal: When a user is experiencing AR on a mobile device in a public place.
  1. Public: When a user is experiencing AR by using the whole body to interact with software.

 

  • Applications of Augment Reality: There are enormous possibilities of using AR technology in various fields ranging from entertain, gaming, business education etc. Following are a few of the emerging areas that define the use of AR applications:
  1. Archeology: AR helps in augmenting archaeological features in the modern landscape. AR helping archeologists to rebuild possible site configuration from extant structural data. http://www.arkeotekno.com/image/genel/big/gnl2018204039.jpg
  2. Architecture: AR helps in visualizing the construction projects even before starting the work. Computer-generated 3D models in real life can be augmented on the property before physical building is constructed. https://vimeo.com/269930252.

The University of Canterbury released CityViewAR after the Christchurch earthquake to visualize the destroyed building and other structures. https://www.youtube.com/watch?v=fdgrXxJx4SE

  1. Medical Training: The best thing using AR in medical training is, the students can try this n number without harming the actual subject. Some, institutes like Cleveland Clinic at Cas Western Reserve University using AR headset to taught anatomy to students.
  2. Urban Design & Planning: AR applications are being used to design, develop and planning the urban landscapes before actually implementing the policies by augmented reality maps and 3D building models
  3. STEM Education: Augmented reality has been used to implement a standard curriculum by providing supplement interactive content to the learners.
  4. Industries: Paper manuals can be transformed to AR manuals, which are overlaid on the manufacturing operator’s field of view. Troubleshoot problems can be solved quickly.
  5. Commerce: Use of AR in business has unprecedented growth, especially in marketing. Marker-based cards can be designed to trigger various advertisements. The 3D content will be overlayed on the card to augment the view of shopping and also decreases the cost of paper advertisement and display boards as users can use their own smartphone to scene the marker.

 

  1. Virtual Reality (VR): As the name suggests, Virtual means “near,” and reality means what we experienced. So, literary “Virtual Reality” means “near-reality.”

Virtual Reality means feeling the imaginary (virtual) world. It is basically an experience taking place within a simulation, which can be similar to or completely different from the real world. The immersive artificial environment created by using software and presented to the user in such a way that the user accepts it as a real environment. Unlike traditional user interfaces and viewing a screen in front of them, users are immersed and able to interact with 3D and the 360-degree world, which can further simulate by using as many senses as possible, like hearing, touch, vision, even smell.

Currently, Head Mounted Devices (HMD) required to feel the immersive user experience in Virtual Reality. HMD may be stand-alone like Oculus rift, HTC Wibe etc. or commonly available HMD with supported mobile devices.

Depending upon the immersive experience, VR may be of following types:

  1. Non-Immersive VR: This type of VR generally disseminated by using the Desktop system or any project screen. Users can see only the VR on-screen, and interaction is possible by using traditional keyboards, mice, and trackballs. To render non-immersive VR, one didn’t require high-end devices or specialized hardware; any regular desktop, monitor, laptop or mobile phone can be used. Users can be navigated through various markers and other information in VR.

In some cases, if non-immersive VR be seen with HMD devices, they can provide semi-immersive experience. Like 360degree VR tours can be seen through simple desktop and through HMD to have a semi-immersive effect.

  1. Semi-Immersive VR: This type of immersive effect allows users to experience VR in a three-dimensional world while remaining connected to real-world surrounding visuals like smells, auditory, and haptics. Users can interact with both the real world as well as the virtual 3D world but to maximize the immersion, focus on the digital world.

Semi-immersive VR set requires specialized hardware and high-end software to render 3D imaginary. In most of the cases following crucial elements are required:

  • 3D virtual environment to generate a realistic effect.
  • High-quality Dolby sound to sync with the 3D environment.
  • Simulator hardware.
  • Haptic feedback

  1. Fully-Immersive VR: In such type of VR, the user experience full emersion and unable to differentiate between real-world and virtual world. As compared to semi-immersive VR, in fully-immersive VR; user completely cut of from the real world. They wear Head Mounted Devices unable to interact with physical things surrounds them. In simple words, it is a perception of being physically present in a non-physical world.

Ernest W. Adams separated fully-immersion into further three categories:

  1. Tactical immersion: being experienced when performing tactile processes.
  2. Strategic immersion: When user associated with mental challenges and feel more cerebral immersion like playing a chess game in VR.
  • Narrative immersion: When user experience the works like story reading, watching a play or movie.

Depending upon the process of experiencing moveability, VR may be of following types:

  1. Static VR: Static VR can be developed by using 360 “Panoramic” images. The images captured stitched together by using software to make a VR tour. The simple example is street-view tours available on “Google Map”.

1In this, the user can experience the immersion by sitting or standing at one place and turning their head around.

  1. Motion VR: In this case, the user can move to experience the full immersion. He/She feels that they are moving inside the scene.

Depending upon the development process, VR may be developed by using:

  1. Software: As the process that we can’t see with the naked eye like inside the body, process inside the vehicle engine
  2. VR Cameras: This type of VR can be developed by using 360 degrees/VR cameras. VR for physical places like monuments, environment degradation can be captured and disseminated through the software or app to render the 360-degree content.

 

  • Technology to deploy VR: Technology to deploy the VR is dependent on various factors. Immersive and semi-immersive VR requires different hardware and software. But in most of the cases a Head Mounted Device (HMD) is required to experience.

 

  • Applications of VR: Virtual Reality applications has been used in many fronts like education, architectural and urban design, digital marketing and activism, engineering and robotics, entertainment, fine arts, healthcare and clinical therapies, heritage and archaeology, occupational safety, social science and psychology etc. Following are the options discussed here:
  1. Entertainment: For entertain purpose, VR currently being used in video games, cinemas (video/documentaries) and for family entertainment. The Head Mounted Devices ranging from few hundred of rupees to thousands of rupees are available in the market which may stand alone or used with smart phones. Like Google cardboard, Oculus rift, HTC vive, Samsung gear etc.
  2. Healthcare: Healthcare is one of the biggest adopters of VR which encompasses surgery simulation, phobia treatment, robotic surgery and skills training. One of the advantages of this technology is that it allows healthcare professionals to learn new skills as well as refreshing existing ones in a safe environment. Plus, it allows this without causing any danger to the patients.

Popular use of this technology is in robotic surgery. This is where surgery is performed by means of a robotic device – controlled by a human surgeon, which reduces time and risk of complications. Virtual reality has been also been used for training purposes and, in the field of remote telesurgery in which surgery is performed by the surgeon at a separate location to the patient.

  1. Sports: VR is used as a training aid in many sports such as golf, athletics, skiing, cycling etc. It is used as an aid to measuring athletic performance as well as analyzing technique and is designed to help with both of these. It also used in clothing/equipment design and as part of the drive to improve the audience’s experience.
  2. Education: Education is another area which has adopted virtual reality for teaching and learning situations. The advantage of this is that it enables large groups of students to interact with each other as well as within a three-dimensional environment. It is able to present complex data in an accessible way to students which is both fun and easy to learn. Plus, these students can interact with the objects in that environment in order to discover more about them.
  3. Military Training: Virtual reality has been adapted in Military training. A simulated environment is being created to train the soldiers without the trauma of being injured. It has been adapted in various scenarios like flight, warfare, disaster management etc.
  4. Scientific visualization: Virtual reality is being increasingly used in the field of scientific visualization. This field is based upon using computer graphics to express complex ideas and scientific concepts, for example molecular models or statistical results.
  5. Engineering: Virtual reality engineering includes the use of 3D modelling tools and visualization techniques as part of the design process. This technology enables engineers to view their project in 3D and gain a greater understanding of how it works. Plus, they can spot any flaws or potential risks before implementation. This also allows the design team to observe their project within a safe environment and make changes as and where necessary. This saves both time and money.
  6. Heritage conservation: This refers to the use of virtual reality in museum and historical settings, e.g. visitor centers. These settings employ interaction as a means of communicating information to the general public in new and exciting ways.

There has been a move away from the traditional type of experience associated with museums, galleries and visitor centers. The old model was that of passive engagement in which people viewed the exhibit/s but did not get involved to an experience in which interaction is the main feature.

So, with the changing time and changing behavior of the learners towards technologies, the teaching-learning process should also change. The immersive technologies like Augmented and Virtual reality will be helpful not only in providing the quality education but also save millions of rupees for training and developing infrastructures. In next module we will discuss the Mixed reality, its applications and implication of Augmented Reality, Virtual Reality and Mixed reality in education.

 

References:

Sharma, Y.P. (2015). Response of Teachers towards the use of Educational Technology in Schools in India. Microcosmos International Journal of Research, 1: 6-9. https://mijr.org/wp-content/uploads/2019/12/MIJR_121405.pdf 

Kesima, M. and Ozarslanb, Y. (2012). Augmented reality in education: current technologies and the potential for education. Procedia – Social and Behavioral Sciences.47: 297-302. https://doi.org/10.1016/j.sbspro.2012.06.654

Azuma, R., Baillot, Y., Behringer, R. and Feiner, S. (2001) Recent advances in augmented reality. Computer Graphics and Applications. IEEE, 21 (6) (2001), pp. 34-47. CrossRefView Record in ScopusGoogle Scholar

Pantelidis, V.S. (2009) Reasons to Use Virtual Reality in Education and Training Courses and a Model to Determine When to Use Virtual Reality, THEMES IN SCIENCE AND TECHNOLOGY EDUCATION Special Issue, Pages 59-70. Klidarithmos Computer Books, USA. http://earthlab.uoi.gr/ojs/theste/index.php/theste/article/ view/22

Mehryar, N. and Ron, W. and Tek, M. (2004). The use of virtual reality in education. In: American Society of Business and Behavioral Sciences (ASBBS) 2004 Seventh Annual International Conference, 6-8 Aug 2004, Cairns, Australia. ISBN: 0-646-43717-8. https://www.researchgate.net/profile/Tek-Maraseni2/publication/228724825_The_use_of_virtual_reality_in _education/links/0fcfd50c7b72225ecb000000/The-useof-virtual-reality-in-education.pdf

.