VR offers a virtual blast for new miners

Global mining company Anglo American is harnessing the virtual reality and motion capture expertise of STS3D and Vicon Motion Systems to create a state-of-the-art virtual environment to safely train mining initiates on the accurate use of explosives for underground development.

Anglo first set out to solve a truly difficult question: how can one appropriately and safely train recruits to blow things up thousands of feet beneath the ground?

At Anglo Platinum operations in South Africa, unearthing this precious industrial metal requires blasting deep into the ground with accurate and powerful explosives.

If a charge were poorly placed, it could mean a shallow advance into a rockface, uneven faces, rocks breaking well under or over planned excavation, or even just falling rocks.

Mistakes like these could misalign the timing of other scheduled blasts and with a daily target of 133 blocks of rock, it is not an uncommon event.

It takes highly trained personnel to avoid this inefficient and unsafe situation, yet providing effective blast training has many difficulties.

For real practice, blast trainees must go 900, 1,500 or even 3,000 feet underground – a descent that takes time and ends in a dark, dusty and damp destination that can be deafening and potentially dangerous for the uninitiated.

Conversely, on the surface, propagating decades of practical experience while also coordinating the nitty-gritty by giving trainees a proper grasp of scale, distances and measurements without any hands-on experience can be a nearly impossible task.

So Anglo approached STS3D, a company that offer mine training environments using virtual reality technology, to see if VR could provide the realities of underground blasting in the safety of a virtual environment.

World's first VR stope

Using a location-based virtual reality system to create a virtual environment representative of being in a mine 3,000 feet underground, STS3D set out to create a truly unmatched experience that would leave no room for errors.

This led to a custom-built VR solution for Anglo American, the Amandelbult Training Complex Virtual Reality Stope, which is the world's first facility that virtually recreates situations miners could encounter at real blast walls.

This massive complex – housed in a 2,300-square-foot training space, surrounded by an 800-square-foot rear projection screen, all held together with roughly 2,000 feet of aluminum tubing – provides a one of a kind LBVR environment.

All powered by two computers, one to run 12 digital projectors and another for 20 cutting-edge motion capture cameras.

And the challenge to accomplish all this was similarly massive – although designing and constructing the hardware, building the virtual stope, and coding the numerous functions one may find on the job caused the STS3D team a few headaches, the really tricky part was how to track the users in real-time within this multi-dimensional virtual stope.

This required breaking new ground, so STS3D, being new to motion capture, turned to the best in the industry – Vicon Motion Systems.

Vicon has worked with leading production studios, game developers and universities, names like Disney, Sony, Dreamworks, Activision, Ubisoft and even Square Enix to provide top of the line, cutting edge motion capture technology.

With games like The Witcher 3 and Destiny 2 or movie franchises such as Star Wars or Marvel, Vicon has proven it is the undisputed champion when it comes to motion capture.

And so, Vicon was able to quickly assess the present problems that STS3D was having and the solution was found in motion capture.

Readjusting the previous system that was not capable of tracking based on markers being blocked by trainees, Vicon convinced STS3D that their solution could track a subject accurately as long as a marker was visible to a single camera.

This new system is now capable of tracking up to five people at once, enabling teams to learn and operate together, enhancing the experience.

This ability to track multiple objects in a space provides as close to real training one can get without descending into the ground, preparing trainees for the actual conditions in the field.

"During entry examination, for example, miners must decide whether a zone is safe for operations. If they get it wrong, they get a realistic simulation of what could happen. A big loud bang and sudden darkness simulate a potential rock fall on the spot where they are standing." said Johan Bouwer, head of VR and new technology at STS3D. "Make no mistake, this gives the miners a real fright. They'll remember their error."

Serious game

The VR stope is a life-sized "serious" game, providing two modes, training and assessment.

Training mode users are instructed step by step on how and where to mark using a virtual paint brush. This hands-on training allows trainees to recall and adapt once in an actual mine environment.

The system also provides immediate visual feedback on correct or incorrect actions with the additional capability of undoing or redoing actions for immediate effective feedback to consolidate learning.

After a trainee completes marking and timing, they soon experience "art of the impossible" offered by VR – a detonation being set off directly in their face. This experience, when they see the firing sequence and fragmentation of the rock it provides a normally impossible scenario.

Afterwards, they will be able to inspect the quality of the blast first-hand - over and over again - in real time, slow motion or even reverse.

This way, the miners can understand best how and why to apply the correct blasting procedure.

"Since you are fully immersed in the virtual mine, there's no room for distraction, and the learning sticks," said Bouwer. "You're on high alert all the time while marking a pattern on the rock face for drill operators. And when you detonate the blast, the rock flies - creating a very vivid, a very memorable experience."

After a blast, the motion capture cameras track hard hat position of trainees, to best adjust the perspective of the projectors. This creates a 3D depth illusion which shows that the wall has been advanced, even if the room they are standing in has not changed in the slightest.

Trainees can then study the results of their blast which immediately lets them see if they have a good or bad advance, something otherwise impossible in the field.

Assessment mode is just that. Trainees mark blasting patterns and time the detonation sequence themselves. Once submitted, the system screenshots the attempt.

This image is then saved to a log of possible previous attempts and accuracy, with time taken and number of times assistance may have been requested. Turning a dangerous job, into an experience to learn in a fun and engaging way.

There is even a top score for trainees to try and set, giving them the ultimate gaming credentials – bragging rights.

Presently, STS3D is ramping up to deploy the next phase in instructional design, safe entry examination, which will test for ventilation and noxious gasses, support installation and rock strata control which will provide more impossible scenarios that one can learn from before it becomes the most costly mistake.

Despite the slow uptake of VR, owing to costs and possibly the risk of the unknown, STS3D has noted a steady growth in interest towards VR training by several of its clients using robotics and VR.

"Early adopters, such as Anglo American, diversified miner Glencore and engineering company Murray and Roberts, have jumped in, and they all agree that VR training is very effective, so more and more companies are starting to see the benefit of VR in training and assessment," said Bouwer.