Jumbo Revolutionizes Deckmarking

By Otto Savenije
 
Jumbo’s journey with Microsoft’s HoloLens – mixed reality smartglasses – began with a team whose end goal was an improved deckmarking solution. This led them down an innovative and creative path that posed several challenges along the way, yet now they have succeeded in developing a pioneering solution that is already innovating the maritime industry.
 
Prior to loading complex heavy-lift items on board, the footprint of the cargo needs to be marked on deck. This is usually carried out by hand, often by two people armed with blueprints and measuring tape, making it an arduous and time-consuming task.
 
Jumbo needed a speedy deckmarking solution and ultimately found it in Microsoft’s HoloLens. Using augmented reality, or AR, glasses, the AR software is programmed to precisely locate the cargo positions by projecting digital technical drawings onto the ships deck, allowing one person to mark the deck easily and efficiently.
 
The main challenge in the development of the HoloLens application was to reach a projection accuracy of no more than a 1-centimeter or 2-centimeter deviation from the design. As the HoloLens was originally developed for indoor usage and for smaller objects, it was a trial for the developers to maintain the accuracy when projecting larger images in sunlight. Sunlight filters were added to the application, solving this issue for the team.
 
Another test was to ensure the projection could accurately follow the roll and pitch movements of a ship. The projection needs to remain flat on the surface of the ship’s deck, but with a floating vessel naturally the roll angle can vary. Extra menu options to (manually) correct the projection when required were developed to overcome this challenge.
 
Further, the processor had difficulty in calculating and projecting a whole image of 17 meters by 13 meters at once, so a method was developed to project only a part of the image. A conversion method was also established to transform the standard drawings, already used within Jumbo, to a layout that could be processed by the HoloLens software with minimal disruption to the engineers.
 
With every element thought through, there were even various designs tried and tested to make a good rigid connection to a safety helmet, which is of course mandatory onboard.
 
With this innovative application, Jumbo is improving its way of working and has used the AR glasses effectively on a recent project. The HoloLens allows for a significant reduction in the duration of a complex loading process, which minimizes time in port and the associated costs. As the projection can be combined directly with the layout of the deck/frames/structure it gives an immediate verification of correct placement/scaling of the projection, which also speeds up the process.
 
Overall, we have found that the HoloLens increases the reliability of deckmarking because the risk of human error is reduced and, now with easy cloud-upload, updates and corrections can be shared from a distance at the click of a button. No more printing and improved efficiency for clients.
 
Jumbo is now taking the next steps in implementing the existing application throughout its fleet. We also look forward to newer hardware that can give a wider “view window” and better processor speed to project larger parts of the whole image with more stability.
 
As well as this, Jumbo is also thinking of other scenarios in which they can implement this or similar applications in a pioneering way.
 
Otto Savenije is senior project engineer and senior port captain at Jumbo.

See the HoloLens Deckmarking Solution in action here.