Using Augmented Reality (AR) and The Internet of Things (IoT) to create a killer app?

Using Augmented Reality (AR) and The Internet of Things (IoT) to create a killer app?

Last week PTC held Thing Event an hour long webcast to display some of their new innovations in the Internet of Things (IoT) space using one of their recent acquisitions Augmented Reality (AR) company Vuforia, a business they acquired from Qualcomm back in November.

One of the statements that really stood out to me was from PTC CEO Jim Heppelmann who said “Service would be the first killer app in The Internet of Things”. While I agree with the sentiment, I think it’s probably safe to say based on existing Industrial solutions in M2M & IoT that some level of service improvement has always been one of, if not the primary driver, it’s not something that suddenly became relevant overnight.

What PTC perhaps should have been saying is that combining AR & IoT could well create a killer app, and based on the demos from KTM Racing, Sysmex & Caterpillar I’m inclined to agree.

sysmex AR demo

Sysmex using AR to overlay video instructions on product maintenance

The demos all showed relatively the same application: At a simple level, it’s using the IoT and a network of sensors for proactive maintenance and fault detection and then using augmented reality to overlay instructions on how to repair or engage with the fault.

The fact that this application could be applied to technologies as diverse as motorcycle repair, medical device management and industrial hardware is part of the beauty of it – You can apply the technology displayed at Thing Event to an almost unlimited number of industries.

Thinking about the implementations of this type of AR integration really got me excited, a few major benefits I can see are:

  • No language barriers – If you look at the KTM Racing demo while it does carry English subtitles in the application you can quite easily understand the instructions just by looking at the visuals. You could ship products all over the world with a simple AR-enhanced visual set of commands.
  • Centralized support – I imagine having a central server handling the distribution of the AR application (If it’s done wirelessly) can eliminate many of the problems current hardware manufactures have with version control for technical documentation.
  • Reduction of Waste – Can you imagine if instead of having pages and pages of instructions in different languages printed and included with a product you could simply scan a QR code and you could be given a full up-to-date visual instruction on how to do anything related to the product. Imagine getting furniture from IKEA and scanning a QR code with your IKEA App and watching your furniture pieces come together in your living room in real-time, you just follow the visual prompts.
  • Harvesting Knowledge & Reducing Training Time – As industries change and grow inevitably people will move on, retire or simply be unable to perform the tasks they were able to do before. Currently, unless someone does a thorough exit interview or the employee is amazingly diligent with documentation so much knowledge and experience is lost forever. By capturing experienced people doing tasks and relaying it back through AR visuals you have that persons’ knowledge in your organization forever.
    This in turn reduces the amount of time and education needed to perform maintenance and service tasks, especially in none specialized roles, rather than having someone read a text book or watch a video on their training they can use AR to view and perform basic tasks instantly with little to no guidance.
  • The Developing World – If you imagine the reduction of language barriers, the transition of knowledge and the reduction of training times it’s not difficult to see how this might be a potentially life-changing technology for solutions being delivered in developing regions.
    Let’s look at something like a water filtration system being shipped to a remote village to supply fresh water. If you add wireless connectivity through sensors and either local or long-range communications, you can monitor the health and performance of the device from anywhere in the world. If you detect a fault, then someone locally can use AR to watch a series of visual instructions all overlaid on the very machine they are working on to solve the problem. These instructions can be recorded by an expert on the product but relayed so clearly that someone who speaks a different language and has a comparatively reduced skillset can perform the same task.
    With some creative thinking, you can envision how this can not only reduce the cost of deploying key infrastructure and services in these countries but also provide increased income to local workers who can perform more valuable maintenance tasks.

While some of this may still sound a bit “pie in the sky” the technology is here now and available to create solutions like this. Hardware to make this more effective like AR glasses and more elegant overlays than tablets will slowly come down in price but you could make something functional with a basic Android tablet and small sensor network and raspberry pi or other low-cost board for under $100 at volume – Myself and Bill Zhou from M2M Connectivity recently spoke to Dev Diner about creating a low-cost IoT solution (Read the full interview at Dev Diner)

Do you think Augmented Reality is going to be a game-changer in IoT?  I’d love to hear what you think, Let us know on Twitter @M2MOneAU, on LinkedIn at M2M One or visit our website at www.m2mone.com.au.

James Mack
General Manager – M2M One

The views and opinions expressed in this blog are solely those of the author and do not necessarily reflect the opinions of M2M One or M2M Group.

M2M Tips – Building a cellular M2M or IoT solution? How to maximize your up time.

M2M Tips – Building a cellular M2M or IoT solution? How to maximize your up time.

Over the past 15 years the team at M2M Connectivity & M2M One have coached, consulted, celebrated and at times commiserated with thousands of hopeful developers trying to make their mark on the Machine to Machine/Internet of Things landscape.

This series of blog posts is designed to pass on some M2M 101 style tips to developers looking to get into the space.

A big question we get asked is “How do I keep my device online as long as possible”, while it’s almost impossible to provide 100% uptime for any wireless device we’ve provided a few tips below to help you get closer.

 

A few things to consider:

  • Expect outages! – Despite their best intentions every single mobile operator in the world still has to have periods where they restart or change a network element either planned or otherwise.
  • ‘Mobile’ Networks – Most people aren’t in the same spot using their phone or iPad for 24 hours a day so they don’t notice network changes or intricacies, but m2m devices are a bit different. A data-logger measuring the water level of a reservoir isn’t going anywhere so if the network goes down, even for a moment it’s communication is going to be interrupted.
  • Is coverage available? – In countries like Australia you can have vast regions of unpopulated land, at some point you’re going to run out of cellular coverage so you need to think of how to cope.

 

Worst case scenario... - Photo courtesy of http://www.engineeringradio.us

Worst case scenario… – Photo courtesy of http://www.engineeringradio.us

So how can you preempt a lack of cellular coverage?

1. Store and forward – Make sure you have enough memory on your hardware to store at least one or two days of data, that way if you experience an outage or your device moves out of cellular coverage temporarily nothing is lost.

 

2. Build a smarter modem – This is particularly important for devices that don’t move between cell towers. Previously we advised customers building M2M devices that they should factor in an automatic reboot to any stationary M2M device at minimum once a day. This is still a great tried and tested solution, but if you need something that communicates 24 hours a day an unnecessary restart may be impractical.

A majority of modem manufacturers include a ‘watchdog’ feature in their hardware which can be set to ping an IP address at certain intervals, this can either be a public known IP like Google for example, or your own network. If the device is unable to ping the ‘watchdog’ address then it will automatically power cycle the modem, without you having to do anything.

For older devices without a ‘watchdog’ we still recommend an automatic reboot at least once a day, and having some kind of failover like being able to restart the device with an SMS.

Why is restarting the modem important? Most raw M2M hardware is reasonably ‘dumb’ if it can see a network and can communicate with that network it assumes everything is ok. This however is not the case, have you ever been in a situation where you can make calls or send an SMS but can’t surf the Internet on your phone? Normally if you put it in and out of airplane mode it fixes the issue. The same principals apply here, the modem assumes everything is normal but really it can’t make a data/IP connection. By restarting the modem you force it to create a brand new connection which 99% of the time brings voice, SMS & data back.

 

3. Can you feel my heart (beat)? – For those of you who are building and deploying devices using public Internet APNs (those commonly shared with mobile phones and tablets). We highly recommend putting in a ‘heartbeat’ to remind the network your device is still there. Public networks are highly aggressive when it comes to closing sessions, this is to maximize capacity and reduce congestion.

Unfortunately networks around the world have different timeouts ranging from minutes to hours to days. So we can’t prescribe the perfect solution, but as most heartbeats send a minimum amount of data (around 1 byte per beat) you can usually be reasonably safe with every few minutes.

 

4. How critical is your data? – Most of the time store and forward will be ok; when a device is back in cellular coverage your data is sitting there waiting for you. However if you absolutely need 24 hour uptime you need some kind of failover, just like cellular became standard failover for Ethernet or Wi-Fi.

Some devices can get away with utilizing multiple SIM cards or connecting to multiple carriers via a ‘roaming SIM’ – This is a great option for areas of high cellular coverage with multiple operators. Remote locations on the other hand tend to have a single operator or none at all, this is where utilizing satellite technology becomes critical.

A lot of people cringe when I start talking about satellite communications, their minds instantly go back to the days of $10,000+ phone bills and hardware that cost and weighed as much as a car. This really isn’t the case anymore, using satellite as a failover for cellular is fast becoming commonplace in OH&S solutions in the remote operations, utilities and mining sectors.

Sometimes satellite is your only option - Photo courtesy of Wikimedia

Sometimes satellite is your only option – Photo courtesy of Wikimedia

Hopefully these tips have inspired you to build some extra smarts or redundancy into your next or current M2M project.

Did you find this tip useful? Do you have some questions on the Machine-to-Machine and Internet of Things industry we can answer for you? Let us know on Twitter @M2MOneAU, on LinkedIn at M2M One or visit our website at www.m2mone.com.au.

 

James Mack

General Manager – M2M One

 

The views and opinions expressed in this blog site are solely those of the author and do not necessarily reflect the opinions of M2M One or M2M Group.

Subscribe To Our Newsletter

Join our mailing list to receive the latest news from M2M One.

You have Successfully Subscribed!