Control a Droid With Your Mind and IBM Bluemix Internet of Things

Control a Droid With Your Mind and IBM Bluemix Internet of Things

 

 

We’ve all heard about how the Internet of Things (IoT) allows us to control light bulbs, thermostats and security alarms remotely from our smartphones, but is there another side to IoT? I can do all the aforementioned things from my phone today (except for some reason my thermostat recently stopped connecting to the Internet), but less commonly do we imagine how IoT can change the way we interact with the physical world.

It turns out, with the help of IBM Bluemix and some inexpensive consumer hardware, you can direct a BB8 droid with your thoughts. Seriously.

I really didn’t know that this was possible until Envato Tuts+ asked me to write the piece.

 

Perhaps the guy with the best job in the world, Joshua Carr, Technical Liaison IBM Bluemix, has perhaps the perfect demonstration, which you’ve probably just watched in the video above.

 

IBM Bluemix IoT Emotiv BB-8 Demo - Picture of Joshua Carr holding BB8 Droid

 

While robots have always been the realm of sci-fi, mind-control was even further out there in fantasy land. As Carr said, “After seeing Yoda lift X-wing out of the water, I’ve spent probably far too long than I’d like to admit staring at cars to see if I could move them.” That didn’t work when he was younger. But now, he shows us how we can do both. And that’s what I’ll be talking to you about today.

And, as always, share your ideas, comments and feedback in the comments below, or reach me directly on Twitter @reifman. For today’s tutorial, you are also invited to reach Carr directly @josh_schwaa or @IBMCloud.

Before we dive in, I want to mention that the Emotiv headset Carr uses in the video is the work of a particular woman genius, Tan Le. Working in technology, we’re often surrounded by men and don’t often know the contributions of women in science and technology. I’ll highlight more of Le’s work below. Of course, IBM Bluemix allows you to connect all kinds of hardware and is manufacturer agnostic.

 

Bluemix is IBM’s open standards cloud platform for building, running and managing apps and services. It offers a vast catalog of services.

 

 

If you want to use your mind to make changes in the real world, like directing droids, then you’ll want to leverage Bluemix’s IoT services. Let’s look at how Carr did it.

Here are the big-picture elements he used to create the video:

 

IBM Bluemix IoT Emotiv BB-8 Demo - Graphic of components of todays tutorial

 

In the real world, that diagram looks like this (don’t worry, you can also get the headset in black):

 

IBM Bluemix IoT Emotiv BB-8 Demo - Photo of Emotiv Headset on table with BB-8

 

Here’s the hardware that Carr used for the video:

 

 

IBM Bluemix IoT Emotiv BB-8 Demo - Sphero Retail box for Star Wars BB8 Droid

 

Here’s the star of our show (the lil’ guy taking Carr’s directions), BB-8, which you all know and love from Star Wars, The Force Awakens.

Here are a few cool things that the Sphero BB-8 Droid offers for about $149.99 USD, weighing in at about 200 grams:

  • Top speed of 4.5 mph (7ft/s)
  • Internal guidance by gyroscope and accelerometer
  • Bluetooth Smart BLE connection (100-foot range)
  • Inductive charging (over 1 hour of play on a full charge)
  • iOS & Android & Windows Phone compatible

Then there’s the headset—the device that receives Carr’s thoughts and sends them to Bluemix.

 

The Emotiv Insight is a sleek, 5-channel, wireless EEG headset that records your brainwaves and translates them into more meaningful data. This high spatial resolution provides in-depth information on your brain activity, but Insight is optimized to produce clean, robust signals for everyday use (see, here it is in black):

 

IBM Bluemix IoT Emotiv BB-8 Demo - Emotiv Insight Headset Website Image

 

Here’s a visual transparent 360-degree spin of the Emotiv Insight’s flexible printed circuitry based electronics:

 

 

Amazingly, the Emotiv Insight began as a Kickstarter project, promising a “headset that allows consumers essentially to read their own brain and understand what’s going on within your head.” It costs about $299 USD.

Here’s co-founder and inventor Tan Le in the original Kickstarter video:

 

 

You can also watch her TED Talk: A headset that reads your brainwaves.

Here are some of the headset’s features:

  • Provides high spatial resolution, ensuring ‘whole brain’ measurement.
  • Provides access to raw EEG data with software subscription.
  • Nine-axis motion sensors for precision measurement of head position and movement.
  • Identifies performance metrics, mental commands, and facial expressions.
  • Built with hydrophilic polymer sensor technology which requires no preparation or sticky gels to use (helpful when you’re just getting out of the shower and need to send BB-8 off to distract stormtroopers).
  • Minimum four hours battery run time.
  • Wireless connectivity to phone, tablet, and PC.
  • Windows, OSX, Linux, Android, and iOS compatible.

 

And, here’s the software you’ll need:

Here’s a visual summary of the data flow from the headset to your droid:

 

IBM Bluemix IoT Emotiv BB-8 Demo - Data Flow Graphic for Data from Headset to Robot

 

Carr describes how he put together the components in the video in Taking your 1st steps to using ‘The Force’. If you’d like to replicate his video demonstration, I encourage you to read his tutorial. I’ve borrowed from his descriptions to provide an overview below.

 

IBM Bluemix IoT Emotiv BB-8 Demo - Bluemix How it All Fits Together from Website

 

 

Carr used Bluemix to create an Internet of Things app from its boilerplates:

 

IBM Bluemix IoT Emotiv BB-8 Demo - Bluemix Boilerplates Menu

 

This creates an instance of Node-RED that you can access from your browser and provides a way to connect the headset and the droid together.

 

“The headset itself uses known technology to look for specific markers, things like excitement and interest,” says Carr. “What we can do is train our headset to recognize certain familiar commands that we think, and the device learns essentially that this means action A. Then we can send this information via Bluetooth off to the Internet.”

He continues, “But once it’s there, what do we do with the data… that’s where IBM technology comes in, we process that data, we flow it through and we make meaningful interactions using small amounts of code.”

“Once you have your Insight, and are familiar with how to wear it to get strong connection (see Emotiv Headset control panel for details), the next step will be to either use the emotions recognized straight out of the box (excitement, focus, et al.) or to train your headset.”

It took Carr several hours to train his headset to a satisfactory level. Emotiv recommends that you should get one command working before trying to add another, and he agrees.

 

“In my demo I got push and pull, but you can always add more commands. Apparently, up to 12!”

 

 

Carr used the Java-based SDK from Emotiv and Java IoT library to connect the headset to Bluemix. These enabled him to receive data from the headset to run code there.

IBM’s Internet of Things Foundation helps you set up credentials for sending and receiving data in the cloud. Each application has an Organization ID and an API key to register your device.

As he describes, a lot of configuration is required, but the IoTF docs will guide you. We wouldn’t want the cloud randomly sending our thoughts to someone else’s droids.

As events are identified by the headset, they are published to your Bluemix instance. You can learn the basics of Node-RED by perusing its official documentation. Once data is being sent, you will see it appear in your Node-RED instance:

 

Depending on how you structure the data you are publishing from your IoT Foundation client, you may need to use Node-RED to create instructions that will be received by the Pi that’s connected to the BB-8.

The Node-RED visual editor is a wiring tool for the Internet of Things that allows you to easily and efficiently take data and run functions against it, sometimes with code and sometimes without explicitly having to write code yourself.

Here’s a screenshot of Node-RED in another IBM BB-8 Droid demonstration:

 

 

IBM Bluemix IoT Emotiv BB-8 Demo - Nodered Snapshot from Bluemix IoT

 

Carr restricted his command set to pushpull, and neutral for the video.

 

The Raspberry Pi will need the code that will connect to the BB-8. You will essentially need to replicate what you did with your code for the headset, but instead of publishing data, you will be subscribing to that data. In other words, the Pi takes in events transmitted to the cloud from your headset and offers them to the droid.

To connect the Pi to the BB-8, Carr used BlueZ to perform a Low Energy Bluetooth scan. This helps the Pi find the MAC address of local BTLE devices such as BB-8.

He also added the IoT Foundation Python library onto the Pi. This allowed him to create a new client in Python that connects to IoT Foundation and imports the BB-8 Driver.

He then used simple conditional statements against the received data to work out which command should be issued, such as the roll() command to move the BB-8.

Below, he instructs to the BB-8 to return to him:

 

IBM Bluemix IoT Emotiv BB-8 Demo - Still from video of Carr Controlling BB-8

Again, I recommend you visit his tutorial for more detail: Taking your 1st steps to using ‘The Force’.

 

 

IBM Bluemix IoT Emotiv BB-8 Demo - Jedi or not quote any developer can build this

 

“If I could make any technology from Star Wars real, it would not be lightsabers,” Carr declares. “Can you imagine giving a light saber to somebody the first thing they would do is chop their arm off.”  He suggests we take baby steps. “First faster than light space ships, and then lightsabers.”

Well, Envato Tuts+ has asked me to write a followup tutorial about controlling BB-8 with arm gestures. Here’s an introductory video by Rob Peeren which we’ll cover next week:

 

 

Peeren mentioned to me that the best part about Bluemix IoT is that nothing is ever very complicated. He’s able to accomplish his goals by combining the building blocks which Bluemix and third party providers offer. Carr echoed this: “It was such an easy step for me to see how can I use my brain to control robots around me.”

We don’t have to be Jedi to do this.

So I say, bring on the spaceships and lightsabers. Self-driving cars seem so passé (sorry Elon).

As for Carr, he’s exploring using the IBM Cloud and different APIs to go deeper—using Watson’s Speech-to-Text to light the BB-8 up in different colors or having the BB-8 reflect his current sentiment by having IBM’s Analytics Service, Watson, analyze his Twitter activity.

He’d also like to hear your thoughts, questions or suggestions on putting Bluemix and BB-8 to greater work @josh_schwaa or @IBMCloud.

Carr says, “We’re going to take a huge leap in feeling truly connected with other people,” and I agree. I’m looking forward to seeing this technology evolve and grow.

If you have kids, especially daughters, tell them about this. Show them the video. Boy, girl, self-chosen gender nomenclature, tell them a woman led the invention of the headset—and let them know it’s okay to dream big about their future.

 

I know this is pretty amazing stuff. I had to go back and re-watch the video to double-check that the Emotiv Insight is analyzing brainwaves and not just facial gestures; it is.

If you can’t wait for the armband tutorial accompanying Peeren’s efforts, here are some additional Bluemix IoT resources to keep moving forward with:

I suggest watching the video describing how SilverHook Powerboats uses IoT to help monitor and optimize engine performance during races:

 

 

Check out Scaling from 1 to 100K+ connected devices with IoT on Bluemix, which describes how to plug your smartphone into Bluemix running a web application named IoT Phone. The app accesses smartphone acceleration, orientation, and location; think of it as a portable motion and movement tracker. It’s pretty easy to set this up today.

 

IBM Bluemix IoT Emotiv BB-8 Demo - Bluemix IoT HTML5 Phone app

 

 

You can also use Bluemix with these $49 IoT sensors from Texas Instruments; see IBM IoT with your Smartphone sensors:

 

 

 

Here’s another IBM Developer Works RecipeControlling your Sphero-BB8 using IBM Watson and Node-RED:

 

 

And there’s more sample code to control a BB-8 Droid from anywhere in the world (GitHub).

 

Finally, for a more general overview of IBM Bluemix and its broader capabilities, watch David Barnes’s Create, Deploy and Manage Your Applications in the Cloud:

 

 

Stay tuned for the upcoming armband droid control episode. In the meantime, please let us know which IBM Bluemix topics you’d like to see more written about in the future.

You can post them in the comments below or reach me directly on Twitter @reifman. I hope you’ll also check out my other Envato Tuts+ series How to Program With Yii2 and Building Your Startup With PHP.

I hope you had as much fun reading this as I did writing it. The future will be big!

 

IBM Bluemix IoT Emotiv BB-8 Demo - IBM Bluemix Logo and Link

 

If you wish to learn more about IBM Bluemix and Internet of Things, visit http://ibm.com/bluemix.

 

[Tutsplus]

October 3, 2016 / by / in , , , , , , , , ,

Leave a Reply

Show Buttons
Hide Buttons

IMPORTANT MESSAGE: Scooblrinc.com is a website owned and operated by Scooblr, Inc. By accessing this website and any pages thereof, you agree to be bound by the Terms of Use and Privacy Policy, as amended from time to time. Scooblr, Inc. does not verify or assure that information provided by any company offering services is accurate or complete or that the valuation is appropriate. Neither Scooblr nor any of its directors, officers, employees, representatives, affiliates or agents shall have any liability whatsoever arising, for any error or incompleteness of fact or opinion in, or lack of care in the preparation or publication, of the materials posted on this website. Scooblr does not give advice, provide analysis or recommendations regarding any offering, service posted on the website. The information on this website does not constitute an offer of, or the solicitation of an offer to buy or subscribe for, any services to any person in any jurisdiction to whom or in which such offer or solicitation is unlawful.