What is a digital assistant?

We’ve all seen advertisements like these…

“Siri, what is the highest mountain in the world?”
“Alexa, order some flour.”
“Ok Google play some music.”

These digital assistants listen to what you say, then send that information off to a remote server, where it is processed before sending a response back to you.

Why do Apple, Amazon and Google provide these services? Well typically they get data from our enquiries. Amazon knows what products we buy, what music we like. Google learns about how we live, our calendars and search histories. These devices offer a great service, and we use them on a daily basis to make our lives easier.

However, some people feel weird letting companies listen in on everything they’re doing in their homes or apartments. And these companies keep your voice recordings long after they’re processed. We can’t control how our voices are used.

How can we make our own smart speaker that offers a similar set of features, but is entirely under our control?

Well here is where the Mycroft project steps into the light. It offers a similar experience to other digital assistants, but with a much more open approach to data. Mycroft devices can be purchased via Kickstarter (crowd funding) or we can make our own using a Raspberry Pi.

What will we make?

In the tutorial we shall make our own Mycroft smart speaker using a Raspberry Pi 3. This device is called a Picroft and it has all the functionality found in the Google, Apple, and Amazon speakers. However, we will have much more control on how things work.

For this project you will need

• Raspberry Pi 3
• 8GB micro SD card
• USB microphone
• Analog speaker (3.5mm speaker)
• Internet connection (Ethernet and WiFi)
• A computer running Windows / Mac or Linux

Setting up Mycroft

To use Picroft we first need to download a copy of their open source software image. From the Mycroft website https://mycroft.ai/ click on Download and then in the next screen look for the Raspberry Pi logo then click on it to download the image. If there are any issues with this link, try this alternative link https://github.com/MycroftAI/enclosure-picroft/

With the software image downloaded we next need to write that image to a blank SD card, and the best tool for the job is called Etcher (https://etcher.io/) as it provides a consistent and easy to use interface across Windows, Mac and Linux platforms.

Download Etcher then run the downloaded file to install and open Etcher.

To use Etcher we follow a three step process. First we tell it where our software image is, this should be in our Downloads folder. Second, we tell Etcher where to write the image, so make sure you insert your blank micro SD card and ensure that you select the correct drive. Lastly we Flash the image on to the card, this step may take some time, but ensure that it completes correctly before moving onwards.

Once Etcher has finished we have an additional step to take and this is to go to the BOOT partition of the micro SD card. This will be mounted in your file manager. Inside the BOOT partition create a new file simple called ssh. This will enable us to remotely connect to our Picroft using SSH.

You can now safely eject the micro SD card from your computer, insert it into your Raspberry Pi 3, and then insert into your Pi the USB microphone, the 3.5mm speaker an Ethernet connection and finally the micro USB power connection. However, don’t turn on the Pi’s power just yet!

Create an account

In order to use Mycroft / Picroft, first we need to have an account on the https://mycroft.ai/ website, so go ahead and click on LOG IN as it will open up a new page where we can sign up.

Once signed up, we now need to add our Picroft device, using the Devices menu at the top of the screen.

Now turn on the power to your Picroft device and wait for it to boot.

Picroft will go through a process of updating its software, and then it will tell you that it is connected to the Internet, but it needs to give us a six digit code to connect this device with our Mycroft account.

Picroft will speak the code, and we need to enter that code into the Registration Code section of the Devices page on their website. Then we tell the page that we have a Picroft device, we give it a name, and tell the page more about the device, in this case it is on my desk.

When ready, click on OK, LET’s PAIR > and after about 1-2 minutes your Picroft device will start talking about what it can do. So lets have a go: ask your Picroft what the weather is like where you live. For example:

“Hey Mycroft, what is the weather like in London?”

You will then hear Mycroft explain the weather in your hometown! Excellent, our Picroft device works!

We can also control the volume at which Mycroft speaks, by setting the value between 0 and 10 as so.

“Hey Mycroft volume 5.”

Handy because sometimes he can get a little too loud.

You’ve now got a smart speaker that you control. Mycroft claims voice recordings are deleted after they’re processed and that no third party handles your recordings. Your privacy is secure. Their software also is open source which means anyone can see how their code works.

There’s a lot more you can do with your smart speaker. You can teach your speaker new skills and operate your speaker with a wi-fi connection instead of ethernet cable.

Skills?

As well as the built in abilities, we can add some extra skills to the device but we need to do a little work in the terminal.

Skills are how Mycroft can learn new abilities. To teach Mycroft new skills, we need to log in to the Picroft device using SSH from our computer. We will need to know the IP address of Picroft on our network, so ask Mycroft:

“Hey Mycroft what is your IP address?”

Remember to write down your IP address as we will need it later!

For Windows users, download PuTTY https://portableapps.com/apps/internet/putty_portable and then install. Open PuTTY and look for the Host Name (or IP address) section, then type in the IP address of Picroft followed by pressing Open. You will then be asked for the username, this is pi and the password is mycroft.

Linux and Mac users will need to open a Terminal, and type in:

ssh pi@<IP ADDRESS OF PICROFT>

Again the password is mycroft.

Once successfully logged in we can now issue commands directly on the Picroft device.

Our first screen is the Mycroft CLI (Command Line Interface), used to monitor and debug Mycroft. For now we do not need it, so press CTRL + C to close the CLI.

With the CLI closed we are now in the Linux terminal, and in here we can control every aspect of the Picroft device. Our first goal is to see what skills we can teach our Picroft, and to do that we type the command, and press Enter.

msm list

This will list all of the skills available for use, so lets go ahead and install a skill. This skill is called skill-dice and we install it from the terminal using the following command.

msm install skill-dice

Now this will take a while to install, but when completed control of the terminal will be handed back to you. Now to start the skill we need to ask Mycroft to roll one, six sided dice.

“Hey Mycroft, roll 1d6.”

Mycroft will tell us what the roll achieved. If you are an avid role player / board game fan then this app can be used to roll the dice in your favourite game. For example a role player may wish to roll three six sided dice.

“Hey Mycroft roll 3d6.”

Or roll a D20:

“Hey Mycroft roll 1d20.”

Just remember to state the number of dice then the number of sides that the dice have.

You are now able to install other skills and use them directly with Mycroft. Try the Internet Radio skill.

msm install internet-radio

You can read more about this skill via their webpage: https://github.com/normandmickey/skill-internet-radio

Setting up WiFi

Right now our Picroft device is tethered to the Internet via an Ethernet cable, which is great, but let’s try and remove one cable from the project.

The Raspberry Pi 3 has a built in WiFi adapter that we can use to connect to the Internet, but we do not have a graphical tool to help us connect. So we need to use the Terminal to edit a file called wpa_supplicant.conf which contains the necessary configuration for our WiFi.

Our first step is to make a copy of the file, just in case something goes wrong.

sudo cp /etc/wpa_supplicant/wpa_supplicant.conf 
/etc/wpa_supplicant/wpa_supplicant.conf.backup

Now we can edit the original file.

sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

In the original file we can delete everything and start fresh. All we need to enter in the file is as follows. You will need to replace the ssid and psk with the WiFi access point, and the password used to get online.

network={
        ssid="YOUR WIFI ACCESS POINT NAME"
        psk="THE ACCESS POINT PASSWORD"
}

To save the edit, press CTRL + O, then Enter. And to exit press CTRL + X.

To ensure that these changes have been written we need to reboot the system.

sudo reboot

Now go ahead and talk to Mycroft!

What have we learnt?

We have successfully created our own digital assistant, that will respond to our commands and act accordingly. Along the way we learnt that digital assistants respond to commands from the user, convert this audio to text and then use that text to launch skills that are preloaded into the device, or to run searches on the Internet. This means that our data is analysed by a remote device, and a response is delivered to us in a timely manner. We also learnt that while this is still not a perfect solution when compared to Amazon or Google’s projects, it is a viable open source tool that does not invade our privacy.

Taking this further, you can write your own skills for Mycroft. Skills are written in Python and you are free to look at the code behind the skills we used in this tutorial, and there is an extensive documentation project supporting the Mycroft project https://mycroft.ai/documentation/

Enjoy your new digital assistant!

Learn More

Mycroft Official Website

https://mycroft.ai/

Mycroft Community Forums

https://community.mycroft.ai/

Hackster.io: “Hey Mycroft, Where Is the International Space Station?”

https://www.hackster.io/gov/hey-mycroft-where-is-the-international-space-station-5f2f04

PuTTY download

https://portableapps.com/apps/internet/putty_portable/

Digital Trends: “Finally, an A.I. voice assistant that doesn’t collect and monetize your data” (Mycroft Mark 2)

https://www.digitaltrends.com/cool-tech/mycroft-mark-ii-assistant/