The Case for Voice Biometrics in Virtual Assistants

I was reminded again today of the inherently flawed system that enables our virtual assistants to do our bidding. While voice recognition is one thing (not trivial in the slightest), actually identifying the source of that voice is another.  Siri, Alexa, Google, and Cortana are all pretty good at responding to your voice request with an action or an answer to a question.  This is incredibly convenient for allowing me to stay on the couch and ask Alexa to turn the thermostat up a few degrees or to dim the living room lights.  However, she could care less WHO asks her, just that the request was made.

This has been a minor nuisance in the past, even in our house.



Other examples have ranged from humorous to disgusting.. really, who adds "big fart" and "hunk of poo" to their shopping lists?  Oh right.. kids.

This seemingly comical behavior that Alexa or Siri will do whatever you ask her to, becomes less so when you combine it with your physical security systems or home automation which can trigger things like the unlocking of deadbolts, the opening of garage doors, or summoning your fancy Tesla S.

Posted on Reddit, comes the story of a man who automated most of his house including the August door locks and used Siri and HomeKit for voice commands.  This enabled him to unlock the door by asking Siri to do it. Now instead of 'sled dogs' being added to his shopping list by the kids, the locks could be activated by anyone within shouting distance of the iPad.  While Redditors were quick to point out that he should have required a PIN to use HomeKit devices, that would make the virtual assistant pretty useless.  Get up, type in a PIN on the iPad, then issue the voice command?  Sha, right.  As if...

 It's clear that unauthenticated voice commands are problematic.  So what can the developers of these virtual assistants do about it?  Why not look at speaker recognition to identify the individual making the request and authenticate based on voice characteristics?  This is moving forward for other sensitive transactions like voice banking.  There's a great infographic from Nuance that talks about the science and debunks some of the myths. This would reduce or eliminate rogue requests from unauthorized parties and allow you to use voice-triggered HA systems more securely.

In the meantime, if you're on a conference call with a number of people who work from home, I encourage you to ask Alexa to play Jingle Cats or order diamond earrings from Amazon and see how many Echos you can get to respond.

Wink Hub 2 News

Not only has Flextronics seemingly saved the Wink from the PR nightmare that was brick-a-thon 2015, but they've really made some significant strides to push the HA market forward.  Recently they announced the new Wink Hub that touts the addition of BLE, enhanced local support, wired connectivity, and simplified device discovery. It also look much nicer meaning I can probably take it out of the cabinet that it's been relegated to.

Probably no need to use the old hub as a Zwave hub since mesh networking takes care of any remote connectivity issues that may arise.  Guess it's destined for eBay to be picked up for pennies on the dollar.  Actually, I think I only paid $20 for it at the time when it was part of some promotional bundling deal with bulbs.  

Noticeable absences are HomeKit and maybe some cloud-to-cloud support, but honestly i don't think that Apple is going to prevail here if they require an AppleTV or iPad to be left at home for on-the-go automation.

Cobbling together Voice Control with Echo, IFTTT, and Wink

Due to Echo's limited ability to control anything other than lights in the realm of Home Automation, I've been using a combination of Wink Shortcuts triggered by a voice command via IFTTT to accomplish kicking off other devices around my house.  I'd used this in the past after IFTTT finally added US-based Honeywell Thermostat control.  Echo has since added things like native thermostat control with Wink that does work with the Honeywell.

My blinds are Bali blinds that use the Somfy ZRTSI Z-Wave to RF Control tied to Wink, 9 in total.  I replaced the ones that were in the house when we moved in which were a mix of Hunter Douglas PowerRise (unreachable from the landing) and pull string (any blinds that can be reached).

With the new They work well when driven from the Wink app on the phone, but who wants to actually pull out their device when they want the blinds up? I could just use the RF remote for that... honestly.

Since Alexa doesn't understand how to raise the blinds when asked, I've been using Shortcuts to accomplish this.

I first added all of my blinds to a group called, not surprisingly, "All Blinds" then created a shortcut that tells Wink to open all of the blinds.

From IFTTT, you can now add a new recipe to tie Echo to one of these Wink Shortcuts.  

It's not very elegant, but it does work. I use similar Shortcuts to lock the doors, make sure the garage's are closed, and turn off all the lights at night, with my "Goodnight" Shortcut.  While I'd love some more native Echo functionality into Wink, this fulfill my laziness requirement for the meantime.

Integrating Amazon Echo with Kodi

While initially Alexa was more of an amusement and a way for kids to try and covertly put cookies on the shopping list and do their homework, she has become an integral part of our home automation system.  Subsequently, I'm now on a quest to see how many thing I can voice automate even if it would probably be easier to do with my hands or a remote.  Controlling Kodi, our primary interface for entertainment, seemed like an obvious choice.

We "cut the cord" 3 or 4 years ago now and have never looked back. We still have a collection of movies and TV shows on the NAS that provide a welcome distraction every once in a while.  Kodi, or XBMC previously, has been our primary interface to that content since I first softmodded our Xbox more than a decade ago and loaded up XBMC 1.1 sometime in 2005.

Enough backstory, let's get to the build.

Credit - makermusings and m0ngr31.  I'm currently using m0ngr31's fork of this skill.

Components
Clients running Kodi
Ubuntu Server, either in the cloud or on premise
Echo (obviously)

Getting Started

You'll need a Echo Developer Account to make all of this work, you can create an account here, or use your Amazon account. Next you will be creating a new skill in the skills kit.  This build also makes use of a server in my house.  I highly recommend at least a Raspberry Pi running in your home if you want to really get into home automation.  Lots of projects have server-based installations that are difficult/impossible from the cloud.

Configuring the Server

This assumes you have some familiarity with Ubuntu, Raspian, or another distribution.  Lots of other resources can help you get it to the point where it is installed, statically addressed, and ready for configuration.

Install apache2 and add mod_wsgi.  You can use this guide to accomplish this requirement.

Generate a self-signed certificate that will be use to securely exchange information with the Amazon cloud.  This guide was helpful for that.  It's possible that you can use letsencrypt.org to generate a trusted certificate for free to use on your home server.  This isn't the way that I went, but if you're using your web server for anything else, you'll probably want something with some credibility.

Clone down m0ngr31's repository into your /var/www/html directory, or wherever you want your skill to live.

 cd /var/www/html/   
 git clone https://github.com/m0ngr31/kodi-alexa.git  

Change permissions to allow the web server to be able so serve up the content.

 chmod 755 /var/www/html/kodi-alexa/*.py  

Add a line to your default-ssl.conf file for Apache.  This should be in the /etc/apache2/sites-enabled directory.

 WSGIScriptAlias /kodi-alexa /var/www/html/kodi-alexa/wsgi.py  

This will trigger the wsgi.py when a call is made to the /kodi-alexa virtual directory.

Edit the wsgi.py file and remove or comment out lines 4-9.  These lines were put in by the author to work in Red Hat's cloud service Openshift, but will prevent the script from working properly on your own server.

     #!/usr/bin/python

     import os

  

     # virtenv = os.environ['OPENSHIFT_PYTHON_DIR'] + '/virtenv/'

     # virtualenv = os.path.join(virtenv, 'bin/activate_this.py')

     # try:

     #     execfile(virtualenv, dict(__file__=virtualenv))

     # except IOError:

     #     pass

     #

     # IMPORTANT: Put any additional includes below this line.  If placed above this

     # line, it's possible required libraries won't be in your searchable path

     #

     #!/usr/bin/env python

Ensure you have port forwarding turned on your router to allow your apache server to be accessible from the Internet.

The documentation from your router should show you how to port forward.




Building the Alexa Skill



Open the Developer Console and sign in.
Navigate to Alexa Skills Kit.







Create a new skill by clicking on the Add New Skill button.



Skill Information:
Name: Kodi
Invocation Name: Cody
Endpoint: HTTPS - https://<Your DNS or IP>/kodi-alexa

Interaction Model:

Intent Schema: The contents of the alexa.intents file in github, raw content here.

Sample Utterances: The contents of the alexa.utterances file in github, raw content here.

SSL Certificate:

I will upload a self-signed certificate in X.509 format and paste in the contents of the apache.crt file generated above.  If you followed the guide, it will be in /etc/apache2/ssl/apache.crt.

Configuring Kodi

This skill relies on the Kodi API which is enabled when you turn on the Kodi web interface.  

Navigate to Settings --> Services --> Web Server and enable "Enable Remote Control via HTTP"

Specify a username and password to authenticate the API calls.  Since this call will be made from your server on your same network, it will not be sending the authentication across the Internet in clear-text.  

Back on your server, you will need to return to the /var/www/html/kodi-alexa/ directory and edit the kodi.py file.

 cd /var/www/html/kodi-alexa  
 vim kodi.py  

Edit the following section with Kodi's IP address and the username/password information.  If the device you are running Kodi on has a DHCP address and the address changes your skill will no longer work, so either give it a static or use DNS on your network.

    # Change this to the IP address of your Kodi server or always pass in an address

    KODI = os.getenv('KODI_ADDRESS', '192.168.1.25')

    PORT = int(os.getenv('KODI_PORT', 8080))

    USER = os.getenv('KODI_USERNAME', 'kodi')

    PASS = os.getenv('KODI_PASSWORD', 'notmykodipassword')

Enjoy!

You can ask her to play a specific or random movie or TV show, ask about what new content you have, and use basic navigation commands (up, up, down, down, left, right,left right...) you get the idea.  Check out the sample utterances file for more examples about what you can ask here.