In this blog, I describe how to run the Google Assistant SDK on a Raspberry Pi with an external speaker and microphone. After you get things configured and gather key pieces of information you'll have a simple, portable and easy to reproduce Google Assistant appliance.
Running the Google SDK on the LmP is extremely simple, just run the following docker command and you'll have the demo running in a matter of minutes.
docker run --rm -it --device /dev/snd \
-e MIC_ADDR=$MIC_ADDR -e SPEAKER_ADDR=$SPEAKER_ADDR -e PROJECT_ID=$PROJECT_ID \
-e MODEL_ID=$MODEL_ID -e CLIENT_SECRET=$CLIENT_SECRET -e CLIENT_ID=$CLIENT_ID \
-e REFRESH_TOKEN=$REFRESH_TOKEN \
--name ok-google-cli-demo opensourcefoundries/ok-google
Ok, maybe the challenge isn't running the sample, but figuring out the hardware addresses and Google Cloud variables necessary.
MIC_ADDR="hw:2,0"
SPEAKER_ADDR="hw:2,0"
PROJECT_ID=voice-kit-01
MODEL_ID=voice-kit-01-number2-flc5yj
CLIENT_ID=7508-5d3.apps.googleusercontent.com
CLIENT_SECRET=kF4ZLu-X
REFRESH_TOKEN=1/Ux0k2DAjHYb6jbA
In the rest of this blog, we'll show you where to collect all of these values so that running the Google Assistant will be easy, repeatable and portable.
Get all the latest software by using the latest microPlatform release. Sign up for a subscription @ https://app.foundries.io and download the latest artifacts.
Download a LmP pre-built binary for your Raspberry Pi 3b, write it to a microSD card or decompress and write it to the flash device using the dd command.
If you are unfamiliar with the Linux microPlatform, more detailed instructions on using the Linux microPlatform can be found at Documentation on how to install the Linux microPlatform
ssh [email protected]
In order to route the audio to the right device we will need to find the hardware addresses (and possibly adjust the volume a bit).
docker run -it --rm --device /dev/snd opensourcefoundries/ok-google arecord -l
setup authorized credentials file
setup alsa configuration
+ exec arecord -l
**** List of CAPTURE Hardware Devices ****
card 2: USB [Jabra SPEAK 510 USB], device 0: USB Audio [USB Audio]
Subdevices: 1/1
Subdevice #0: subdevice #0
docker run -it --rm --device /dev/snd opensourcefoundries/ok-google aplay -l
setup authorized credentials file
setup alsa configuration
+ exec aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: vc4hdmi [vc4-hdmi], device 0: MAI PCM vc4-hdmi-hifi-0 []
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: ALSA [bcm2835 ALSA], device 0: bcm2835 ALSA [bcm2835 ALSA]
Subdevices: 7/7
Subdevice #0: subdevice #0
Subdevice #1: subdevice #1
Subdevice #2: subdevice #2
Subdevice #3: subdevice #3
Subdevice #4: subdevice #4
Subdevice #5: subdevice #5
Subdevice #6: subdevice #6
card 1: ALSA [bcm2835 ALSA], device 1: bcm2835 ALSA [bcm2835 IEC958/HDMI]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 2: USB [Jabra SPEAK 510 USB], device 0: USB Audio [USB Audio]
Subdevices: 1/1
ÅSubdevice #0: subdevice #0
Now we know that the MIC is 2,0 and the Speaker is 2,0 we will be able to pass this information into the container for further adjustments. (Note: the card ID may may change over reboots)
docker run -it --rm --device /dev/snd -e MIC_ADDR="hw:2,0" -e SPEAKER_ADDR="hw:2,0" opensourcefoundries/ok-google speaker-test -t wav
docker run -it --rm --device /dev/snd -e MIC_ADDR="hw:2,0" -e SPEAKER_ADDR="hw:2,0" opensourcefoundries/ok-google alsamixer
OR
docker run -it --rm --device /dev/snd -e MIC_ADDR="hw:2,0" -e SPEAKER_ADDR="hw:2,0" opensourcefoundries/ok-google alsamixer --card=2 --view=all
After you adjust the volume, you can retest using speaker-test command from step 1 above.
Note: Much of this blog was excerpted from Google's SDK documentation at https://developers.google.com/assistant/sdk/guides/library/python/
A Google Cloud Platform project, managed by the Actions Console, gives your device access to the Google Assistant API. The project tracks quota usage and gives you valuable metrics for the requests made from your device.
To enable access to the Google Assistant API, do the following:
Click on Add/import project.
To create a new project, type a name in the Project name box and click CREATE PROJECT. If you already have an existing Google Cloud Platform project, you can select that project and import it instead.
Click the Device registration box.
Enable the Google Assistant API on the project you selected (see the Terms of Service). You need to do this in the Cloud Platform Console.
ENABLE THE API in Console.developers.google.com
Then Click Enable.
In order to use the Google Assistant, you must share certain activity data with Google. The Google Assistant needs this data to function properly; this is not specific to the SDK.
Open the Activity Controls page for the Google account that you want to use with the Assistant. You can use any Google account, it does not need to be your developer account.
Ensure the following toggle switches are enabled (blue):
Use the registration UI in the Actions Console to register a device model.
The credentials.json file must be located on the device. Later, you will run an authorization tool and reference this file in order to authorize the Google Assistant SDK sample to make Google Assistant queries. Do not rename this file.
Download this file and transfer it to the device.
scp ~/Downloads/credentials.json [email protected]:/home/osf/
In the Google samples you would now install software onto your target device, however the container you have been using to query the audio hardware is configured and ready to run the sample with no further modification.
In order to connect your device to the Google services you will first need to generate the appropriate credentials. We'll do this in a more interactive version using the bash shell and mapping in our credentials.json file from above.
docker run -it --rm --device /dev/snd -v ${PWD}/credentials.json:/credentials.json opensourcefoundries/ok-google bash
google-oauthlib-tool --scope https://www.googleapis.com/auth/assistant-sdk-prototype \
--scope https://www.googleapis.com/auth/gcm \
--save --headless --client-secrets /credentials.json
Running google-oauthlib-tool will return a HTTPS URL that you will need to copy and paste into your browser to complete the authorization steps.
Copy the authorization code back into your terminal window to complete the authorization dance and get your credentials.
Note three values from your credentials file
cat /root/.config/google-oauthlib-tool/credentials.json
{
"client_secret": "kF4ZLu-X",
"scopes": [
"https://www.googleapis.com/auth/assistant-sdk-prototype",
"https://www.googleapis.com/auth/gcm"
],
"token_uri": "https://accounts.google.com/o/oauth2/token",
"client_id": "7508-5d3.apps.googleusercontent.com",
"refresh_token": "1/Ux0k2DAjHYb6jbA"
}
Now we can exit the container and re-launch it to run the ok-google sample and pass in all of the variables we have discovered.
MIC_ADDR="hw:2,0"
SPEAKER_ADDR="hw:2,0"
PROJECT_ID=voice-kit-01
MODEL_ID=voice-kit-01-number2-flc5yj
CLIENT_ID=7508-5d3.apps.googleusercontent.com
CLIENT_SECRET=kF4ZLu-X
REFRESH_TOKEN=1/Ux0k2DAjHYb6jbA
docker run -it --rm --device /dev/snd \
-e MIC_ADDR=$MIC_ADDR -e SPEAKER_ADDR=$SPEAKER_ADDR -e PROJECT_ID=$PROJECT_ID \
-e MODEL_ID=$MODEL_ID -e CLIENT_SECRET=$CLIENT_SECRET -e CLIENT_ID=$CLIENT_ID \
-e REFRESH_TOKEN=$REFRESH_TOKEN \
--name ok-google-cli-demo opensourcefoundries/ok-google
setup authorized credentials file
setup alsa configuration
+ exec bash -c 'googlesamples-assistant-hotword --project_id ${PROJECT_ID} --device_model_id ${MODEL_ID}'
device_model_id: voice-kit-01-number2-flc5yj
device_id: FB5DB23
https://embeddedassistant.googleapis.com/v1alpha2/projects/voice-kit-01/devices/FB5DB23 404
Registering....
Device registered.
ON_MUTED_CHANGED:
{'is_muted': False}
ON_START_FINISHED
ON_CONVERSATION_TURN_STARTED
ON_END_OF_UTTERANCE
ON_RECOGNIZING_SPEECH_FINISHED:
{'text': 'what time is it'}
ON_RESPONDING_STARTED:
{'is_error_response': False}
ON_RESPONDING_FINISHED
ON_CONVERSATION_TURN_FINISHED:
{'with_follow_on_turn': False}