Hey people! Welcome to my blog.

This post has clear explanation on creating a action for google assistant in Actions Console using DialogFlow and Firebase tools. Alternatively you can scroll to end for project sample in GitHub.
Let’s get started.

Let’s create an action and this example action is about ‘Booking tickets for a movie‘ and see video below to understand and know what we are building.

1. Setting-up Actions Console

Actions console is a tool that simplifies creating your actions in google assistant. It assists to create, build, add and publish your actions.

To create an action, it requires to set up console as follows:

  1. Go to Actions Console.
  2. Click on New Project, followed by accepting Terms of services.
  3. Enter Project Name and other credentials in New Project dialog box. Click Create Project.
  1. To create conversational actions, choose the appropriate category of Action to build which is Custom in this case as shown in the image below. Select Next.
  2. Start with a blank slate by selecting the Blank project card to build action without using sample data. Click on Start building.
  1. In Overview section of actions console, click on Decide how your action is invoked. It directs to Invocation part (Settings) of Develop section.
  2. Enter invocation phrases under Display name, which triggers your action, when user invokes your action.
  3. Click play button to hear pronunciation and it can be modified by clicking on edit icon below. We can also change google assistant voice.
  4. Save the changes.
    Note: To edit project name, click on vertical ellipsis icon in actions console page, select project settings and change the name.
  1. Go to Activity Controls page. Make sure that all the following permissions are enabled
    – Web & App Activity (also enable ‘Include Chrome history’)
    – Device Information
    – Voice & Audio Activity
  2. Go back to Overview section, click on Build your action followed by Add action.
  3. You are directed to actions part of Develop section. Click on Add your first action, it displays CREATE ACTION dialog box. Choose Custom intent and Click BUILD.
  1. A google sign-in is requested automatically, choose your account to work with actions (to continue with Dialogflow).
  2. Google accesses your dialogflow account, click Allow to accept the terms and services. Select Back to Dialogflow, which takes to dialogflow console.

2. Setting-up Dialogflow Console

Dialogflow is a web service agent provided by google that helps in Natural Language Understanding (NLU), it uses machine learning to understand context of what user says, in order to respond in a most useful way. This is where your conversation is built for action and give better conversational experience to the user.

After completion of setting up actions console, you should already be in Dialogflow Console. Actions project name will be displayed in dialogflow page, click CREATE to create your dialogflow agent. You can see that actions project name appears on the left panel of dialogflow.

Understand how it works

  • When a user triggers an action, HTTP request is sent to google assistant from user’s device. Once the action gets triggered, it entirely runs in cloud.
  • Google assistant communicates with dialogflow directing to corresponding action according to user’s utterance.
  • In dialogflow, the response is generated by matching of phrases from user with the intents present in a trained dialogflow agent.
  • If any of the intents are enabled with webhook call (i.e., Fulfillment with webhook, for additional functionality of an action), a callback to webhook is done by dialogflow agent and the fulfillment code that is deployed to cloud functions of firebase handles logic related to corresponding intent and response is generated.
  • This response is collected by dialogflow agent and the agent forwards the response to google assistant.
  • Finally, google assistant sends the fulfillment response to the user.

3. Setting-up Firebase Console

To make your actions deploy to cloud functions for firebase you need to set up by giving the following commands:

  1. Download and Install Node.js.
  2. A Sublime text editor must be installed in your environment to work with javascript.
  3. Open Node.js command prompt and run this command to initialize Firebase CLI (Command Line Interface), which deploys actions project to cloud functions.
    >>> npm install -g firebase-tools
  4. Firebase version can be known by the following command.
    >>> firebase –version
  5. Authentication of firebase to your google account is done with this command.
    >>> firebase login
    Note:
    (a) Check that Firebase CLI Login Successful message is displayed.
    If using more than one google account, see that you are using same account across all consoles i.e., for Actions, Dialogflow and Firebase.
    (b) You can also logout of default account chosen by firebase with this command.
    >>> firebase logout
    Now, you can login again by choosing account of your choice.
  6. Initailizing of firebase has to be done to set up actions project. Before running the command below, make sure that you add a separate new folder in your drive, to host your actions project dependecy files.Now, move to command prompt and navigate to directory of newly added folder and run the command.
    >>> cd <cloud_function_dir>
    >>> firebase init
    Confirm to proceed with the given directory.
  7. Firebase CLI requests to choose which features are to be added to your folder. Choose Functions.
    (*) Functions: Configure and deploy Cloud Functions
  8. Select Use an existing project, which displays your actions project list. Choose current project to work with.
  9. Use Javascript language to write to cloud functions and allow ESLint to catch probable bugs.
  10. Also allow installing dependencies with npm.
  11. If set-up is successful this message is displayed. (Firebase initialization complete!)
  12. Install the actions-on-google dependency with this command.
    >>> npm install actions-on-google
  13. Go to actions project directory in your drive and check that packages are downloaded within folder.
    A functions directory will be created in your project with a Node.js package pre-configured. Functions can be deployed with firebase deploy. You can see index (JavaScript) file inside functions.
  14. Open index.js file
  15. You can see the ‘Hello from Firebase!’ program. Uncomment the code and save it in file.
  1. Open the Firebase Console. The project name is created automatically with same name as in actions console.
  2. Open the Node.js command prompt and run the following commands to deploy to firebase cloud functions.
    firebase use – It selects the active project for deployment
    firebase deploy – Deploy complete! message is seen finally.
  3. Open firebase console and click on Develop section in the left panel.
    Select Functions option, a function URL is created in Dashboard tab.
  1. Copy URL, go to dialogflow console and choose Fulfillment section. Enable Webhook option and paste URL. Click save.

4. Create Intents in Dialogflow

By using intents in dialogflow, a specific activity can be performed as an action. In actions on google, responses of user for an action triggers the corresponding intent. An activity is performed conversationally with these intents in actions, when user triggers it and service that handles user responses in case of logical responses is done through fulfillment.

For creating a simple action, the following steps are necessary:

Go to Dialogflow console, in intents section there are two default intents. A Welcome intent and Fall back intent, both of these are built-in intents. For a better user experience with your action, it needs to have rich responses, handle user silence, conversational flow and design.

  • Select Default Welcome Intent, you can scroll down to see that all the options are presetted already and welcome intent is buit-in helper intent which allows to invoke your action.
  • There’s a welcome intent event under Events, as it is built-in and pre-defined in google assistant and trigger intents without the need for users to say anything. We can also create our own custom events by clicking Add event.
  • Under the Training phrases, some of the phrases are defined to train the dialogflow agent. If any of the phrase uttered by user matches the training phrases given, it triggers that particular intent and response will be generated by google assistant. Whereas, events are optional way to trigger intent without need for matched text or input from user.
  • In Responses, there are certain text responses. When this welcome intent is triggered by user due to matching of phrases trained, any one of the text responses will be delivered by dialogflow to the user.
  • In Fulfillment, clicking on drop down arrow, you can see a couple of options. It is used to call the webhook by enabling webhook call for intent. To extend the functionality of action and handling of logics, fulfillment is used by dialogflow to generate best responses.
  • Click on intents, in the left navigation menu to go back. Select Default Fallback Intent, these are triggered if a user’s input is not matched by any of the regular intents or if it matches the training phrases provided in it. As usually, text responses can be given to respond.

Before starting to create intents, test your action once. You can test it by selecting Integrations in the left navigation and click on INTEGRATION SETTINGS, a dialog box appears.

Check that Auto-preview changes are enabled. Select TEST. Actions console (Test) simulator is opened in another tab, where you can test changes. You can tap on suggested input (invocation) or type in the invocation phrase for your action.

BookTickets – Main Intent

  1. Go to dialogflow console, click on the intents section and choose default welcome intent.
  2. Change the intent name to ‘BookTickets’ . Scroll down to Text responses and delete all response. Now, type in the new response ‘Welcome! Do you want to book tickets for a movie?‘ and Save.

BookTickets – Follow-up Intents

To continue the conversation of a particular intent, you can add follow-up intents that gets triggered based on response from the user. Add follow-up intent to BookTickets (parent) intent.

  1. To create follow-up of an intent, place the cursor over BookTickets intent. You can see ‘Add follow-up intent’. Tapping on it, a dropdown box appears to choose the type of follow-up to be added. Select yes.
  2. A BookTickets – yes follow-up is added. Now again add another follow-up, but selecting no follow-up that creates BookTickets – no intent. Make sure that both the follow-ups are set with the BookTickets intent as parent. (You may accidentally create no follow-up to BookTickets – yes intent, which goes wrong).
    Note: Both these yes and no follow-ups are pre-defined. You can also create follow-up with your own names by clicking on custom, while creating your own action according to its functionality.
  1. Select BookTickets – yes intent, you can see an input context is generated automatically. Contexts control the flow of conversation by defining specific states that a conversation must be, for an intent to match.
  2. Also training phrases are auto-generated due to choosing of built-in follow-up. Add text ‘How many tickets would you like to book? One! or Two!‘ under Text responses and Save.
  3. Go back and select BookTickets – no intent. Add text response as ‘No problem! Let me know when you want to book tickets!‘ and Save. Here the conversation ends, if user opts no.
  4. Move back to intents section. Place the cursor over BookTickets – yes and as usually add two follow-up intents keeping it as parent intent. You must see ‘BookTickets – yes – custom-2‘ and ‘BookTickets – yes – custom‘ are created.
  1. Select ‘BookTickets – yes – custom-2‘ , replace the name as BookTickets – yes – oneTicket. Add training phrases this time, as it is custom ones. Give phrases as “one” and “1“, so that if user gives the input as one either in number or word, this intent is triggered and response is generated. Under Responses, add the text ‘Alright! Your ticket is booked.‘ and Save it.
  1. Select ‘BookTickets – yes – custom‘, replace the name as BookTickets – yes – twoTickets. Add training phrases as “two” and “2” and text response as ‘Alright! Your tickets are booked.‘ , so that this response is sent if user chooses two tickets to book. Click save.
  1. Test your action.

5. Build fulfillment responses

For rich conversational responses and enhancement of user experience, additional functionality can be given in the form of conversational design. It can be suggestion chips (options given as a hint to user without any need of typing), basic card display instead of normal chat bubble, tables, lists etc.

But these conversational designs cannot be created within dialogflow alone, because in dialogflow we can only add text responses to deliver to the user. So, we use Text editor (Sublime) where code is written in Javascript and Node.js client library to process requests and return responses. This request and response is defined by the Dialogflow webhook.


When an intent in our action uses fulfillment i.e., if enabled as “Enable webhook call for this intent“, the webhook code in index.js file is requested by dialogflow and logical response is generated for corresponding intent.

Go to your Sublime text editor in index.js file. Replace the following code.

const functions = require('firebase-functions');

// Create and Deploy Your First Cloud Functions
// https://firebase.google.com/docs/functions/write-firebase-functions

exports.helloWorld = functions.https.onRequest((request, response) => {
    response.send("Hello from Firebase!");
});

With this :

'use strict';

// Import the Dialogflow module and Suggestions dependency from the Actions on Google client library.
const {
    dialogflow,
    Suggestions,
} = require('actions-on-google');

// Import the firebase-functions package for deployment.
const functions = require('firebase-functions');

// Instantiate the Dialogflow client.
const app = dialogflow({debug: true});

// Handle the Dialogflow intent named 'BookTickets'.
app.intent('BookTickets', (conv) => {
    conv.ask('Welcome! Do you want to book tickets for a movie?');
    conv.ask(new Suggestions('Yes', 'No'));
});

// Handle the Dialogflow intent named 'BookTickets - yes'.
app.intent('BookTickets - yes', (conv) => {
    conv.ask('How many tickets do you like to book?');
    conv.ask(new Suggestions('One', 'Two'));
});

// Set the DialogflowApp object to handle the HTTPS POST request.
exports.helloWorld = functions.https.onRequest(app);

Understand the code :

  • We use the string ‘use strict’, which enables strict mode, thereby helps to catch errors and also prevents using undeclared variables.
  • From actions google client library, we are importing the Dialogflow module and Suggestions dependency. This dependency allows your webhook include suggestion chips in your responses.
  • Importing the firebase functions package for deploying webhook code (in index.js file).
  • Instantiation of Dialogflow client includes creating an instance to the variable(app) i.e., DialogflowApp object.
  • The app.intent() function is used to declare a callback to handle both the “BookTickets“ intent and “BookTickets – yes“ intent . A Dialogflow Conversation object parameter (conv) is used with method (ask) that contains text response. We call the ask( ) function with our own response messages to ask information or respond towards the user.
  • Setting the DialogflowApp object to handle the HTTPS POST request, requires exports module to export functions.

You can observe in code that both these intents ( ‘BookTickets‘ and ‘BookTickets – yes‘ ) are called-back with their intent names referring to DialogflowApp object as app.intent( ).
These intents are called-back to add more functionality to enhance user experience. Here we are adding suggestion chips along with text response using conv.ask( ) function.

Follow the steps below :

  1. Go to Dialogflow Console. Select BookTickets intent, delete the Text Response under Responses section by clicking on trash icon.
  2. Scroll down to Fulfillment section and click on drop-down arrow, toggle on ‘Enable webhook call for this intent‘. Click Save.
  3. Go back to Intents menu, select BookTickets – yes intent, delete Text Response as in above step and toggle on ‘Enable webhook call for this intent‘. Click Save.
    (Now, text response for both of these intents are generated by making webhook call to corresponding intents, as they are enabled with fulfillment).

Finally, test your action after deploying the code in index.js file. To deploy,

  • Open Node.js command prompt. Navigate to functions in your project file directory. Enter the command firebase deploy to deploy the project.
    Note: Every time you make changes in the text editor, run firebase deploy command to deploy changes into your actions.
  • Open actions simulator and test your action.

That’s it! We are done.
I have put lot of information and time writing this. Give a like and subscribe to blog for being notified when I write new post.
And check out other articles before you go.

6. References

Build Actions for Google Assistant – Codelabs
Google Actions Console
DialogFlow Console
Github Project

Here We Go Again : (

if (article == helpful) {
    println("Like and subscribe to blog newsletter.")
} else {
    println("Let me know what i should blog on.")
}

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.