More than a month since I haven’t posted… I definitely don’t have as much time as I’d like to work on my personal projects. Turns out working is more time-consuming than I expected! But Gaia is evolving, and it is high time I gave you an update.
Before digging into details, here’s a video showing what my app can do for now:
(I tried using an app that captures the screen but it’s in conflict with the Dialogflow api and it doesn’t work, so I made a video instead)
So, it is now possible to add/modify/delete reminders and relations directly from the app, and the chatbot can interact with this information.
I connected my app to firebase in order to be able to store data. To do this, I used the plugin angularfire2 and bumped into some issues: with the newest version (v5), most of what I could find on the internet didn’t work. Fortunately, I found this video tutorial explaining very clearly how to store shopping items in firebase with a Ionic app, and I adapted it to my needs.
The tricky part was the uploading and storage of photos for relations: they’re not just text you can pass along, you need to store them somewhere. Again, I found this very helpful tutorial from the same guy, and I added the displaying photos part. I store the images in firebase storage, then display photos using their download URL. So I don’t actually store the images in the database, I just store a link to them.
Then, the even trickier part was letting the chatbot access the database. As I mentioned, if you want your chatbot to be able to perform operations and not just respond to the user, you need a webhook. I had already used a webhook the last time I posted, but this webhook didn’t have to write or read data in the database.
What I could find on the internet was either outdated or didn’t work with Ionic, but after a while I remembered that in the Dialogflow console there was an inline editor where a webhook was already coded, meant to be powered by firebase cloud functions. So I took this code and wrote my own functions before deploying them. In order to do that, you need to have the dialogflow-fullfilment plugin installed. Creating entries for reminders was fairly easy (the firebase doc was helpful), but I admit making the “Who is” function work was a bigger challenge (because you need to search for a specific name in your database without having a key instead of just creating a new item).
Now that I have done this, I consider the hardest part behind me. I’ve developed the basic components, and from now on, all I add will be easier to implement (at least I hope so). There’s still a lot to do, of course, but I think this project might not need my full attention anymore and I’m hoping I can take on new projects if I want to.
These are my next steps:
Details on reminders
When the user wants to add a reminder, the chatbot will ask if there are any details or a deadline to remember. When a deadline approaches, it will remind the user what they have to do.
Details on relations
When the user asks who is a certain person, if known the photo of this person will appear in the interact view (replacing the photo of Gaia which is currently a photo of myself). The chatbot will also be able to give details about that person and to store facts (in a list) when the user says something about them. For example, if the user says their daughter just had a new job, the chatbot will store this information to be able to talk about it later.
Statistics on user’s behaviour
Recording of certain questions (such as “who is X”) to keep track of the user’s behaviour and be able to have a better understanding of their mental state if needed.
After having answered a question, instead of just stopping as regular chatbots would do, Gaia will keep making the conversation by picking questions or fun/informative facts at random.
Improving the UI in numerous ways and creating a visual for the interact view.
Allow users to connect to a personal account. Really not a priority as I don’t plan on releasing this app to the public, at least not for now.
Adding the planning of the user, ideally connected to an outlook/google calendar (authentication needed).
It will take time for my app to be “finished”, but I’m sure it’s nothing I can’t handle!
A girl in tech