This is a project for team HCMUTCSE in Kambria Hackthon II, 2021.
View Demo »
Table of Contents
In this project, we will develop applications on an Ohmni healthcare assistant robot, which helps outpatients in hospitals with registration, thermal measurement, navigation, ...
This section is for the web application which is deployed to the Ohmni robot via https://app.ohmnilabs.com/my-robots. The following subsections give instructions on setting up the project locally.
Clone down this repository. You will need node and npm installed globally on your machine. You can find the download page at Nodejs Homepage
-
Make sure your npm is the latest
npm install npm@latest -g
-
A Firebase project for the web framework, you can create one at Firebase Homepage
-
MQTT Broker, e.g. Adafruit
Change directory to web.
cd webIf you are running the project for the first time, install all the dependencies
npm installFor the Firebase project, register your web app with steps follow Firebase register app. For the Firebase config variable, go to Project Setting, under Your Apps section.
Prepare your environment by renaming the .env.example to .env with your variables
| Variable | Value |
|---|---|
| REACT_APP_FIREBASE_API_KEY | apiKey |
| REACT_APP_FIREBASE_AUTH_DOMAIN | authDomain |
| REACT_APP_FIREBASE_PROJECT_ID | projectId |
| REACT_APP_FIREBASE_STORAGE_BUCKET | storageBucket |
| REACT_APP_FIREBASE_MESS_SENDER_ID | messagingSenderId |
| REACT_APP_FIREBASE_APP_ID | appId |
| REACT_APP_MQTT_SECRET | <your MQTT Secret, if any> |
When they're all set run the project locally
npm startWhen you first access the page, you need to provide the MQTT config: input the topic name to subscribe, host, port, clientid, username, and password (if any). There is a pre-filled example in the Demo so you can quickly use it to test.
Next, choose the medical check-in type, there are two options:
- Register: register for a new patient
- Retrieve: for the old patient, retrieve medical checkup
In the Register option, fill all the needed information and continue. In the Retrieve one, input the phone number to retrieve all the previous information.
At the very bottom, there are the status of the MQTT Client and the current accuracy of the robot. There is a button "MQTT Test", link to another page, to explicitly test your connection.
This part is dedicated to the code used to run and command the behaviours of the Ohmni robot. For software, we use python3 and the pycoral library. With hardware, we utilized the Ohmni® Telepresence Robot as well as the Google Coral USB accelerator for tensor support.
Installation of pycoral is required and you can run this for a quick pip install:
python3 -m pip install --extra-index-url https://google-coral.github.io/py-repo/ pycoral~=2.0If there are still problems regarding the pycoral library, check out the pycoral installation guide for further explanation. Note: We do not use the edgetpu library because it is deprecated and will be hard to reuse for future projects so do not install it and its following libraries.
It is extremely important to plug in your Google Coral USB accelerator into the robot prior running the program.
After cloning the Github to your robot's Linux environment, run it by executing the ohmni.sh file. It shouldn't be possible for the application to run correctly since there are 2 things also required:
Filling all of these 3 should be enough to run the program and subscribe to a feed.
ADAFRUIT_IO_KEY = ''
ADAFRUIT_IO_USERNAME = ''
FEED_ID = ''It is needed to capture pictures from the camera for the application to process and work correctly. Hence, create another terminal, this time in the android environment of the robot, initiate the OpenCamera application to begin capturing:
monkey -p net.sourceforge.opencamera -c android.intent.category.LAUNCHER 1It is possible to stop the OpenCamera application via physical actions or terminal:
input keyevent KEYCODE_HOME