Below is more details and guidance for our project submission to the 2017 IEEE ComSoc Student Competition “Communications Technology Changing the World”.
This page contains information and implementation details about the project, The project consists of a device that sits inside of a vehicle and constantly monitors car's information. It can determine speed, g-force, and location. Using these data, the device can detect a car crash (or pothole on the road). The data collected from the car is forwarded to a server for more in-depth analytics. If there is an accident, the server promptly contacts the emergency services and contacts with the location of the crash. The pothole information is used for analytics of road conditions for smart city operators.
Arduino Uno, SIM808 Module, SIM card, GPS Antenna, GSM Antenna, Arduino 101 (Intel Curie), LEDs, 220 Ohm resistors.
Amazon AWS cloud IaaS EC2, Java IDE (Netbeans and Intellij), NodeJS, Javascript, MySQL, HTML, CSS
JDBC, Twilio, Google reverse geocode
The main "brains" of the ARAM is a Java server that obtains a stream of data from the ADM. As the server receives incoming data, it writes the data to a MySQL database with JDBC.
// Open a TCP port for the Arduino to establish a connection
// ...
// Connection with the MySQL database
try {
// Register JDBC driver
Class.forName("com.mysql.jdbc.Driver");
// Open a connection
conn = DriverManager.getConnection(DB_URL, USERNAME, PASSWORD); // DB_URL, USERNAME, and PASSWORD are constant variables
stmt = conn.createStatement();
} catch (SQLException se) {
se.printStackTrace();
}
Here is the main parts of the Server.java file. There are two port numbers, one for the java server, and one for the NodeJS server that shows the accident information on the web portal. In the code, we first connect to the JDBC server (for writing accident and pothole information to the database). Then the java server and NodeJS server are initialized.
private static final int PORT = 43005;
private static final int WEBSERVER_PORT = 43006; // port number of NodeJS server
public static void main(String[] args) throws Exception {
try {
//STEP 2: Register JDBC driver
Class.forName("com.mysql.jdbc.Driver");
//STEP 3: Open a connection
conn = DriverManager.getConnection(DB_URL, USER, PASS);
stmt = conn.createStatement();
} catch (SQLException se) {
...
}
ServerSocket ServerSocketlistener = new ServerSocket(PORT);
ServerSocket NodeJSSocketListener = new ServerSocket(WEBSERVER_PORT);
try {
NodeJSHandler NJSH = new NodeJSHandler(NodeJSSocketListener.accept());
NJSH.start();
new Handler(ServerSocketlistener, NJSH.getNodeJSSocket()).start();
} catch (Exception e) {
System.out.println(e.getMessage());
}
...
This is how the NodeJS server and java server look like. When the Java server receives a pothole or accident data from the ADM, it first generate an appropriate meta data for the event (e.g. "accident from passenger side"), then it obtains a humar readable address of the location, then writes it to the database, and finally it sends the data to the NodeJS server, so that the pothole or accident is shown on the website.
The two functions callnTextEmergencyContacts() and generateMessage() are shown below as well.
private static class NodeJSHandler extends Thread {
public void run() {
try {
out = new PrintWriter(socket.getOutputStream(), true);
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
...
private static class Handler extends Thread {
// ...
public void run() {
while (true) {
try {
...
while ((input = in.readLine()) != null) {
...
if (input.length() > 10) {
String[] parts = input.split(" ");
// parts[0] = [latitude], parts[1] = [longitude] parts[2] = [speed], parts[3] = [shock] parts[4] = [gforce], parts[5] = [axis], parts[6] = [type], parts[7] = [driverId]
if (parts[6].equalsIgnoreCase("1")) { // if type is 1 (Accident
message = generateMessage(Integer.parseInt(parts[4]), parts[5]);
} else {
message = "Pothole";
}
...
String humanReadableAddress = ReverseGeocode.getStreetAddress(parts[0], parts[1]);
int eventID = writeEventToDB(parts[0], parts[1], parts[2], parts[3], parts[6], parts[7], message, humanReadableAddress);
try {
sendEventIDtoNodeJSserver(nodeJSSocket, eventID, parts[0], parts[1], parts[2], parts[3], message, humanReadableAddress, parts[7]);
} catch (UnknownHostException e) {
System.out.println("Node server is down!");
}
if (parts[6].equalsIgnoreCase("1")) { // if type is 1 (Accident
callnTextEmergencyContacts(parts[7], parts[0], parts[1], humanReadableAddress);
}
}
}
...
}
// ...
private static void callnTextEmergencyContacts(String DriverID, String latitude, String longitude, String humanReadableAddress) throws SQLException, IOException{
String sql = "SELECT * from drivers where id='" + DriverID + "'";
ResultSet rs = stmt.executeQuery(sql);
while (rs.next()) {
//Retrieve by column name
String name = rs.getString("name");
String EmergencyName = rs.getString("emergencyName");
String EmergencyContact = rs.getString("emergencyContact");
String plate = rs.getString("licensePlate");
String emergencyMessage = "Hello, We are sorry to deliver this urgent message. " + name + ", driver of the car with licence plate number " + plate + " was involved in an accident now, at" + humanReadableAddress;
Calling.textPhone(DestinationPhone, emergencyMessage, AuthorizedCallerPhone); // text the emergency contact
Calling.createXML(emergencyMessage, "/var/www/html/call/v1.xml");
Calling.callPhone(DestinationPhone, "http://ec2-54-202-247-75.us-west-2.compute.amazonaws.com/call/v1.xml", AuthorizedCallerPhone);
}
rs.close();
}
private static String generateMessage(int gforce, String biggestShockAxis) {
String message = "Accident from the ";
if (gforce < 0) {
if (biggestShockAxis.equalsIgnoreCase("x")) {
message += "front";
} else if (biggestShockAxis.equalsIgnoreCase("y")) {
message += "driver side";
} else {
message += "top";
}
} else {
if (biggestShockAxis.equalsIgnoreCase("x")) {
message += "back";
} else if (biggestShockAxis.equalsIgnoreCase("y")) {
message += "passenger side";
} else {
message += "bottom";
}
}
return message;
}
}
The Java server also sends text messages and phone calls using the Twilio API with reverse geocode latitude, longitude coordinates (for a human-friendly address). This feature is intended for notifying the emergency contacts of the driver who is involved in an accident and emergency authorities (e.g. 911).
Here is how Twilio API is used for making automated calls and sending predefined text messages.
public static final String ACCOUNT_SID = "....";
public static final String AUTH_TOKEN = "....";
/**
* Texts the DestinationPhone from the registered phone number with Twilio
* (TwilioAuthorizedCallerPhone), and sends the message that is specified in
* textMessage
*
* @param DestinationPhone
* @param textMessage
* @param TwilioAuthorizedCallerPhone
*/
public static void textPhone(String DestinationPhone, String textMessage, String TwilioAuthorizedCallerPhone) {
Twilio.init(ACCOUNT_SID, AUTH_TOKEN);
Message message = Message.creator(
new PhoneNumber(DestinationPhone),
new PhoneNumber(TwilioAuthorizedCallerPhone),
textMessage).create();
}
/**
* Calls the DestinationPhone from the registered phone number with Twilio
* (TwilioAuthorizedCallerPhone), and speaks the message that is specified
* in XML file stored in XMLPath
*
* @param DestinationPhone
* @param XMLpath
* @param TwilioAuthorizedCallerPhone
*/
public static void callPhone(String DestinationPhone, String XMLpath, String TwilioAuthorizedCallerPhone) {
Twilio.init(ACCOUNT_SID, AUTH_TOKEN);
try {
Call.creator(new PhoneNumber(DestinationPhone), new PhoneNumber(TwilioAuthorizedCallerPhone),
new URI(XMLpath)).create();
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* Creates XML file on the server. The server uses this XML file
* as the input to its automated call.
*
* @param s Message
* @param fileLocation location where the XML file will be created
*/
public static void createXML(String s, String fileLocation) {
try {
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.newDocument(); //
Element rootElement = doc.createElement("Response"); //
doc.appendChild(rootElement);
Element supercar = doc.createElement("Say");
rootElement.appendChild(supercar);
// setting attribute to element
Attr attr = doc.createAttribute("voice");
attr.setValue("alice");
supercar.setAttributeNode(attr);
supercar.appendChild(doc.createTextNode(s));
// write the content into xml file
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer();
DOMSource source = new DOMSource(doc);
StreamResult result = new StreamResult(new File(fileLocation));
transformer.transform(source, result);
} catch (Exception e) {
e.printStackTrace();
}
}
Here is how Google reverse geocode is used to get the street address (human readable) from the latitude-longitude coordinates.
private static JSONObject readJsonFromUrl(String url) throws IOException, JSONException {
InputStream is = new URL(url).openStream();
try {
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(is, Charset.forName("UTF-8")));
String jsonText = readAll(bufferedReader);
JSONObject json = new JSONObject(jsonText);
return json;
} finally {
is.close();
}
}
public static String getStreetAddress(String latitude, String longitude) throws IOException {
String googleMapPath = "https://maps.googleapis.com/maps/api/geocode/json?latlng=" + latitude + "," + longitude + "&key=our-google-api-key";
JSONObject json = readJsonFromUrl(googleMapPath);
JSONArray results = (JSONArray) json.get("results");
JSONObject firstComponent = (JSONObject) results.get(0);
return firstComponent.get("formatted_address").toString();
}
There are two Arduinos that run concurrently and asynchronously. The Arduino Uno is responsible for networking and obtaining GPS data. The majority of connections that the Arduino makes is done through the Adafruit FONA library. Before anything can be done, the Arduino must establish a Serial connection with the FONA module.
#include
//Pins used for software serial
#define FONA_RX 2
#define FONA_TX 3
#define FONA_RST 4
//SoftwareSerial is used to communicate with FONA
SoftwareSerial fonaSS = SoftwareSerial(FONA_TX, FONA_RX);
SoftwareSerial *fonaSerial = &fonaSS;
Adafruit_FONA fona = Adafruit_FONA(FONA_RST); //Fona object
//...
fonaSerial->begin(4800);
After connection with the FONA Shield, GPS and GPRS are enabled. Then a TCP connection is established with the Java server via TCP.
const char* server = "54.202.247.75"; //IP address of our server
const uint16_t port = 43005; //Port of server
//...
fona.TCPconnect(server,port);
After successful connection to the server, the Arduino Uno continuously polls for GPS location and speed. If a GPS signal was found, the GPS coordinates and speed are stored into variables. If no GPS signal was found, the approximate GSM coordinates are used instead along with estimated speed. This is the main loop which is only broken when the Arduino 101 sends the intterupt signal.
// ...
fona.enableGPS(true); // Enable GPS
// ...
if (foundGPS){
//Stores coordinates and speed into variables
}
else {
//...
fona.getGSMLoc(&latitude, &longitude); // If GPS is unavailable, we can use GSM location instead
//Stores GSM coordinates into variables
}
The loop is broken out of via an interrupt signal from the Arduino 101, only sent when a crash is detected. When the interrupt signal is received by the Arduino Uno, the compileData() function is immediately entered. The Arduino Uno then listens for data sent from the Arduino 101 via the I2C protocol. It is listening for crash G force, the axis of impact, and the type of impact sent from the Arduino 101.
//...
while (Wire.available()) { // loop through all but the last
char x = Wire.read(); // receive byte as an integer
data[dataIndex] = x;
dataIndex++;
}
//...
This data is then compiled into a string along with GPS coordinates inside the compileData() function. The string contains GPS coordinates, speed, G force, axis of impact, type of impact, and total magnitude. This final string is then sent to the Java server and the Arduino Uno enters into its polling loop again.
void detectCrash(){
if (crashDetected){
memcpy(dataToSend,data,lastIndex(data) + 1);
fona.TCPsend(dataToSend,lastIndex(data) + 1);
Serial.println(dataToSend);
}
crashDetected = false;
}
The need for two separate Arduinos is due to the fact that the networking functions block the program from doing anything else, causing it to potentially miss an accident. Arduinos do not support multithreading so it is necessary to use two separate Arduinos and rely on interrupts. The purpose of the Arduino 101 is to constantly poll accelerometer data to detect a crash. This polling is independent from the Arduino Uno's networking functions, so there is no blocking.
void loop() {
float gx, gy, gz; //G-Forces on all axes
CurieIMU.readAccelerometerScaled(gx, gy, gz); //Get accelerometer data
//...
}
If there acceleration on any axis exceeds the crash threshold defined at the beginning of the program, then the axis of impact, the max G force, and the shock are all sent to the Arduino Uno via I2C. The Arduino 101 then continues to poll acceleration, repeating the same process.
//...
char maxAxis = findMaxAxis(gx,gy,gz); //Finds the axis with highest g-force
float gForce = findMaxGForce(gx,gy,gz,maxAxis); //Finds highest g-force
memset(gForceChar,0,sizeof(gForceChar)); //Clears gforce array
dtostrf(gForce,2,2,gForceChar); //Converts float to array gForce -> gForceChar
sendWire(shock,gForceChar,maxAxis,'0'); //Sends data byte by byte to the other Arduino via I2C
//...
The NodeJS server is responsible for handling real-time display of crash data. It obtains data from the Java server and uses a socket.io connection to displays the traffic data. When a web client connects, the NodeJS server sends recent traffic data within the past hour given the driver ID from the Java server. In order to obtain this previous data, the this server also needs to connect to the MySQL database.
let con = mysql.createConnection({ // Define the connection parameters
host: "localhost",
user: "anrl",
password: PASSWORD,
database: "traffic"
})
con.connect(function(err) { // Connect to the MySQL database
if (err) throw err
})
// ...
io.on('connection', function(socket){ // On new web client connection and send the most recent traffic data to the client
// Query the database
con.query("SELECT date, latitude, longitude, speed, shock, type, message, address, name, licensePlate FROM events, drivers WHERE events.driverId = drivers.id", function (err, result, fields) {
if (err) throw err
let records = []
records = result
socket.emit('client-connection', {'records': records})
})
// ...
const io = require('socket.io')(web_server)
The NodeJS server uses ExpressJS to route server static webpages - the console webpage and the analytics webpage.
// Route to the console webpage
web_app.get('/console', function(req, res, next){
res.sendFile(path.resolve('/path/to/console.html'))
})
// Route to the analytics webpage
web_app.get('/analytics', function(req, res, next){
res.sendFile(path.resolve('/path/to/analytics.html'))
})
// ...
const web_server = https.createServer(options, web_app);
web_server.listen(443); // Server with SSL enabled (via Let's Encrypt)
We used the Yarn package manager, but you can still use NPM. Below is the manifest.json configuration:
{
"name": "node-server",
"version": "0.1.0",
"private": true,
"dependencies": {
"cors": "^2.8.4",
"express": "^4.15.4",
"express-http-proxy": "^1.0.6",
"mysql": "^2.14.1",
"request": "^2.81.0",
"socket.io": "^2.0.3"
},
"scripts": {
"start": "sudo node server.js"
}
}
The main portion of the console webpage is a Google map that displays markers in the order of real time data from the NodeJS server. This real time interaction is facilitated by a TCP-based connection via socket.io. The user can also toggle when he wants to view traffic data for the past 24 hours in hourly increments, or select a date range.
<div id="main">
<h1≷Traffic Console</h1>
<div id="user-input">
<div id="slider">
<input class="bar" type="range" id="rangeinput" value="0" min="0" max="24" onchange="rangeChangeCallback(this.value)" oninput="rangeInputCallback(this.value)"/>
<output id="rangevalue">Now</output>
</div>
<span class="example-spacer"≷</span>
<div id="selection">
<span≷From:</span>
<input id="startDate" class="datepicker" type="text" placeholder="start date" onchange="startChangeCallback(this.value)"/>
<span>To:</span>
<input id="endDate" class="datepicker" type="text" placeholder="end date" onchange="endChangeCallback(this.value)"/>
<button id="submitBtn" onclick="onSubmit()"≷Get results</button>
</div>
</div>
<div id="map"></div>
</div>
<script>
// ...
socket.on('client-connection', function(data){ // When client is connected with web server
<!-- ... -->
let marker = new google.maps.Marker({ // Create marker
position: {lat: accident.lat, lng: accident.lng},
map: map,
icon: crashIcon,
title: 'Crash'
})
})
// ...
function initMap() { // Callback function when Google map is loaded
map = new google.maps.Map(document.getElementById('map'), {
center: {lat: 32.984, lng: -96.752},
zoom: 13
})
// ...
}
<script/>
<!-- ... -->
<script src="https://maps.googleapis.com/maps/api/js?key=API_KEY&callback=initMap"></script>
This webpage displays the accident speed in sections of a donut chart. It also displays a comprehensive line chart comparing the potholes to accidents given a date range.
<div id="main">
<h1>Analytics<h1/>
<div id="user-input">
<span>From:<span/>
<input id="startDate" class="datepicker" type="text" placeholder="start date" onchange="startChangeCallback(this.value)"/>
<span>To:</span>
<input id="endDate" class="datepicker" type="text" placeholder="end date" onchange="endChangeCallback(this.value)"/>
<button id="submitBtn" onclick="analyze()">Analyze<button/>
<div/>
<div id="donut-container">
<h2>Accident Speeds<h2/>
<canvas id="donutChart"><canvas/>
<div/>
<div id="line-container">
<h2>Accident-Pothole Comparison<h2/>
<canvas id="lineChart"><canvas/>
<div/>
<div/>
<script>
flatpickr(".datepicker", {}); // Instantiate datepicker object
// ...
// Donut chart
// ...
myLineChart = new Chart(lineCtx, { // Line chart
type: 'line',
data: {
labels: labels,
datasets: [
{
label: "Crashes",
fill: false,
lineTension: 0.2,
borderColor: 'rgb(255, 99, 132)',
data: crashData
},
{
label: "Potholes",
fill: false,
lineTension: 0.2,
borderColor: 'rgb(66, 134, 244)',
data: potholeData
}
]
},
options: {
responsive: true,
maintainAspectRatio: true
}
})
<script/>
Run the Java server first, then run the NodeJS server, then start up the Arduino. Open the console webpage and jolt the Arduino to see the system in action! (Make sure you changed the DestinationPhone in Server.java to your cell phone number, so that don't call 911 for this project each time you jolt the Arduino! Make sure your MySQL database is running by the way.
java -cp ~/path/to/libraries/\*: Server # In the Java server directory
yarn start # In the NodeJS server directory
This project has been a rewarding experience for all team members. In the future, we consider inplementing connections between the Arduino and Java server as well as the Java server and NodeJS server with UDP, so we can avoild the delay for TCP handshake (connection setup). We may also update the project with additional features, such the ability for parents to track their children's driving habits (average speeds, turning habits, acceration habitis, etc).
The code of this project is open-source and available to everyone!