[some notes and headaches on data acquisition]

Current working hardware - Raspberry Pi 3, Copperhill PiCAN, Arduino Nano, Sparkfun Venus GPS, and a SanDisk USB drive

And now for something completely different.

New objective. Instead of recording the data to a thumb drive, we're going to stream the data from the can to a websocket. From there I can create an iOS app to connect to the Rpi and parse the data as needed. Node.js and socket.io will be used for the code. I'll also need to determine why I cannot use cansend() to display text on the dash yet. That is irking me lately.

There is also a new version of the PiCAN for the Raspberry Pi 4 here or here. It has a built in RTC which is freaking rad. I'll have to order a new Pi and one of these.

So programming when gps lines are crossed is not that simple. Been hammering on this for a few days now. If I've got two GPS points that make one line it's not really that simple. If there are four points it's not so bad. At some point you'll be inside the box. I think what needs to happen is I need to store the previous coords and evaluate the path drawn. If only there was a way to draw lines in html/css. I'm going to need this image. https://commons.wikimedia.org/wiki/File:Mid-Ohio.svg
Mid-Ohio race track, fun.

Been a little while. I found something very interesting specifically for a Ducati. PZRacing Plug and play lap timer. From what I can tell it doesn't log the times just displays them on the dash. The price isn't horrible but I'm kinda interested in what it does. The functionality can be mimicked with and arduino and a GPS. Set up some predetermined GPS way points. At system start up, start a timer. When ever a way point is crossed, log the time and reset the timer. With this device it just displays it on the dash. Seems pretty simple.

A successful test. PTR day at Mid-Ohio was really good. For the data, I'm going give race render another shot. It seems that importing the the gps and the can data will be a bit easier to get it all together. With the Virb software I currently have to convert the nmea data to gpx then import it. RR will will simply import the nmea without converting it. I have a script that will conver the can data. That will need processed into a format that RR will accept, CSV, XLS, tab.

Real world test at Mid-Ohio on Monday the 21st. Hopefully the solar exlipse won't jack things up!

I have the Venus GPS running in 8Hz mode and it's humming a long. I'll have the code posted soon. The whole package seems to working pretty smoothly. I did have one issue with the usb stick when it was not present on start up all of the datafiles get placed into a directory on the system drive. Not that big of a deal but kinda of a PITA. I've ordered a RTC for the RPi Zero W. I'm planning on moving to that device. Half the size and just as good as the larger Pi.

Test two will have some camera footage. Tested sending the commands to bring the chip up to 20hz and it doesn't look like it's done anything. We'll find out later today. Most likely, I'm doing it wrong. The googles turned up this little gem today. 50hz! Insane. With the CANBUS on the same board. That's a lot of data. I'd like to get one to test it out. It may not have the flexibility of the RPi but still.

Soldered together, so far the arduino and gps are working well. I had a small problem on the first run, I had the baud rate for the serial terminal on the arduino set incorrectly. But a simple update of the code and burn and it's turning out some good data. With the powered antenna it looks like it's picking up a signal really quickly after getting power. It's not running at the full 20hz yet. I think it's closer to 2hz by default. I'll update that and retest.

Received both the new GPS chip and the new Arduino Mini to power it. The Arduino code is pretty light weight.

Got a first 'real' test with the CAN + GPS together. Just a ride to work and it seems to be a pretty good success. A 40+ minute ride generated 25M of data. The GPS file is not finalized yet. After the new GPS unit shows up I should have that nailed down. Then what to do with all this data. What I'm looking for is a video over lay to display the throttle, rpm, and reported wheel speed / reported gps speed. I should be able to take that data and generate an After Effects script file. Or create a Garmin Virb plug in. I'm not really sure that GoPro's app will accept custom plug in but I haven't really looked into it yet.

And my AdaFruit uGPS is dead. Completely dead. Next purchase is a unit from Sparkfun with an AdaFruit Trinket. SparkFun's Venus has 20hz update rate which is double that of the uGPS. I also ordered a 3.3volt Trinket. GPS → Tinket → RPi. This may get interesting. If the GPS is set to blast out data at 20hz and the Trinket runs at 8hz there could be some issues. I will most likely run the gps off a FTDI to usb into the RPi.

The past weekend I got a few miles in with the PiCan. Three plus hours of riding generated a 104M file. I've yet to parse through all of it. When the bike is turned on the the can bus will power the PiCan An RPi. A start up script checks for a USB storage device and creates a log file there. This process seems to work pretty well. I'm close to getting the gps (which is just an Arduino with the Adafruit uGPS shield) to fire up and dump it's data on the drive. The serial port isn't playing nice yet. The gps is now dumping data into a datafile. The trick was to set the baud rate of the port before attemping to read from the port (stty -F /dev/ttyUSB0 115200). On the other hand, Copperhill who make the PiCan have created a PiCan with gps built in It can't yet power the RPi from the canbus but the board has the traces for that module. The other nice thing here is it has an RTC with battery. If they could only get this thing down to Pi Zero size.. they'd have my money. I may also connect a heart rate monitor to the Pi since it has bluetooth. Just to track "pucker" moments. Also, the cases that I have everthing stuck in are from C4 Labs.net. For both the Arduinos and the Pi's

And I've killed one RaspberryPi. I replaced it with another. The board may have gotten zapped with static. The new board is humming along. Currently, I'm looking at switching back to the Arduino with an RTC and adding a GoPro Hero 5 with gps.

The Raspberry Pi B+ 2 & the PiCan Hat are working so far. It's kinda of a mess of data. I'm also using the Arduino & the Adafruit uGPS shield to handle the gps for now. So far the data is getting save to a USB thumb drive and so far it looks pretty good. The only issue that I really have is keeping time. The Pi doesn't include a real time clock. Sp I ordered a small one called a PiFace shim. It sits on the gpio pins between Pi and a hat. Shipping from the ebay seller is pretty slow, I'm not sure when this thing is going to show up.

Back on the RaspberryPi wagon. I ordered one of these. I'm still have some issues recording both the gps and the canbus data. The arduino just chokes on the data coming in. The arduino can do one or the other with no issues. But both and dump that data one a usb drive seems to be an issue. I'm also running into low memory issues when adding in the usb lib. I think a RPiA+ should do the drink. I also ordered an RTC to use with. Timecode issues are a PITA.

I've got a bit of a back log of video. The current software still has some issues. I can collect either the can data or the gps data but when combining them all hell breaks looks. I've also bought a usb host card to log all the data on a usb stick rather than the tiny microSD card. Usb keys are just eaiser to handle that those really really small and loseable sd cards.

First video with both the canbus and the GPS shield on youtube. It's almost a complete fail. My GoPro has eaten two SD cards and rendered them useless but I did get out and picked up a new one that is working. The Arduino was able to capture both the GPS and canbus data as expected. I'm still working on decoding that data. The GPS, well, I've got the code pulling only minimum requred NMEA sentences get a location. That text file get processed though GPSBable and imported into the Garmin Virb application. GPS Bable will take the nema sentences and create a .gpx file. Somewhere in that mix the ground speed was totaly jumbled.

I've had another brain fart, I'm really thinking of picking up a Garmin Virb Ultra 30 and using the Arduino 101's BLE to just stream the data to the camera. The canbus doesn't work with the 101 yet. There is a chunk of avr code in the canbus library that would need ported to compile with the 101. That could be next up.

The Current stack is The UNO, CANBus, and Ultimate GPS w/ extrn antenna. There are some issues. On the ugps shield pin 10 is used for the chip select for the SD card. That pin on the CANBus is used for the actual can chip. To remedy this I've ordered a USB interface card that will be used inplace of SD cards. I'm not fan of the SD cards anyway. They're pretty small and you need to carry an adapter to used them. Thumbdrive will be a better choice. Looks like a simple addition but there is a LOT of code. I may be pushing 66% or so of memory on the UNO. I could hit some walls there.

Changed things up a bit. Received a CANBus shield from Sparkfun. It will not work with the 101 (yet). So I'm back to using the UNO. I don't have any real working samples yet. I'm working on the sketch. It's not playing nice yet. I also recieved a gps antenna from Sparkfun. It will pick up a signal in seconds. It light years faster than the smaller antenna.

I'm looking at possibly moving the platform to a Raspberry Pi. I'd really like to be able to dump all the data onto a usb drive rather than the SD cards and leave the entire unit on the bike. With the Arduino the SD cards are the way it is. If there is a simple way to use a usb drive; I haven't found it yet.

I'm thinking the RasPi B+ is the way to go. The PiZero only has two usb, one for power and one for data. It's got the bluetooth and wireless. I would also like to just dump the data to a usb drive or a database. It would also handle both GPS and CAN data in multiple threads making possible to capture more data. More data is better, yea.

The lastest hardware used was the Adafruit Ultimate GPS with an Arduino Uno. It was not very successful in track testing. I'm not entirely sure how to boost the accuracy yet. I may need lower the sampling frequency to get better results. It's set at 10hz now. I can drop to 5hz or 1hz. And after looking at the notes I may have been doing it wrong. There may be some hope for this card yet.

Next purchase is the Canbus shield from Sparkfun. This will allow me to pull data directly from the bikes ODBII port. Here's the sample code. As 7/27/16 Sparkfun has these on back order. There are a few no name boards on the eBays. There is also some good info here. This is another library that uses the ODBII port with gps.

Here is the wiki on ODBII codes. I have not coded or tested any of these yet.

I'm now using the Arduino 101 with the uGPS from Adafruit. Currently street testing.

I think I've got a pretty good system so far. Here's a link to the latest overlay test with Garmin's Virb Edit. The quality of the is ok. I'll need to run it through Compressor before uploading.

Here is another video scooting around by work in Akron.

Raspberry π Zero W

Just received this little bugger in the mail. I've only yet to get the OS loaded and configured. I'll need to order and solder the headers and find a case that I like. So far it looks like this one might just work.

PiCAN2 for RaspberryPi [info]

I'm hoping this board will simplify the hardware so I can dump all the data directly to a usb drive. I will also be able to to remotely code the software without lugging the device between the bike and the pc. I should also be able to do more data prep on the device before importing it.

Adruino 101

This is a cool little board. A six axis accelerometer, gyroscope, & a Bluetooth for $30. Huge potential here.

First issue I ran into with this card and the gps shield below is that from what I read the board didn't or doesn't support the Software Serial. The GPS shield sends it's data over 7 & 8 with the softserial.h library. A link to the forum post at Adafruit.

Second issue is that this board is 3.3v on all the I/O pins. It really wasn't a problem but some shields can be incompatible with this board.

Adding the GPS shield from Adafruit. And some issues. Under some heavy vibration this board will loose it's bearings. It seems that with the board and the shield on the bike it wouldn't work for more that a few seconds. I can't determine which part was failing; the card reader, the gps signal, the gps sheild, or the arduino. In testing with just the arduino plugged into a pc while logging just the info coming from the gyro and accel. If you were to just shake the shit out of the board it will not read out consistant readings. Example, if you let it calibrate itself the device would read that it was right-side-up. Shake the hell out of it and place it back on the desk upside down. It will read that it's right-side-up still.

I updated the version of code installed on this to include the correct location update frequency. The resulting files will be up soon. -- Bugs, it'll have to wait --

Bugs worked out. I think I've got working a bit better now. It's not using Adafruits class now. It's just sucking down all the info that is coming off the radio. It's not in a production ready state but better for testing. I have one file so far that has captured a pretty good amount of data from the ride to work.

I have some more sample files with the Arduino on the bike. Data looks good. I did have one instance where there was zero data recorded. Not sure if it was user error yet.

Adruino UNO

This is the basic starter board. It's pretty bullet proof. 5v I/O pins. Standard hardware serial. Works pretty good with the Adafruit GPS shield.

Adafruit Ultimate GPS shield

Includes a standard GPS radio and a micro sd slot for data logging. All of the raw data that I pulled off of this device can be found here: 5/23 & 6/30 The text data can be entered into a NEMA data parsers. Some of the data from 5.23 is not valid NEMA sentences and will not parse. Those lines begin with $AGYRO. They were ment to include the gyro data with the gps data and I fail at programming. The standard NEMA sentneces are generated using the UNO and this sketch with some simple changes.

Sparkfun CAN Bus Arduino shield

No Frills, it just sucks up all the data that is spewed out the data port on the back of the bike. I had to solder up a db9 cable to plug in. The other end of that cable is a Delphi connector. I had to find the terminal pin for it on ebay. There is also a seller in the UK on ebay that sell kits for this plug for Ducatis. The software library seems to work OK. However, none of the ODB codes in the library actually work with the bike.

This shield is also not compatible with the Arduino 101. So it's being tested with the Uno.

Purchased the GPS module to go with this board. It's in the mail.

ELM 327 ODBII with Wifi

Only tested this one in the truck so far and it's pretty cool. I bought this one because of the ease of use with the iPhone. I've tested with two applications TrackAddict & FourStroke. Both seem pretty cool so far. TrackAddicts will allow you to export the raw data into a csv file.

So far, I can't get this work with an iOS application. It looks like it's powering up and I can connected to the Wifi network. It ends there. The applications for the phone will not get any data from the device. I will need to connect directly to the ELM device on a COM port and see if there is data actually be transmitted or if the proper initial initialization code is getting sent.

Ducati DDA for the 1098 (older version)

I can't speak for the new version of this thing but the one made for the 1098 (from 2008) is pretty much a turd. The data is locked into a binary format in the file. The device is not reconized on a Mac. At the time the company ProsSA had no way for a Mac to pull the data off the device or even use the data from the deivce. It was windows only. They do have a Mac application now but it will not open the old files and I'm not sure if the device I have for my bike can be updated with newer firmware to use the new software.

Race Render

I've yet to test this application but they say they can overlay NEMA and Trackaddict data on to a video file and it's avaliable for Mac & Windows. I've got one or two videos and gps info that should link up.

Tested with a video from Mid Ohio with sample gps data. It's not super easy to link up. Getting timecode from a GoPro is not a simple thing (yet). The gps data was this. It's not a very good reference. There is a break in the middle of the file and at one point it has me doing something like 200 mph. But combining the actual video and sample data works pretty well. Syncing them up is the tricky part.

This software is kinda of dead to me now. I tested once. It seemed to work ok. Here's the result. I downloaded a copy of Garmin's Virb software and I'm going to test that out. It requires that you convert he gps data to a gpx format.

Garmin Virb Edit

These guys may just eat GoPro's lunch. Not only is the overlay software pretty good; the camera looks pretty good as well (I haven't used one yet). I'll have a link asap. This software requires a gps file from a Garmin device or GPX formated file. It's just xml. Here's a DTD for the file type

General Notes

Spent a minute looking at wether a RaspberryPI would be a better choice than an Arduino. The first thing is the software overhead on the PI. The linux and the linux dependency hell. Win for the Arduino. However, the googles are turning up a few results of the PI working with some vehicles and it looks ok. I'd like to have the wifi and apache with php installed. That'd be pretty cool.

Some more google foo whipped up this info for Ducati's canbus info. Good stuff.