What's new
Van's Air Force

Don't miss anything! Register now for full access to the definitive RV support community.

Small Portable Serial SD Card Logger

Brantel

Brantel

Well Known Member
Background:
I have been working with Dynon on some project that require me to log the raw serial data being used by my Dynon legacy system from my NMEA GPS. Up till now every time I have had to do this, I have had to lug a laptop to the airport, secure it in the cockpit and fly around with it in the passenger seat while trying to see the screen and keep it working while in flight. Painfull to say the least!

So I decided to make a go at creating an inexpensive, small, very lightweight and easy to use SD card logger that can log any serial stream onto an SD card with little to no effort. The logger must be capable of logging data at typical NMEA GPS data rates all the way up to ultra high speed logging of the Dynon EFIS data stream of 53bytes 64 times a second @ 115,200 baud. Yes that is a ton of serial data at a high rate of speed!

What I came up with is an Arduino based logger that can be built for around $40 using off the self kits. So far it is working well and is capable of meeting my original goals for logging the data without missing a byte. If anyone is interested in more details let me know. It is easy to build for someone that can solder. I can provide you with my code to operate it or give you a head start on your own code. The unit is very simple. Connect the data line and power and it starts logging. 2 LED's provide feedback about the SD card initialization and incoming data flow status.
 
Last edited:
Just so people are aware, the D100 series has internal datalogging so they only reason you need this is if you want to store the serial output which gives you 64Hz updates instead of 1Hz. It's much easier in 99% of cases to just download the flight from inside the unit.
 
Yes, this is true. No reason to normally log the Dynon EFIS stream unless you need to play around with the high rate data for some reason. An example was my remote AOA indicator project development. While I listed logging the Dynon EFIS stream as a possible application, it was not the primary goal. I figured if I could get the hardware and code efficient enough to handle that much data at that rate, the primary goal would be a snap.

The main reason I built this is to log the actual NMEA data flowing from the GPS to the Dynon. This data is not logged by the internal logger.

Also keep in mind that this logger is generic. It can log RS232 serial streams from just about anything that spits out a stream. It is not Dynon specific.

An example: this could be used to log PMag Electronic Ignition data in flight in real time. The data could then be viewed later on the ground.


Oh yeah, there is another reason: It is fun!
 
Last edited:
I used a similar arduino based logger and TTL shifter from Sparkfun. It has automatically logged my first 100 hours of flight on a 4GB microSD card. I have viewed and downloaded the data once, and it blew up MS excel! There is just too much data to process. I haven't bothered since then. I would love to see support on cirrusreports.com.

What I find more useful is taking screen snapshots periodically. I wish the Skyview system could do it automatically for me, instead of me having to remember to do it manually.

Just my 2c!
 
I better understand what you are doing now. I was liking not having to take the laptop to the airport, not that carrying a laptop is difficult.
 
I would like to have the plans for this logger you have developed. I have been seeking a small form factor board to log both EFIS and EMS data to function as a "poor boy's black box". My goal is to locate this in the tail and provide data and power to a "hardened" box containing the logger. Optimily this logger would write continuously and rewrite as required to keep the most current one or two or three hours, whatever the storage size of the given SD card. Thanks, Dan
 
I need engine data from SkyView

As many of you know I have been troubleshooting an intermitten engine problem and really, really need to be able to record engine data from my SkyView. Unfortunately, even if I figure out how to record the data, according to Dynon it still requires conversion, which they currently are not providing a method to do.

Brantel - I have dabbled with Arduinos and have also tried to do a logger using a SD card read/writer and shifter but without success. I'd love some instructions for setting up the spare Arduino sitting on the shelf to do this.

jchang10 - how about posting exactly how you did it? I think we're trying to do the same thing but I've not successfully recorded any data
 
John,

I really did not know the Skyview did not log internally yet.

Hmmm....well by looking at the manual, they do a good job explaining their serial output streams. It would not be that difficult to decode the data with that info.

I see they are sending 222 bytes with the EMS setting alone. This will require the use of some buffering in the code to keep from losing data. The UNO has a 128 byte receive buffer out of the box so it won't hold a full set of EMS data without dropping bytes.

Today I flew for over an hour with my logger logging the GPS stream to the SD card. Works great!

You could always use the old fashioned way of storing the EMS stream onto a laptop with a terminal program. If you did this, I could help you get the data into a format you can read and graph with Excel.

Tomorrow I will post details on what I have that works.
 
Last edited:
More Info..

To those that wanted more info:

To make this logger here is what I used:

arduinounor3_MED.jpg

Arduino UNO, available at tons of places online. Just use your favorite source.



datalogshield_MED.jpg

Adafruit SD Logging Shield


13z4mfo.jpg

RS232 to TTL level shifter, Use one made from an IC. The resistor types do not work well at high speeds.

Put it all together and end up with something that looks like this:

vijjiw.jpg


Now you will need to download the latest Arduino IDE.

And install the following library add ons:

SdFat
Serialport

And then copy this sketch into a project, edit the baud rate to suit your needs and download it into the UNO.

Code: 
const uint32_t BAUD_RATE = 9600;
// Maximum time between sync() calls in milliseconds.  If Serial is always
// active, you must provide a way to stop the program and close the file.
const uint32_t MAX_SYNC_TIME_MSEC = 1000;
#include <SdFat.h>
#include <SerialPort.h>
SerialPort<0, 1024, 0> NewSerial;
SdFat sd;
SdFile file;
//------------------------------------------------------------------------------
// pin for error LED
const uint8_t ERROR_LED_PIN = 7;
const uint8_t DATA_LED_PIN = 8;
// Error codes repeat as errno short blinks with a delay between codes.
const uint8_t ERROR_INIT   = 1;  // SD init error
const uint8_t ERROR_OPEN   = 2;  // file open error
const uint8_t ERROR_SERIAL = 3;  // serial error
const uint8_t ERROR_WRITE  = 4;  // SD write or sync error
void errorBlink(uint8_t errno) {
  digitalWrite(DATA_LED_PIN, LOW);
  uint8_t i;
  while (1) {
    for (i = 0; i < errno; i++) {
      digitalWrite(ERROR_LED_PIN, LOW);
      delay(200);
      digitalWrite(ERROR_LED_PIN, HIGH);
      delay(200);
    }
    delay(1600);
  }
}
//------------------------------------------------------------------------------
void setup() {
  pinMode(ERROR_LED_PIN, OUTPUT);
  pinMode(DATA_LED_PIN, OUTPUT);
  digitalWrite(ERROR_LED_PIN, HIGH);  
  NewSerial.begin(BAUD_RATE);
  if (!sd.init()) {
    errorBlink(ERROR_INIT);
  }
  if (!file.open("SERIAL.TXT", O_WRITE | O_CREAT | O_AT_END)) {
    errorBlink(ERROR_OPEN);
  }
  if (file.fileSize() == 0) {
    // make sure first cluster is allocated
    file.write((uint8_t)0);
    file.rewind();
    file.sync();
  }
}
//------------------------------------------------------------------------------
// time of last sync
uint32_t syncTime = 0;
uint8_t buf[32];
void loop() {
  if (NewSerial.getRxError()) {
    errorBlink(ERROR_SERIAL);
  }
  uint8_t n = NewSerial.read(buf, sizeof(buf));
  if (n > 0) {
    digitalWrite(DATA_LED_PIN, HIGH);
    if (file.write(buf, n) != n) {
      errorBlink(ERROR_WRITE);
    }
    // don't sync if active
    return;
  }
  if ((millis() - syncTime) < MAX_SYNC_TIME_MSEC) return;
  if (!file.sync()) {
    errorBlink(ERROR_WRITE);
  }
  syncTime = millis();
  digitalWrite(DATA_LED_PIN, LOW);  
}

The credit for most of this code goes to Bill Greiman, the developer of the libraries used in the sketch.

I have tested this logger with normal NMEA data out of a Garmin portable at 9600 baud. Works great.

Also have tested this logger with the D100 serial stream at 115,200 baud. It works but this is on the edge of how much data can be handled. The better SD cards have more overhead built into them. The D100 spits out over 3.3Kb/sec of data! The Arduino Mega would handle this with ease.

I have tested with EMULATED Skyview data (ADAHRS+SYS+EMS) at 9,600, 19,200, 38,400 baud and it works great. I also tested 57,600 and it is marginal with the full data set being output. It would work with a reduced data set selected.

I have no idea why one would need more than 9,600 or 19,200. Those two baud rates still produce a ton of data fast. At 19,200 it logs the full Dynon data set @ ~ 10 times/sec!

Good Luck!
 
Last edited:
Bubblehead, sorry to hear about your issues.

I used the 2 products from sparkfun.com listed here:
http://www.sparkfun.com/products/9530
http://www.sparkfun.com/products/449

I did have to solder a few pig tails to the boards. You could call it done with just some wire and a soldering iron. However, I did add a molex connector inbetween so i could more easily debug and access things.

It was a while ago, but I believe it only required 3 wires to be connected: Power, Ground, Tx or Rx. The Rx and Tx wires are the trickiest, because it must be the right one. Thus, SV serial Tx to sdlogger Rx but i was confused on their labelling of Tx or Rx. I can't remember which it was off-hand. I had issues with this initially and had to debug and swap wires around until the board was happy.

However, the board works well once working. There are several indicator lights of different colors and blinking levels to determine whether data is being logged just by looking at it.

I have to go back and review details further. I have been meaning to take a photo, which I can come back and add if wanted.

Jae

Bubblehead said:
jchang10 - how about posting exactly how you did it? I think we're trying to do the same thing but I've not successfully recorded any data
Click to expand...
 
Last edited:
I have updated post #10 to show the latest testing I have done for max EMULATED logging of the Skyview data set based on info from Dynon and the Skyview manual.

It looks like this logger could be used to effectively provide a good logging solution for the Skyview until Dynon gets their internal logging feature out.

If someone attempts to use this to log Skyview data, I suggest setting your baud rate at 9,600 just to keep the amount of raw data down.......Dynon says that if the baud rate supports it, they can spit out updates at 16 times per second max.
 
Last edited:
Brantel - outstanding write up. I'm traveling this week but looking forward to tackling it next weekend. I actually have the data logging shield in kit form already so I just need to solder that up and upload the libraries and code to the Arduino. I already have a data cable setup from the SkyView with ground, RX and TX so should be good to go. I used R/C model servo extension wires and connectors to do that so it will be easy to plug this in for testing.

Was the emulated data downloaded from SkyView in text format or was further conversion needed?

jChang10 - OpenLog is exactly what I used, and the shifter is the same chip I used but in a board without connector.
 
Thanks for posting this. I'm neck-deep in (re)wiring right now but will make the time to dig into it.
 
Progress on the project

I assembled the SD card logger shield and have it up and running. Need to get the level shifter up and going now. I have the same shifter as jchang10 used but without the DB-9 mounted on it.

I'll download the libraries and sketch and be ready once the shifter is done.

I've really enjoyed dabbling with the Arduino. I have been working through the "Sparkfun Inventor's Guide" to get started but have a couple more advanced how to books to work on too.
 
Question on Libraries

I downloaded the two libraries which unzipped as sdfablib20111205 and SerialPortBeta20120106 with subdirectories of SdFat and SerialPort.

Do I rename sdfablib20111205 and SerialPortBeta20120106 to SdFat and SerialPort or do I move subdirectories SdFat and SerialPort into the libraries directory.

I had read that the .h files need to be no more than on level down from "libraries."
 
Move the subfolders to the library folder.

The name of the folder and the library have to match.
 
Hooking up the serial data

Brantel - I built the data logger and ran through the on-line tutorial. Everything seems to be working right. I just ordered the shifter but looking at your pictures of the logger shield I noticed that the straight logger shield looks a bit different from the last picture in your post. There appears to be some other components added.

What's the rest of the circuit, or do I just need to connect the output (TX), +5V and ground from the shifter to certain pins on the board through a connector?

John
 
jchang10 said:
Bubblehead, sorry to hear about your issues.

I used the 2 products from sparkfun.com listed here:
http://www.sparkfun.com/products/9530
http://www.sparkfun.com/products/449

I did have to solder a few pig tails to the boards. You could call it done with just some wire and a soldering iron. However, I did add a molex connector inbetween so i could more easily debug and access things.

It was a while ago, but I believe it only required 3 wires to be connected: Power, Ground, Tx or Rx. The Rx and Tx wires are the trickiest, because it must be the right one. Thus, SV serial Tx to sdlogger Rx but i was confused on their labelling of Tx or Rx. I can't remember which it was off-hand. I had issues with this initially and had to debug and swap wires around until the board was happy.

However, the board works well once working. There are several indicator lights of different colors and blinking levels to determine whether data is being logged just by looking at it.

I have to go back and review details further. I have been meaning to take a photo, which I can come back and add if wanted.

Jae
Click to expand...

I have been pursuing parallel paths; yours and Brantel's. Here is the circuit I tried.

ScreenHunter01-Sep-24-1703-L.gif


I used the same shifter but without a DB connector, and used a 7805 voltage regulator to be able to use either 12 or 5 volt from the aircraft to supply 5V to the shifter and the OpenLog.

I have yet to get any data successfully to the logger. The LEDs on the shifter show data from the Skyview is being received but nothing is going to the card. As you can see from the diagram, the data TX line from Skyview is going to the RX pad of the shifter, and the TX connection on the shifter is connected to the RX pad of the OpenLog.

Perhaps I should try swapping RX and TX lines on the shifter?
 
Bubblehead said:
Brantel - I built the data logger and ran through the on-line tutorial. Everything seems to be working right. I just ordered the shifter but looking at your pictures of the logger shield I noticed that the straight logger shield looks a bit different from the last picture in your post. There appears to be some other components added.

What's the rest of the circuit, or do I just need to connect the output (TX), +5V and ground from the shifter to certain pins on the board through a connector?

John
Click to expand...

Its just a connector and some jumper wires. I played around with some other parts during prototyping but they are not needed.
 
How much data? Can't be more than 4gb, right? Send me a zipped text file and I'll send the data back to you in Microsoft Access. Also the new Excel format (XLSX) can support a million rows of data.

Just for fun.

Dkb


jchang10 said:
I used a similar arduino based logger and TTL shifter from Sparkfun. It has automatically logged my first 100 hours of flight on a 4GB microSD card. I have viewed and downloaded the data once, and it blew up MS excel! There is just too much data to process. I haven't bothered since then. I would love to see support on cirrusreports.com.

What I find more useful is taking screen snapshots periodically. I wish the Skyview system could do it automatically for me, instead of me having to remember to do it manually.

Just my 2c!
Click to expand...
 
Brantel - what pins do I connect to on the Arduino? I would think GND and TX from Skyview go to GND and something else on Arduino, and that the Arduino is powered from a small battery pack.
 
The Skyview RS232 output must connect to the logger shield thru a RS232-TTL level shifter.

The output of the level shifter is connected to the logger shield/Arduino pins 5v, GND, Pin 0 and 1 (although this is configurable) for TX and RX, and optionally the reset pin if you want to upload the program to the Arduino thru the serial connection.
 
Keep in mind that what I have is based on the Adafruit logging shield. I see over on the Dynon forum that you have been playing with a Sparkfun Open Log logger. My setup may not work the same with that logger.
 
I'm working both sides of the street. I have an Arduino with the logger shield working on your method and working on jchang's method too. I am learning a lot and perhaps will succeed at one.
 
Yahoo! I figured out I had the data from Skyview going into the shifter in the TTL side not the RS-232 side where it belonged. I changed one wire, added a ground to the RS gnd pad and successfully recorded data! Now I need to figure out how to make sense of it! Here is what one line of data looks like.

!3112074208000+02900000000292000000011140162263123XXXXXXX0703800674XXXXXXXXXXXXX
XXX+030+029+030+029+030+029+030+029XXXXXXXXXXXXXXXXXXXXZZZZZZ+0011P+4998VZZZZZZZ
ZZZZZZZZZZZ+0140G+0162G+0000P+0287CXXXXXXZZZZZZZZZZZZZZZZ24

There is a thread on the Dynon forum site that is covering a lot of the same territory as here. Here's a link http://dynonavionics.com/cgi-bin/yabb2/YaBB.pl?num=1344201894/0#14

On that other forum a member posted a utility to decode the data but that utility is set up to decode ADAHARS and EMS data and I only recorded EMS data. I am hoping he (or she) can modify the utility to handle just EMS data. Alternatively I will go to the hanger tonight, change the setup in Skyview and record some more data.

Major breakthrough today, and that feels good. I have been putzing around with this on and off for 6 months.
 
I was able to record ADAHRS plus EMS data and convert it using lolachampcar's utility. When I get time this weekend I will correct the schematic, put a bill of material on it and write some instructions. Hopefully that will help people get some data until things come down the pike from Dynon.
 
Here is the corrected schematic with a parts list. Set Skyview data output to "ADAHARS and EMS" and use the utility that lolachampcar posted on the Dynon forum to convert the data, then inport it into Excel as a comma delimited file.

schematic-M.jpg
 
With help from a person on the Dynon forum we've simplified the data logger even more. The level shifter is eliminated and replaced with a 2N7000 MOSFET and a 2.2k resistor. I did not have a 2N7000 so bought the only thing the local Radio Shack, a IRF510, shown here http://www.radioshack.com/product/index.jsp?productId=2062618

It only took a few minutes to add to one of the breadboards that had an OpenLog and voltage regulator already on it. I tried it this morning and it recorded data just fine.

Total cost is really low, probably under $40 for the whole thing.
$24.95 for the OpenLog
$2.19 for the Radio Shack MOSFET
about $2 for the pack of resistors
about $3.50 for a perf board to mount it all on
about $3.00 if you want to add a 5V voltage regulator 7805.

I used a RC servo extension to wire it into the plane. I cut the extension in half, soldered the male end onto the perf board and the female end was connected to +12V, GND and TX serial line from SkyView. I will probably rewire to the +5V source from SkyView and eliminate the voltage regulator.

Here is the schematic.

SkyView-Data-Recorder-2012-L.jpg
 
Bubblehead said:
With help from a person on the Dynon forum we've simplified the data logger even more. The level shifter is eliminated and replaced with a 2N7000 MOSFET and a 2.2k resistor. I did not have a 2N7000 so bought the only thing the local Radio Shack, a IRF510, shown here http://www.radioshack.com/product/index.jsp?productId=2062618

It only took a few minutes to add to one of the breadboards that had an OpenLog and voltage regulator already on it. I tried it this morning and it recorded data just fine.

Total cost is really low, probably under $40 for the whole thing.
$24.95 for the OpenLog
$2.19 for the Radio Shack MOSFET
about $2 for the pack of resistors
about $3.50 for a perf board to mount it all on
about $3.00 if you want to add a 5V voltage regulator 7805.

I used a RC servo extension to wire it into the plane. I cut the extension in half, soldered the male end onto the perf board and the female end was connected to +12V, GND and TX serial line from SkyView. I will probably rewire to the +5V source from SkyView and eliminate the voltage regulator.

Here is the schematic.

SkyView-Data-Recorder-2012-L.jpg
Click to expand...

Ever thought of building a few and selling them? I for one would buy one. :cool:
 
GalinHdz said:
Ever thought of building a few and selling them? I for one would buy one. :cool:
Click to expand...

I am refining the PC board now and if that goes well could have a few in about 10 days. It would be sooner but I will be on the road for most of the next 5 days. I am planning on a DB9 connector because we are all familiar with them and most of us have the tools for putting the DB9 male together. The DB9 would have power, serial Data line and ground. The power could come fromeither the Dynon 5V supply soit would record data whenever Skyview is on, or 12V and record whenever the Master switch is on. I am undecided which is better but with the 7805 voltage regulator the aircraft owner could use either and the circuit would handle it fine.

I appreciate the suggestion.
 
Bubblehead said:
I am refining the PC board now and if that goes well could have a few in about 10 days. It would be sooner but I will be on the road for most of the next 5 days. I am planning on a DB9 connector because we are all familiar with them and most of us have the tools for putting the DB9 male together. The DB9 would have power, serial Data line and ground. The power could come fromeither the Dynon 5V supply soit would record data whenever Skyview is on, or 12V and record whenever the Master switch is on. I am undecided which is better but with the 7805 voltage regulator the aircraft owner could use either and the circuit would handle it fine.

I appreciate the suggestion.
Click to expand...

PM me when you have them so I can buy one and "Beta" test it for you. :cool:
 
We are making progress

DB9 board mount connectors should be here at the end of the week. I designed the PCB today using software called http://www.diptrace.com/. It is very easy to learn and use and much better than the other schematic and PCB layout programs I have tried in the past.

I am getting ready to export the files needed to get quotes on boards but will prototype up a board Saturday and give it a try.
 
3 prototypes ready Saturday

The first 3 boards showed up. I assembled one and tested it yesterday, and everything worked fine. It recorded data and a small utility program split the data into blocks which were easy to import into an Excel spreadsheet.

I will build up the next two tomorrow night and test them Saturday. Anyone wanting to record data from their Skyview can now do it, and I'll sell these three for $50 each. The data logger alone was $25 plus board and a few components and some development time went into it so I think it's a fair price.

To hook it up just connect the supplied cable to +5V from Skyview, ground, and a Skyview serial data TX line, then setup Skyview to output EMS plus EFIS data. When you power up Skyview the recorder will start recording data on a micro SD card.

If you can wire up an RV you can wire this thing up. Contact me at [email protected] if you are interested. This weekend I will tweak the board layout and order more components.
 
Have you thought about using a Raspberry Pi for this application?

It has all the required ports, a plenty fast CPU, needs zero soldering, runs on 12VDC, and costs about $30.

Combine it with a Keyspan USB serial port (or any cheaper equivalent that's capable of 115200 bps) and you're in business.

For an extra five bucks you can even get a nicely made plastic enclosure to keep it in.

- mark
 
Newt - there are lots of ways to skin this cat. Brantel shared an Arduino method complete with real time clock that is great. I have the Arduino and shield but need to wire it in and load software etc. I would love to see a Raspberry Pie version. I suspect once the SD or micro SD card portion is on there it will cost more than the $50 I am offering plus need some programming and an uploader and the right libraries in the right place etc. etc..

The device I am offering is simple and cheap and ready to go for $50. Plud and Play! If someone wants to collect EMS and EFIS data from Skyview next week, they can buy this now and be good to go.

I appreciate your suggestion though.
 
You must log in or register to reply here.
Back
Top