#296020 - 29/03/2007 19:47
using GPS to record a trip
|
new poster
Registered: 29/03/2007
Posts: 9
|
So, here's my plan. I'm doing a cross-USA road trip over the summer with some friends. Being the geeks that we are, we're bringing along a GPS, and plan on recording our trip so we can upload it to google earth, geosynch our pictures, fun stuff like that. I have a GPS unit (Garmin eTrex Venture) that does your standard NMEA output to the serial port. I want to record the data it outputs so that we can later upload it to a computer and play with it. Using a laptop, this is easy, as there are all sorts of GPS apps that let you configure every minute detail of what you want to do. I know I won't be able to do all that on the Empeg, but I'm pretty sure I can accomplish everything I want just by recording the raw data that comes in from the serial port. I can later parse it on the computer and do whatever with it. Seems simple enough, but i'm not sure how to go about it. Would running a shell script that just did something like "cat /dev/ttys0 >> gps.log" work? My other problem is that according to my (probably wrong) calculations, assuming 115 kilobits/sec coming in thru the com port and 12 hours of driving a day (an overestimate), I could end up with as much as 50 gigs of data if it's always recording (the trip is 6 weeks long), which is too much for me to set aside. Is there any way to only record data every 10 seconds, with the data still being useful? I don't know much about NMEA, so I'm not really sure how that works. And although I can hack my way through shell scripting, if it turns out I have to write REAL code, I'm pretty much screwed. I appreciate any input. Thanks!
|
Top
|
|
|
|
#296021 - 29/03/2007 19:58
Re: using GPS to record a trip
[Re: kilbasar]
|
carpal tunnel
Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
|
NMEA is 4800bps, so that would be less than 5% of your original estimate based on 115kbps, so maybe 2GB.
That said, nothing is mounted r/w on the empeg while it's in the car. You could make it mount something r/w, but you risk corrupting data when you shut it down.
Also, the empeg will spit out data on the serial port on boot. It probably won't brick your GPS, but I don't know that I'd trust that. I think Mark has added some stuff to Hijack to allow app support for the serial port, which is usually the console, but I don't know how that all works.
Honestly, I'd be inclined to either use a laptop, if laziness is the top concern, or get some single-board PC with a big-ish flash disk running Linux if it's not.
_________________________
Bitt Faulk
|
Top
|
|
|
|
#296022 - 29/03/2007 20:00
Re: using GPS to record a trip
[Re: kilbasar]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14496
Loc: Canada
|
Most (all?) consumer grade GPS units report position over serial once per second. So the amount of data to be recorded is actually quite small, even over a long period.
|
Top
|
|
|
|
#296023 - 29/03/2007 20:40
Re: using GPS to record a trip
[Re: wfaulk]
|
new poster
Registered: 29/03/2007
Posts: 9
|
Hmm... so recording everything isn't that much data. That's good, and simplifies things. But yeah, I forgot that the empeg keeps the hard drives read-only, and spun down most of the time. I guess there's probably a way to hold the data in memory until a certain amount has been recorded, then write it in chunks to the hard drive. That would help with keeping the drive from spinning all the time, which wouldn't be good for it. How much memory is there for apps to use? Probably not much. As for the R/W thing, that's a bit trickier...
|
Top
|
|
|
|
#296024 - 29/03/2007 20:43
Re: using GPS to record a trip
[Re: kilbasar]
|
carpal tunnel
Registered: 20/12/1999
Posts: 31602
Loc: Seattle, WA
|
I notice that was your first post. Welcome! Did you just buy the empeg?
If your upcoming trip is Very Soon, I'd say forget it and do it all with a laptop. Getting the kinks of the system ironed out for an empeg-based application will be tricky and take some time and some dry runs, with questions answered by the BBS members here.
Your biggest issue is that you'd need to do some hacking to enable a partition on the player to stay read-write the entire time. I think there's an extra partition that might be big enough that you could format and use for your own nefarious purposes. But it'd be tricky to configure.
|
Top
|
|
|
|
#296025 - 29/03/2007 20:53
Re: using GPS to record a trip
[Re: tfabris]
|
new poster
Registered: 29/03/2007
Posts: 9
|
Actually, I've posted here before (only once or twice), but I guess I forgot my login, so I'm brand-new again. I've had my empeg for a little over 2 years now, and have loved every minute of it. Although I'm not sure if I like it more for its actual features, or just for how much it impresses people who get in my car. My trip isn't until July, but I'm trying to get things ready well in advance so that it'll all work out. Originally I planned on just using a laptop, but then I thought "Y'know, I've already got a linux computer with a serial port in my car, why don't I just use that?" It also seems like a fun project.
I couldn't just re-partition one of the drives however I like? I don't mind losing my data, as I can put it all back fairly easily.
|
Top
|
|
|
|
#296026 - 29/03/2007 20:55
Re: using GPS to record a trip
[Re: kilbasar]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14496
Loc: Canada
|
Quote: Hmm... so recording everything isn't that much data. That's good, and simplifies things. But yeah, I forgot that the empeg keeps the hard drives read-only, and spun down most of the time. I guess there's probably a way to hold the data in memory until a certain amount has been recorded, then write it in chunks to the hard drive. That would help with keeping the drive from spinning all the time, which wouldn't be good for it. How much memory is there for apps to use? Probably not much. As for the R/W thing, that's a bit trickier...
The whole job can be done with a moderate bash shell script running on the empeg. You can use the Hijack ;@EXEC_ONCE feature to (re)launch it at boot. The script could create a ramdisk device, and then use dd to copy from the serial port to a temp file on the ramdisk.
Each time the file hits a defined size (eg. 1MB, determined by parameters to dd), the command would exit, and the script would then continue: remount the disk RW, use cat to append the temp file's contents to a master log file, and then set the disk back to RO, nuke the temp file, and loop again.
The tricky bit is ensuring the drive gets set RO before any power-off of the player, as otherwise a possibly lengthy fsck will cripple it at next boot. There's a config.ini setting somewhere that controls power-off after the ignition turns off, and one could tie into that.
This could take a couple of hours for a reasonable UNIX script writer to get working correctly.
Me? I just use a laptop for stuff like this!
|
Top
|
|
|
|
#296027 - 29/03/2007 21:03
Re: using GPS to record a trip
[Re: mlord]
|
carpal tunnel
Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
|
Actually, something like this would be a great thing to use ext3 for.
_________________________
Bitt Faulk
|
Top
|
|
|
|
#296028 - 29/03/2007 21:13
Re: using GPS to record a trip
[Re: kilbasar]
|
carpal tunnel
Registered: 20/12/1999
Posts: 31602
Loc: Seattle, WA
|
Quote: I couldn't just re-partition one of the drives however I like? I don't mind losing my data, as I can put it all back fairly easily.
The player is picky about its partitions when you apply firmware upgrades. If you mess with the partitions, you might mess that all up. Supposedly there is a spare 32 meg partition reserved for stuff just like this. I don't know what the details are regarding how to access it.
|
Top
|
|
|
|
#296029 - 29/03/2007 21:36
Re: using GPS to record a trip
[Re: tfabris]
|
addict
Registered: 02/04/2002
Posts: 691
|
Why not just pickup one of those nifty IDE>CF adapters, grab a decent size cf card, and setup the scripts to write directly to the flash device? Then after the trip they could use the card for those nifty new hijack features that are in the works
_________________________
Oliver
mk1 30gb: 129 | mk2a 30gb: 040104126
|
Top
|
|
|
|
#296030 - 29/03/2007 21:52
Re: using GPS to record a trip
[Re: oliver]
|
new poster
Registered: 29/03/2007
Posts: 9
|
Quote: Why not just pickup one of those nifty IDE>CF adapters, grab a decent size cf card, and setup the scripts to write directly to the flash device?
that could work, except i'd have to take out one of my hard drives (i have a 2-drive setup)
Quote: The tricky bit is ensuring the drive gets set RO before any power-off of the player, as otherwise a possibly lengthy fsck will cripple it at next boot.
The GPS will mostly be running during long, well-defined drives, so I wouldn't mind if I had to manually start/stop the script (as long as it's something I can do from the player menu, ie something in the Hijack menu). This way, the script can set it back to RO before it closes, and then I just have to make sure I close the script before I power off the player.
Quote: Supposedly there is a spare 32 meg partition reserved for stuff just like this.
According my revised calculations (this time somewhat based in reality), the NMEA data will come to about 2 megabytes per hour. Not a lot, but when you're talking about 6 weeks of driving, 32 megs won't be enough. On the other hand, assuming I go the path of making a RAM drive for temp storage, even if the RAM drive is only 1 meg in size, that means I'll only need to write to the hard drive every half hour or so (plus right before the script closes). In which case, it doesn't seem like it would be too risky to have the normal data partition set to RW for a couple of seconds every half hour. Does that make sense?
Edited by kilbasar (29/03/2007 22:10)
|
Top
|
|
|
|
#296032 - 30/03/2007 06:58
Re: using GPS to record a trip
[Re: kilbasar]
|
veteran
Registered: 01/10/2001
Posts: 1307
Loc: Amsterdam, The Netherlands
|
Are you stuck with the GPS you have? I am doing the logging using my TomTom 700 - remember it runs Linux too.
|
Top
|
|
|
|
#296033 - 30/03/2007 11:55
Re: using GPS to record a trip
[Re: kilbasar]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14496
Loc: Canada
|
Quote:
According my revised calculations (this time somewhat based in reality), the NMEA data will come to about 2 megabytes per hour. Not a lot, but when you're talking about 6 weeks of driving, 32 megs won't be enough. On the other hand, assuming I go the path of making a RAM drive for temp storage, even if the RAM drive is only 1 meg in size, that means I'll only need to write to the hard drive every half hour or so (plus right before the script closes). In which case, it doesn't seem like it would be too risky to have the normal data partition set to RW for a couple of seconds every half hour. Does that make sense?
Yes, that makes sense.
As Bitt suggested, you can also simplify handling of the RO/RW status by installing one of mcomb's ext3 enabled Hijack builds (or build it yourself from the standard Hijack source code -- it's just a config option that needs to be enabled). With ext3, the drive could be left mounted RW without much chance of harm, giving less risk of something bad happening should the empeg power down suddenly.
Cheers
|
Top
|
|
|
|
#296034 - 30/03/2007 12:08
Re: using GPS to record a trip
[Re: kilbasar]
|
carpal tunnel
Registered: 13/07/2000
Posts: 4180
Loc: Cambridge, England
|
Quote: According my revised calculations (this time somewhat based in reality), the NMEA data will come to about 2 megabytes per hour. Not a lot, but when you're talking about 6 weeks of driving, 32 megs won't be enough. On the other hand, assuming I go the path of making a RAM drive for temp storage, even if the RAM drive is only 1 meg in size, that means I'll only need to write to the hard drive every half hour or so (plus right before the script closes). In which case, it doesn't seem like it would be too risky to have the normal data partition set to RW for a couple of seconds every half hour. Does that make sense?
If you're prepared to do a bit of coding for it, you can eliminate remounting altogether by pre-preparing a file of large-enough size, working out which disk sectors the file occupies using ioctl(FIBMAP), and writing to those sectors directly in the partition block device. This reduces the risk of a power failure hosing your filesystem, though of course it increases the risk of a bug in your own code hosing the filesystem...
Peter
|
Top
|
|
|
|
#296035 - 30/03/2007 12:19
Re: using GPS to record a trip
[Re: peter]
|
carpal tunnel
Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
|
From OP: Quote: And although I can hack my way through shell scripting, if it turns out I have to write REAL code, I'm pretty much screwed.
_________________________
Bitt Faulk
|
Top
|
|
|
|
#296036 - 30/03/2007 14:16
Re: using GPS to record a trip
[Re: kilbasar]
|
old hand
Registered: 09/01/2002
Posts: 702
Loc: Tacoma,WA
|
If cost isn't much of a concern just get one of these for $150 and your set.
|
Top
|
|
|
|
#296037 - 30/03/2007 15:38
Re: using GPS to record a trip
[Re: siberia37]
|
carpal tunnel
Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
|
Or, since he already apparently has a serial-out GPS receiver, one of these for $60 and an SD card.
_________________________
Bitt Faulk
|
Top
|
|
|
|
#296038 - 01/04/2007 21:56
Re: using GPS to record a trip
[Re: siberia37]
|
new poster
Registered: 29/03/2007
Posts: 9
|
Quote: If cost isn't much of a concern just get one of these for $150 and your set.
cost is a concern. i already have the GPS, i already have the Empeg, so why spend another $60 for a device I'll only use once?
From what I've heard here, it seems like I should be able to get this working with a not-too-complicated script. I'll be out of town for the next two weeks, but will start writing the script when I get back. Thank you all for your help, and I'll be sure to bug you again once I start the code writing!
|
Top
|
|
|
|
#296039 - 02/04/2007 00:13
Re: using GPS to record a trip
[Re: kilbasar]
|
enthusiast
Registered: 20/08/2002
Posts: 340
Loc: Pittsburgh, PA
|
The etrex venture already logs a track, so really the only thing you need is something like gpstrans on the empeg to download the log once a day or so.
Oh, and bring lots of batteries.
Edited by jaharkes (02/04/2007 00:25)
_________________________
40GB - serial #40104051 gpsapp
|
Top
|
|
|
|
#296040 - 02/04/2007 00:33
Re: using GPS to record a trip
[Re: kilbasar]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14496
Loc: Canada
|
Quote: i already have the Empeg, so why spend another $60 for a device I'll only use once?
Exactly. And where's the fun in just spending one's way to a solution?
Quote:
From what I've heard here, it seems like I should be able to get this working with a not-too-complicated script. I'll be out of town for the next two weeks, but will start writing the script when I get back. Thank you all for your help, and I'll be sure to bug you again once I start the code writing!
I think there'll be a few tricks and traps along the way with this, but just keep updating us here as things progress, and I (and others?) will help you out with it.
Cheers
|
Top
|
|
|
|
|
|