Before setting out on this trip I knew that I wanted to track our location. This wasn't really so people could find us wherever we were (though it doesn't bother me; come visit if you are close by!). Rather, it was so I could plot the route on a map to help memorialize it. My original plan was to buy a cheap phone, write a tracking app for it, and throw it in a cabinet somewhere to do its thing.
Hold that thought.
I mentioned in an
earlier post that my fancy modem has the ability to post its location every few seconds. It has this ability because the modem is intended for mass transit vehicles like buses or trains. This functionality allows a dispatcher to keep tabs on the vehicles. I didn't actually know it could do this when I ordered it, but once I found out it could...
See where this is going?
I've been tracking our location since we pulled out of our driveway a month and a half ago, but I wasn't doing anything with that information. Until now. You can now pull up
https://rv-location.thejohnsfamily.org and see were we are, roughly in real-time (hover over the pin to see a timestamp). As long as we have cell service, the location should be accurate to within 60 seconds. One caveat: This is the
RV's location. We may or may not be there at any given time, but close enough.
I also embedded the map into the bottom of this blog.
The final product is about a simple as it gets: a pin on a map, but it wasn't nearly as straightforward as I hoped it would be. Read on if you want to know how the sausage was made. Perhaps my experience will be helpful if you ever think of trying something similar.
If your eyes glaze over at technical talk, do yourself a favor and stop now.
The modem's administrative console offers me the following configuration options:
And the following documentation to go along with it:
This is perfect! I just need to stand something up to listen for these locations, point to that something, and let it go! I don't know what a "NMEA Sentence Type" is, but I knew the internet would help fill in the gaps for me there. Once I had the data, I was confident I could do what I needed to with it.
And that turned out to be true. Using the data was easy. Capturing the data, not so much. After all, I wouldn't bother writing this up if it were, would I?
Disclaimer for the rest of this post: I'm using AWS services to pull this off. So instead of constantly saying "AWS Lambda" or "AWS API Gateway," I'm going going to drop the "AWS."
I figured I'd stand up a Lambda to catch the event posted by the modem. This is a perfect use case for a Lambda: infrequent data that doesn't need a whole lot of processing once it's received. I'd just catch it and slam it into a Dynamo DB table to deal with later.
So I did it. I stood up a Lambda, pointed the modem to it and waited. And waited. And waited. It quickly became apparently that no data was reaching my Lambda, but there was no useful logging on the modem, so I had no clue where in the process the breakdown was happening. Did I configure the modem wrong? Did I configure my Lambda wrong? Was I sending GPS data to some other random server on the interwebs?
I wasn't sure, but I decided to shelve that for a bit.
I knew I eventually wanted API Gateway to front my Lambda, so I decided to work on that. I also wanted it all behind TLS (remember this), and I needed API Gateway to pull that off. And maybe I could turn some logging on at the gateway and glean some more information.
I have rolled this exact setup multiple times in the past, so putting API Gateway in front of my Lambda was uneventful. Once I had that in place, I configured a Route 53 record to point to it so I could put it behind a proper domain name. No sweat.
Time to try again. Still nothing. I was hoping for an easy win, but this certainly wasn't the first time something I wrote didn't work, so I shifted into debugging mode. If this weren't a day or two before departure, I may have even enjoyed it. But I wanted to capture our trip from the very beginning, so I was on the clock. That makes debugging decidedly less fun.
Given the limited configuration options in the console, there wasn't a whole lot to try. I messed with ports. I messed with the URL. I messed with the reporting interval. No luck. Maybe this thing just doesn't work. I paid too much, so that didn't seem right. Plus, I could see my location in the admin console, so I knew the GPS portion of it worked. It was just the "post the location to a server" portion that wasn't working.
Maybe it doesn't support TLS. This is 2021, so that didn't seem right either.
Spoiler: My fancy expensive modem doesn't support TLS, at least not for GPS location posting. (It also doesn't support paths in the URL. I guess that's why they label it "IP Address / Host Name," so points to them for being precise.)
Second spoiler: API Gateway requires TLS for external connections (there is a way around this internally; more on that later).
Ok, so I can't post directly to API Gateway. To what can I post? I decided to spin up a micro EC2 instance to set in front of it. I could slip it into the AWS free tier, at least for the first year. I'll just put a simple NGINX proxy on there to catch the post and forward it on to API Gateway using TLS. Easy, right?
Come on, you know better by now.
I also have a fair bit of EC2 and NGINX experience, so standing up the proxy was easy. Pointing the modem to it was also easy. But wait... why was none of the data showing up in API Gateway? Ah, turns out the modem doesn't send the data in a proper HTTP request, so NGINX rejects it. I'm positive that this is due to having to be compliant with legacy GPS data processing systems, but it sure wasn't helping me.
All I needed was a simple GET request, is that too much to ask?
[Insert montage of me mucking around with NGINX configuration for a few hours, interspersed with me punching slabs of meat in a freezer, all set to "Eye of the Tiger."]
Got it! I'll just configure NGINX's error handler to forward the thing on. Normally an error handler would just log the error, but there's no reason I can't use it to actually keep things moving. That was my thought at least, and this time, I was right! Behold, my NGINX error handler:
# The Router sends invalid headers, which always results in a 400
error_page 400 /redirect;
location = /redirect {
resolver 8.8.8.8;
proxy_method POST;
proxy_set_body "{ \"payload\": \"$request\" }";
}
In layman's terms this says "If you get something that would normally result in a 400 error, like invalid headers, use the /redirect location as the error page." And then, "When someone hits /redirect, POST the whole request to API Gateway." If you are thinking that this is a highly unusual use case for an error handler, you are correct.
But you know what?
It worked gloriously! My Lambda (via API Gateway) was now receiving data. It wasn't a slam dunk, but it was more or less downhill from there. I needed to figure out how to interpret the data, and as I suspected it would, the internet helped me out.
Once I had the data in a usable format, I threw it into a DynamoDB table. Actually, two. One table stored all locations posted, keyed by date. The other table just stored the latest location. That set me up to trace our overall route as well as quickly respond to a request for the latest location.
I enhanced my Lambda to not only know how to process POSTs of location data but also GETs for data. As of now, it only knows how to GET the current location. It will eventually know how to get locations for a time range so that I can show trip progress. Baby steps.
I then whipped together a dead simple React application that hit the Lambda (via API Gateway, as before) to get the latest location. It marked that location on a map with a pin.
All so that you, dear reader, can know where we are.
When I get up the energy to dive back in, I'm going to attempt to put CloudFront in front of API Gateway. CloudFront does support non-TLS connections, so I may be able to cut out the NGINX proxy. What I don't yet know (because I haven't tried) is what it will do with the invalid request. If I can get it to ignore the fact that it's invalid and send it on to API Gateway, it might just work. Oh, and I also need API Gateway to ignore the fact that it's invalid and send it on to Lambda. I'm skeptical, but I'd really like to do away with the EC2 instance, so I'm going to give it a shot.
Just not now.
Update: I didn't update the original post with this information because I wanted to preserve my original thought process and not create revisionist history by pretending I understood this all along. But I did want to add this coda to put a bow on things. My hat's off to those of you that figured out the problem while reading the original post.
About a month after writing this (yes, an embarrassingly long time), it finally hit. The reason AWS doesn't know how to handle the request and the reason NGINX thinks the request is invalid is because it's not a request at all. It's a raw socket connection. I've done a bit of socket programming in the past, but it's been a long time. We live in a GET and POST world now, so it never even crossed my mind that it would be making a socket connection. But it was. Look closely at the label in the screenshot above. See how it says "IP Address / Host Name" and not "URL"? Also see the "Port" and "Protocol" fields? Yep, sockets.
Frankly I'm a bit surprised that NGINX even handled a socket connection to the point where I could grab the content via the error handler, but I'm sure glad it did! I would have eventually figured it out and written my own little socket handler, but the NGINX kludge was easier.
Comments
Post a Comment